JPN-92-024, Responds to Requesting That Util Review FitzPatrick IPE to Assess Accuracy of Ipe'S Underlying Evaluations & Conclusions & Reconcile W/Findings of Det

Responds to Requesting That Util Review FitzPatrick IPE to Assess Accuracy of Ipe'S Underlying Evaluations & Conclusions & Reconcile W/Findings of Det

ML20097A244
Person / Time
Site:	FitzPatrick
Issue date:	05/28/1992
From:	Ralph Beedle POWER AUTHORITY OF THE STATE OF NEW YORK (NEW YORK
To:	Murley T Office of Nuclear Reactor Regulation
References
JPN-92-024, JPN-92-24, NUDOCS 9206020235
Download: ML20097A244 (70)

Text

. - <

m Mam sneet Wh.1e Piamt. New Y:;r k 10601 914 fel f 84f-

1. > NewYorkPower no m .o....

4# Authority =;g=;r"'

May 28,1992 JPN 92-024 Dr. Thomas E. Murley Director - Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission 11555 Rockville Pike Rockville, MD 20852

SUBJECT:

James A. FitzPatrick Nuclear Power Plant Docket No. 50 333

[Dilividup! Plant Examination

Reference:

NRC letter, T.E. Murley to J.C. Brons, dated February 6,1992, requesting a review of the FitzPctrick IPE with respect to the NRC's Diagnostic Evaluation Team Report.

Dear Dr. Murley:

The Authority has evaluated the FitzPatrick Individual Plant Examination (IPE) to consider issues raised by the NRC's Diagnostic Evaluation Team (DF1). This evaluation is in response to the referer.::ed letter which requested that the Authority review the FitzPatrick IPE to assess the accuracy of the IPE's underlying evaluations and conclusions and reconcile them with the findings of the DET.

The evaluation of a FitzPatrick IPE is detailed in Attachments I through !!!

to this letter. Attachment IV provides a copy of Section 2 of the DET report annotated to provide a cross reference between the IPE evaluation and the specific findings of the DET. The more significant findings of the Authority's evaluation are described below.

1.

The FitzPatrick IPE models end data bases were constructed consistent with state of the art Probabilistic Hisk Assessment (PFsA) technology. (See Attachment I)

2. The plant records used in the FitzPatrick IPE are much more recent than from the early 1980s. Two data bases have recording periods ending in December,1990, and the initiating event data base is current througn December,1989. The least current data base, " Equipment Failure Rates and Unavailabilities", provides information through September,1986. (See Attachment 1).

9206020235 920528 ),h PDR ADDCK 05000333 j p PDR .

q ll

, 3. The Core Damage Frequency (CDF) calculated for the FitzPatrick p' ant is comparable to that calculated by the NRC for the Peach Bottom plant in NUREG/CR 4550. The FitzPatrick CDF is lower than the Peach Bottom CDF hy about a factor of 2, almost entirely bect.use of the reduced frequency of anticipated transient without scram (ATWS) initiated core damage events at the FitzPatrick plant. Comparisons to other boiling water reactors (BWR) were H!so made. Many of their CDF values fall within the uncertainty band of the FitzPatrick IPE. The Duane Arnold plant, another BWR with a Mark I containment design similar to FitzPatrick, has a virtually identical CDF to the FitzFatrick plant. (See Attachment II)

4. Manv issues contained in the NRC's DET report would not appreciably affect the FitzPatrick CDF. This was determined by applying a series of screening tests. The results of these screening tests and specific studies of OET issues are provided in Attachment 111. Approximately three quarters of the DET comnients that were evaluated through the screening tests and specific

studies were judged to have little or no impact on the FitzPatrick CDP.

A few DET :omments wo'o related to issuec already considered in the FitzPatrick IPE Some DET comments focused on oxternal events such as fires, v.tich are beyond the scope of the IPE. These issues would be part of the plant d Individual Plant Examination for External Events (IPEEE) analysis.

Initiating vents which are the subject of other DET comments were already captured oy thn relatively current initiating event data base issues that portain to human error were encompassed within the conservative human reliability analyses used in the FitzPatrick IPE. Finally, certain DET concerns such as offgas system problems, do not affect systems modelad in the IPE and would not affect the FitzPatrick CDF value.

In addition to thr, scraening tests, specific issues relating to emergency service water (ESW) and resideal heat removal service water (RHRSW) pump room ventilation failure, design errors, and the EDG air start system design 3 deficiency were examincd in more detail. None of thesa DET issues change the FitzPatrick CDF appreciably.

Two types of PRA CDF wnsitivity analyses were performed. The first PRA sensitivity analysis showed that many systems can undergo large increases in their unavailabilities without approciably affecting the CDF. Comments on these systems would not lead 1 $ significant CDF increases. To resolve the remaining issues, a second sensitivity study was performed. Results of this second study show that of tha systems of concern to the DU the ESW system has the most potential for increasing the CDF. A .toser review of ESW DET issues revealed that they do not relate to those aspects of ESW operation that affect the CDF. Additionally, the NRC ESW SSFl team found the ESW system operable during their review at FitzPatrick which was completed on May 1,1992.

The findings described in the DET report do not represent significant deficiencies in the models used in the FitzPatrick IPE. The equipment data base would require updating fcr the post-September,1986 time period to numerically 1

- - _ _ _ _ _ _ _ - - _ __ J

e .

- quantify the effect of some DET issues. The Authority plans to update the l relevant IPE data bases as part of the Authority's "living PRA" process. However, the PRA sensitivity studies indicate that this update is unlikely to cause significant changes in the FitzPatrick CDF.

If you have any questions, please contact me.

I Very truly yours,

^

g ,L p,'/}rg '1 .j '

4 Ralph E. Boodle-Executive Vice President Nuclear Generation Attachments:

, t

l. Data Bases and Processes Used in Producing the JAF IPE II. Comparisons of the JAF CDF to other BWR CDFs Ill. Screening of DET Comments, Sensitivity Studies, and Recommendations IV. Annotated DET Report cc: U.S. Nuclear Regulatory Commission ATTN: Document Control Desk Mail Station P1 137 Washington, D.C. 20555 Regional Administrator

, U.S. Nuclear Regulatory Commission l 475 Allendale Road King of Prussia, PA 19406 Office of the Resident inspector U.S. Nuclear Regulatory Commission P.O. Box 136

- Lycoming, NY 13093 Mr. Brian C. McCabo Project Directorate 11 Division of Reactor Projects l/11-U.S. Nuclear Regulatory Commission

. Mail Stop 14 B2 Washington, DC 20555-

ATTACHMENT I l DATA BASES AND PROCESSES USED IN PRODUCING THE JAF lPE Questions were raised about both the data bases and processes used in the development of the JAF IPE. While these are described in detailin this Attachment,  ;

it is appropriate to note that JAF IPE data are more recent than the early 1980s, as shown in Tablo 1-1.

Table 1 1 Data Base Recording Period Equipment Failure Rates August, 1980 to September, 1986 and Unavailabilities*

Initiating E' rents January, 1976 to Ducember, 1989 Plant Modifications through December, 1990 Procedures through December, 1990

'Some of the generic data in this data base may be more recent than September, 1986.

l l

1 I-1

. s i .

1. IPE METHODOLOGY AND REVIEW 1.1 The Modelina ProceJJ The James A. FitzPatrick (JAF) IPE was performed to meet the requirements of the Nuclear Regulatory Commission's Generic Letter No. 88 20'. It did so by employing state-of the art methodologies prepared or reviewed by or for the NRC".

A Loveli PRA was performed using the small event tree /large fault tree approach':

the containment performance analysis followed the general guidance given in Appendir 1 to the Generic Letter.

Event Trees. The identification and evaluation of accident sequences entailed the evaluation of event soquences described by the occurwnce of an initiating event and the subsequent responsos of systems. These responses determine the status of the core and containment the delineation of the sequence terminates with a determination of whether the core is safe or damaged. The ,

procedures used to perform the event tree modeling and accident acquence analysis are those described in NUREG/CR-4550, Vol.1, Rev.1".

Systems Analysis. The systems analysis comprised the development of fault tree models for the systems depicted in the event troos and for their support systems, and the development of a plant specific data base, humun-error data base, common-cause failure data base, with which to quantify the event sequences that lead to coro damage.

l The fault trees were developed as suggestod in the PRA Procedures Guide (NUREG/CR 2300), Analysic of Core Damage Frequency: Internal Events Methodology (NUREG/CH-4550, Volume 1, Revision 1), and other recent PRAs.

Particular attention was paid to the assumptions made in creating the trees'.

I united states Nuclear Regulatory Commission.

• Individual Nant Examination for seve o Accident Vulnerabilities -

10CFR 50.5410,* Generic Le:ter 88 20, Noveinber 23,1986.

2 Now York Power Aisthdty, ' James A. Fit Patrick Nue: ear Power Plant, indMdual Plant Examination,'

stember 1991, sectics 2 page 2-1.

lbid, section 2.1, page 2 2 lbic, section 2.3, page 2 3.

D. M. Ericson et al., ' Analysis of Core camage Frequency: Internal Events Methooology.' Gandia National Laboratories, NUREGICR-455o, s AND86 2004. Vol.1. Rev.1 January 1990.

6 New York Power Authority,

• James A. FitzPatrick Nuclear Power Plant, Individual Plant Enmination,*

september 1091, section 2.3.1, page 2 3.

i Ibid, section 2.3.2, page 2-3 I-2

l

han.k End Analysis. The back end analysis focused on the propagation of occident sequences subsequent to the start of coro damage'. As the back-end analysis and its conclusions do not appear to be of immediato concern, they will not be discussed here.

DSneInfEGY_Tmahtinnt. Both dependent and subtle failures were addressed in the IPEU . Explicit dependencies were included in the fault tree models; other dependencies were addressed as common-cause failures.

Subtle failures occur as design related inadequacies. A review of plant operating experience was made to determino it subtle failures had occurred. A walkdown and interviews were also conducted to identify potential subtle failures. Subtle failures to which other plants woro subject " were also reviewed to ascertain their applicability to JAf yyhntahllity identification and Treatr.nent. Accident sequences developed in

• the event trees were quantified by linking fault troos, and solving the resultant accident sequenco equations to obtain minimal cut sets"", A truncation value of 10', excluding initiating event frequency, was used in accident sequence quantification. This represents a scrooning criterion that is more conservative than that proposed by the NRC and ensures that the causes of at least 95 porcent of the accident sequence frequency are computed. After quantification, dominant accident sequences woro reviewed to ensure their vstidity and to ascertcin if recovery actions are feasible. The validity check was made to confirm that sequenco success criteria were not violated and that sequence cut sets did not imply the violation of Technical Specifications The dominant accident t.equence cut sets were also reviewed for potential human recovery actions, and, whero recovery is possible, requantified to account for recovery, Vulnerabilities were identified from both a review of the dominant accident sequences and a determination of three measures of importance: risk reduction, risk increase, and uncertainty. In identifying vulnerabilities to which JAF is susceptible, particular attention was paid to unresolved safety issues, and in particuler to the evaluation of decay heat removal and the potential for failuro of non saloty related I

lbid, section 2.3.3, page 2-4.

lbid, section 2.3.4, page 2 4.

Ibid, section 3.2.3, pages 3 355 to 3 363.

U

o. M. Ericson et al.. ' Analysis of Core Damege Frequency: Internal Events Methodology,' Sandia N6tional Laboratories. NuREGICR-4550, S AND86 2084, vol.1, Rev.1, January 1990.

U New York Power Authonty. " James A. Fit: Patrick Nuclear Power Plant, Individual Plant Exansnation."

september 1991, section 2.3.b. pages 2 4 to 2-5.

" lbid, section 3.3.6. pages 3 430 to 3-459.

I-3

f I control systems creating or complicating transients. The evaluation of loss of decay heat removal ertailed the quantification and review of accident sequences that result in a loss of containment heat removal.

l!1f9Enstion Assembly. The bulk of the plant layout _and containment information used in the IPE is contained within system descriptions, design drawings and documentation, procedures, and the Final Safety Analysis Report (FS AR)" " ". Where documentation was revised or issued while this IPE was being prepared (i.e., prior to December,1990), the analyses were changed to reflect any changes affecting 'he models or the results of this IPE.

While no PRA or IPE had previously been performed for JAF, PRAs had been performed under the auspices of the NRC on nuclear power plants similar in design to JAF. These PRAs were reviewed in the course of this IPE, particular attention being paid to the analyses performed for Peach Bottom Unit 2 under the NUREG-1150 program".

The intent of these reviews was to:

o Ascertain whether the assumptions made in the event tree modeling and systems analysis were appropriate o Provide a basis for the generic data base and an object of comparison fcr other plant specific, common-cause, and human error data bases o Gain insights into the detail and emphasis of previous reports and submittals.

Plant walkdowns were performed as an integral part of the IPE to clarify information and to identify spatialinteractions or dependencies". A total of 13 walkdowns were made: two for familiarization and to examine system interactions; two to examine instrument functional tests and calibration; six as part of the internal flooding analysis; one to examine heating, ventilation, and air conditioning (HVAC) dependencies; and two to examine human recovery actions. Extensive preparations were made prior to each of the walkdowns.

" Ibid, section 2.4, page 2 6

" Ibid, section 2.4.3, page 2 7.

" Ibid, Table 2.4.3.1, page 2-8.

U Ibid, section 2.4.2, page 2-6.

O ibid, Section 2.4.4 page 2 7.

I-4

-l

f 1.2 Rata The development of the plant specific failure data base entailed establishing a generic data base, determining component failure rates or demand probabilities from plant data, and the combining of the generic and plant data into a plant specific data base using Bayesian updating. The time periods over which plant specific data were gathered were as follows:

initiating events 1/1/76 to 12/31/89 Component failures, demands and unavailabilities 8/11/80 to 9/30/86.

The gathering of initiating event data was terminated when sequence evaluation began. The gathering of component data was terminated when the systems analysis began. The data collected represents the plant conditions at the time the sequence evalustion and systems analysis began, By extending the period for which plant-specific data were gathered and incorporating more recent data the quality of data willimprove. However, combination of generic data and increased time span for plant specific data will dampen the effects of any short term change in f ailure rates, i

The human error data base was compiled using the methodology described in the Accident Sequence Evaluation Program - Human Reliability Analysis Procedure (NUREGICR-4772). Plant operators were also observed in simulated accident scenarios, These observations provided qualitative and quantitative information nec,essary to obtain reliable estimates of human error probabilities. =

Initiatina Event Dale, Initiating event data were derived from both plant specific and generic data. As noted above, plant trip data were for the period from January 1976 to December 1989. Gene..; data for initiating events were taken from the ASEP Program Peach Bottom Study' , NUREG/CR-38628', and NSAC-14722 Where plant data were available, the frequency presented was a Bayesian update of generic data. Otherwise, generic data were used.

" A. D swain, " Accident sequence Evaluation Program Human Rehab+ty Analysis Procedure,'sandia National Laboratories, NuREG/cR 4772, s ANo86-1996, February 1987, 20 A. M. Kolaczkowski et al., " Analysis of Core Damage Frequency: Peach Bottom, unit 2 Internal Events,'

Sand a National Laboratories, NuREGICR-4550, s ANDB6 2084, Vol. 4, Rev.1 August 1989, 21 D. P. Mackowiak, ' Development of Transient Initiating Frequencies for use in Probabiliatio Risk Assessmente."

EG&G Idaho, NuREG/CR-3862, May 1985.

22

" Loss of offsite Power at u. s. Nuclear Power Plants Through" Ns AC 147, EPRI, March 1989.

I-5

- Failure and Unavailability Data.

Generic Data Failure data are required to quantify system unavailabilities and the sequences of events that lead to core damage. Component failure data can be obtained from generic data bases, from industry wide  ;

experience, or from JAF plant records. Data acquisition is highly subjectivo.  !

Particularly since there is little JAF plant specific failuro data, and generic data may not be necessarily applicable to JAF systern snedals.

In the IPE, generic data were gathered for cornponent failure rates and demand probabilities". Plant specific data were used for repair times and the unavailability of components due to maintenance and testing, in addition to the generic data ,

sources, plant specific data bases developed in other nuclear power plant PRAs l were also used. Like generic data, they provide indication of the data that might bo j anticipated for JAF. '

While generic data clearly do not always apply, plant d,ta cannot be relied upon exclusively since most comoonents have experienced few failores,if any".

Furthermore, even if a sufficient number of failures has occurred to make a statistically sound prediction of a failure rate, subsequent redesign or now equipment might have rendered the predictiolirrolovant.

The approach adopted in developing component failure data in the IPE was that used in all recent PRAs: the data are composites of generic data, other plant data, and JAF plant specific data, combined where appropriate with Bayesian analysis to provide a quantitatively consistent representation of expected equipment performance at JAF.

To obtain single values for component failure rates or demand probabilities for tho IPE, the values obtained from the various sources were first screened for their applicability. If, after screening, several possible values remained, a single genc ic data source was selected for each component or failure mode based on specific

. technical reasons for selecting that source. Whenever no single source was uniquely applicable, the source with the widest acceptance was selected (particularly if used in PRAs of plants similar to JAF)". Data from the various generic data sources were not combined.

Plant soecific Data. The development of a plant specific data base entailed using plarit data sources in the collection, compilation, and systematic evaluation of U Now Yo,k Power Authority, " James A. Fit Patrick Nuclear Power Plant. Individual Plant Examination,*

september 1991, section 3.3.2.1, page 3 364.

Ibid, page 3 366.

" Ibid. page 3 368.

I-6

component failure records, and data analysis and aggregation". The parameters used in the calculation of failure rates are described in the Interim Reliability Evaluation Procedures Guido".

The actual data cc,:lection and compilation activities were limited to a specific time period for which records are available and that represented plant operations pertaining at the timo the iPE commenced. The poriod, therefore, excludes the initial start up and extended outages. For the IPE, the porlod extended from 8/11/80 to 9/30/86, with the systems analysis and data gathering activities beginning in October 1986.

Component failure and operating data (i.e., component domands, operated hours, and hours in standby) were gathered for the components and failure modes included in the system logic models". To calculato plant specific failure data, the following information was obtained from the plant records for each component modeled:

o Failuro history: failure modes, start a,,d end times, descriptions, soverity, and cause o Exposure history: hours opertted, hours in standby, the number of actuation demands from operations, tests, and maintenance o Plant and system operating history: detailed histories of surveillance testilig, operations, and maintenance, Data for the plant specific failure data analysis como from plant maintenance work orders (MWO) , licenseo event reports (LER), JAF events reported to the nuclear plant reliability oata system (NPRDS), scram reports, monthly operating status reports (MOSR), and daily status reports (DSR)". In addition, the shift supervisor (SS) and nuclear control operator (NCO) logs provided the component, system and plant operating histories used to estimate compr>nent exposure data. To develop component demand data, surveillance test (ST) proceduros, system operating procedures (OP), maintenanco procedures (MP), the P & ID drawings, and system descriptions, were used, Discussions with operations and maintenance staff helped ensure that the data compiled reflect the actual component actuation and f ailuro history and resolve uncertainties as to component functions and failure modes.

Ibid, section 3.3.2.2, page 3 369.

U

o. o. Carlson.

• interim Rel.abihty Evaluation Program Procedures ouide," NuREG/CR 2728, January 1983.

j New York Power Authority, " James A. Fit: Patrick Nuclear Fower Plant, indMdual Plant Examination,'

september 1991, section 3.3.2.2, page 3 370.

Ibid, Appendices B and D.

I-7

~

Data Analysh. The data were analyzed by summing the failures for a specified basic event and determining exposure times for each component typo using system and component operating history, and surveillance, and maintenance histories. Demand spectra were developed for each component type using reactor statt up and shutdown information, appropriate STs, operating and maintenance domands, and operator input **.

D11AAgar12A1!0.0. Data aggregation followed data analysis. Data aggregation involved the Bayesian update of data for which both generic and plant data were available using the generic failuro rates or demand probabilities as prior distributions8 '. For components and failure modes for which no plant data wcro availablo, the data reported are generic.

System Unayalfability Durina Testand MainttRaDEg. System unavailability due to unscheduled tests and maintenance was quantified using a descriptive statistic: the ratio of downtime to the time tho system should have been available*'.

Scheduled testing of equipment was included in the models only if it contributed to significant system or subsystem unavailability most test activities do not impair system performance.

The main sourcos of data for calculating system maintenance unavailability were the DSRs, tht, ST histories, and the MWO data base. Information on the downtime of the analyzed systems was recorded including downtime caused by unscheduled tests and maintenanco. Refueling periods were determined from the SS logs, and system unavailabilities due to test and maintenance were then calculated for the modelod systems.

Uncertainty. The uncertainty in component failure rate and demand probability and system unavailability data is characteristic of the statistical uncertainty of the plant specific data and those factors that may affect component failures in the various uses and environments from which the generic data have been gathered.

A major source of error in plant data is the inadvertent omission of some component failures and demands. However, becauso a methodical and conscientious offurt was made by experienced personnel to ensure that the ple t data baso is as accurate as possible, the data base is as good as is reasonably achievable for the periods covered 83 Ibid, section 3.3.2.2. pages 3 370 to 3 371.

N lbid, page 3 371.

N lbid. section 3.3.2.3, page 3 373.

U Ibid, section 3.3.2.5. page 3 377 I-8 a, ---

Human ReliabilityJeta. The methodology for identifying and quantifying potential human failures was based primarily on the Accident Sequence Evaluation Program Human Reliability Analysis Procedure (ASEP HRAP, NUREG/CR 4772")*5 In addition, suggestions made in the Systematic Human Action Reliability Procedure (SHARP) concerning the representation of complex diagnosis events were included.

The goal of the HRA was to incorporate all the critical human error events into the models and to determine their nominal human error probabilities (NHEP). A team of systems analysts, human reliability analysts, and plant personnelidentified risk-relevant events that result predominantly from the action or inaction of plant personnel. The critical human errors identified were then characterized and quantified.

Different procedures were applied to determine human error probabilities (HEPs) for pre and post-accident tasks. Pre accident HRA focused on the return of system components to their normal states af ter maintenance, calibration, or testing. Errors in such tasks could result in unavailability of system components or inaccuracies in the data needed to respond to an accident. In extreme situations, such errors also could lead to human actions that could initiate an accident.

Post-accident HRA dealt with the activities of operators after an abnormal event".

These activities are usually undertaken to ensure that appropriate protective actions are accomplished. When an abncimal event is detected, operators must correctly diagnose the situation in time to take corrective action. Diagnostic tasks, and the performance tasks that follow, were examined separately in HRA. The effects on HEPs of task differences and the frequency with which tasks are practiced were handled in HRA by further categorizing post accident tasks into Rasmussen's skill-based / rule based / knowledge based schemes and adjusting error probabilities according to the ASEP HRAP guidelines.

In addition to the effects of task complexity and practice, the HRA also considered the affect of recovery factors (RFs) on HEPs, other performance shaping factors, '

emergency operating procedures (EOPs), and dependencies within and among systems and components". Actual data for the probability of restoring the plant l after electrical faults and restoring the power conversion systems were also used.

N A. D. swein, Accident sequence Evaluation Program Human Reliability Analysie Procedure,' Sandia National Laboratones. NuREG!CR 4772, s AND861996 February 1987.

New York Power Authority, " James A. Fit: Patrick Nuclear Power Plant. Individont Plant Examination,'

september 1991, section 3.3.3.1, page 3 377, and Appendix E.

Ibid, page 3 378.

" Ibid, sections 3.3.3.2 and 3.3.3.3, pages 3 378 to 3 382, and Appendix E.

I-9 1

i

_ - - _ _ _ _ _ . . _ - - 1

SigmlAllDD. Operators were observed in accident simulation exercises to ideritify the operator tasks and associated activities, to evaluate the tasks and operator performance, and to ensure proper integration of the EOPs into the IPE analyses". These simulations provided valuable information for the HRA, especially with respect to:

o Interaction among operators o The manner in which operators implement the EOPs o The time required to perform various steps in the EOPs.

The effectiveness of operator performance, and thus the HEPs, in the potentiMiy stressful conditions of concern to the HRA, is a function of crew structure, communication, and procedural compliance.

in the simulation exercises and the HRA, a minimum control room staffing level at the start of an accident sequence was assumed. The identity and arrival time of additional staff were then determined from discussions with JAF personnel. The minimum control room staff assumed comprises one senior reactor operator (SRO)-

(either the shift supervisor or assistant shift suporvisory), who also acts as the shif t technical advisor (STA), and one reactor operator (RO), (either the senior nuclear operator or nuclear control operator). The latter is always in the control room within the main control panel " horseshoe" area, in addition to the minimum control room staff, the shif t crew on site also includes an additional SRO, an additional RO, and four auxiliary (non-licensed) operators.

These problems were assumed to be outside of the control room when a reactor scram occurs. They return immediately to the control room to form an " accident response" team and become available to implement plant procedures.

1.3 Review

{

The methodology, data, results, and conclusions of the IPE were reviewed at several levels":

o NYPA Systems Analysis Group staff and consultants examined each other's work at each stage of development. These reviews focused on the accuracy and consistency within their areas of expertise.

o tiYPA Corporate staff from the Nuclear licensing, operations and maintenance, and engineering departments were kept apprised of the

" Ibid, section 3.3.3.4, page 3 382.

" tbid section 5.2. pages 5-7 to 5 8.

1-10

_ . _ . - _ _ ~ _ _ _ _ . - . _ . _ . __ . . _ - _ . . . _ _ _ .= _ - ~ _ - _ _ _ _ _

. progress made; they reviewed the methodology and guidelines document, the system work packages and accident sequences.

J Cognizant departments at JAF including licensing, operations, maintenance, training, instrumentation and control, planning, and technical services reviewed the system work packages ar'.d accident sequences at two formal site reviews. They also reviewed the insights and recommendations derived from the study at a third formal review, o A formal, independent review was made of the draft final report.

The independent review committee comprised both NYPA staff (the Technical Advisor to the Executive Vice President of Nuclear Generation; the Manager, Nuclear Safety Evaluation (Chairman of the Safety Review Committee); the Director, Quality Assurance; and a Senior Nuclear Licensing Engineer from JAF) and three prominent outside experts:

o Dr. Norman C. Rasmussen, McAfee Professor of Engineering, Messachusetts Institute of Technoloby.

Professor Rasmussen provided an overview of the methodology, the application of fault trees and event trees, and confirmed the

" reasonableness" of the results when examined both in isolation and in comparison with Peach Bottom, o Dr. Gareth W. Parry, NUS Corporation Dr. Parry confirmed the adequacy and applicability of the event tree models and accident sequences and reviewed the scope of the analysis of subtle dependencies and data.

l o Dr. Alan D. Swain Dr. Swain validated the human reliability analysis described in the draft .

report with respect to its methodology, adequacy, and accuracy of results.

The consensus of the reviewers was that the report was "welllaid out and clearly written." Professor Rasmussen also noted that he was able to take one of the dominant sequences and follow it through tho study. Technical comments were both detailed and general. Professor Rasmussen found the methodology used to l- identify the accident initiating events " logical and consistent with l current ...... practice." He also found the review of common-cause failures, data l gathering, and human error probabilities to be good.

Dr. Parry expressed two principal concerns; the initial assumption that operator 40 lbid, section 5.3. pages 5 9 to 5-10.

I-11

recovery actions in ATWS sequences were independent, and the use of a time-based failure probability rather than a demand based failure probability for battery failure in station blackout sequencs The first concern was resolved bv the conservativo assumption of complete dependency between operstor recovery actions in ATWS sequences. The second was resolved by a recalculation of the probability of battery failure by treating it as a failure on demand. Dr. Party's other comt.. ants concerned details of the human reliablity analysis and the evolution of the accident sequences. These were resolved in discussions with Dr. Party and changes were made as appropriate to the analysis and report.

Dr. Parry concurred that there would be no significant dif ference between the common cause data used in this study and that derived using the methodology presented in NUREG/CR 4780, " Procedures for Treating Common Cause Failures in Safety and Rollability Studios."

Dr Swain's comments addressed the derivation of human error probabilities. His concerns were resWed in discussions with NYPA staff and consultants. Again, appropriate changes were made to the analyses and report. He did stato, however, that his overall impression was favorable in particular, he noted the use of information from simulator exercises and the fact that the use of ASEP HRAP methodology provided a built in conservatism that would serve to counter any concern for the levels of dependence and other assessed performance aspects. He was also impressed with the conservatism in tho human reliability analysis of pro-accident tasks. For example, the assumption that an error in one action weald be repeated wnere operators perform the same actions on different components in the same system.

The internal review team's comments largely pertained to details of the analysis and the analysts' interpretation and depiction of systems and sequences of evt.nts.

Their comments wore incorporated into the report.

In summary, the independent review team concluded that the study had been performed in a logical, reasonable, and thorough manner. Although certain changes were recommended, none of these changes would require a major revision of the analysis or the results obtained. The recommended changes were examined with the review team and appropriate changes were made to the analysis and the report.

I-12 k_ .. .. .

4 ATTACHMENT 11 COMPARISONS OF THE JAF CDF TO OTHER BWR CDFs in the NRC's February 6,1992 letter to the Authority on the JAF IPE, the NRC expressed the concern that the estimated CDF may be unusually low The Authority nas compiled a number of CDF results from other BWRs and finds that many of their mean CDF values fall within the 95 percent confidenco level of the JAF IPE (See Figures 111,-ll 2, and 113), in fact, one other BWR similar to JAF, Duane Arnold, has a CDF that is virtually identical to the JAF CDF. The sources of tuo CDF numbers for these other plants are various publications, including NUREGs aiid a survey of CDP values recently assembled by the Severe Accident Evaluation Committoo of the Boiling Water Reactor Owner's Group. Please note that the CDF values compiled by the Owner's Group are preliminary.

Although we can compare "mean ostimate" CDF values from plants plant, detailed reasons for similar and dissimilar results are not generally available at this timo. Some IPEs have higher CDFs than JAF because of internal flooding events, internal flooding is a negligible concern at JAF because of a favorablo plant design.

(Soo Appendix H of the JAF IPE.)

Although detailed comparisons of the JAF IPE to numerous other BWR IPEs are not feasible at this timo, the Authority made a detailed comparison to the Peach Bottom study (NUREGICR 4550, Volume 4 Rev.1). This was be particularly relevant sinco both plants have Mark I containments. This comparison was included in Section 1.4.3 of the JAF IPE.

The totalinternal mean CDF for peach Bottom is 4.5 x 10*/ year". This CDF value is not significantly different from JAF's 1,42 ' 10*/yr CDF. The CDFs due to station blackout at the two plants are very sim!" nd are the dominant contributors to their total CDFs. At Peach 8t,.. . :o station blackout contribution to the CDF is 2.2 x 104/yr, or about 49% of the total CDF. At JAF the station blackout contribution to the CDF is 1.7 x IO*/ year, or about 91% of the total CDF.

Peach Bottom and JAF were also compared for the contr;bution to CDF of transients with stuck-open mfety relief valves with loss of all ECCS injection,

, LOCAs with the loss of all ECCS injection, and trans;ents with the loss of long term containment heat removal. While there are plant specific differences among these throo accident groups, in aggregate they are similar for both plants, Approximately 11 percent of Peach Bottom's CDF is attributable to the sum of these 'hree accident groups, whereas the sum for JAF is about 9 percent of the CLF.

A!though thero are significant similarities between the CDFs for Poich M

Analyses, more recent then NUREGICR-45bo. indiceto a higher Peach Bottom CDF.

II-1

. ~ . . ,

Bottom and JAF, the ATWS contributions are markedly different. Virtually all the difference between Peach Esottom's overall CDF value and the JAF CDP is attributable to the ATWS contribution The ATWS contribution at Peacn Bottom, 1.9 x 104/yr, is by itself as large as the total JAF CDF. ATWS provides 42 percent of total CDF at Peach Bottom, but less i.i.sn 1 percev at JAF.

There are four principal reasons for the much lower ATWS CDF contributinn at JAF. First,is a dosign difference between the two plants. At JAF there are hose connections with a dedicated hose to connect the standby liquid control (SLC) tank to the control *od drive (CRD) pump. Procedures are in place for an operator to connect this hose to the suction of an CRD pump to inject the boron solution from the SLC tank into the reactor vessel to mitigate a postulated ATWS. No similar capability exists at Peach Bottom.

Second, credit was taken in the JAF IPE analysis for boron mixing within the reactor vessel using the RCIC system. The Peach Bottom analysis did not take credit for thim. Third, in the JAF analys, a higher probability was assumed for operators to initiate S'_C than was assumed in the Peach Bottom analysis. The value used for JAF was based on simulator observations. Finally, the Peach Bottom analysis always assumed closed MSIVs. The JAF analysis took credit for bypassing the MSIV interlocks and reopening a main steam line as called for in plant procedures. Use of this pathway to remove reactor energy reduces the importance of ATWS events.

In summary, the Authority does not consider the JAF CDF values significantly different from those published for Poch Bottom in NUREG/CR 4550.

Many important accident groups, such as station blackout, have quite similar CDF contributions, and almost all differences are attributable to the effect of ATWS sequences.

II-2

Figure il-1 .

Lamulative Probability vs. Core Damage Frequency 4 -

0.9 -

mean ce ..

m + x e A

>s ,7

~O

.c 0.6 --

2 o_

0.5 -- n a

_O O4 -- /

E j 0.3 --

02 --

/

0. , --

l

-0 l l l l 1.0DE48 1.0CY47 1.00ff4 I OCE45 1.0DE41a Internal Core Damage Frequency (/YR)

I

- - m- sr + ou.ne Arne + 3 mones x 2 ms e ivm A 1m j a

i

_ _ _ _ -- M _/

Figure 11-2 Core Damage Frequencies of Various BWR Plants 1.60E 45 -

h..-:' ,;J, h'f g := si 1.40E-05 --  : s

.'. 'g.-?. l

.: ?J I :.; . :Q f:;i %~ gy mc - V

.: s  %,

x

.,N'.

{ y&' 4)

g

..,,,. s's;.

1.00645 -- .rWS i-u'

..'( .:]. ' 5)'

n..

g ,.'

g 'r:' I N-D 6 00E46 -- tv;: 4:;-d  ?  ?

a r - m' s pf g O

ss -

[-

t. ,s . .f; ;4 ..ee I, '

6.00E46 - ' fg 'i[fg  : x 1.s'? "'a s fin's . ..

h "

x s k ;w"M  %

s.3 p gf s <

.s ;pg;[ .

"# --[ s .  %

F--

MMi .

k ;yr, r .. {;$. , ~  :

~x \El

-n 2z. N

'...,^1 vy:MA

.s,,," , . .

.+c',.

s.

) s ',- r -

-h '

ny:TA-7 .....,i-'E'i '~';2' -

Nj N , x e . 5:1 ('j N 2ME46 -- -

d'c : ' W' s'..

b. Ok:

p' .;s .jts f .y

% ' ;';-J m.

f.-';,

};;; ;z:.5 ! - s4 f.y (g'_y:3i -

At:

' i

-w  ;:.M: > t .. x s..s- *:% - .

( :*' l'

J+x: ,.

%W e,hN . y(

g . . , , , y { x-p:

Ni s Iq.' t a i f'  :

s g

3.s J v 2

, ' +. :-'

' .1 1 e r 11J. .

1

~

l e,

I. "

. ; [

x:)

I; I i i i i 1 4 i i 4 i Fazpatrkk Duane Arnold l

?

1

~

Figure 11-3 Core Damage Frequencies of Mark 1 Containment Plants 1.00E45 --

g,g NEi$dhd4 9.00EM -- [Qfy^pf h- x.fy - 9 w -

~ ' *.

8.MM --

v -

r'

,hllkk,.gs .h

' " * ~~

emy wa n%

y ?(

ema --

a pg 2 .

fmi+

s.-- 1. I gw(6'jf 8

mEm--

n

?

wl!$N$i ay

': $x, . ?q;jgll

) N~ gp p u-+qqq@ e jr 3~

su

s.

• 3MG -- . $$$ '

I :r - m  ;

.j O u

.w

.K  :

. Y dq$n E

2 COEM -- .

.y  :

l

~

! IBUjjp;;/ L Nt3i)  : -..

I- -2delp! ,

a sae n w sq 1.00E4 -- kh - 7 uge e Egy

, 34 ggsp P

mg(4:2 3 u@.n sj &h j q er ,aq

1. 1 e 7 i t _t

, , , , ,t , i Fftzpatrick Duane AmoL1 lw

_ - - - . . - - _ - . _ . . _ _ _ _ _ . _ . - _ _ ~ _ ~ - , _ _ - - . - . - . - . ~&-

- ~

I 4 4-ATTACHMENT ll1 SCREENING OF DET COMMENTS, SENSITIVITY STUDIES,-

- AND CONCLUSIONS AND RECOMMENDATIONS

1. INTRODUCTION The inoividualiterns of concern to the Diagnostic Evaluation _ Team.[DET) were divided into forty issues, as annotated in Attachment IV.

These forty issues were then examined in two ways. First, five screening tests were applied to eliminate issues which could readily be shown to have little or no impact on the present JAF CDF c :lculation.- Two series of PRA sensitivity-studies were then performed on the remaining DET issues to identify the DET comments which could have a significant impact on CDF. Those DET comments which, through PRA sensitivity analyses, were shown to have limited impact on the CDF were then set aside.

About three quarters of the DET comments were found to have a limited impact on the JAF CDF based on the screening tests and the first sensitivity study.

The second sensitivity study showed that the remaining issues were unlikely to result in a_large CDF increase.

2. SCREENING TESTS Five screening tests were applied to the DET issues.

The tests were:

A. - Do the issues relate to external events? Such issues were beyond the scope of the IPE submitted in _ September,--1991.

B. Are the issues addressed in the present data bases? -

C. Which issues have little or no impact on core damage frequency?

D. Are the issues bounded by the JAF IPE conservative human reliability analysis (HRA)?

E. Are the issues already addressed in the JAF IPE?

III-1

a The results of these five screening tests are summarized as follows:

2.1 Summary of Screenina Test Results

[A] Items beyond the scope of the IP_li. These items will be addressed in the IPEEE. Items 16,30,35 and 38 and part of item 6 are beyond the scope of the internal events IPE. (Items numbers reflect DET comments annotated in Attachment IV.)

(B) Items thalrould be reflectestin the oresent data bases, item 5 would be placed in this category.

[C] Items that can be exoticitiv excluded. These items are explicitly excluded from further consideration because of their low impact on core damage frequency. Items (4,11,14,17,18,19, 20, 21, 22, 24, 26, 21, 29, 31, 32, 34, 36, 37, and 40).

[D] Items bounded by JAF IPE human reliability analvsgs, items 1,8,9, and 10 are items that might contribute to human errors. They would be reflected in the human reliability analyses which were conservatively derived.

[E !tems Already included in the JAF IPE. Items 7 and 28.

Application of these screening test to DET issues is detailed below:

2.2 Qelpiled information A. Items Bevond The Scoce of The Present IPE

[6] (Sections 2.1.4 and 2.1.5, pages 8 to 10)---Poor material conditions and housekeeping. Such issues relate mostly to fire hazards, and would be addressed in an external events IPE.

[1 81 (Section 2.2.4, pages 17 to 18, item 1)-Failure to test or inspect check valves in the fire protection system for corrosion and sitting. Fire protection issues would be part of an external events IPti.

[30] (Section 2.3.2, page 26, item 8)--Fire protection weaknesses. Such issues would be part of an external events IPE.

[35) (S9ction 2.3.3, page 28, item 2)-Lighting fixtures not seismically insta; led.

This item relates to a seismic deficiency which would be addressed in the external events IPE.

[38) (Section 2.3.3, page 29, item 6)--Electrical cable separation.

This fire hazard issued would be addressed 'in the external events IPE.

III-2

- . . - . - - . - _ . - ~ . .- -- . . - . . . . - - . ~ . . -.- . . - ~ ~ . - - . . .

'.- 9 B. [tems That Would Be Reflected in the Present Data _ Bases 151 (Saction 2.1.2, page 7, item 3)--Oscillations in the main turbine EHC system.

~

These problems would_ha've'been manifested as trips and would have been -

included in the fairly up-to-date initiating event database.

-t C. Items That Can Be Exolicitiv Excluded (4) (Section 2.1.2, page 6, item 2)--Offgas system problems.

These problems are not relevant to the internal events IPE.

Ill) (Section 2.2, page 12, final paragraph)--A failure to identify the root cause -

of the failure of two motor-operated valves in the LPCI system.

Failure of the LPCIinjection valves will have a negligible offect on CDF since the low pressure reflood function can be performed by the core spray _

system. Failure of the LPClinjection valves would have no effect on decay heat removal function since decay heat is trannferred to the suppression pool (torus) by the loss of coolant accident or by the core spray system. Core spray transfers decay heat to the torus by reflooding to above the main ;

steam (ines and then through open safety relief valves (SRVs) to the tores.

Once decay heat is transferred to the torus by any means, the RHR/LPCI-system and RHR service water transfers the decay heat to the ultimate heat sink (Lake Ontario) by operating RHR in the containment _(torus) cooling mode.

[14] (Section 2.2.3, pages _15 to 16, items.1 and 2)--

Unqualified gear casa lubricant may have been used on environmentally qualified MOV's .

The use of unqualified gear caso lubricant would mainly affect two types of L

sequences: LOCA combined with loss of ECCS and transients (e.g. a stuck-

- open SRV) combined with loss of ECCS. ~These situations are of limited-concern to the CDF for the same reason as given in [11], above.

[17) (Section 2.2.4, page 18, item 2: Section 2.3.2, page 28,Litem 3)-Failure to t inspect small-bore piping in'the RHRSW system for corrosion and silting._ ,

Silting of.the RHRSW system could only impact the CDF high percentage if-

- the RHR beat exchanger were restricted.JHowever, periodic testing in the RHR heat exchanger precludes this type of failure.-

- III-3 I

L

1. . , _ , . . . . - , - , . . - . . . . . . . , _ . am,-.. ., ..m.,~ , - - , _ _ . . -,..,-. . y... ,, . , , - , . , , - - - , . . - , - . , ,,

e a i -

(18] (Section 2.2.4, page 18, item 3)--The PASS cooler has never been tested in an emergency configuration.

This equicment is not germane to the internal events IPE.

[19] (Section 2.2.4, page 18, item 4)--There was no independent testing of each of the EDG air start motor sets.

See response to issue 40.

[20] (Section 2.2.4, page 18, item 5)--l&C surveillance were completed after the due date.

Delays in performing surveillance would not affect point estimates of-predicted component unavailabilities.

[21) (Section 2.2.4, page 18, item 6) -Instruments uncalibrated/out of calibration.

Common cause failures of instruments due to miscalibration were modeled in the JAF IPE. The miscalibration of four reactor low pressure transmitters-and their associated trip units would be required to impact the CDF. This is unlikely.

122] (Section 2.2.4, pages 18 to 19, item 7) Temperature control and surveillance of the LPCI MOV uninterruptable power supply battery spaces were inconsistent.

The timing of battery degradation caused by elevated temperatures would have no impact on the ability of the batteries to function under accident conditions. Accordingly, temperature-induced battery degradation was not modeled.

l (24) (Section 2.3.2, page 23, item 1)--ES'N system water-hammer ' damage.

The portions of the ESW system susceptible to water hammer damage are not germane to the IPE. As such, the damage and its consequences were not modeled.

12 6) (Section 2.3.2, pages 24 to 25, item 4)--Small-bore piping failures.

Small-bore piping failures were not modeled as they will not lead to significant flow diversions.

127] '(Section 2.3.2, page 25, item 5)--Flow reversal tunnel problems.

No credit was'taken in the IPE for using the flow reversal tunnel. Therefore inability to use the flow reversal tunnel would not affect the CDF calculated in the IPE.

III-4 i

~

1 j

[29) (Section 2.3.2, page 26, item 7)--Condition of the intake and discharge tunnels.

Silting of the tunnels and suction bays have not caused any system failures (reduction of flow or pump damage). The prot: fem was detected and corrected, preventing future failures. The IPE accurately modeled the observed performance of the service water systems.

[31] (Section 2.3.2, page 26, items 9 and 10)--LPCI inverter and battery capacity inadequacies.

LPCI batteries A and B would perform their functions within a short time after a demand on them was made. Therefore the impact,if any, of subsequent radiation fields is not risk significant.

13 2) (Section 2.3.2, page 27, item 11)--ESW and RHRSW pump room ventilatico l failure.

I Pump room cooling was examined in the manner specified in NUREG/CR-4550, Vo!.1, Rev.1. That is, the objective was to perform a reakstic

~

analysis rather than the traditional design basis approach. Therefore, in addressing the ESW/RHRSW pump room ventilation dependencies, credit was taken for all systems and recovery actions where appropriate. During

! l an IPE walkdown, it was noted that the ventilation / fire dampers were normally open, the pump motors were the only heat sources, and that the fire doors can be opened (and are opened per operating procedures). At that time it was postulated that natural convection through the open dampers, i doors and exhaust would provide adequate neat removal. Subsequent calculations based upon rangervative assumptions have shown that the maximum heat rise within the room with all pumps running and the dampers open is 67' F. A preliminary calculation using the NUMARC 87-00, Appendix E models has shown that even with the fire doors and dampers closed, the steady state temperature would not exceed 204* F. These temperatures should not cause a pump or an MOV failure during the mission time required for these components in the IPE analyses, The case where fire causes the dampers to close will be a'ddressed in the External Events IPE.

[34) (Section 2.3.3, page 28, item 1)--Design errors.

The IPE models relied upon the rnost recent versions of controlled drawings.

In addition, walkdowr's were conducted and any apparent discrepancies

- between drawings were resolved. Accordingly, we contend that the models are as good as can be reasonably achieved. The 15 LERs cited in the DET as examples of design errors were reviewed. None of the cited design errors was risk significant.

III-5

_ . _ _ _ _ _ _ _ _ - ------_-J

13 6) (Section 2.3.3, page 28)--Use of de-icing heaters l Design basis events that require the availability of at least 18 de-icing l heaters have limited bearing on the JAF CDF.

[27) (Section 2.3.3, page 29, item 5)--EDG air start system design deficiency /FSAR deficiency.

The EDG air start system was modeled as designed and not as described in the FSAR. While a failure of the line connecting the two banks of air l receivers with the air start motors would disable them, such a failure has a low probability and, as such was explicitly not modeled.

I 14 0) (Section 2.3.4, page 30, item 8)--EDG fan failure.

This failure was modeled. Failure of a single EDG fan has no appreciable impact on CDF due to a success criterion of one out of four diesels.

D. Items Bounded by the JAF IPE Human Reliability Analyses The Power Authority believes that the HRA analyses in the JAF IPE are conservative and bound DET concerns. There are several reasons for this belief.

First, actual measurements were made of human performance during stressful conditions while at the simulator when severe challenges were being simulated.

DET concerns such as a tendency of not relying on plant indicators and informality wera not observed in such simulator tests. Second, the HRA models were based on minimum staff levels. Lastly, conservative assumptions were made for repeated errors.

[1] (Section 2.1.1, page 4, bottom paragraph)-Staffing levels were not the minimums necessary.

(Section 2.1.1, page 5,2nd paragraph)--The ope.ating shift structure was inefficient and impacted the ability of the control room shift supervisor to control activities.

The human reliability analysis made in the IPE assumed a minimum sts .

level. Only one licensed operator was assumed to be present in the control room within the " horseshoe" at the onset of an accident: the shift supervisor was assumed to be inside the control room but not necessarily within the horsc;hae.

[8] (Section 2.* ,6. page 11,2nd, 3rd, and 4th paragraphs) -Control room activitics are ir. formal.

Insofar as informality in control room activities would lead to problems in j responding to an accident, it is conservatively addressed by data developed l III-6 l

s in the human reliability analysis. Much of these data were gathered under stressful conditions where accident conditions were being simulated. During these simulator exercises no informality was observed.

191 (Section 2.1.6, page 11, bottom paragraph)--Noise levels in the control room are higher.

The human error data utilized in the JAF IPE is a composite of generic data and plant specific data. Noise levels that distract operators could affect human reliability, but should already be one of the many factors captured in generic human error data. The plant specific high noise levelissue noted by the DET should ciready be factored into the human reliability values recorded at JAF during the simulator tests.

[101 (Section 'l i.3, page 12, top paragraph)- Operators had a tendency not to rely on PMot indications.

The response to questicn 9 also applies here, it should be noted that this tendency was not observed to occur during simulator exercises.

E. Items Already included in the IPE

[7] (Section 2.1.2. page 7,3rd paragraph)-Operators were content to work with 400 temporary procedure changes.

The operating procedures on which the IPE models were based included tempc.ary procedures when applicable.

[28) (Sectinn 2.3.2, page 25, item 6)--Service water swing check valves. The potential flow diversion patt. that would result if the check valves were to fail open was modeled. In crder for there to be an impact on CDF, three check valves would have to tail; this is unlikely. Additionally, the generic data base upon which the IPE analyses of these valves was calculated is thought to have rather hi0h (conservative) failure rates.' Therefore the JAF IPE nuy a' ready be somewhat conservative in this regard.

2.3 S_gnsitivity Sindi_ea 1

At this point about 75% of the DET's origina! 40 issues have been shown to l have little cr no impact on the JAF CDF calculation. The remaining issues are 2,3, 12,13,14.15,23,25,33, and portions of 6 and 39.

A precise evaluatiom of these remaining issues would rer;uire extending the equipment failure rate and unavailability data base to more recent time periods as woaid be done in a "living PRA" update, however it is possible to acquire significant insights into the CDF impacts of these iemaining issues by using the JAF IPE itself to perform various sensitivity III-7

\

studies. The intent of performing these studies is to estimate the impact of postulated equipment degradations on JAF's CDF.

PRA sensitivity studies were performed on the JAF IPE to correlate the impact of increased system unavailabilities on the JAF CDF. The results of one-such study are displayed in Figure ill 1. Here, a number of JAF systems were assumed to undergo a doubling of their base case unavailabilities, and the resultant increases in CDF were observed. Such assumed increases in unavailabilities were taken one system a ~ a time. Many systems were examined and the top 11 are presented in Figur' . '. This analysis was a simple hand calculation and is thought to be conservative, i.e., it may overstate the increase in CDF for a change in unavailability.

Two important observations can be derived from this figure. _First, the CDF shows the greatest sensitivity to changes in the unavailability of the emergency service water [ESW) system. Second, the core spray system did not rank among the top eleven systems when its base case' unavailability was doubled. This second observation addresses item 25: -(Section 2.3.2, page 23, item 2)--Chronic core spray system pump deficiencies. Since the whole core spray system could undergo significant increases in its unavailability without impacting the JAF CDF, then deficiencies in a core spray pump should not affect the JAF CDF in an important way, in order to ovaluate the remaining DET issues, additional sensitivity studies were performed. Based upon the above screening process, the most important remaining DET/IPE systems are: ESW, RHR (including RHRSW), EDGS, and the HPCI systems.

Because the CDF is rnost sensitive to increases in ESW unavailability (see Figure 1111), higher multiples of increased ESW unavailabilities were examined.

Figure ill-1 considered an ESW base case unavailability increase of a factor of 2, whereas Figure 1112 used multiples of 3 and 5.

The above ESW studies were then expanded to develop insights on additional systems. Whereas Figure lil 2 represents CDF versus ESW unavailability, Figures lil-3 and 111-4 considered groups of systems in which base case -

unavailabilities were all simultaneously increased by 3 and 5, respectively. These additional sensitivity studies were more sophisticated than the hand calculation utilized for Figure 1111. Here, the sensitivity studies were performed by PRA computer analyses.

These latter calculations were first done with a group of three systems, ESW-RHA-HPCI, all simultaneously experiencing elevated unavailabilities. These analyses were then expanded to two four-system groups, ESW-RHR-HPCI-EDG and-ESW-RHR-HPCI-DC, then to the five-system group ESW-RHR-HPCI-DC EDG.

Although the DC system was not the subject of DET concern, it was included isecause of its high importance ranking. By subtracting the CDF results of the 3-system group from the CDF results of the 4-system group, the worth of the III-8

. . . . . . . . -- . . . :...~..v .. . . - . . .. a additional system can be determined. For example, using Figure Ill 3, a threefold increase in the unavailability of the EDG would increase the CDF by 8.33 x 104 -

6.87 x 10 ' = 1.46 x 10 4/RY.

With an unavailability multiplier of 3, the JAF CDF would increase from 1.9 x 10 8/yr to about 7 x 10 4/yr, for the three-group and fivo-group situations, respectively. As expected, the ESW contribution to the overall CDF increase dominates. Just increasing the ESW unavailability by a factor of 3 would raise the JAF CDF to about 6.2104/yr. The ESW portion of the three-group increase in CDF is 87% For the five-group situation, ESW contributes about 52%.

If a very large (five fold) increase in baseline unavailabilities were assumed, then even larger CDF's would occur, ranging from about 1.7 x 104/yr for the ESW-RHR-HPCI group to about 2.9 x 10-5/yr for the ESWR-RHR HDIC-DC EDG group.

Again, the ESW coritribution alone would be the important contributor. A five fold increase in the ESW unavailability itself results in a CDF of 1.45(10)~5/yr. This ESW increase would account for 84% and 46% of the 3-system group and 5-system group increases, respectively.

Because of the dominant role of the ESW in potentialincreases in the CDF, a closer review of the DET/ESW comments was made. The priicipal role of the ESW system during accidents is to provide cooling water to the emergency diesels during loss of offsite power conditions and to provide cooling water to the crescent area coolers. Loss of the ESW concurrent with the loss of offsite power leads to station blackout, the dominant contributor to the CDF at JAF, However, there are no DET concerns about ESW operation with respect to its critical function of cooling the EDGs. With respect to crescent area coc!ers, the issue concerns the effects of silting impairing this cooling capability.

Crescent area coolers are normcIly supplied with normal service water.

Silting resulted from of low flow velocity through the coolers due to cooling water.

flow throttling for needed temperature control. Modifications of the cooler temperature controls (on 8 of 10 coolers) to provide full cooling water and modulated air flow (by turning the fan on or off) has corrected this design deficiency. Trending of crescent area cooler cooling water flow and cooler thermal performance since the modifications were completed in the Fall 051990 indicates the modifications were effective in controlling silting. Some cooling water flow reduction (since original construction) is also due to flow restriction in the normal service water piping as a result of microbiological induced corrosion (MIC).

Chemical cleaning of the piping is scheduled to be completed prior to start up following the 1992 refuel outage to reduce this flow reduction. ,

Cooling water flow to crescent area coolers, when supplied from ESW,is not affected by the MIC problem or silting because ESW flow to the coolers is cnly in service for testing or during those rare events when normal service water is not available. Silting has not been shown to affect operability of the check valves at the normal service water and ESW interface at the crescent area cooier inlets.

III-9

_ _ _ _ _ _ _ _ _ _ _ . _ _ l

l Problems with the check valves found during in Service Test (IST)' inspections has, in most cases, been limited to binding of the moving part of the check valves as a result of corrosion and/or infrequent exercising of the vat'vo. Replacement of the internals with different materials (generally stainless stoe!) has, based on recent IST inspection and testing, corrected the problem, in addition, activities for the control of Zebra Mussels (chlorination of service water, ESW and fire water) will also seduce or eliminate MIC in these systems.

The IPE data base time period included a portion of the check valve and-cres:ent area coolcr problems discussed above. Modification of the cooler temperature controls and check vt,1ve internals has corrected the deficiencies.

System reliability (availabi'ity) et this time is likely to be greater than it was during the iPE date base time patiod. Without dagradation of the ESW, decreases in-plant performance that raight affect the JAF IPE CDF are likaly to be lirnited. For example, if the RHA. HPCI and.EDG systems all underwent a simultaneous five fold increase in their base case unevailabilities, the CDF would increase from 1.92 x 10-'

to about 7.4 x iO-e, assuming no increase in the ESW contnbution. Even_ if this simultaneous large increase in the unavailabilities of three major systems occurred, the JAF CDF would still be within the range of the CDFs reported for other BWRs.

In additivi to the issues identified'by the DET, a obsequent issue related to z the ESW wcter inlet temperature has arisen. Thu issue' concerns the possibility that the ESW water inlet remperature could rise to unacceptable levels because of the discharge of worm ESW water back into the ESW pump A bay. This issue has been analy7ed for its potentialimpact on the JAF IPE,-as described below, o Operation of RHRSW pumps would limit the ESW! pump "A" bay temperature rise by drawing fresh water into the bay, thereby eliminating the adverse consequences of higher ESW inlet ternperatures it is anticipated that RHRSW pump operation to ,

provide primary containment control will commence about 1/2 hour after the EDGs started, Furthermore, a detailed event-tree analysis based on assumptions.-

-utilized in the bay water temperature heatup analysis revealed no new dominant-accident sequences that could impact the ~JAF core damage frequency profile.

Therefore, this post DET concem would not effect CDF values, a

s]

III-10

3. CONCL_QSIONS AND RECOMMENDATIONS This review has reached a number of conclusions:

o The JAF IPE was generated using up-to-date P'R A techniques, o The calculated JAF CDF vaiue is comparab;e to other Mark l BWR CDFs.

o The data bases used in the IPE are more current than the early 1980's.

o Approxirnately three quarters of the DET comments can be directly screened out btscause they would not significantiy affect the calculated JAF CDF value, o PRA sensitivity studies revealed that the most important system to examine would be tha ESW. However, there are no DET comments on the ESW that would impact the JAF CDP.

o Specific other issues beyond the DET review were examined and were also determined to have little impact on the JAF CDF.

!n addition to specific concerns, the DET has raised some very broad issues such as management practices, housekeeping, operator informality, and poor maint lance. Such concerns cannot be directly quantified using existing PRA techniques. These issues would be discernable in the IPE only if they affect the plant's reliability by showing up in the PRA data bases. For example,if poor management affected plant performance, this would have been been reflected in one or more cf the PRA data bases. The influence of these DET concerns on the CDF would already have been considered due to the recent data used in most of the IPE analyses.

The one iPE data base which may not reflect theso DET concerns is the

! equipment availability data base. As noted earlier, this data base is current through l September,1986. However, sensitivity studies show that even significant changes l to system availability would not appreciably change the CDF, To quantify the more recent operating experiences, this data base will be updated as part of the Authority's "living PRA" process.

Programmatic changes are being implemented as part of the FitzPatrick Results improvement Program to reverse the decline in equipment performance.

This will be verified in updates to the FitzPatrick living PRA.

l III-11

Figure 111-1 ^

l

% CHANGE OF CORE DAMAGE FREQUENCY

• OF.JAF 130 --

120 --

- 110 --

100 --

90 _ _

'O 30 .-

E5 m__

d g so --

5 so __

a--

. so .

O 7

j +  !  ;

;  ! -1 ESW 125vDC SRV ' EDG AHA VENT HPV - 115 RCIC 60CVAC APS' SYSTEMS WHICH CONSTITUTE 90% OF CDF

• Change in Core Damage Frequency if system unavailability is doubled

Figure 111-2 Calculated CDF Increase DueTo increased ESW System Unavailability 3.00E45 --

250E45 -

2,00E45 --

1ASE45 1 M 45 --

o 1.00E45 ' --

5.00E46 --

c.cos 00 1 1 3 Times Base Case , 5 Times Base Case p Unavailability . Unavailability ESW System Unavailability' Cases

~

Figure 111-3 Calculated CDF Increase Due To Threefold increase in Unavailabilities Of DET Observed Systems uan --

2.50E45 --

- 2.00E45 --

1.50E45 --

192EM

'~

P.L65EM g3yg

.C l j l l l }

JAF APE Ase Case '- ESW ESW4tHH@Ct ESW-RHR-N ESW-RHM*CHDG ESW-MHR+4PCM)CM DET Observed System Groupings l = _ -_ -

i

^

Figure lil-4 Calculated CDF Increase Due To Fivefold increase in Unavailabilities Of DET Observed Systems 3.00EC -- 2.91EG IMO " 2.37EM 2.00EW --

1.69E-05 igg __ 1.45EM l

1mEe --

5.ooEm --

1.92EM

^

0.00E+00 --  ; i [ [ [ ]

JAF Base Case ESW cSW4%A4 PCI ESW-RH4HFO-DC ESW44R4@C16 C5W-RHMFCIOC6 DET Observed System Groupings

. . ATTACHMENT IV ANNOTATED DET COMMENTS 2.0 EVALUATION-RESULTS 2,1 Operations and Training -

The team evaluated the operations department in the~ areas of staffing and-shift -

organizational structure, the ability to identify and correct problems and their-root causes, and the ability to plan, schedule and control work activities, in addition, the team evaluated the integration of training into operational practices and policies.

In the area of operations and training, the team found the_ shif t organizat '

structure, as utilized, limited the supervision capabilities of the operat -

staff. To meet the required Technical Specification-minimum staffing leveis, and to avoid an overtime problem, the -licensee was forced to assign six operations department managers and other staff to routinely stand shift watches.

The thoroughness of control room- shift turnovers was inconsistent. Shift turnovers ranged from ~ discussions among operators to entire shif t _ briefings.

Communications between the shifts were also hindered because control room logs-lacked detail-. The operations department was-not-consistently aware of plant problems, and those= problems that did gain -its attention were ' resolved-inconsistently and, at times, ineffectively. Operators tolerated degraded plant _j equipment-and systems: examples ' included chronic suppression- pool in _ leakage from the main steam system; repeated offgas system hydrogen burns; ESW silting i and repeated water hammers; large numbers of components that leaked -oil; EHC pressure oscillations;;a large HPCI system work backlog; and LPCI system valve problems . Numerous team observations ~ were noted that' indicated poor plant material condition. Inadequate planning, scheduling,. and control of work flow centributed to a backlog of more than 2,000 outstanding work requests _. _ Often the plan-of-the-day schedule did not accurately reflect ongoing work-because scheduled work was not conducted, and work-was-not3 effectively prioritized.

Management displayed . ; inadequate _ oversight- of- _ the 11icensed -operator-requalification program in that imany operators' failed - to comply . with the licensee's program. _ The team also found that ott.er training activities were- 1 not supported by management; examples included routine maintenance staff training and supervisory' fitness-for_ duty training.

The team observed the shift crews and noted- high levels of experience and  ;

knowledge of the. plant, and good manipulation =of the-controls. .The level of i utilization of the plant-specific simulator:inilicensed operator training and

'its-fidelity to the plant were both considered strengths.

2.1.1 Minimum Staffing and Shift Organizational Structure Limitations 1

Staffing levels were 'at -the TS minimcms necessary to support - a-- six-shif t (1)-

rotation. 'In order to avoid an overtime problem and because of the low staffing - ,

levels, the licensee was forced to assign sixl operations department managers and other staff to routinely stand shift watches when two licensed individuals -were-

'4 e

a s

.. 9 . .

taken off licensed duties. The licensee was forced to reduce to a five- shif t rotation during refueling outages. This condition was not expected to change until the end of 1992 when a new class of operators was scheduled to take their license exams.

Ineffective use of supervisory resources within the operations organization W resulted in an operating shift structure that was inefficient and impacted the ability of the control room shift supervisor (SS) to oversee control room activities. Each operating crew consisted of two licensed reactor operators (R0s) and two licensed senior reactor operators (SR0s) 'and some non-licensed auxiliary operators; this was conststent with the minimum levels.specified in

,the TS. One SR0 functioned as the SS, the other as the assistant-SS and shift technical advisor (STA). The assistant SS monitored the work control center (WCC) in preparing tagouts, processing work requests, and giving approval to start work. The WCC was physically adjacent to the control room but a wall separated the two rooms. The assistant SS could not observe the main control panels directly and was also cut off from the communice' ions taking place in the control room. The assistant SS was required to obserw the auxiliary operatt rs performing their rounds, to conduct a brief training session on the swing shii '

of each training cycle as part of the on-the-job training requirements for licensed operator requalification, and to act as fire brigade chief. As a consequence of these conditions, the assistant SS could lose touch with control room activities; therefore, the SS also served as control room SRO. This impaired the ability of the SS to monitor all of the activities vc:Frino throughout the plant.

The team also observed that one of the two R0s on each shift spent considerable time out in the plant. With only one R0 in the control room and the SS allowed to be behind the control panels or in adjacent office areas, onlj one licensed operator remained at the controls,' monitoring the reactor. This structure limited the ability of a minimum shift crew to respond to a scenario involving activation of the plant fire brigade, implementation of the emergency operating l procedures, and implementation of the emergency response plan, including l assessing emergency action level; and making protective action recommendations, l

The licensee informed the team of its plans for increasing the number of licensed operators and the number of individuals on each shift. The licensee planned to add one licensed operator (RO) and an STA to each shift.

2.1.2 Limited Awareness and Resolution of Problems The operations department was not consistently-aware of plant problems. Those (2) problems that gained its-attention were resolved inconsistently and, at times, ineffectively. This condition was exacerbated by management's tolerance of degraded equipment. Three examples of this problem, observed by the team, follow.

1. Since the-reactor startup in August.1991, several thousand gallons of water per day had been leaking from the main steam system into the suppression pool, and the licensee had not aggressively pursued a solution to the 5

s

, problem until the team raised the issue. The water was leaking into the (3) suppression pool from the main steam system via a residual heat removal heat exchanger (RHRHX). The leakage flow path was from the reactor via the high pressure coolant injection -(HPCI) steam supply line and then through the RHR heat exchanger's steam condensing mode supply isolation valve, lhis situation affected one RHR train operability, heat exchanger level, suppression-pool level control, and radwaste system capability.

Operation's management was not proactive in addressing this problem. When the leak began, operations considered the problem to be only a level-indication problem and did not thoroughly investigate an RHR system operability issue. The final analysis of system operability performed by the licensee's plant operating review committee (PORC) determined that the RHR keep-fill system was not full of water during the period that steam was leaking through the isolation valves and into the RHRHX. The PORC determined period.

that train A of the RHR system was inoperable during this The licensee informed the team that in the past, water hammer events of significant magnitude (to dis;) lace pipe hangers) had occurred when the RHR system was being placed in service.

On the basis of a request by the team, the licensee performed a root cause analysis of this problem. The analysis determined that one probable cause of the problem was that the HPCI to RHR cross-tie isolation valve 10MOV-70A had been cut by steam because the valve was not being fully closed.

The steam cutting of the valve's disc and seat created the leakage path.

The licensee determined that one possible reason for the. valve not being fully closed was that wiring in the valve logic had been changed in 1972 and that the change had removed the seal-in circuit.

Cortrol room operators were not aware that this change had been ade.

The team's review of the root cause analysis (JSEM-91-070) indicated that while it addressed the hardware cause of the nroblem, the analysis did not address other programmatic and communications problems. The problem with the error in valve 10M0V 70A's logic was identified in March 1991.

Unusually high leakage into the suppression pool was identified in August 1991, after the reactor startup. In both cases, the licensee did not act to resolve these discrepancies.

2. A large number of problems in the offgas system had progressed to the point (4) that the licensee could not place the hydrogen recombiner and charcoal filters-in service until the reactor was at 90-percent power. The licensee had been able to tolerate this situation because of advancements in fuel cladding integrity resulting in low fuel leakage rates. In addition, management was slow to recognize the seriousness of the hydrogen burn event in the offgas piping in March 1991, and-had chosen to implement only a small number of the General Electric recommendations to minimize the possibility of hydrogen fires. The licensee believed that it had implemented all the recommendations that were applicable. After discussions with the system engineer, the team concluded that some additional modifications and repairs would enhance overall system reliability and performance.

6 1

3.

Pressure oscillations in the main turbine electrohydraulic contrul (EHC) (5) system were considered significant- by some operating shif ts and trivial by others when, in fact, .the system had degraded to the point that pump A was out of service and pump B exhibited large pressure oscillations.

Subsequently the EHC pump A had to be replaced and- its relief valve had to be repaired.

Management informations 'ystems utilized by the licensee were weak. Performance (6) monitoring of such items as control room deficiencies, work request backlog, procedure revision backlog, missed or late preventive mair+enance, and safety-related and important-to safety system unavailability was ineffectual. The team concluded that a low awareness and utilization of industry experience and initiatives combined with little exposure to other nuclear plants had led to an atmosphere of isolation and inconsistent standards of acceptance in '. h e operations department.

Operations department management did not fully utilize data on the status of (6) degraded equipment, the ".ontents of control room logs, information on repetitive problems involving equipment, and reviews prformed by system engineers. In many instances, shift supervisors did not tell operations department managers about plant problems because acceptance standards varied. Operators were content to (7) work with approximately 400 temporary procedure changes that had never been incorporated into the operating procedures.

As a result of these condition > operators had become accustomed to working around equipment problems or dei .ciencies, procedure eroblea, and operations management. The operating staff focused its attention and energy on just dealing with the immediate problem rather than on getting to the root of the problem and effecting permanent change.

2.1.3 Ineffective Planning, Scheduling, and Control of Work Equipment remained deficient because of inadequate planning, scheduling, and control of work flow. Often the plan of-the-day schedule did not reflect the ongoing work, or. scheduled work was notl conducteo, or the schedule did not provide for effective prioritization of the work. Essentially, each department controlled its own priorities without being given direction'on what work should be given the highest priority. This had contributed to a backlog of more than 2,000 outstanding work -packages, of which approximately 150 were priority 1 status. Priority 1 work requests were supposed to be completed in seven days.

The team also observed 85 equipment deficiency tags on the main control panels, some of which were several yea-s old.

The WCC was not effective in coordinating planned work. The team noted that the SS would occasionally negotiate with technicians over their ability to perform specified work. As a result, the shift supervisor's time was consumed with <

reviewing nearly all work packages. For example, a SS discussed repairing a feedwater heater drain valve with a maintenance technician when the plant was not in an operational situation to permit this repair, nor were extra auxiliary operators available, as was required to support the activity. There was no previous coordination or scoping of. the work between planning, operations, and 7

1 maintenance supervision for this activity; consequently, the SS was distracted from his duties while explaining this situation to the maintenance technician, Given the existing minimal staffing levels, these situations further interfered I with the SS's ability to monitor overall plant conditions.

At times, the SS's ability to initiate equipment repairs was ineffective. On one occasion, three different shifts requested that an average power range monitor (APRM) recorder on the main control board be repaired. It took several days to initiate this maintenance activity. On another occasion, annual preventive maintenance was scheduled on a fire hose-located in a high radiation area. Shift supervisors suggested that this maintenance be performed during an outage or under improved as low as reasonably achievable (ALARA) conditions.

The licensee was evaluating these alternative selutions.

2.1.4 Poor Material Condition and Housekeeping The team observed numerous deficiencies and poor material condition during plant tours and system walkdowns. Two examples of this problem, observed by the team, follow.

1. Although the licensee's individual plant evaluation (IPE) stated that the HPCI system was the most important safety system, a review of records (6) indicated more than 115 open work requests on this system. A Walkdown of the system revealed: broken gauges, corrosion, ' debris lying on top of equipment, scaffolding erected permanently over the pump, and contaminated water from numerous. leaks standing on the floor around the pump.

2. The liquid nitrogen storage tanks 'or the containment air dilution (CAD) system were leaking nitrogen and frost-covered piping was observed 8 to 10 feet from the tanks, with more than 4 inches of frost encasing relief valves. Work requests had not been initiated on these problems and senior operations management was unaware of this condition.

Other examples of poor material condition were numerous over-ranged instruments, I

offgas system problems, and the periodic- trip Jing of the offgas hydrogen and oxygen cnalyzers.

The team was particularly cc.ncerned about management's willingness to tolerate i degraded equipment that directly affected plant operations. An example of this l was- the daily entry into emergency operating procedure E0P-5, " Secondary l Containment Control," which was necessitated when the temperature in the reactor water cleanup (RWCV) heat exchanger room exceeded the entry condition of Il9'F -

during late summer and early fall. This_ condition was caused by' inadequate cooling of the RWCV heat exchanger room because of degraded performance of the room cooler and loose or missing insulation on the hcat exchanger. The need to evaluate. entry conditions, classify the event, and enter -the E0P on aidaily basis, placed an unnecessary burden on the SS. Of further concern to the team was a meeting with senior station management, to discuss corrective actions lfor l

this problem, in wh_ich one senior manager stated that the insulation.on the heat

-exchanger had been repaired and that since Lake' Ontario temperatures had dropped, the degraded room cooler could be " lived with" until a modification to relocate 8

i

. the temperature detector could be analyzed. Another senior manager had been working with the system engineer to fix the problem permanently by chemically cleaning the heat exchanger ard modifying the piping to allow on-line cleaning.

Dif ferent managers, working on different solutions, were not coordinating the solution to the problem, and were not providing fcr a permanent solution of the root cause.

A large number of components were leaking oil. Examples of this were the reactor recirculation motor generator sets, the reactor feedwater pumps, and the condensate booster pumps. Used oil-absorbent pads were replaced and oil collected in temporary containers each shift. Two SS-gallon drums had been placed next to the reactor feedwater pumps, one to collect the leaked oil and the other for replenishing the lubricating system with new oil. Condensate booster pump C had been leaking oil for more than 5 years and the present leakage rate was 2 quarts per shift.

The cuxiliary operators had become overburdened with extraneous surveillances g) during their rounds. They were responsible for checking high radiation doors on their rounds to ensure that they were locked. Since some of these doors were inside contamination areas, the operators had to don and remove protective clothing as many as 0 times a shift. All of these conditions consumed considerable time for the auxiliary operators and in numerous instances delayed the completion of the operator rounds until the subsequent shift. Interviews with the operators indicated that many of these problems were of a longstanding nature that fosterad an operator attitude to accept and " work around" problems.

The team ooserved problems throughout the plant concerning the labeling and marking of both safety and non-safety-related components and systems. Many valves and associated piping were observed without any identification tags or labeling (making it difficult to identify system components), and the direction-of-flow labeling for piping was minimal. The licensee had _ implemented two programs to correct this problem. Tne first one utilized small metal tags which 7

were hard to read. Plant operators were not consulted prior to the implementation of this program. The failure of the first program required the implementation of a second program. The other program utilized colored labels, with bar codes and noun names; the operators considered these labels very useful and easy to read.

Housekeeping in high-traffic areas of the plant appeared quite good; however, in low traffic areas (i.e., behind socked, high- radiation or contamination area gates) housekeeping was generally poor. These areas were cluttered with trash or were being used to store maintenance or test equipment. Bags of trash were noted in the reactor building; in one case a bag of trash was observed between the HPCI pump and turbine for more than a month. The presence of numerous oil and water leaks was exacerbated by the plant's design which had no collection systems in some areas. Where collection systems existed, the high inflow of water had placed a large workload on the radwaste system to process all of the water. Additionally, senior management's unfamiliarity with the team's observations indicated that operations and station management were not touring low traffic areas of the plant on a routine basis.

9 1

{

2.1.5 Inadequate Management Oversight of Training Activities The team found training activities, especially refresher or periodic training, were not viewed as high priority tasks, but as tasks that were subordinated to the immediate needs of the moment. Examples of_ this problem were nated in routine maintenance staff training, supervisory fitness-for duty training, and most significantly,_ in the licensed operator requalification training.

Additionally, the licensee had only recently implemented supervisory training for first level supervisors. Training for second-level supervisors, such as the SS, had been deferred because of the staffing constraints (see Section 2.1.1).

The team noted a significant number of R0s and SR0s had either not attended er not completed the required requalification training program for calendar year 1991. The licensee had not complied with its own requirement that individuals who missed training complete a makeup class within 12 weeks, or else were removed m from shift duties, until the training was completed. The team observed that 25 of the 53 licensed R0s and SR0s had not completed their required training.

Deficient training included failed quizzes, classroom training and simulator training. On-shift individuals who were affected included two L SSs, three assistant SSs, two nuclear control room operators, (one a senior nuclear control room operator). In addition, six staff SR0 licensees, five of whom had been performir,g assistant SS duties on shif t 8, had also exceeded the requirement for completing the missed training and should have not been al' owed to perform licensed duties. Considering the current unsatisfactory status of the requalification program, the level of attention that senior licensee management had given to the plant's corrective actions program'was inadequate.

The training coordinator for this program had sent memoranda to the operations department's management to make them aware of the backlog of missed training.

However, interviews with licensed individuals and site management indicated that a lack of understanding existed between the general manager for operations and the general manager for training with respect to the program criteria relating to the 12-week requirement for making up the missed training, along with current

' knowledge of the train.ng backlog. The facility's current 2-year training program cycle was scheduled to end on December 31, 1991, at which time all training was required to be completed.

Management communications between the training and operations organizations had broken down completely on this issue. Had the licensee attempted to correct this problem, it would have encountered significant difficulties due to staffing constraints, vacation schedules, and the estimated work load to- support the January 1992 refueling outage.

The c.ount of continuing training provided to journeyman maintenance personnel had significantly decreased during the past year because of the size of the work load and unplanned outages. For example, two journeyman electricians received more than 100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> of training in 1990 and only about 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> in 1991. The team also ~ observed that 'the . licensee's quality assurance (QA) department had audited supervisory fitness for duty (FFO) training and concluded that 20 supervisois had not received initial or requalification training as was required

-10

by their FFD program. The team concluded that most -training -activities were being given_a low priority because of staffing constraints and little management-attention, 2.1.6 Informal Control Room Activities-Shif t turnovers observed -in the control; room _were int.omplete. The licensee's procedure for shift. turnover appeared.to be adequate; however, the team noted -(g) that the degree of completeness in shift turnover varied among shifts. -Depending:-

on the- individual SS's policy, shift- turnover ranged from discussions among operators to entire shift briefings. - Some operators were permitted to leave the control room after debriefing their replacement without_ informing the-SS. This-led to the oncoming SS being uninformed on equipment status. For example, during investigation of the RHRHX _ leak, some shifts were venting the heat exchanger to -

recover vater level, while others were unaware of the_ ongoing situation-and had to-conduct the'r own investigation into the leak.

Communications between the shifts was hindered by control roa logs that lacked detail. Operators did not maintain logs on a number of_ sign 4ficant control room and plant parameters. For examole,_ control room logs we e not maintained on.

nuisance annunciators', workf reguests~ issued for the c' . troll roomTequipment, inoperative equipment (other than--TS ' limiting conditi es for operation), and abnormal equipment trends. _ The_ team noted that many control room activities could not be reconstructed through the control: room log-system. One example was j the leak from the reactor into the suppression-pool-of several thousand gallons 1 per day (see Section 2.1.2). The team could not -reconstruct-- the operations s

department's-response to-this event-by consulting.the._ logs. Consequently, as a result of the insufficier,t information in the 1_ogs, the shifts often approached-problems inconsistently.

Operators were inconsistent regarding the level of formality in using procedures and in verbally communicating in the control room. During activities such as surveillance testing, operators adhered closely to the procedures and used formal communications; an- example of this ;was testing of APRM. Ouring routine-operations, however, communications 1 were informal- and operators _ 'used?such communications -' as hand gestures and slang, when issuing -directions to' the auxiliary operators. Additionally, instrumentation andlcontrot(l&C) technicians were observed resetting the reactor protection and primary containment'isolationi system trips, during surveillance testing. Relief crew R0s wereTnot performing this duty. Complicating this situation wm the. general lack of licensed operator supervision of the 1&C technicians during these types of; activities. This was a direct consequence of the staffing levels of' licensed operators. :The1 team - '

noted that the-I&C technicians would sometimesHinform the R0 that .they were- 1 resetting the trip signal; however, this appeared to-be little more than a proe I forma approval of_ the I&C technicians' actions.

The team observed that the plant evacuation and fire alarn noise levels ~in the - g i main control room were too high.- .This- wasi potentially . distracting for the operators and could interfere with- their ability toLutilize the emergency operating procedures to communicate, monitor, and evaluate the plant'; response

( during an emergency. 4 11-

Operators had a tendency not to rely on plant indications. During troubleshooting activities, operators questioned the accuracy of indications (1g) before considering an abnormal system condition. Given the extended delays in repairing equipment, abnormal situations might have gone undetected- for significant periods. Several examples of this had occurred while the team was onsite. Operators initiated a maintenance request on the EHC pressure indicator and while waiting several days for maintenance to calibrate the meter, EHC pump A had degraded to a point where it had to be replaced. Similarly, the operators assumed that the RHRHX level indicator was inaccurate, not that it was indicating a problem skepticism Operator with the RHRHX, when an actual leak was present (see Section 2.1.2).

of control room indications appeared to stem from the significant number of outstanding work requests to fix equipment.

2.1.7 Operations and Training Positive Observations Positive observations on the shift crews included high levels of experience and knowledge of the plant and good manipulation of the controls. The shift crews were observed to perform their duties and operate the plant in a safe manor inspite of the minimal staffing levels, numerous degraded or inoperable equipment, and limited resources. The level of utilization of the plant-specific simulator in licensed operator training and its fidelity to the plant were both considered positive attributes.

2.2 Maintenance and Testing The team performed a detailed evaluation of maintenance and testing activities associated with the HPCI and ESW systems, and to a lesser degree, other safety-related and non-safety-related systems. The team also evaluated other aspects of mainten ece and-testing, such as preventive and predictive maintenance: ro.,t cause analysis and corrective actions; motor-operated valves (MOVs); industry operating experience; risk insights; planning, scheduling, and prioritization; control of overtime; maintenance training; work request backlog; maintenance facilities; and in-service testing (IST) of pumps and valves. The team also observed several maintenance activities and surveillance tests.

In the areas of maintenance and testing, the team found that several factor; pertaining to resource availability and utilization and insufficient management oversight significantly reduced the overall effectiveness of plant maintenance.

The preventive maintenance program was only partially implemented and was less than fully effective. The root causes of numerous equipment failures and problems were rarely identified and this resulted in untimely or ineffective problem resolution. For example, two motor-operated valves: (M0Vs) in the low pressure coolant injection (LPCI) system failed, rendering the system inoperable.

These valves had a long history of problems and were also the subject of previous industry operating experience. Although the licensee took action to correct specific problems. associated with these MOVs, it never took action to correct (n) the underlying causes. The team found that many M0V hardware and programmatic weaknesses were still apparent, even though the licensee had made some recent progress in maintaining MOVs. The team also found that industry operating experience was not-incorporated into the maintenance program in a timely manner 12 i

a *

, experience was act incorporated into the maintenance program in a timely manner -1 and that risk insights were not factored into the maintenance program. Instances-of 1&C surveillance procedural adherence problems and inadequacies in safety -

related procedures were noted. Several weaknesses in the area of surveillance-and testing were identified. For example, many fire protection check valves were -

never- tested or cycled. Several other factors also - adversely affected ,

maintenance effectiveness. Among= these were: excessive overtime in the  ;

maintenance department during the past 12 months; an increastag bt.cklog~of work- '

requests; a reduction in the amount of continuing maintenance training; limited-work space for the maintenance crews;;and weaknesses in planning, scheduling, and prioritization.

The team found that maintenance craft personnel were experienced, knowledgeable, and professional,. which led to good: performance in the conduct of maintenance activities. The in-service testing of pumps and.MOVs was s aiso found to be -

functioning well. The licensee recognized the need for improved maintenance and r recently developed corrective action plans to address many of the problem areas observed by the team.

2.2.1 Less Than Fully Effective Preventivs Maintenance Program Although the licensee had a preventive maintenance-task force (PMTF)-in place for the past several years _to identify preventive maintenance (PM) requirements,. (12) the scope of the _ PM program was too narrow. Progress _had been made since-1989 (e.g. safety-related MOV maintenance), but the team found that--several component -

types did not fall within the scope of the PM program,- Exampits of equipment -

not in the PM program hicluded air operated valves (A0Vs), fire protection system check valves, critical safety-related manual valves, transfer and control switches associated with safe shutdown equipment, and 0C control-circuitry for emergency diesel generators (EDGs) and high. pressure coolant injection (HPCI) and reactor core isolation cooling (RCIC) turbine- governors.

Extensive corrective maintenance ' performed on some of these components (e.g., A0Vs) indicated the need for PM. The-team noted that the PMTF had performed several-PM evaluations (PMEs) for several major components, and-was in the: process of' performing a PME for A0Vs which was scheduled to be completed by the end of 1991.

However, no other components had been identified for evaluation.

l In addition, the team found that similar types of! components were mairtained differently even though there was no documented--justification to s',pport the-different levels of PM. For_ example, most non-safety-related MOVs did not have any PM in recent years, and although lube 0i_1 in the HPCI' turbine .was sampled every month, lube oil in the RCIC turbine was not sampled at;all.

Discussions with licensee _' personnel and- a L review -of; licensee records also

' rev:aled that little feedback;was incorporated into the.PM program to further 1 improve its effectiveness. The team-also found that the trending of critical-components _and process attributes _was.not. fully o m loped.

l Although licensee maintenance personnel acknowledged 'that the- P" program was L

not fully implemented, .the PMTF was scheduled to be deleted from the 1992 budget.

13 m

-9w---- w-- ,-

3 rw , ,,,-,.pyw- y. 9 9 y7 9y 9 pg A,,,,79.- 3 isr wW- @ *wt** owwreW Mgr my w 99wW9- jnge 4 h yy 9 ^W-'

. . _ - ~ _ - - - - - . - -. - - = - - .. - . - . _ - .

2.2.2 Inadequate Root Cause Determinations of Equipment Failures Root causes of equipment problems were not usually identified which resulted in the untimely or inadequate resolution of problems. The licensee had no program for performing integrated causal and corrective action evaluations. The licensee had prepared a draf t procedure, but management did not know when it would be implemented. The team found that very few root cause evaluations had been (13) conducted in the past 2 years. In addition, material history of the cause of equ!. ment failure was poorly documented and did not facilitate equipment history reviews. Examples of weaknesses in inadequate root caase determinations or corrective action icclementation included the following:

1. Several safety related MOVs (e.g., HPL inboard torus isolation valve) had damaged hand wheels or broken declutch levers. Although these conditions were corrected following identification, the causes were not determined. .

2. Excessive seat leakage of RHR minimum flow isolation valve 10MOV16B was identified. Because the licensee could not determine the cause of the leakage, the torque switch settings were increased in order to seat the valve disc. The licensee appeared to be repairing the symptom despite prior industry guidance (i.e., an HRC information notice) concerning the unsuitability of just repai-ing the symptom.

3. The core spray holding pumps provide a " keep full" function for the core spray system to prevent water hammer damage. Core spray holding pump A failed because of a loose pump impeller. The condition was corrected but the cause was not determined.

4. During a surveillance test in June 1991, excessive fluid leaked past 46M0V101B, ESW injection valve B. This valve was disassembled and the valve wedge guides were found to be severely worn. The stem had excessive pitting throughout i ts surface and the valve seats were also worn.

Although ESW injection valve A (46ESW101A) was suspected of being similarly affected, it was not disassembled during this period. The licensee reassembled 46ESW1018 using the same worn parts, and declared both ESW injection valves acceptable for continued use until the January 1992 refueling outage, at which time the internals parts for both valves would I be replaced.

5. In May 1990, the motor of the RHR shutdown cooling inboard isolation valve actuator (10MOV18) tripped while the valve was being stroked closed. The MOV motor and torque- switch were replaced, but no cause was identified.

This also appeared to be an example of just repairing the symptom.

2.2.3 Motor Operato:' Valve Hardware and Programmatic Deficiencies l

, Recently implemented PM activities (since 1989) associated with safety-related l L MOVs appeared to have reduced the number of MOV failures. However, a review of i maintenance history revealed that many continuing problems had bec, discovered as a result of equipment operation, PM, testing, and trouble shooting. Examples 14 I

,,,-,---w-, ,- - . -v-, -,..+,w.- = , , _,--s -o r

e included: failure to stroke against high system differential pressure; packing leaks requiring motor operator backseatingt environmental qualification (EQ) deficiencies; valve position indication problems; separated grease grease ,

seepage and migration, hardened grease, insufficient grease, wrong grea, set many  :"

local leak rate test (LLRT) failures because of seat leakage and packing leaks; missing handwheels, bent handwheel shafts, broken declutch levers; failure to '

stroke within required technical specification (TS) limits; improperly installed (14)  !

torque switches, defective torque switches; motor control center (HCC) load

• center breaker trips; and torque switch setting discrepancies. In adlition, even

• though many activities associated with Generic letter (GL) 8910 had been initiated, the licensee's overall progress in implementing GL 8910 appeared limited.  !

t

1. Licensee actions were ineffective in preventing the failures of LPCI MOVs, 10MOV25B and 10MOV27A. These MOVs were declared inoperable on May 7,1991, '

following surveillance testing, thus renderina both trains of the LPCI (14) ,

system inoperable. Valve 10MOV258 failed to fulhy open upon demand because e of stem nut binding, caused by stem nut wear due to excessive loading and l wear associated with aging. Valve 10MOV27A was damaged internally ts a- i' result of years of excessive vibration resulting in excessive seat leakage, thus preventing it from performing its containment isolation function, b

A review of maintenance history revealed that these and their companion  ;

valves (10MOV25A and 10MOV278) had a long history of MOV motor failures  !

(potentially indicative of valve disc -binding) or. vibration induced ,

external valve damage. Over the years,.the licensee corrected the specific 4

problems, but did not identify the root causes of the problems. In addition, even though 10MOV27A was not intended to be used as a throttlin valve (which caused the excessive vibration in the vicinity of the valve)g.

• it was used in this capacity despite. the fact that other utilities had previously recognized the misapplication of valves used in the same ';

capacity. Although the licensee -took action- in 1983 to reduce the i

throttling time of 10MOV27A and 10MOV278 while RHR was-in service, they '

did not correct the root cause.

Some Itcensee personnel suspected that there might be internal valve damage associ.ited with 10M0V27A.- - They recommended in late 1989 or early _1990 to ,

l inspect 10MOV27A during the 1990 refueling outage for inte nal (vibration- t induced) damage. However, this recommerdation= was _not _ approved by management.

Industry operating experience dating back to 1984' indicated.the potential for valve disc binding because of bonnct over pressurization. Even though - '

10MOV258 was identified in the late 1980s as being potentially susceptible.

to such and other valve--disc binding . phenomena s no recommended modifications were implemented during this period to address this concern.

2.- The licensee's proceduralf guidance for lubricating MOV gear cases was deficient and . some environmentally. qualifled -(EQ): MOVs1may have been (14) _,

lubricated with an . unqualified: gear case lubricant. - The licensee was '

using at least three different types of M0V gear case lubricant, of which i

15

?

,-,._,--,-,._,-.,.,..,,u,,-,.___,-.

- - . - . , , . - - , _ - . - . _ - - w . m ,- - , , , . - - - - . , . _

• only one (Exxon Nebula EP ) was acceptable for use in EQ MOVs. HOV lubrication procedural guid:nce was such that a potential existed for mixing lubricants of dif ferent soap bases. Mixing lubricants of dif ferent soap bases (calcium and lithium) can result in grease separation which, in tJrn, can result in MOV failure. A review of Itcensee records re,ealed (14) one example of grease separation in o non-safety related MOV. In one other instance, hardened grease was found in a safety related MOV. As many as 36 EQ MOVs may be lubricated with Sun Oil SOEP; however, the licensee could not produce any test report documentation that demonstrated the acceptability of using this grease in an EQ application. Because of a lack of traceability, the licensee could not specifically ascertain whether these MOVs had been lubricated with Sun Oil SOEP or the qualified Exxon Nebula EP 0,

3. The team noted some examples of MOV torque switch setpoint discrepancies.

Although the licensee established procedural guidance for setting and (15) changing MOV torque switch setpoints, the team fot.nd discrepancies between the actual and the documented (in a controlled MOV database) torque switch settings for one of six safety-related MOVs that it inspected. A review of licensee records revealed several other recent licensee identified discrepancies between the as found and documented torque switch setpoints.

in no case did the discrepant setpoint exceed minimum and maximum values established by the vendor, and the team did not identify any operability concerns for the subject valves. However, the licensee did identify two errors that resulted in the discrepancy noted by the team. First, a torque switch setpoint document change had been processed for the wrong MOV.

Second, the actual torque switch setting of the subjact MOV was incorrectly performed. The licenseo attributed this to insufficient procedural guidance which had subsequently been corrected.

4. The licensee was testing some MOVs without measuring the full stroke of s the valve. The team identified at least three safety related MOVs that had their vd ve stroke lengths reduced to less than full stroke so that the valve strcke time would be less than the TS required limit. This was accomplished by adjusting NOV limit switches. Two of these MOVs were containment isolation valves in the containment purge system, and the third was the RCIC outboard steam isolation valve.

In each of the three cases, the licensee performed a safety evaluation demonstrating that the MOVs would perform their intended function while being positioned less than fully open. In the case of the containment purge valves, however, this safety evaluation was performed approximately 10 years after the valve stroke time problem was first identified 6nd subsequently corrected by adjusting limit switches. The RCIC outboard steam isolation valve limit switch was adjusted pending.the installatien of an adequately sized MOV actuator during the 1992 refueling outage. The limit switch was adjusted during -the -1990 refueling outage after the licensee discovered that the newly installed actuator for this M0V was the wrong size. This action was taken even though some licensee personnd ,ere concerned about the potential for valve disc degradation because the valve disc-would be partially entrained in the process flow path and would be 16

)

exposed to high pressure steam. A damaged disc could compromise the ability of the valve to perform its containment isolation function.

5. A review of licensee records revealed that 19 safety-related MOVs did not have torque switch limiter plate installed. The limiter pla*.e prevents the torque switch from being set at a value that exceeds the capability of the MOV actuator or the associated valve. Discussions with licensee personnel and a review of licensee records revealed that the licensee was installing torque switch limiter plates when corrective or preventive maintenance was performed on a specific HOV actuator.

6. The licensee had not fully developed the program description required by GL 8910. " Safety Related Motor Operated Valve TestW and Surveillance."

Supplement 2 of GL 8910 required that such gogram description be developed by January 1, 1991, so that licensee rogress c in implementing GL 89-10 could be assessed during NRC Inspections. The licensee's program plan (description) that was intended to meet the requirements of GL 89-10 was not developed until May 1991. On the basis of a review of this program plan and discussiont with liennsee personnel, the team determined that the program plan did not provide sufficient information pertaining to the schedule for completion of items B through H of GL 8910, or sufficient descriptions, in every case, to accomplish the actions specified in items B through H.

. 2.4 Surveillance and Testing Deficiencies The licensee had identified numerous problems (with erosion, corrosion, and silting of various raw water cooling system piping and components) and had acted to correct them. However, the team found that'several check valves and small bore piping in some of these systems had not been inspected or periodically testad even though such inspection and testing appeared warranted. in addition, the team noted other examples of testing or surveillance weaknesses. These examp?es follow:

1. Some check valves in the fire protection sysum (FPS) were neither tested (16) nor inspected for corrosion or silting. These included:

Two check valves in the headers to the reactor building NE and NW standpipes were not tested, r e: lure of these check valves would limit or prevent flow to hose su.' ions at four levels of the north half of the reactor building.

i Emergency diesel generator (EDG) suppression system pre-action check valves were neither tested nor inspected, except for one which may

have been cycled during a fire that damaged one of the EDGs in 1977.

Several check valves in the flow paths to turbine oil sprinkler i

piping were neither tested nor cycled, except for two valves checked in 1988 when a sprinkler actuated inadvertently. These suppression system flow paths covered the highest combustible loading in the olant.

17

i The check valve in the flow path to the radwaste handling and laundry '

areas was not tested. A contaminated combustible fire have could (16) resulted in an unmonitored release as a result of fire fighting efforts, if sprinklers did not extinguish the fire. ,

2. No small-bore piping in the residual heat removal service water (RHRSW)  !

system (with the excepti in of the RHRSW pump heat exchanger and keep + full (17) I; system) has been inspected for corrosion or silting. During the diagnostic evaluation, an RHRSW gauge line was found blocked with silt.

3. The post accident sampling system (PASS) cooler had never been tested in its emergency configuration. The licensee did not wish to degrade PASS [

(18) system cleanliness by using raw water to periodically test the emergency-supply to the cooler. The team noted, however, that no post installation test was conducted to verify the emergency cooling supply to the PASS cooler following system installation.

4. There was no independent testing of each of-the EDG air start motor sets (t3 air start motors per set) even though each air start motor-set is (19) l des.oned to have 100 percent capacity. There was no procedural requirement to af ternate EDG air start banks (two . banks -per EDG) during EDG-surveillance testing. Because of procedural weaknesses, the team could not fully determi e what bank was in service during the performance of EDG surveillance t, sting.

5, Many I&C surveillances, both safety and non safety related, were completed (20).

after the administrative due date. The IfC department routinely missed the scheduled due date, established new dates, and in at least one case, changed the grace period. It appeared, based upon the data available, the licensee routinely used .the grace peried, rather than- striving to- i perform the surveillance within'the established period.

6. Some instruments both safety and non-safety related,:were found-out of (21) calibration, not calibrated at all, or over-ranged.

7. Temperature control and the method of, surveilling the ' station battery 122) spaces and the LPCI MOV uninterruptible power supply (UPS) battery spaces-l were inconsistent. . Safety-related temperature sensors existed for the L station battery spaces. The operability range of.the station batteries "

was from 60 F to 110 F, as document 9d in a-TS' interpretation. However, '

no similar documented temperature-related operability criteria existed for the LPCI MOV UPS batteries, A. review of prior operations shift logs and-

operations reports indicated occasional low ambient temperatures in the ,

I

' LPCI MOViUPS battery spaces. . During~ these periods : of low temperature,  !

shift personnel used the same operability criteria for the LPCI batteries that existed for the station batteries safety related temperature-indication or(i.e., alarm 60existed F-110 for F).the

. However, LPCI MOV no UPS batteries and spaces. The only indication in the control room of LPCI:

MOV. UPS area temperatures;was) on - the emergency :and plant information computer (EPIC). The-EPIC had low and-high alarm setpoints of 631F and 104 F, respectively. The operators typically made shift rounds.of the LPCI 18 '

4 I -

_._,_. . ....m. . _. .  ;,--,,,m._.-__ _ _ _ , .,_,s.~,- . ~ . _ . - , - - - -

• HOV UPS spaces and recorded local temperatures from uncontrolled or uncalibrated hand 4 eld thermometers positioned in the LPCI battery rooms. '

In addition, a review of temperature sensor and alarm calibration data sheets and team observations associated with station battery temperature indicators revealed that the instruments were subject to drift in the non-conservative direction.

2.2.5 Untimely implementation of Industry Operating Experience '

In many instances, the licensee failed to incorporate industr (23) experience (OE) into the maintenance program in a timely manner.y operatin9 The team ,

reviewed the OE backlog and found it to be large (several hundred items regarding '

OE). Also uany OE items were either past due for review or implementation of '

action, some dating back to the early 1980s. The team sampled the licensee's review of 24 NRC information notices, which dated back to 1988. The team found that the quality of the licensee's review had improved substantially in_the past 2 years after implementin, OE program changes. .The team also noted that all the NRC information notices pertaining to MOVs (7 of 24) were still open. A review of the open OE backlog revealed that many other MOV-related OE items dating back to 1984 were etill open, including some pertaining _to stem nut failures -and lubricatior m a

• i.: nreviously discussed in section 2.2.3. Items l'and-2.

In two inst snm, -

aan Nnd that tevaral years passed before substantive action was he N c A .:gnificant issues pertaining to check valves and motor operato, gp e es.

2.2.6 1&C Surveillsnce Procedural Adherence Problems and Inadequacies (23) ,

Safety related I&C f ant.tional test and calibration procedures were inadequate  ;

for the circumstances or were not adhered to. -The team observed three'different surveillance tests performed by I&C technicians. In one case, the procedure for --

a "high-risk" test did not provide detailed guidance to the !&C technician'. As a result, the technician had'to rely on " skill of the craft" to successfully complete the test. During the performance of another surveillance test, the team observed that none of the EPIC points associated with the instruments tested were verified on the control room alarm printer.- During another surveillance test, the 1&C technician did .not perform all_ steps of the procedure in sequence because the procedure, as written, was wrong. Although the nature of the non-adherence was minor and the I&C- technician performed the test as intended. he did not stop and correct the procedural deficiency (as required) before completing the test.

2.2.7 Limited Use of Risk Insights in Maintenance Processes (23)

The licensee had not made much progress-in ' incorporating risk insights into maintenance processes. The team attempted to _ assess _ the _ availabil_ity and reliability of several safety systems and other systems that are important to.

safety,- but found that .the licensee could not determine. system availability -

(except for four systems: HPCI, EDGs, . RCIC, . and- RHR).- - For . example, no-availability data existed for the emergency service water (ESW) system and the.

station batteries, even though these1 systems were identified as two of the 19 i-

-,m,., .s -,o-~.., ,, .w. , y de ,r .~- w , , ~ , - ,,,,,.w,.--mm -.~-,,,,ar-c-.-y, - + .

. . ' systems that are needed to mitigate some of the dominant accident sequences identified in the individual plant evaluation (IPE) specific to fitzPatrick.

Low or decreasing system availability would serve to flag an adverse condition so that management could focus attention to resolve the causes of the reduced availability. In addition, the team found that the licensee had no procedural guidance for controlling vehicles and cranes in the switchyard, even though station blackout scenarios contribute 91 percent to the core damage frequency.

Prior to completing the onsite evaluation, the team received a PA$fH memorandum (dated September 19, 1991) that specified the initial actions and information to implement the IPE . The licensee did not address the two issues discussed above in this memorandum.

2.2.8 Other Factors Adversely Affecting Maintenance Effectiveness The team identified several other weaknesses, some of which were longstanding problems, that also reduced the overall effectiveness of the maintenance program.

Licensee management was apparently aware of these problems, but at the time of (23) the evaluation, had made little progress in addressing or correcting them.

1. The maintenance department overtime rate (electricians and mechanics) had been excessive. The team reviewed the work hours for maintenance ,

department craft personnel during the period of September 1990 through August 1991. This review revealed that the aggregate average overtime rate was excessive during this period. In many instances, maintenance department personnel worked more than 350 hours0.00405 days <br />0.0972 hours <br />5.787037e-4 weeks <br />1.33175e-4 months <br /> in a month. The overtime rate for ISC technicians during this same period was moderate.

2. The work request bucklog has been increasing. Since January 1990, the maintenance department (electrical and mechanical) work request backlog increased by approximately 44 percent even though there was a planned maintenance mini outage and an extended forced outage in 1991, whict offered opportunities to reduce the backlog. During 1991, as the work request backlog continued to increase, the maintenance department overtime rate was high while the number of hours devoted to continuing maintenance training decreased. While the -overall work request backlog for the maintenance department appeared to be relatively low, the work request backlog for all plant departments appeared' to be relatively high for a

single-unit facility like FitzPatrick. In addition, several hundred work request packages, some dating back to the mid-1980s, had not been closed I out even though the actual work was completed. These packages were being reviewed by the coordinator for the Nuclear Power Plant Reliability Data System and by the QA department personnel.

3. Several weaknesses in planning, scheduling, and prioritization led, in part, to inefficiencies in the work control process. Numerous priority I work -requests were not completed within the- 7 day period that was specified in the governing procedure. A sample of these open priority I work requests (those more than 1 year old) revealed that the majority had not been handled because they required some type of engineering department-action (e.g., modification, engineering review or suitable spare parts identification). The team reviewed the work request backlog and found that l

20-l

4 8 priorities had been assigned inconsistently. For example, identical MOV work had been assigned both priority 1 and 2 for different MOV actuators, even though all such work was intended to be accomplished during the next refueling outage. in one instance, two work requests were written for the same apparent problen. The first work request was worked af ter the second one was closed even though no problem was identified.

4. The maintenance craft had limited work space. The team observed that the electrical maintenance technicians routinely removed circuit breakers from service for testing in the electrical maintenance cage located in the turbine building. The space was about 8 ft x 15 ft, including equipment, storage racks, and work benches. It was observed that as many as 10 technicians worked in the space at the same time, in addition, there were not enough work benches for all the l&C craft in the !&C shop. The licensee was aware of these space limitations and planned to resolve the problem by constructing a new building in 1993. Discussions with maintenance personnel indicated that the condition was a longstanding problem.

2.2.9 Maintenance and Testing Positive Initiatives The team identified positive initiatives as well as some areas of strength. For

'xample, the licensee recently developed maintenance and 1&C improvement plans,

l. The IST program for pumps and valves appeared to be functioning well. The team reviewed the records of tests for all pumps under the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code,Section XI. Records were properly maintained in accordance with the requirements.

In several cases, the recorded test data indicated possible pump degradation. In each of these cases, the licensee had properly resolved the apparent pump problems. A sampling of IST pump test procedures revealed that they were of good quality. The licensee, with the assistance of a contractor, performed a high quality audit of the valve IST program in mid-1990. The audit revealed numerout weaknesses pertaining to valve testing, including numerous safety-related valves that were required to 1

I be tested but were not being tested. The licensee implemented or was continuing to implement appropriate actions to - correct these audit findings.

2. A high level of electrical and mechanical craft knowledge, experience, and professionalism was noted during the observance of plant maintenance activities. The team observed several electrical and mechanical maintenance activities and found t' hat they were performed well. Interviews with maintenance craft personnel revealed that their experience level was high. The craft were observed to be knowledgeable of the tasks. Further, it was observed that the craft foremen monitored maintenance activities in the plant, particularly when the tasks involved troubleshooting.

l l 21 i

g v, w- -

y y v  % i-- -y e-- - - - - ,. -- r

l

', 2.3 Engineering Support ' '

The team evaluated the effectiveness of engineering and tech, al support functions by reviewing such er.gineering products as engineering modifications, '

engineering evaluations, work backlogs, root cause determin*tions, and event responses. The evaluation also included an assessment of the ESW and associated systems to ascertain the consistency of design basis documents with the plant's as built condition. ,

L in the area of engineering support, the team concluded that four significant performance weaknesses continued to exist at the time of the evaluation:

insufficient engineering and technical support oversight, weak or non existent engineering evaluations, weak engineering design activities, and configuration control deficiencies. Contributing causes to the four main weaknesses included:

limited breadth and depth of site and HQ engineering, HQ engineering generally not aware of technical needs of site, site engineering not utilizing HQ expertise, acceptance of non rigorous engineering evaluations, untimely 1 evaluation of chronic system or component deficiencies, weak engineering design activities resulting in operational concerns, several fire protection program inadequacies, and numerous drawing and component labelling inconsistencies.

These conclusions were confirmed by review of recent performance, and engineering responses to the three events that took place while the team was onsite where the HPCI, EDG and RHR systems were declared inoperable. In each instance, work performed by engineering contributed to the event that took place.

The licensee had recently initiated programs designed to improve engineering support including the ongoing design basis document program, initiation of drawing improvements, root cause training, acquisition of computer analytical tools and the reorganization of engineering. Although many of these programs had been initiated, they were not fully implemented or effectively utilized.

The team's conclusions are based upon the following findings:

2.3.1 Insufficient Engineering and Technical Support Oversight (23)

1. Limited breadth and depth of site and HQ engineering organizations appeared to affect problem recognition, problem resolution and timely corrective actions. This problem was further compounded by: (1) a growing backlog l

and postponement of engineering work (modifications, improvement programs, studies), (2) missing-critical expertise at .the corporate level, (3) an insufficient number of system engineers to adequately cover systems, (4) the reactive nature of_ both site and HQ engineering.- (5) being unresponsive to resolving adverse quality condition report findings that were. assigned, (6) the lack of adequate _ staff causing some managers to perform their own technical / engineering work, rather than delegating the

• work to their subordinates, and (7) the licensee's' failure to benefit from

" lessons learned" or good engineering support practices from the nuclear industry, including inJian % int 3 (IP3), to provide improvements in engineering support.

22

2. HQ was generally not aware of current technical needs at the site, and was not proactive in offering its expertise to the site. Conversely, the site engineering organization did not pursue engineering expertise at HQ.

The licensee recognized that communications and teamwork needed to be strengthened, and was beginning to make improvemtnts in this area.

3. A primary concern of any organization should be the management of its resources, products, and services. The team requested information regarding human and financial resources expended such as staffing levels (both direct and contracted), resource allocation, resource trending, and comparative statistics regarding engineering support provided to fitzPatrick and IP3 which should have been readily available. This information was not available from PASNY until the team raised the concern, and the licensee expended considerable time before accurate information was obtained. Once obtained, the information indicated very small support from some HQ individuals or groups to the site. The data also indicated a decided preference to support IP3 rather than FitzPatrick.

4. HQ engineering was not aware of significant events at the site such as:

six recent water hammer events in the ESW system, that occurred since August 11, 1990; chronic leakage into the suppression pool and the formation of a steam bubble in a RHP.HX; and Rosemount transmitter issues being followeri and resolved by site staff.

5. Management acceptance of non rigorous engineering evaluations to dismiss safety concerns was noticed. Evaluations related to silting and corrosion problems in the service water (SW) and. ESW systems contained erroneous information and calculations concerning system or component operability invalidating conclusions made in licensee event reports (LERs) (see section '

2.3.2).

2.3.2 Weak or Non existent Engineering Evaluations

1. The safety significance of ESW system water hammer (WH) damage was not adequately assessed. Repeated WH occurrences had resulted in damages to the ESW system. Over the last year there were six WH occurrences, four of which resulted in the rupture of rubber expansion boots installed in (20 the control room chiller condenser. Regarding surveillance testing (ST),

i systems were - tested under controlled conditions, rather than under l conditions simulating ESW actuation. To reduce the- probability of additional WHs, the licensee recently installed a keep full system to make up for back-leakage through the ESW injection valves which was the main mechanism for establishing conditions susceptible to WH. The licensee did not perform any analyses to assess the potential . Severity' of the WH l occurrences (magnitude of WH forces) for any part of the system, including the part that experienced the WH regularly.-

I 2. Root cause analyses of chronic core spray pump deficiencies and potential WH were not addressed. Core spray- holding pumps provide .a " keep full" function for the core spray system to prevent WH damage. The pumps and (20 motors have had vibration problems and had been unreliable (i.e., failed-23

.. -- .~ - - - . .. - _ .

t

. +

to operate, l'npeller came loose, etc.). Engineering invol ve. ten t was minimal.

3. Engineering /operstions did not evaluate leakage from the reactor to the suppression peol, steam bubble formation in RHRHX A, or potential WH concerns. Several concerns were raised as the team studied the licensee's response to leakage of 6000 to 10000 gallons of water into the suppression pool over a period of 60 hours6.944444e-4 days <br />0.0167 hours <br />9.920635e-5 weeks <br />2.283e-5 months <br /> since August 1991. To summari:e: (1) operations did not believe RHRHX level instrumentation, (2) site engin7ering did not get involved until the team requested an evaluation, (3) HQ was not involved in the event analysis, (4) the licensee accepted the deficient condition and did not actively pur;ue corrective action, and (5) operibility concerns were not addressed in a timely manner.

Coincident with leakage into the suppression pool, was the formation of a steam bubble in RHRHX A which should have rendered train A of RHR system inoperable, in accordance with Technical Specification 3.5-G. During a PORC meeting, held in November 1991, it was concluded that the system was inoperable. Valve seat leakage past a series of valves allowed steam from the reactor to heat water within the CHRHX, producing a steam bubble, which produced RHRHX level fluctuations varying from 40 to 100 percent. The licensee assumed that RHRHX A instrumentation was not working properly during this period. The leakage of water into the suppression pool was also increasing the amount of radwaste that needed to be processed. After the team asked the licensee to evaluate the NRC's concern regarding the leakage and bubble formation within the RHRHX, an engineering evaluation and root cause analysis was performed and documented in memorandum JSEM-91-070, issued on October 17, 1991.

The following findings of poor material and design condition findings were determined in the engineering evaluation: (1) the air line to val"e 10PCV69A had fractured, the coupling between the valve and the operatur was loose, and the spring adjustment for the operator, whi h determined the seating force, was set too low (the licensee didn't know how long this cundition had existed); (2) the spring pack was replaceo on valve 10MOV70A in hopes of achieving a greater seating force, although this action failed to reduce leakage; (3) electricians discovered that the logic circuitry seal-in feature had been defeated by lifting the leads in 1972 through a field change (the operators were not aware of this change for the last 19 years); (4) the drawing showing the seal-in togic feature had been as-built at least three times without anyone noticing the lifted leads; (5) the seating surfaces of valve 10MOV70A were steam cut; (6) the RHR pump discharge check valves were gross leakers; (7) a steam bubble was allowed to form in RHRHX A; (8) the amount of leakage into the suppression pool prior to August 1991 was-unknown; (9) operations and engineering failed to question whether the system was inoperable; and (10) the licensee was evaluating other leakage paths to the suppression pool.

4. Repeated small-bore piping failures had recently occurred but root cause evaluation and corrective actions were inadequate. Six documented piping (26) failures had occurred since April 1988. _The most recent failure, a HPCI 24

_ = _ - . __ _ - - . . -_ _ . - - - -.

g line break, occurred during the performance of a surveillance test while

+

the team was at the pitnt. The line had been installed as part of a modification completed in 1983. Visual inspection of the failed pipe indicattd that the original weld may have been defective. However, the licensee stated that the weld had passed a liquid penetrant inspection and should have been acceptable, previous pipe failures were attributed to 1

l either poor weld quality or vibration induced fatigue. It was not until i the team raised the issue of small bore f ailure rates, that the licensee recommended that small-bore piping attachments be analyzed. This review would establish guidance and inspection criteria for existing piping lines that would be likely to experience fatigue failure.

5. An engineering evaluation of the flow reversal capability of the ultimate heat sink contained several errors. Several weaknesses were discovered ,

when reviewing safety evaluations regarding operation of the flow reversal tunnel which was found to be clogged with sand in March 1991. One of these safety evaluations indicated a lack of understanding of the flow rever;al design capability and contained erroneous . statements. This safety evaluation was referenced in an attachment to a letter to the NRC, and was used to justify that sand piled up against gate 36G 4 constituted no safety problem. A subsequent evaluation, performed shortly after, assumed that this gate could be lifted. (21)

This evaluation also contained several additional weaknesses which would make it invalid. One should note that since the plant began operation in 1975, reverse flow gate 36G 4 had never been operated and the reverse flow capability had never been tested. The gate was later determined to be inoperable during the team's review. Other weaknesses in evaluations associated with the reverse-flow capability included: (1) the power source for the overhead crane that would be used to reposition certain gates to achieve flow- reversal was powered from a non safety-related bus and would not be available during a loss of offsite power and (2) even with no loss of offsite power, using the same crane to move the three intake gates and gate 36G 4 durincj an a:cident condition, would be an extremely difficult task.

L 6. A calculation (JAF 091-046) used to justify SW swing check valve closure and operability presented in LER 9012 revision 1, was determined to be incomplete and overly si;nplistic. Further, the evaluation relied on interviews with maintenance personnel who were uncertain what means was used to close the valves. Calculation JAF 091-046, used to justify (20 operability of SW swing check valves identified in LER 90 25, revision 1, contained unsubstantiated assumptions tnd used incorrect fata. Examples included: (1) use of a low coefficient of static friction, (2) unsubstantiated assumption for corrosion ' joint efficiency,

, (3) incorrect data for the corrosion joint, and (4) minor errors in the ,

water hammer calculation. These discrepancies were indicative of a less than rigorous engineering evaluation. Evaluations did not exist for ESW check' valves which were found in the stuck-closed position. Based on a review of these calculations,- it was determined that justification of SW l

swing check valve operabil'ty was not provided by the licensee for the conditions that existed as identified in LER 90 12 01 and LER 90 25 01.

However, based on the licensee's corrective actions and improved check l 25 l

l

_-_ , - - . , ~ . - . . . - - - _ , ,

valve surveillance program, it is determined that the SW swing check valves are currently operable.

7.

The condition of the intake and discharge tunnels was not adequately evaluated. The tunnels were last inspected in October 1986. Substantial (29) accumulation of silt in the SW, ESW and RHRSW bays was noticed between 1988 and 1990, indicating that low velocity locations within the tunnels may also have experienced a similar buildup.

8. Several fire protection weaknesses were identified by the team which needed to be addressed by the licensee. Examples included: (1) the assumption of no offsite power for fire scenarios, (2) no high-trpedance fault analysis, (3) lack of guidance to operators in fire response procedures to achieve a safe shutdown and to assist with diagaosis of significant spurious actuations of equipment, (4) assignment of only one individual part time to walk down the plant for transient combustibles and evaluate the condition of the fire protection system, (5) 1ack of a design basis (30) document for fire protection. (6) failure to include spurious actuation vulnerabilities in fire response procedures for communications and indication circuitry, (7) lack of original or subsequent verification of illumination levels of lighting, (8) no procedures governing fire watches.

(9) unreviewed potential common mode failures of electrical cables due to lack of separation, and (10) uncontrolled storage of flammables in safety-related equipment rooms. Although the licensee developed an improvement plan for fire protection issues that had been identified, the team found the plan incomplete, and corrective actions did not appear to have been developed.

9. The operability of the LPCI battery inverter following a loss of-coolant-accident (LOCA) was not evaluated. The licensee determined in 1982 that the inverters would have to last for a period of 30 days following an accident. However, when radiation damage was considered, the inverters .

were expected to fail within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />; this would eliminate the ability to operate certain RHR valves during the accident sequence. More recent calculations put the expected lifespan of the inverters between 8 and 96 (31) hours. Given information available at the time, a common mode failure (radiation damage), would render the valves inoperable much sooner than 30 days. The licensee failed to consider such u scenario until 1991', and had not yet determined the best course of_ action. While the team was onsite, the licensee did determined that the 30 day requirement was wrong and once GE had concurred, the licensee planned to change the requirement.

10. The team reviewed the LPCI battery capacity from an accident loading perspective, and determined that an operating temperature of 65 F would be more prudent than the 60 F currently used by the shift personcel for determining operability. Similar comments also apply to the station (31) batteries. A 1987 modification and calculation recommended that the minimum operating temperature be raised from 60 F to 65 F. Because priorities were improperly assigned, this change had not been implemented.

26

- - - - _ - . - . . . . - - .. .- - - - .. -. .. - - _ ~ _ -

i h

• 4

,o

• An Engineering calculation showed that the ESW and RHR$W pump rooms would experience high temperatures should the room ventilation system f all. This design deficiency had been in existence for a long time, and was currently being reviewed by the licensee. Operability of MOV actuators and other conv enents for such high temperatures had not been determined. Moreover, (32) the calculation did not consider all the heat loads and the maximum outside temperature, and assumed, without adequate ju*tification, that thermal links on the fire dampers would not be activated id that the dampers would remain open. ,

12.

i Evaluations of General Electric Service Information Letter (CESil) applicability required by site procedures, and subsequent corrective actions were untimely.

1 Thero were a total of 542 outstanding GESit reports, of which 313 were overdue. Considering the current GESIL backlog i reduction rate, it will tske approximately 7 years to address the current

outstanding GESils, l 13. ' Root cause evaluations during LER analysis were superficial. The root

cause analysis of sevi.al events, involving plant transients such as scram and system isolations and as reported in LERs, was not adeqeate enough to

identify the real root causes. In these instances, the an& lysis did ..ot (33) go beyond identification of immediate or intermediate causes. The corrective actions based on such immediate or intermediate causes did not cure the real problems. This weakness resulted in recurrence of the same events several times over a period of a few years which could have been otherwise avoided. The licensee was beginning to address this issue, as demonstrated by the recent completion of root cause courses by more than i i 100 plant personnel.

I 14.

i It was difficult to track the status of corrective actions associated with i;

LERs. Such corrective actions were reviewed during the evaluation, and (33) they were generally found to have been prioritized, worked, and tracked using the computerized plant Action Commitment Tracking System (ACTS).

However, the process of finding the status of these corrective actions at 4

' any given time was cumbersome, because the items in ACTS were not cross-referenced to the LERs and, therefore, could not be sorted by LER. The licensee indicated that the ACTS would be improved to enable corrective-

actions to be sorted by LER number.

15. The licensee generally reviewed NRC bulletins, and generic letters in a timely manner, and in accordance with site procedures. However, corrective (33) actions emerging from operating experier.ce reviews was insufficient in some cases (See section 2.2). The team found two deficiencies relating to experience reviews:- (1) there was a conflict between site procedure (PS0-

28) and HQ procedure (NGP 12) regarding the necessity for a formal review of NRC notices -in that the site did not require a formal review, Site engineering was unaware of the HQ procedure, and (2) a review performed for notice 9104, indicated that Operating Procedure OP 65 was to be revised by June 30, 1991. However, the action was not completed.

T 27 I

_._ -. _ , , - , , , m.m., , _m - _ _ _

- _ _ _ - . .. - - - -_ - - _ . . _ = - _ _ . - .

, 16. The licensee was in the process of downgrading some :afety celated ESW components to non safety related, and reducing the required flow requirements. These actions appeared questionable. The draft design basis document for the ESW system did not accurately reflect the new, or (33) proposed, safety classification of components and equipment. Consequently, the team was unable to make an adequate determination of the licensee's '

safety evaluation, JAF-SE 90 067. Rev 1, regarding the new design basis for the ESW system. As a result of the downgrading, the required flow to safety-related equipment, provided by the ESW pumps. would be substantially lower than currently specified in the FSAR. A thorough review of this change should be accomplished.

2.3.3 Weak Engineering Design Activities Resulting in Operational Concerns

1. A relatively high number of design errors for FitzPatrick. (twice that of peer group plants), was reported by the Nuclear Operations Analysis Center (NOAC). Its report (ORNL/NOAC 266, " Review of the Operating Experience for FitzPatrick from January 1989 April 1991,") dated August 1991, indicated in section 3.2, that 16 events involved problems arising from design errors found duri'.g plant operation, or during reviews conducted (34) by the utility staff, or during reviews conducted by the NRC. This appeared high for a plant in commercial operation for almost 15 years.

The 16 events involving design errors were reported in 15 LERs: 333/89-004,008, 010, 011. 012, 013, 015, 018, 025, 90 012, 019, 020, 021, 029, and 91004. The licensee indicated that this was caused partially by increased efforts through increased engineering reviews. However, other contributing causes might be present. The licensee had not performed a root cause analysis of this issue.

2. Lighting fixtures installed directly above safety related batteries in each of the battery rooms were found by the team not to be seismically (35) installed. During a seismic event, the fluorescent bulbs and metal sh& des could fall onto the batteries, possibly damaging or shorting out the batteries. The licensee was developing a modification to resolve this concern.

3. The licensee had experienced repeated small-bore line breaks without evaluating the causes. While the team was on site, a HPCI test connection, including two valves and a pressure gauge installed by a modification in 1988 broke off during a surveillance test requ! ring the system t) be inoperable. Similar small-bore pipe failures had o: curred at least six times since 1988, due pessibly to weak engineering design, defective welds, or vibration fatigue. A root cause analysis was not performed on any <f these line breaks. In addition, the as-built conditien of the HP'Jl modification differed fro;n the design, in that an addit!onal vdve, pressure gauge and associated piping were added.

4. The design basis did not account for a single failure of either 6! the

. two trains of intake de-icing heaters. Each EDG train provides power to (36) 44 intake de icing heaters, for a total 'of 88 heaters. The Techniccl Specifications require a minimum of 18 (total) de icing heaters to oe l

28 l

1 - -. . .. -

- - _ - - . -- -- -- - . . _ - -- . - - - _~ . . .-.

I

• operable, the basis for which could not be located by the licensee. To ensure that at least 18 heaters are operable when considering a single 1

failure of one of the EDGs. the Technical Specifications :hould require '

) that at least 18 heaters be operable for each train.

] 5. Several design deficiencies were noted by the team involving the EDG air start system. For example, a common header between air tanks made the (373 i

system not entirely independent or redundant as required by the FSAR. -

l The team also noticed several design modifications initiated as early as 1988 to correct many of these deficiencies that had not yet been implemented because of improper priority assignments.

6. The team found that electrical cables located in the control room to 1

operate the intake de icing heaters did not meet separation criteria. I38I

The cables were actually bundled together in the control room.

Consequently, a common mode f ailure could remove both trains from service, 1 resulting in the loss of de icing capability. To recover from this event j would require that extraordinary actions be taken which were not covered j

by emergency procedures. The potential for many additional similar -

conditions exist at the plant. For example, after the team left the site,

' the licensee reported in November 1991, that a fire could di$able both .

trains of the control and relay room heating and ventilation system.

7. The team questioned whether cabinets located in the control and relay i rooms were seismically installed. The licensee had also identified this i same issue and was continuing its evaluation. Preliminary documentation i

received on the seismic qualification of the cabinets was deficient, in ,

i format and content, rendering the review inconclusive. Consequently, the DET was unable to determine whether the seismic qualification was properly j

done required or the extent this to resolve of corrective issue. action (modifications) that 'would be 2.3.4 Configuration Control Deficiencies

1. The ultimate heat sink flow reversal design capability as described in the FSAR and surveillance test OP 4 was not maintained in an operable- (39) condition since initial startup.

i 1

2. Control of setpoint information was. weak and fragmented. The licensee is currently considering which setpoint information should be controlled, (39) the method used for control, and the time period for initiation.

[ 3. Design calculations were not properly controlled. Some calculations performed as part of the modification process.were not included in a list (39) of controlled calculations. Actions were being taken to collect all design calculations and to provide proper control. . However, the licensee did not know when this effort would be completed. 1

4. Drawings contained numerous errors and inconsistencies. Controlled flow -(39)

, diagrams of the ESW system contained conflicting or erroneous information.

1 Some instrumentation isometric sketches did exist, but were not "as built" 29 i

l i

+- -

w-w.ey y r ye aw r9-, eD w

,9-g -wT-r N ' - ' i t F-

__ . _ _ . - - ~ _ . _ - - - - - _ . ._ - --_

a**=

.' and also contained errors. Using the diagrams to perform walk downs was also difficult, because many components did not have labels, and the labels that did exist had been made with a felt tip marker and did not necessarily agree with the diagram. A new series of drawings was t,eing developed to minimize confusion. For a period of time, however, as many as four different types of controlled drawints will exist, containing similar information on the same system.

5. More than 550 installed modifications were not closed out as complete, of which, approximately 514 were installed more than a year ago. The licensee (39) stated that management had assigned a low priority to close out process, because of l'mited resources and changing priorities.

6. Post modification tests were not performed for 6 V de and 125 V de 10 CFR (39)

Part 50 Appendix R lighting, as is required by procedures.

L

7. A temporary modification was installed without proper authorization and was subsequently removed without an evaluation. Temporary electrical cables and a fan were installed (installation date unknown) to cool a (39) security electrical panel located in the screenwell area which had clogged ventilation ports. Once the team identified this issue, the licensee cleaned the panel ports to improve air circulation and removed the ,

unauthorized modification.

8. During the onsite evaluation period, a 115 kV line was lost which required an EDG to start. Due to inadequacies caused by a recent EDG fan modification (M1 91 171), the fan failed to start, rendering the EDG (40) inoperable. The same failure occurred during post modification testing '

of the fan, but the problem was not corrected. This problem was cau:ed by the lack of instruction in the modification package regarding reinsertion of the fan starter cubicle into the motor control center (MCC) cell. The MCC cubicles did not have e mechanical locking device to ensure complete insertion before operation, as does a 600 V or 4-kV breaker.

9. The sitt procedures used to accomplish engineering modifications lacked specificity and referred to organizations and engineering work policies g39) and practices that no longer existed. Ambigucus wording and multiple procedures allowed too much discretion for staff to initiate, design, install and close out modifications. Much of the procedural confusion was due, in part, to the site and HQ engineering organizations being in a state of transition.

2.4 Management and Organization The team evaluated the effectiveness of management and organization at both FitzPatrick and HQ. This included an evaluation of management oversight and direction, corrective actions, quality verification,- staffing, programs, ano improvement initiatives.

30

. _ - _ _ _ _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ - _ - _ _ _ _ _ - _ _ _ - _ _ --_ _-_ _ -__ ___--____-__ ___- __ _____-_-____-____-