IR 05000413/1997011

Insp Repts 50-413/97-11 & 50-414/97-11 on 970720-0830. Violations Noted.Major Areas Inspected:Operations,Maint, Engineering & Plant Support

ML20211L346
Person / Time
Site:	Catawba
Issue date:	09/26/1997
From:	NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II)
To:
Shared Package
ML20211L256	List: IR 05000413/1997011 ML20211L251 ML20211L305
References
50-413-97-11, 50-414-97-11, NUDOCS 9710100165
Download: ML20211L346 (31)

Text

_ _ .

,

4 .

U.S. NUCLEAR REGULATORY COMMISSION -

REGION II

Docket Nos: 50-413, 50-414 License Nos: NPF-35. NPF-52 Report Nos.: 50-413/97-11. 50-414/97-11 Licensee: Duke Energy Corporation Facility: Catawba Nuclear Station. Units 1 and 2 Location: 422 South Church Street Charlotte. NC 28242 Dates: July 20 - August 30, 1997 Inspectors: P. B31 main.-Acting Senior Resident Inspector R. L. Franovich. Acting Senior Resident Inspector M. A. Giles. Resident Inspector (In Training)

W. Holland. Senior Reactor Inspector (Sections 01.1. 02.1, M1.1, M1.2 M7. M8.2)

N. Merriweather Reactor Inspector (Section E2.1)

Approved by: C. Ogle. Chief Reactor Projects Branch 1 Division of Reactor Projects Enclosure 2

[k Soo$ Obo 3

.

- _ _ _ _ _ _

,

EXECUTIVE SUMMARY Catawba Nuclear Station. Units 1 and 2 NRC Inspection Report 50-413/97-11, 50-414/97-11 This integrated inspection included aspects of licensee operation maintenance. engineering, and plant support. The report covers a 6 week period of resident inspection. as well as the results of announced and reactive inspections by regional reactor safety inspector Ooerations

. A good cuestioning attitude and operating crew safety sensitivity were observec during routine shift turnovers and operations in the control rcom. (Section 01.1)

. A non-cited violation was identified for failure to follow the control rod movement testing procedure. The licensee's reactivity management evaluation process following the occurrence was effective. (Section 01.2).

. Two manual Unit 2 reactor trips were initiated in response to Digital Optical Isolator (D01) failures that caused the D Steam Generator (SG)

Main Steam Isolation Valve (MSIV) to close. The failure to correctly develop a DOI replacement plan and to perform DOI replacement activities in accordance with controlling procedure was identified as a violatio (Section 01.3)

. Following the second manual trip, the decision by the Operator at the Controls to control the D SG level within 1 percent of the lo-lo level setpoint for auxiliary feedwater system auto-start contributed to an unnecessary Engineered Safety Fedture actuation and associated minor Jost-trip complications. The inspector raised concerns regarding the 31 ant Operations Review Committee's initial recommendation that Unit 2 was prepared for restart contingent upon successful DOI test result However, the subsequent decision to delay restart to re) lace potentially implicated components that had the same date stamp as tie failed components was appropriate. Efforts to obtain additional information on the DOI failures and develop a strategy and procedures for online testing of the MSIV D01s until a root cause could be determined, were also considered appropriate. (Section 01.3)

. During plant tours. operating and standby equi) ment appeared to be in generally good material condition. However, o) served material condition of some balance of plant equipment indicated a need for additional maintenance attention. (Section 02.1)

Maintenance

. Testing of the 1B emergency diesel generator was conducted in a good manner. The procedure provided clear instructions and operations personnel conducting the test performed the evolutions in a thorough and professional manner. (Section M1.1)

Enclosure 2

,

a 2 1

. Evaluation of the emergency diesel generator output breaker trip problem and troubleshooting was conducted as required by licensee procedures and i processe In addition the Failure Investigation Process provided a methodical method for cause determinatio However, only an apparent cause was identified. (Section M1.2)

. An unresolved item was identified, pending the licensee's completion of a metallurgical analysis and root cause determination of an emergency diesel generator turbocharger mounting bolt failure, to determine if the root cause and corrective actions for previous failures should have prevented recurrenc (Section M1.3)

.

The licensee's evaluation to determine the root cause of condensation in the 8 and 2B pump motor breaker cubicles was thorough, and corrective actions to prevent recurrence were appropriate. (Section M2.1)

. A violation was identified for failure to identify conditions adverse to quality and take corrective actions during reviews in accordance with Generic Letter 96-01. (Section M7.1)

Enoineerina

. The licensee's planned and completed actions to determine the root cause for the failure of the E-max digital optical isolation devices were adequat The 1icensee took conservative steps by replacing five Unit 2 MSIV D01s that had the same manufacturers date code as the two that previously failed. (Section E2.1)

. Communications between McGuire Nuclear Station (MNS) and Catawba Nuclear Station (CNS) regarding the degraded condition af the ice condenser floor at MNS was prompt. Based on the licensee's inspection of Unit 2 and review thus far, this issue does not appear to be a problem at Catawba. Inspection of the Unit 1 ice condenser has been placed on the licensee's forced outage list. (Section E2.2)

. An issue involving excess aluminum in containment, which was identified at MNS. was communicated to CNS engineering personnel in a timely manner. and engineering support from the corporate office to determine the impact was responsive. An unresolved item was identified pending the completion of a root cause evaluation to determine why inappropriate filters were used inside containment. (Section E2.3)

Plant Succort

. The licensee's efforts to monitor ammonia concentrations in closed cooling water systems were proactive in minimizing the risk of stress-corrosion cracking of copper-alloy heat exchanger tubes. Appropriate actions were taken to reduce elevated levels ammonia in these system (Section R1.1)

Enclosure 2

- _ - _ - _ -

.

a

. The licensee's response to the temporary loss of access to the Unit 2 auxiliary feedwater pump room and auxiliary shutdown panels was timely and appropriate. The root cause evaluation was adequate. and the-conclusion that tampering was not involved in the incident was well-reasoned. Actions taken to prevent recurrence were appropriat (Section $1.1)

i

,

Enclosure 2 i

- o

-

-. -f ;

c

.

Reoort Detail SummarFof-Plant Status

!

- Unit 1 operated at or near 100% power during the inspection perio !

- Unit 2 operated at or near 100% power until July 26, when a manual = reactor trip was initiated in response to the closure of the D Steam Generator (SG)

Main Steam Isolation Valve (MSIV). The immediate cause was attributed to a

failed Digital Optical Isolator (D01) in the seal-in circuit associated with -

the D SG MSly "Open" pushbutto The D01 was replaced. and the unit restarted on July 28. The unit reached 100% power on July 29 and operated at that power

= level until August 17, when a manual reactor trip was initiated following the closure of the D SG MSIV due to the failure of a second D01. Like the firs this second DOI was also located in the MSIV seal-in circuitry for the "Open" pushbutton. The unit restarted on August 18 following testing of other D01s

'

in the Unit-2 MSIV control circuits and replacement of 6 001s (including the one that had failed). The unit returned to 100% reactor power on August 19

!'

and operated at or near that power level for the remainder of the inspection perio Review of Uodated Final Safety Analysis Report (UFSAR) Commitments-p While performing inspections discussed in this report, the inspectors reviewed the applicable portions of the UFSAR that were related to the areas inspecte The inspectors verified that the UFSAR wording was consistent with the

. observed plant prdctices. procedures, and/or parameter I. Operations Ol' Conduct of Operations 01.1 General Comments (71707)

The inspector observed several operations shift turnover meetings in the control room. The inspector also observed that the turnover meetings involved discussion-on unit status and upcoming evolutions by each oncoming crew member. The inspector noted the briefings of shift personnel were generally: formal and followed a process which allowed for ,

crew interaction to assure all items were discussed. During one of the briefs, an abnormal condition annunciated on Unit 2. Briefing -

activities were stopped and operator attention was focused on the understanding of-the condition annunciated. The briefing did not recommence until this issue was appropriately-addressed by the operators. The inspector considered these operations crew actions demonstrated a good questioning attitude of an abnormal conditio The inspector conducted several control room tours and observed each unit's operators performing evolutions. The inspector noted that few annunciators were lit and when operators were questioned about lit annunciators. they fully understood reasons why annunciators were in alarm. The inspector specifically noted good sensitivity by the Enclosure 2

~

-. -

.

.. Operations Shift Manager. The inspector considered the operations crews

--

demonstrated-a good operating safety-sensitivit = -

01.2 Error Durina Unit 1 Control Rod Movement Surveillance Insoection Scooe (71707.61726)

On July 30. during the 3erformance of the Unit 1 monthly control rod movement surveillance-()T/1/A/4600/01. Section 12.11.5), the reactor

'

operator performing the test erroneously withdrew control bank C control rods during a portion of the test requiring the bank be inserte The-

,

inspector reviewed procedure guidance for performing the test and the l

!-

effect this error had on the reactor core and rod control system. The inspector also reviewed the licensee's assessment of this error as a potential reactivity management event, i Observations and Findinas The control rod movement test is a surveillance required by Technical Specifications (TS) that verifies pro)er operation of the control rods 'i and rod _ position indication system. Juring the test.-each individual bank increments) of control rods is inserted and~ subsequently a total

_ withdrawn (in of five10 steps step (in 5 step) to its increments initial location. When control bank C was selected for testing the

. reactor operator verified.all prerequisites, but inadvertently withdre the bank from its initial position of 229 steps to 231 steps. The reactor operator recognized the error and stopped rod movemen Concurrently. the control room Senior Reactor Operator (SR0) recognized t.te error and verbally instructed the reactor. operator to stop. The control room staff determined that control bank C was in a safe position at 231 steps. - After consultation with engineering - the bank was returned to-its-initial position and the test was complete At-229 steps, control bank C was' essentially fully withdrawn from the core and outside of the active fuel region. Moving to 231 steps had little or no reactivity effect. .The 231 step position is thelast valid

-

control rod position. Moving control bank C to this position did not impact operation of-the rod control system, which would occur if outward rod movement is demanded beyond 231 step The-licensee initiated Problem Investigation Process (PIP) report 1-C97-2484 and a root cause evaluation to investigate the 3roblem. The inspector attended a reactivity management meeting w1ere this problem was discussed., The meeting was we R supported by 0)eration Maintenance. Chemistry and Engineering personnel. )iscussions involving the assessment and characterization of-this and other reactivity related PIPS were in dept Failure to insert the control bank C control rods is a violation of TS 6.8.1. Procedures and Programs, for failing to follow procedur Enclosure 2

.. .. ..

. .

.

.

.. .

.

.. .. .. _

. . .. .

.

.

However, based on the immediate corrective actions and the safety significance of the circumstances, this non-repetitive. licensee-identified, and corrected violation is being treated as a Non-Cited Violation (NCV) consistent with Section VII.B.1 of the NRC Enforcement Policy. NCV 50-413/97-11-01: Failure to Follow Control Rod Hovement Testing Procedur Ccoclusions A non-cited violation was identified for failure to follow the control rod movement testing procedure. The licensee's reactivity management evaluation process following the occurrence was effectiv .3 Manual Reactor Trio Followina Closure of the 20 Steam Generator Main Steam Isolation Valve 1 Insoection Scone (71707. 37551)

On July 26 and August 17, the Unit 2 Operator at the Controls (0ATC) l responded to control room indications that the D Steam Generator (SG)

Main Steam Isolation Valve (MSIV) had closed. The 0ATC manually tripped the reactor, and the Failure Investigation Process (FIP) was initiated to determine the root cause. The first MSIV failure was attributed to a failed Digital Optical Isolator (D0I) whereas the second MSIV failure was attributed to a degraded DOI. The inspector responded to both unit trips, inspected.the electrical cabinet that housed the failed 00l ;

discussed the FIP team's progress with Engineering personnel, and reviewed the associated PIPS and Licensee Event Report-(LER) 50-414/97-0 Observations and Findinas First Manual Trio On July 26. the D SG MSIV failed closed for no apparent reason. A control room annunciator alarmed, and the 0ATC referred to the valve position indications to verify that the valve was closed. The 0ATC then manually trip 3ed the reactor, and a FIP team was organized to investigate t1e cause of the MSIV failure. The licensee reported the manual actuation of the Reactor Protection System (RPS) in accordance with 10 CFR 50.7 The FIP team identified a failure of one of two DOIs that separate the nonsafety-related control circuitry from the safety-related control i

circuitry in the seal-in circuit associated with the MSIV's "Open" pushbutton. The voltage across the DOI did not meet acceptance criteria during a simple test using a Fluke multimeter. The DOI was replaced on July 27. and work requests were generated to similarly test the DOIs associated with the control circuits of the other Unit 2 MSIVs. The-Enclosure 2

"

,

Ltesting'did not reveal _any other failed 001 During. replacement of the failed DOI. technicians-incorrectly _ isolated

-

the component and caused the MSIVs for the other three SGs-to clos Control. room operators manually cycled SG Power-Operated Relief Valves (PORVs) to relieve-the resulting pressure increase. The technician had

-

jumpered around the input of the failed DOI. which was in series with three D01s associated with the other SG's MSIVs. Instead of jum)ering around the output. the technician isolated it, which disrupted.tle-circuit and caused the other three MSIVs to close.

The ccdrolling procedure for the DOI replacement was IP/0/A/3840/003, p " Calibration. Checkout, and Replacement of Optical Isolators." Revision 1 Prere drawings,details, quisite'4.1-of theprocedures, manuals, procedure directs thereference and other technician to-review material as necessar Implicit in the prerequisite-is that these activities be E

performed correctly. The inspector interviewed the technician -

performing the replacement activities, who indicated that he had misread -

a connection diagram when developing-the DOI replacement pla The= error was self-revealing, affected the safety-related contro circuits of three MSIVs. caused the MSIVs to close, and induced a SG pressure transient that required manual mitigation by control room operators. Therefore, this failure to correctly develop a DOI replacement plan and to perform DOI replacement activities in accordance -i with the controlling procedure is identified as a violation of TS 6. '

Violation-50-414/97-11-02: Failure to Correctly Develop a DOI Replacement Plan and to Perform DOI Replacement Activities in Accordance With the Controlling Procedur The licensee sent the failed DOI to the Qualification and Testing Facility at the McGuire Nuclear Station to determine which subcomponent caused the failure. Unit restart commenced on July 2 Second Manual Trio On.Adgust 17, the D_SG MSIV failed closed. The 0ATC manu' ally tri the reactor in response to indications that the MSIV..had closed. ppedand a-FIP team was organized to investigate the cause of the MSIV' failur The licensee reported the manual actuation of the RPS in accordance with 10 CFR 50.7 An Auxiliary Feedwater (AFW) system auto-start signal was generated when-

-

the DATC was. manually cycling the SG PORV to relieve pressure-and controlling' D SG level to avoid overcooling of the loop. The Unit 2 SG- ,

Llo-lo. level setpoint for AFW auto-start, htich is an Engineered Safety !

Features-(ESF) actuation, is 37 percent. The controlling procedur EP/2/AM000/ES-0.1. " Reactor Trip Response Procedure." Retype Number 1 dr. provide specific guidance for controlling SG level above the-

% in le.e' setpoint for AFW actuation. The OATC decided to control. the Enclosure 2

.

_

..

.

0 SG 1evel at 38 percen Although manual cycling of the PORV was proactive, executed in anticipatiori of an automatic action and performed in accordance with applicable procedures and management expectations, the resulting change in SG inventory caused SG level to drop to the AFW autostart setpoin Controlling SG level so close to the setpoint contributed to an unnecessary ESF actuation and associated minor post-trip complication The ESF actuation generated an auto-start signal to the AFW pumps (which were running at the time) and caused the AFW flow control valves-to fully open. The 0ATC throttled the flow control valves back, and ro appreciable Reactor Coolant System (RCS) cooldown resulte The licensee-included the ESF actuation in the 10 CFR 50.72 report for the manual RPS actuatio The inspector questioned the appropriateness of controlling SG level so close to the 10-10 level setpoint and discussed the question with Operations management. Operations management asserted that the 0ATC was o>erating the plant in accordance with procedures. When questioned a)out expectations for preventing unnecessary ESF actuations. 0)erations management expressed an expectation that ESF actuations should )e l avoided. Operations management indicated that some procedural

! enhancements and augmented training would be considered and addressed l accordingly in station PIP 2-C97-268 The FIP team investigating the MSIV closure determined that the other DOI in the seal-in circuitry associated with the MSIV's "0)en" pushbutton had degraded. The voltage across the DOI met t1e acceptance criteria during the test using a Fluke multimeter. However, over time, spikes in the output voltage were detected by the Fluke multimeter. An oscilloscope was used to perform an extended test to confirm this behavior. Engineering personnel speculated that a spike in voltage had caused resistance to increase and a downstream, normally energized relay to de-energize; thereby opening the seal-in circuit associated with the

"Open" pushbutto The licensee replaced the degraded DOI and successfully tested its replacement. In light of the earlier DOI failure in the MSIV circuit on July 26. the licensee also )lanned to conduct an extended test of the D01s associated with the otler Unit 2 SG MSIV A restart Plant Operations Review Committee (PORC) meeting was conducted via teleconference on the evening of August 17: the inspector was s present via telephone. Potential causes of the DOI failure, including environmental conditions, degraded power supplies, and defective components or manufacturing processes, were discusse Engineering aersonnel indicated to the PORC that a root cause of the DOI failures lad not been determined and that the vendor was not available for assistance. Engineering personnel recommended extended testing of the Unit 1 MSIV DOIs and subsequent weekly testing of all MSIV D01s until a p

Enclosure 2

,

-..

.

root cause of the DOI failures could be' determined.-

The PORC recommended-that the remaining Unit 2 MSIV 001s be tested and that unit restart commence contingent upon succea ful test results: the

-

station manager approved the recommendation.- The inspector raised concerns about restarting the-unit when: (1) the root cause of the DOI-failures had not been determined: (2) common cause failure had not been ruled out; and (3) the licensee had not involved the vendor in its

-

investigation of the second MSIV DOI failure. The licensee delayed-restart to replace five D01s in the other Unit 2 MSIV control circuits that had the same dates stamped into the components as those that had faile A second restart PORC was conducted on August 18, Since the first restart:PORC.-the licensee had: (1) ruled out certain potential root causes: (2) developed a testing plan with formal testing procedures and test acceptance criteria for online testing of the MSIV circuits: (3)-

begun to address generic implications for similar 00ls in critical

control-applications other than MSIV circuits: (4) determined that a failed resistor had caused the HSIV COI to fail: and (5) contacted Performance Improvement International to arrange for a component failure analysis, The PORC recommended unit restart, and the recommendation was approved.. Unit-restart commenced on August 18. Both manual reactor trips and the ESF actuation associated with the second manual trip were documented in LER 50-414/97-06. which was submitted to the NRC on August 25. 1997. (Details of the failure history of D01s and the licensee's:

root cause investigation are discussed _ in Section E2.1. of this Inspection Report.)- Conclusions Two manual Unit 2 reactor trips were initiated in response to DOI

.~

failures that caused the D SG MSIV to close. The failure to correctly develop a DUI replacement plan and to perform DOI replacement activities-in accordance with the controlling procedure was identified as a violation. Following the second manual reactor trip. the OATC's decision to control the D SG level within 1 percent of the 10-10 level setpoint for- AFW auto-start contributed to an-unnecessary ESF-actuation and associated minor post-trip complications. The ins)ector raised concerns regarding the PORC_'s initial recommendation tlat Unit 2 was' '

arepared for restart contingent upon successful DOI test result owever, the subsequent decision to delay restart to_ re) lace potentially implicated components that had the same date stamp as t1e failed components was appropriate. Efforts to obtain additional information on

'the DOI failures and develop a strategy.and procedures for online-testing of tk MSIV D01s until a root cause could be determined, were -

also considered appropriat Enclosure 2

, . , _ _ . - - . _ - , _ _ , , -J...,--, .---_W, m ~.m -,

.

.

02.1 Observed Plant Doerational Conditions InsDection Scope (71707)

The inspector conducted several tours of selected areas of the plan Areas or components observed included:

.

Unit 1 and Unit 2 Main Turbine Decks

. Unit 1 and Unit 2 Main Feed Pumps

. Unit 2 Condensate Booster Pumps

. Unit 1 and Unit 2 Hotwell Pumps

. Instrument Air Compressors

.

Unit 1 and Unit 2 Condenser Waterbox Areas

. Unit 2 Heater Drain Pumps

. Unit 1 Emergency Diesel Generators

. Unit 1 and Unit 2 Vital Inverters

. Unit 1 Emergency Shutdown Boards Observations and Findinas The inspector observed that housekeeping in most plant areas toured was good. In addition most of the operating and standby equipment appeared to be in good material condition. However, oil leaks were noted on the Unit 1 and 2 hotwell Sumps and the Unit 2 condensate booster pump Also, minor water leacage was noted for both the Unit 1 ar.d Unit 2 condenser waterboxes where the waterboxes attach to the condenser. The ins)ector noted a body to bonnet leak on Valve 2-HW-65 (2C1 Heater Drain Tan < Pum) Recirculation Line Control Valve). Operators had identified this leat as a packing leak. After questioning by the inspector, operators reclassified the leak as body to bonnet, Conclusions During plant tours, operating and standby equi) ment appeared to be in generally good material condition. However caserved material condition of some balance of plant equipment indicated a need for additional maintenance attentio Enclosure 2

_ . _ _ . - _ - _ _ - _- _ _ _ _ _ - - _ - - - _ ____-___ _ - _-

'

V :. - '_ _

i

'

[

8 l l08' Hiscellaneous Operatinns Issues (92901)-

08.1 _(Closed) Violation (VIO)'50 413/95-16-01: Inadequate OperNing Procedure

for Reestablishing Normal I.etdown Resulting in a Water Pommer

" r The violation cited an example where a. procedure failej to maintain proper system configuration and led to a challenge of the-letdown

, systen. -The . ins j August 31.1995,pector

-

which reviewed addressedthe thelicenset's procedural violation response inadequac dated control of-L troubleshooting activities, and management and engineering review needed i

to. implement troubleshooting activities. Through review of the

corrective action documentation (PIP 1-C95-058) the inspector verified F that the licensee revised procedures for establishing letdown, required management approval prior to opening the most risk-sionificant valve- in the letdowr. system, and communicateo expec%tions to Operations Shift-Managers for including engineering for inde)endent review and obtaining '

station management concurrence for troubleslooting plan .

II. Maintenance M1 Conduct of Haintenance M1.1 General Coments - Insoection Stone (62707)-

The inspector observed and reviewed complete test documentation for all-or portions of the following work activities: 4

)

--*

-PT/1/A/4350/02B DIESEL GENERATOR 1B OPERABILITY TEST.,

performed.on August 26, 1997

-. PT/1/A/4350/02B DIESEL GENERATOR 1B OPERABILITY TES performed on August ~27.-1997

' Observations and Findin.q1 The inspector reviewed.the surveillance sackages and discussed the test activities with operations personnel. Tie inspector noted the work-instructions were appropriately-filled out and functional testino accomplished-as required. The procedure provided clear instructicas~to accomplish the work activity. On August-26c after the IB EDG was loade .

. to approximately 5700KW. the output breaker, from the EDG tripped open  ;

due to a high current condition. This issue is further. discussed in section M1.2 of this report. After corrective actions were accomplished for PIP 1-C97-2796. the IB EDG was tested in ac,:ordance' with PT/1/?d4350/02B on August 27, 1997.' No. deficiencies were note _

-.

Enclosure 2

.. ---

.

.

c. Conclusions The inspector concluded the testing of the IB EDG was conducted in a good manner. The procedure provided clear instructions and o)erations personnel conducting the test performed the evolutions in a t1orough and professional manne M1.2 Unit 1 B Emeroency Diesel Generator (EDG) Outout Breaker Trio Durina Testinc Inspection Stone (62700)

The inspector monitored licensee activities associated k "1 emergent work order generation. EDG troubleshooting, and subsequent corrective maintenance associated with the subject proble b. Observations and Findinas The inspector was observing testing of the IB EDG when the output b eaker tripped open. The initial indication of the cause of the trip was an overcurrent condition (Instantaneous Overcurrent Trip Relay 50DTG) caused by loss of control of the machine load (drift). Operators

[ exited the test and shut down the machine. A work order (W/0 97074038)

and PIP 1 C97-2796 was written to commence troubleshooting of the proble During the next 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, the licensee conducted troubleshooting activities to determine the cause of the overcurrent condition. A failure Investigation Process (FIP) team was established to perform a root cause evaluation. Troubleshooting testing was performed using procedure OP/1/A/6350/02. " DIESEL GENERATOR OPERATION" during the afternoon of August 26. and again during the morning of August 27. 199 Maintenance activities were conducted between the troubleshooting test Maintenance activities included:

. W/0 97074038-06 TEST 50 DGT RELAY

. W/0 97074038-07 CHANGE OIL IN GOVERNOR The inspector monitored troubleshooting activities including EDG troubleshooting test runs and reviewed completed documentation for the maintenance activitie The FIP team used a list of possible failure modes (causes) for both the drift and overcurrent conditions experienced during testing. These possible causes were documented along with a discurtion of eac Although the aroblem could not be repeated and no root cause could be found, the FI) team determined the apparent cause of the event was an intermittent bad connection on the motor operated potentiometer (MOP)

which was used to control EDG speed during testing. The FIP team Enclosure 2 I

.. - . .- -- - . - . . .

- - - . - .- .

l

.

.

l

. l

concluded the MOP oroblem would not affect EDG operation because the MOP returned to the 60' hertz position after each test run and/or initiation-of an emergency start. Tne FIP team review process and apparent cause was presented at a Plant Operations Review Committee (PORC) meeting on August 28, 1997. The PORC agreed that EDG 1B could be returned to an Operable status.

>

The inspector discu: sed the findings with FIP team members and attended the PORC meeting on August'28. 1997. -In addition, the inspector reviewed the FIP team notes, the PIP evaluation, and the PORC minute The inspector also reviewed applicable licensee Elementary Generator Control Panel Electrical Schematic Diagrams and independently verified from the diagrams that the POT would return to the 60 hertz position after each test run and/or initiation of an emergency start. During ,

discussions with the licensee, the inspector questioned whether ,

disassembly of the POT would have provided any additional information which could support their apparent cause. The licensee did not consider this action was necessary at this time, Conclusions The inspector concluded the evaluation of the EDG output breaker trip problem and troubleshooting was conducted as required by licensee procedures and processes. In addition, the FIP provided a methodical method for cause determination. However, only an apparent cause was identifie M1.3 Emeraency Diesel Generator (EDG) Turbocharaer Bolt Failure JnsoectionScone(61726 On August 19. during performance of semi-annual Preventative Maintenance (PM) to check the torque on the 2B EDG turbocharger mounting bolts, the licensee identified a broken bolt from the EDG turbocharger mounting bracket. A similar failure of three mounting bolts for the same turbocharger had occurred in September 1995, resulting in Violation 50-413,414/95-20-01: Inadequate Incorporation of Vendor Information Into Diesel Generator Maintenance Instructions. The inspector reviewed the previous violation. PIPS 2-C95-1495 and 2-C97-2713. the metallurgical-analysis report associated with the previous failures, and turbocharger mounting drawings and specifications. The inspector also discussed this issue with engineering personne Observations and Findina The inspector reviewed the September 1995 event to understand the mounting bolt failure mechanism and the corrective actions that were implemented to prevent recurrence. The metallurgical examination of the three bolts that had failed revealed that the bolt failures were caused by vibration-induced fatigue cracking resulting fror' loosening of the Enclosure 2

i

!

-  ;

-t i

'l 11- t ut bolt The bolts loosened because no locking device had been installed.

,

'

Lock washers were specifieo to be used as shown on the vendor's mounting -

diagrams.- As a corrective action.=the licensee generated Minor --

,

Modification CNCE-7308 and procedure changes to provided for i

.

installatim1 of flat washers and lock washers in accordance with vendor '

recommendatwr.s. and the use of a support plate. The support plate was

installed to accommodate elongated mounting bracket holes. Support

[

-

plates were not required, but could be used as deemed necessary by

-

maintenance personnel performing maintenance on the turbochargers ,

i

Support. plates had not been installed on the turbocharger bank that had

'

the bolt failures, but had been installed on all the other EDG ,

turbocharger After the failed bolt was identified during a torque pass PM on the 2B t EDG right-bank turbocharger on August 19. all four mounting bolts were

,

removed and replaced with new bolts. The failed bolt and the other

.

three mounting bolts were sent for metallurgical analysis. The resident

! visually verified that su) port plates were not installed on the 2B EDG

right bank unlike the otler seven inservice turbochargers. Licensee

! evaluation determined that the failure of a single mounting bolt did-not affect the operability of the 2B EDG. Subsequent to the review of PIP L 2-C95-1495 (which documents the original bolt failures)- and of listed '

.

t

'

possible causes, the inspector questioned engineering aersonnel about the loading of the mounting bolts and the: adequacy of ]olt thread i:' engagement. The bolt torque had been increased as a corrective action for the 1995 failures from 60 ft-lbs to 75 ft-lbs. However, adequate

-

bolt loading;may not exist if the mounting bolt is bottomed out'in the turbocharger casing. If-this condition is present, it could cause the

bolt to loosen. Engineering personnel indicated that no calculations-

had been performed to determine the adequacy of actual bolt loading or j- thread engagemen Conclusions

"

l0nce completed the metallurgical analysis should reveal if this reaeat t

failure occurred from high-cycle vibrational fatigue cracking, whic1 was

the previous failure mechanism. The inspector will review the completed 1 metallurgical analysis and.the licensee's root cause determination-to a determine if the root cause and corrective actions for the previous 1 failures should have prevented recurrenc Pending the completion of

" --

the -licensee's root cause analysis, this item is identified as

, Unresolved Item'50-414/97-11-03: Failure of 2B EDG Turbocharger Mounting

. Bolt.

,

j, I

'

Enclosure 2 i

- - . ~ - - - _ , _ .._ - . , - _ _ , ,

7" ,.  ;

,

,

.-

+

c \

p 12 c -M25 Maintenance and Material Condition of Facilities and Equipment

! 1M2-.1- Moisture-Induced Corrosion of Nuclear Service Water Pumo Motor Breaker

-

~ Insoection Scoce-(62707)-

On July 22. thel licensee identified moisture in the switchgear cubicle

.. associated with the 28 Nuclear Service Water (NSW) pump motor. -The

liceasee determined that the-moisture originated from condensation that

had developed on the inner surface of the cable conduit housing the pump i

motor leads. The inspector discussed the condition of the breaker with engineering personnel, ins)ected the motor breaker cubicle, traced the- !

conduit line from the breater cubicle to the adjacent room, and reviewed L

t station PIP-0-C97-2368. The inspector also discussed the generic implications of the condensation dynamics with engineering personne i

b._ _ 0bservation and Findinas

!. During preventive maintenance activities on the 2B NSW Pump Motor, the licensee identified condensation in ue pump motor breaker cubicl Maintenance personnel-determined that water had been entering the !

breaker cubicle from the cable conduit at the_ top of the cubicle over a l period 'ofl time. A high potential (Hi Pot) test was performed to determine-if the cable had degraded. The initial test was unsuccessfu The motor leads were disconnected.- and the connections were-cleane The Hi Pot test was performed again, and the results were goo The licensee determined that condensation had been--forming inside the cable conduit for the'NSW pump motor leads. The conduit exits the breaker cubicle, runs the length of the switchgear room ceiling. and enters a pullbox in the 2B EDG Sequencer hallway. Engineering personnel saeculated that-warm, humid-air had migrated through the conduit from tie.EDG Sequencer hallway into the cool, air-conditioned essential-

-switchgear room. A ventilation duct vent located above the breaker:

cubicle was blowing cool air onto the conduit just above the cubicle, exacerbating the condensation of moisture inithe ai The other three NSW pump motor breakers were inspected for condensation, and moisture was also discovered inside the 18 NSW pump motor breaker cubicl Immediate corrective actions were taken to clean and dry the-1B and 2B NSW pump motor breaker cubicles and block air flow from the !

ventilation duct that was blowing directly onto the conduit. Water j-samples were.take for chemical analysis to verify _that the water was

~

condensation and not from another source. Past and present operability evaluations', dccumented in PIP OC97-2368, determined that the 2B and IB NSW pump motor. breakers were and had been operable. Corrective actions-to seal the conduit to prevent the migration of warm, moist air through ,

the conduit and into the switchgear room were implemented via work orders (WO) 97063728-05 (for Unit 1) and 97062900-01 (for Unit 2). The inspector verified that _the tasks associated with the work were complete Enclosure 2

-. . . - - _ _

.

.

The inspector asked engineering personnel if the electrica' breaker cabinets associated with other vital equipment were suscep;ible to a similar condensation dynamic. The response was that armored cabling was used to house the motor leads of the other vital components in the switchgear rooms, and that the use of conduit was specific to the NSW pump motor breaker Since armored cabling is insulated and no sir space exists between the cabling and the armored conduit. the potential for condensation is very lo Conclusions The licensee's evaluation to determine the root cause of condensation in the IB and 28 NSW pump motor breaker cubicles was thorough, and corrective actions to prevent recurrence were appropriat H7 Quality Assurance in Maintenance Activities M7.1 Hissed Technical Soarification Surveillance Reauiraments Insoection Scoce (6170H1 The inspector reviewed licensee identified issues associated with inadequate testing of both units solid state protection system (SSPS)

P-11 function and the Unit 2 SSPS P-13 functio Observations and Findinas The licensee identified. during a review of operating experience reports, that some plants were not adequately testing, on a qu6rterly basis the SSPS P-11 function. This issue was identified at another plant as part of the other plant's Generic Letter 96-01 (GL), " TESTING OF SAFETY-RELATED LOGIC CIRCUITS" dated January 10. 1996, review. The Catawba Instrumentation and Controls (I&C) surveillance procedure (Analog Channel Operational Tests ACOTs) were reviewed and found to be inadequate in that they were not testing the SSPS P- function as reauired by Technical Specification 3.3.2. The licensee wrote PIP 0-C97-2554 to resolve the issue. Further review of other circuits determined that Unit 2 SSPS P-13 testing had not been conducted in a condition that adequately tested this function as required by TS 3. due to the testing being moved to an operational (innage) window instead of testing being accomplished during an outage window. PIP report 0-C97-2646 was written to address this potentially generic proble The inspector reviewed corrective actions documented in PIPS 0-C97-2554 and 0-C97-264 He also reviewed procedure IP/2/A/3222/000B. " ANALOG CHANNEL OPERATIONAL TEST CHANNEL II 7300." Revision 52 which was performed on August 26. 1997, and verified the P-11 interlock was a)propriately tested based on correctiva actions for PIP 0-C97-2554 T1e inspector discussed the corrective actions for these issues and the process used to conduct the GL 96-01 review at Catawba with licensee Enclosure 2

__ -__ _ _ _ _ . _ _ _-___ _ _

,

.

engineering and. management personnel on August 28. 1997. The licensee noted that several enhancements to procedures were identified during the reviews and considered the review process was thorough and detaile However, the problems discussed above were not identified as part of the

- GL 96-01 review process.

.

The inspector reviewed a portion of GL 96-01 which requested, in par that licensee's " Compare electrical schematic drawings and logic diagrams for the reactor protection system. EDG load shedding and sequencing, and actuation logic for the engineered safety features systems against plant surveillance test procedures to ensure that all portions of the logic circuitry, including the parallel logi interlocks, bypasses and inhibit circuits, are 6Jequately covered in the surveillance procedures to fulfill the TS requirements." The inspector reviewed the licensee's res April 17. and May 20, 1997.ponses to the 17,NRC forresponsa GL 96 0) stated, dated in

'

The April 199 part, that " Duke Power will initiate a program that implements the requested actions of GL 96 01. This program will include a comprehensive review of each station's logic diagrams and surveillance

'

test procedures to ensure that all portions of the logic circuitry are 4 tested such that the Technical Specifications requirements are i

fulfilled. " The May 20, 1997, response stated, in part. "the purpose of this letter is to confirm the completion of the requested actions of GL 96-01 on Catawba Units 1 and 2. The review program that was recently completed on Catawba Units 1 and 2 confirmed that all Jortions of the effected logic circuitry were being tested such that tie existing station Technical Specifications were fulfilled."

Although the inspector considered the licensee's review of electrical

.

schematic drawings was methodical. the review process did not identify

the P-11 problem. or at that time, the Jotential problem associated with P-1 Code of Federal Regulations 10 C R 50. Appendix B. Criterion XVI requires, in part, measures shall be established to assure that conditions adverse to quality, such as deficiencies. deviations, and

. nonconformances are promptly identified and carrected, Tne inspector considered that licensee actions for GL 96-01 which were

, completed May E0. 1997, did not identify the P-11 problem. This problem was identified after the licensee's review of an o)erating experience report from another plant which was documented in )IP U-C97-2554 on August 4, 1997, Also, the GL 96-01 review did not identify the P-13 problem which was documented in PIP 0-C97-2646 on August 13. 199 This-is identified as a Violation 50-413.414/97-11-05: Failure to Identify

' Conditions Adverse to Quality and Take Corrective Actions During Reviews in Acccrdance With GL 96-0 The inspector noted the licensee took required corrective actions for each TS violation identifie Enclosure 2

_

,

,-

,

. c.-

_

' Conclusions

'

A violation was identified for failure to identify conditions adverse to

. quality and take corrective actions during reviews in accordance with GL

- , 96-01

' M8 -Miscellaneous Maintenance Issues (92902)

- M (Closed) VIO 50-413.414/95 20 4 1: -Inadequdte Incorporation of Vendor Information-Into Diesel Generator Maintenance Instructions The violation was issued because of a failure to incorporate veno:r I

.information into maintenance instructions (procedures), which allowed the introduction of a common mode failure (i.e., broken turbo charger mounting bolts) to the Unit 1-and 2 EDGs. As indicated in Inspection

'

Report 50 413.414/95-20, maintenance procedures were revised to include i- a lockwasher configuration for mounting the turbochargers to the diesel

-

engines. In~ addition a modification was implemented to allow for an F alternate mounting configuration. The inspector reviewed the licensee's i'

' initial res)onse,~ dated December 4, 1995, and a supplemental response, dated Decem)er 21, 1995, which addressed broader corrective actions to e -ensure continued reliability of the Catawba EDGs. The inspector

. verified these actions were performed. As addressed in Section M1.3,

'

i another failure of an EDG turbocharger mounting bolt was identified subsequent'to the licensee's corrective actions. This subsequent

, .

failure and its reflection on previous corrective action adequacy will U be followed up under URI 50-414/97-11-03. Accordingly- Violation 50-

._

413.414/95-20-01 is closed, t

E M8.2 (Closed) URI 50-413.414/94-17-04: Overpressure Protection for Service

_

Water (RN) Pumps

-The-issue involved additional review of the licensee's' administrative controls for precluding the addition of heat to RN pump motor and upper

-

bearing: coolers when the cooling system was isolated for maintenanc The ASME Code section discussing overpressure protection allows for

. positive controls and interlocks. The inspector reviewed applicable sections of the following-licensee processes to verify appropriate positive controls were in place:

. NUCLEAR SERVICE WATER SYSTEM OP/0/A/6400/06C, " VALVE CHECKLIST 4 OUTSIDE CONTAINMENT," Units 1.'and 2

.- NUCLEAR SERVICE WATER PREPLANNED TAGOUT LISTINGS

. ANNUNCIATOR RESPONSE PROCEDURES FOR RN MOTOR OPERATION ~

to- NUCLEAR SERVICE WATER SYSTEM DESIGN BASIS SPECIFICATION

. SPEC. CNS-1754.RNt00-0001. Revision 11

-

__

Enclosure 2

. _ _ ..._ _ _ _ . _ . _ _ . _ . _. _ . - . _

c

.: (

.- FLOW DIAGRAME FOR RN SYSTEM

.- PIP 0-C94-1084 The inspector determined the hcensee used appropriate positive controls and/or interlocks to ensure overpressure protection for the motor and upper bearing coolers-for the RN pumps, inis issue is considered close III. Enaineerina E2 Engineering Support of Facilities and Equipment E2.1. Review of Enaineerino Activities to Determine Root Cause For Dioital 00tical isolator (DOI) Failures that Resulted in Two Reactor Trios Insoection'Scoce (37550)

A regional based-inspector was dispatched to the Catawba site on August 19, 1997, to inspect the facility's plans for determining the root cause for two DOI failures that occurred in Unit 2 on July 26 and August 17. 1997. which resulted in Mair. Steam Isolation Valve (MSIV)

closure events and subsequent manual reactor trip Observations and Findinas Backaround The Unit 2 MSIV closure events that occurred on July 26, and August 1 , involved failure of one of two safety related "0A Condition 1" 001s in the control circuitry for the 2SM 1 MSIV associated with steam generator 2D. The licensee had documented the two events on Problem Investigation Process ~(PIP) reports 2-C97-27681 and 2-C97-242 respectively. Both events involved failures of E-max-Model number 175C156-00Is with ac input and dc output, and.a-manufacturer's date code of18/95. -The inspector found that the Failure _ Investigation Process-(FIP) team was being directed by Instrumentation and Control (I&C)

-Engineering. . The preliminary results of the Engineering investigation had concluded that the failures resulted from failure of a 20.000 chm (5 percent). 2 watt, metal film resistor that was mounted in the Model-175C156 001 circuit board. This resistor was identified as R1 on the DOI schematic drawing The DOI that failed on July 26. 1997 had-undergone evaluation and

. failure analysis testing at the Duke Qualification and Testing Facility just prior to the occurrence of the second DOI failure on August 1 . The failed DOI = circuit board components were inspected and the R1 resistor was discovered to-have a brownish discoloration on the-component. body and visible cracks in the ceramic coating. Subsequentl : the R1 resistor was removed from the DOI circuit board and its resistive

_

Enclosure 2

. - - - - - . . . - . - - - . - . - . - . - . - - - - - - - -

.-  ;

.

measurements were take The measurements showed the resistance was either infinite or an open circuit condition existed. The R1 resistor was replaced with a new resistor.-and the DOI-tested satisfactorily in-

accoroance with-design. The Duke Qualification and Testing facility issued an evaluation report on the test resti.o (documented in Memo To File No.-EV-190 dated July-30, 1997), that indicated the failure of resistor R1 was a random component failure caused by aging from heat dissipation and mechanical stresses, and should not be classified as a-manufacturing ufec The R1 resistor on the second DOI failure also was found to be discolored. The licensee observed that both failed 00ls had the same marf acturer's date code of 8/95. Accordingly, the licensee replaced five other critical D01s installed in the Unit 2 MSIV circuitry that had-the same 8/95 date cod The R1 resistors in these other five D01s with 8/95 date codes also showed signs of discoloratio The licensee shipped the two failed isolators and one " good" isolator with the 8/95 date code to an outside testing facility to conduct

-

.further failure analysis. The licensee had also scheduled an audit-of the vendor (E-Max) to be-performed on August-25, 1997. The licensee was informed by the vendor that the supplier and the design of resistor R1 had changed in the new replacement isolators installed in 1995. Visual-examination by the. licensee and also by the inspector of an older E-Max-Model 175C156 DOI-revealed that the new replacement resistor R1 was

" physically much smaller in size than in the older models. The inspector-noted that other internal components were also different in both size-and color. However, the significance of these differences was act known by the iicense Review of Procurement Documentation The procurement documentation ( purchase orders. packing slip Duke supplier verification release. Receiving Inspection Report, and vendor certificate of compliance) was reviewed for the subject DOIs and was acceptable. The inspector found that the E-Max 00Is were procured as 1 safety-related components in accordance with Duke's "0A Condition 1" procurement classification and purchase Specification No. CNS-1338.00-00-0001. " Optical Isolation Device." Revision 10; dated January 24.~

-1995. The vendor provided a certificate of compliance that certified the devices were manufactured and tested in accordance with the referenced specificatio 'The inspector found that the vendor was on the approved vendor list:

'however, the licensee informed the inspector that, at the time the failed DOIs were being manufactured, restrictions had been placed on the vendor as a result of a Quality Assurance (0A) audit finding regarding a

-

failure.to follow procedures in processing Engineering Change Orders and their failure to implement timely corrective action for the audit finding. A subsequent audit and surveillance verified that the vendor Enclosure 2 o

--

- . - . . _ _ - _._ . _ ._. _ __.__ _ _ __.. _ _ . .. __ _

.

.

[

3 --

,

had corrected the problem..and the restriction was remove Review of Desian Soecifications-for DOIs F The inspector reviewed the electrical, environmental, and mechanical.

i;

'

specifications discussed in the DOI vendor manual and found them to be:

consisteht with the performance requirements outlined in Duke's-L Specification No. CNS-1338,00-00 0001. However, the inspector o questioned whether the D01s were designed to operate continuously at the L maximum abnormal temperature limit of 140 degrees Fahrenheit as

! described in the manual, or were they designad to operate for only 8 i hours as discussed in the specification? The specification required the - :

units to be designed to satisfactorily operate at an abnormal i temperature of 140 degrees Fahrenheit for 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />. The technical manual I indicated the D01s could o)erate in a temperature band oT 32 degrees ,

Fahrenheit to-140 degrees rahrenheit with no limits being placed on 001 h operation at the maximum abnormal temperature of 140 degrees Fahrenheit, i

Engineering was not aware of the discrepancy and did not know if the

-

r temperature band described in the technical manual was for continuous

'

o)eration of the D01s at 140 degrees Fahrenheit. The inspector noted tlat newly manufactured units received a 100 hour0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> burn-in test by the l vendor at 60 degrees Celsius (140 degrees ~ Fahrenheit) as part of the factory acceptance-test. Later the licensee learned from the vendor i that the D01s were qualified only to operate for 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> at 140 degrees

Fahrenheit and that the vendor manual would need to-be clarified to j: indicate that-this was not a continuous rating.

Walkdown of MSIV 001s i The inspector. accom)anied by engineering aersonnel, inspected both Unit

< 1 and 2 MSIV'00ls. Both the Unit 1 and 2 iSIV DOIs are located in a h common-room in the auxiliary building that was ventilated with outside s:

'

- air. but was not air conditioned. The Unit 1 DOIs were mounted in Panel 1SMTC1. and the Unit 2 MSIV D01s were mounted in Panel 2SMTC Both panels were several feet. apart. The ins)ector touched the housing of some of the energized DOI modules on bot 1 units and found them to be warm from self-heating effects as expected. -The licensee had evaluated the internal heat rise in the D01s and deterkned that it should be no more than 10 degrees Fahrenheit above ambient- temperature inside the panel. The electrical grounding of the panels was observed, and no noticeable. concerns were' identified. The internal wiring and general housekeeping inside the panels looked good. ' The licensee indicated to the. inspector that the Unit,2 MSIV DOI circuitry had been checked for power quality and ground problems: no problems were identifie Failure History Trends of E-Max Diaital 00tical Isolators The ins)ector found that there have been a total of three 001 failures in the Jnit- 2 MSIV control circuitry. These failures caused MSIV closure events on two different MSIVs. The first E-Max DOI failure-Enclosure 2

-___ _ _ _ _ __ ._ _ - _ __ ._ _- _

1 . i

'

!

!

4 occurred in February 199 The licensee, in conjunction with the vendor, determined that the root cause was a failure of capacitor C which had reached the end of its service life. The corrective actions taken at that time were to upgrade the C4 capacitor with a more reliable ca)acitor: replace the D01s with old capacitors in MSIV circuitry and

, ot1er identified critical control circuits; and implement a PM

to replace those D01s every 12 years based on the manufacturer'sprogram date

'

code. The Unit 2 MSIV D01 failures that occurred on July 26 and August i 17, 1997, involved the new replacement D01s with the new capacitor Overall, the inspector found that there had been 5 failures of the D01s

'.'

with new capaciters. The other three failures involved different model numbers (two failures of Model 175C180 and one failure of Model

.'

175C1R7t The root cause for one of the two failures of Model 175C180 was a 7 sure of resistor R3. The remaining two failures were

-

consi;ered random, and a root cause investigation had not been performed

by the licensee.

The inspectr was informed that there were ap roximately 5700 001s d

included in the Catawba design. Approximatel 4800 were in indication

circuits, aiid 900 were in control circuit his breakdown between control and indication was considered significant from the standpoint of

+ which isolators were included in the PH program. The D01s in indication

circuits were not included in the PM program for replacement and were

l identified by the licensee as "run to failure " Approximately 432 of

'

the 900 001s in control circuits were in the PM prograr the remaining 483 001s were also considered as "run to failure The licensee indicated that the failure thresholds or goals for 001

'

failures were 0.128 percent per quarter for control failures and 0.521 percent per quarter for indication fcilures. The licer.see also indicated that if the failure goals were exceeded, a corrective action plan or root cause analysis would be developed. The Failure Analysis and Trending System Reports for 1997 were reviewed for D01 centrol and indication failures. The trcnd report for the second quarter of 1997

'

provided the average failure rate for the )ast six quarters for the population of both control and indication )01s. The average quarterly

, failure rate for the D01s was 0.06 percent in control a)plications and 0.29 percent for D01s in indication applications. As t1ese failure rate; +ere below the establisned goal rates. Engineering assessed them

-

as Meg acceptabl Corrective Actions The short-term corrective actions taken and or planned by the licensee included testing the Unit 1 MSIV 00ls: developing a test procedure and conducting weekly testing of the Unit 1 and 2 MSIV D01s (32 total): and performing one time testing of 57 other 001s that were in critical control circuits for a total of 89 isolators to be tested. The licensee indicated that the testing would be completed within 3 to 4 week Enclosure 2

,

u-e m ec., , - . - , _.-r-r-m ----..-. .,y.,.-r .--m,----% -. - - 1.,,+, - - - - - , - , - - - - - , - r,

r

i

-.  ;

i l b 20 b Conclusions

. The licensee's planned and completed actions to determine the root cause !

-

for the failure of the E max digital optical icolation devices were '

-adequate. The licensee took conservative steps by replacin additional Unit 2 MSIV D01s that had the same manufacturer'g fivecode s date ,

as~the two that previously faile E2.2 Ice Condenser Door Ooerabili f i i

' a, insoection Scooe (37551)

On July 18, the McGuire Nuclear Station (MNS) made a 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 50.72 -

notification to the NRC when they determined that 10 of 48 ice condenser i lower inlet doors in Unit 2 would not open within the allowable TS torque limit. The apparent cause of the door failures was attributed to heaving of the ice condenser floor Nuclear and subsequent binding)of evaluatedthe the door Engineering personnel at the Catawba Station (CNS potential for a similar problem at CNS, The inspector reviewed drawings of the ice condenser, discussed the issue with licensee personnel and NRC inspectors at MNS, and reviewed work orders and ice condenser 't refrigeration system diagrams, b, Observations and Findinas

,

Several operational events were identified at HNS durin0 which ice may have melted and the resultin of the ice condenser floor, g froze, water and seeped intothe caused thefloor foamtoconcrete layer expard and i heave, In addition to the operational events, the licensee also -

discovered that undetected floor cooling system degradation occurred as

_

_ a result of instrument drift. -(See Inspection Report 50 369,370/97-16-for more details.)

i-

"

In response to the adverse condition of the Unit 2 ice condenser at MNS, Catawba Engineering personnel evaluated the potential for conditions  !

.that have caused or would have caused water to seep into the foam concrete layer of the ice condenser floor. A Loss Of Offsite Power (LOOP) event occurred on Unit 2 in February 1996. A 'iafety Injection during +he event caused the pressurizer PORVs to lift, and the rupture disk on the pressurizer relief tank (PRT) eventually ruptured, releasing i steam into containment. The ice condenser system engineer indicated  ;

that he had obtained temperatures of the ice condenser floor after the PRT had ruptured during the LOOP event, and that floor temperature had not increased above freezing, To the licensee's knowledge, no other operational events have occurred that could have caused the damage incurred at.HN The system engineer indicated that non licensed operators verify flow through the floor cooling-loop on a weekly basis during containment building rounds. The inspector verified that this was true. A flow Enclosure 2 ,

__

_____ _ ___ ______ _ _ _ -__ _ _ _ _ _ _ _ _

_ _ -

.

,

'

gauge indicates flow rate from the floor cooling coil discharge header of the ice condenser refrigeration system. The licensee also initiated work orders (W0s) to provide for visual inspection of the Unit 1 and Unit 2 ice condenser floors at the next available opportunity. A visual inspection of the Unit 2 floor was performed during the forced outage following the MSIV failure and manual react:r trip on August 1 .icensee personnel inspected the r rete floor under the lower inlet door hinges and around the clevis located at the ends of the turning veins. No cracks were found. The inspection included verification that

, a gap existed between the lower inlet doors 1nd the flashing just above

. the floor. No discrepancies were identified other than a need to

replace caulking around some concrete joints. The inspection results were documented in the task completion comments for WO 97064768 0 Inspection of the Unit 1 ice condenser floor is on the forced outage list to be performed under WO 97064769 01.

4 Conclusions The inspector concluded that communication between HNS and CNS regarding

, the degraded condition of the ice condenser floor was proma Based on the licensee's inspection of Unit 2 and review thus far, tais issue does

not appear to be a problem at Catawba, inspection of the Unit 1 ice

>

condenser has been placed on the licensee's forced outage lis E2.3 Identification of Aluminum in Containment in Excess of Assumed Volume

Insnection Scone A concern was identified at McGuire Nuclear Station (MNS) when the

licensee discovered that the square footage surface area of aluminum insidecontainmentwashigherthanassumedintheUFSAR. Engineering

<

personnel in the licensee s General Office evaluated the aaplicability of the concern to Catawba Nuclear Station and documented t1e results of the evaluation in PIP 0 C97-2602. The inspector discussed the issue with Engineering personnel and reviewed their UFSAR. Design Basis Documentation, and PI Observations and Findinas During a review of the Containment Purge Ventilation (VP) System, which is a non-nuclear safety related system at MNS. engineering personnel discovered that the High Efficiency Particulate Air (HEPA) filters and prefilters used in tb lower containment filtration units contained aluminum separators, which provide structural support to the filters, The engineers calculated the total surface area of aluminum associated with these units (13.186 square feet) and determined that it exceeded t

the surface area assumed in the UFSAR (1.500 square feet). During certain design basis accidents, the aluminum is assumed to interact with the acidic reactor coolant system water, which can cause corrosion of aluminum and generate hydrogen, a combustible ga In sufficient Enclosure 2

_

- , - - - - - - - - - - , . - - . - - . - - - - - . - - - - - - . - . - . , , - - . - - - - - . - - . - - - - - - - - - - - - - . - - . - - . - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - . - - - - - - - - - - - - - - - , - - - - - - - - - . - - - - - - - - - - . - - - - - - - - . - - - - - - - , . - - - - -

'

.

.

4 concentrations, hydrogen can deflagrate (a challenge to equipment) or detonate (a challenge to containment integrity). The MNS finding was immediately shared with CNS engineering personnel, who evaluated the potential for applicability to CNS and initiated PIP 0 C97-2602 to j document their assessment.

'

Catawba's Containment Ventilation (VV) System has two Containment

'

Auxiliary Carbon Filter Units (CACFUs). Each CACFU has six HEPA filters with aluminum se)arators. comprising 8.240 square feet of aluminum surface area. T1e FSAR limit for aluminum was 2.000 square feet of surface area. An engineering group at the licensee's cor) orate office i re evaluated the original calculation used to determine t1e allowable amount of aluminum in containment and the associated generation of hydrogen, ar.d identified assumptions that were overly conservative. The a

licensee performed a calculation based on more reasonable assumptions and determined that the allowable aluminum in containment had increased i from 2.000 to 10.000 square feet. The CACFUs contributed 8.240 square i feet of aluminum, and other sources contributed approximately 800 square feet of aluminum. The licensee concluded that present levels of

'

aluminum were within the revised allowable limits.

The inspector questioned the licensee why the aluminum in the CACFUs had not been recognized until the MNS finding was identified. Procurement

, Engineering personnel indicated that the original HEPA filters on the CACFUs did not contain aluminum separators. The original filter met the

'

'

requirements of Specification CNS 1211.00 00 0003. Paragraph 5.5 which reads " Separators, if used, shall be 304 stainless stee Since 1989, a different HEPA filter (Stock Code 85212) with aluminum separators has

'

been used in accordance # th Specification CNS 1211.00 00 0011, which 3 applies to nuclear safety related filters outside of containmen The 85212 filter had not been evaluated for the VV System CACFUs. The inspector inquired how a filter that is not intended nor appropriate for

the CACFUs could be installed: thereby raising the concern that controls

'

for ensuring appropriate materials are used in containment applications may not be effective or adhered to. To address the inspector's concern, the licensee initiated action to determine the root cause of the inappropriate material usage. Pending the completion of the licensee's a evaluation, this issue is characterized as Unresolved item 413.414/97-4 11 04: Use of Aluminum HEPA Filter Separators inside Containment. The inspector will review the hydrogen generation calculations during followup inspection of this ite Conclusions The inspector concluded that the issue, which was identified at MNS. was communicated to CNS engineering personnel in a timely manner, and engineering support from the corporate office to determine the impact was responsive. A root cause evaluation to determine why inappropriate

filters were used in the CACFUs was initiated. Pending the completion i of a root cause evaluation, this issue is characterized as an unresolved l item.

I Enclosure 2

- -__ _ _ _ - _._____ ._ - __ . ,

_ _ _ _ _ _ _ - _ _ _ _ _ - _ _ - _ _ _ _ _ _ _ - _ - _ - _ _ - _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ - _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ - _ _ _ _ _ . .___

.

.

E8 Hiscellaneous Engineering Issues (92903)

E (Closed) Unresolved item (URI) 50 413.414/97-05 01: Non Conservative SG PORV lechnical Specification

,

This item was opened pending NRC approval of a license amendment to:

(1) require four SG PORVs operable per unit: and (2) allow for the use of manual o)erator action to mitigate a SG tube rupture. On April 29, 1997, the NRC issued amendments 159 and 151 for Units 1 and respectively, to require four (instead of three) SG PORVs to be operabl The use of local operation was credited in the event that remote operation is unavailabl IV. Plant Support R1 Radiological Protection and Chemistry (RP&C) Controls RI.1 Ammonia Concentration Levels in Closed Coolina Water Systems a. Insoection Stone (71750)

,

On July 21. station chemistry personnel determined that ammonia concentration levels for the several closed cooling water systems with copper-alloy heat exchanger tubes were in excess of the recommended limit of 0.5 parts per million (ppm). Corrective actions were taken to decrease the concentration of ammonia in the systems and determine the cause of the increase. The inspector discussed the issue with chemistry

-

personnel, attended a managemut update on the issue. and reviewed associated PIP 0 C97 252 b. Observations and Findinas

.

'

On July 21. station chemistry personnel determined that ammonia concentration levels for the Component Cooling Water (KC) System, the Control Area Chilled Water (YC) System, and the Emergency Diesel Generator Engine Cooling Water System (KD). were in excess of 0.5 pp thestationlimi Industry experience has indicated that stress-corrosion cracking of copper alloys is exacerbated by high concentrations of ammonia and dissolved oxygen. In 1995, the Institute for Nuclear Power Operations (INPO) provided inform 6 tion to the Nuclear Industry via a document titled ' Good Practice CY-708. Treating and Monitoring Closed Cooling Water Systems." The phenomenon also is discussed in an EPRI documen currently in draft form, titled "EPRI Closed Cooling Water Guidelines."

To minimize the potential for piping degradation, the Licensee's General Office imposed a 0.5 ppm limit for ammonia concentration in closed cooling water systems with copper alloy piping. The limit was based on information in "INPO Guideline for Chemistry at Nuclear Power Station ." Revision 1. 1991. On July 21 the licensee determin6d that the ammonia concentrations in the YC and KD systems were 1.3 ppm and opm respectively; ammonia concentration in the KC system was 0.96 pp Corrective actions were taken to feed and bleed the systems on a Enclosure 2

_- .

. - -

,. ~._ _ . . . _ . _.____ _.__.._._ ____ i  !

'

w .

I

$

Y' .

l 24  !

!  !

!

periodic basis in an effort to decrease the ammonia concentrations. The sampling frequency was increased from quarterly to weekly to monitor the

trend and verify that feed and bleed evolutions were effective in  !

i reducing amonia concentrations. The licensee generated PIP 0 C97 2527 i

.to document the issue and is pursuing six potential root causes of the ,

i ammonia level increase. Long term corrective actions will be determined once the root cause has been identified.

j Conclusions j The inspector concluded that the licensee's efforts to monitor ammonia j concentrations in closed cooling water systems were proactive in

,

minimizing the risk of stress corrosion cracking of copper alloy heat .

L exchanger tubes. Appropriate actions were taken to reduce elevated levels of ammonia in thew system '

j P8 Hiscellaneous EP Issues (92904)

I P8.1 -(Closed) Deviation (DEV) 50-413.414/95 18-02: Control Room Habitability j Discrepancies +

i

The deviation cited several instances where the licensee did not comply

! with UFSAR commitments of Regulatory Guide 1.95. Protection of Nuclear i Plant Operators Against an Accidental Chlorine Release (Revision 1, '

[ 1/77). The. inspector reviewed the licensee's response dated September l'

28. 1995, which addressed corrective actions to reperform the analysis of the maximum credible onsite chlorine release based on existing ,

! quantities of chlorine stored onsit Based on the result, the licensee

'

determined that the control room breathing air apparatus was no longer *

required to meet Regulatory Guide 1.95 requirements and revised the

, UFSAR accordingly (refer to November 30. 1995 update). The licensee ,

j continues to' maintain the breathing apparatus o>erational and has

provided additional masks in various sizes in tie main control room.

!- The licensee has also added additional breathing air tanks and masks ,

adjacent to the control room. The inspector considered the licensee's

_ actions appropriate. - ,

3: S1 Conduct of Security and Safeguards Activities (71750)

L S1,1 Loss of Access to the Unit 2 Auxiliary Feedwater Pumn Room and Auxiliarv j Shutdown Panels L a-, Insnection Scone  :

, ,

On August 11. operations personnel were notified that the door to the i Unit 2 auxiliary feedwater (AFW) pump room, which also contains the A 4 and B train auxiliary shutdown panels, was inaccessible. Catawba ,

, Nuclear Site Directive (NSD) 3.1.4. Operational Response to Acts

. Directed Aga1nst Plant Equi > ment. Revision 0. was implemented to

'

determine if tampering of tie door locks was involved. The inspector -

i discussed the issue with compliance and security personnel and reviewed

[

NSD 3.1.4 and associated PIP 2-C97-2624.

' '

Enclosure 2 i

_,-._ . , _ _ _ . _ , _.,_,_._,2.______, _ _ _ _ _ _.,_.,_, - _ _ _ .

_ _ _ _ _ _ _ . _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ - _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

.

.

l 25 l b. Observations and Findinas On August 11. operations personnel were notified that the door to the Unit 2 AFW pump room, which also contains the auxiliary shutdown panels, was inaccessible. Security and Maintenance >ersonnel immediately responded to the door to attempt to access tie room. They were unable to unlock the door using a key and removed the key core to open the <

door. Within 25 minutes the door was opened successfull Maintenance personnel determined that the door had been locked using the push buttons on the door edge. Catawba NSD 3.1.4 was implemented to determine if tempering of the door locks was involved and to ensure that no other doors were adversely affected. Operations personnel did not identify problems with any other Unit 1 or Unit 2 vital area doors.

! The licensee determined that the last person to card out of the AFW pump room was a maintenance technician. The technician was located and

interviewed to determine what may have caused the door to loc According to the technician, he was carrying large loads out of the AFW

'

pump room as he was exiting the area. Security aersonnel speculated

! that the door lock push-button was impacted by tie load and l inadvertently displaced into the locked position. Tampering was ruled l

out as a cause of the locked access, i The inspector questioned the impact of fettered access to the AFW pumps and the auxiliary shutdown panels to determine if timeliness i requirements for manual actions were adversely affected. To determine if assumptions regarding time requirements for responding to these areas

were compromised, the licensee conducted a L...puter search through the

! UFSAR contacted probabilistic risk assessment experts in the l licensees's Corporate Office, and queried emergency and abnormal operating procedures. A requirement to access the rooms within a

'

specified period of time could not be identified. The licensee also indicated that, had there been an urgent need to access the room in response to an event, security and maintenance personnel would have been able to remove the lock core within 10 to 15 minutes. The inspector considered this reasonable.-and could not identify independent of the licensee's review, a time requirement for responding to the room during event respons Because the AFW pump room door was susceptible to inadvertent locking, the licensee proposed a corrective action in PIP 2 C97-2624 to prevent recurrence. Security personnel evaluated the use of other types of locking mechanisms that do not feature push buttons and are less

'

! susceptible to inadvertent locking. Doors to vital areas were evaluated to deterniine which ones had locking mechanisms featuring push-button Work orders (W0s) were generated to replace the locks featuring push-buttons with those requiring key access. The inspector reviewed work

'

orders 97033503 and 97035509 to verify that locks had been changed to key-controlled locking mechanisms for doors associated with the EDG AFW pump rooms, containment, the containment annulus, and doghouses i (containing secondary isolation and relief valves). The inspector l

visually inspected a sample of doors identified for lock replacement to

,

l Enclosure 2

'

.-- -

.. - -. -- , . . _

_ - . _- --

________-________-____ _ _ __ _ _

.

f

verify that key locks had been installed: no discrepancies were identi fie c. [pnclusions The inspector concluded that the licensee's implementation of NSD 3. was a timely and appropriate response to address the potential threat of tamperin The root cause evaluation was adequate, and the conclusion that tampering was not involved in the incident was well reasone Actions taken to prevent recurrence were appropriat V. Manaaement Meetinas X1 Exit Heeting Summary The inspectors > resented the inspection results to members of licensee management at t1e conclusion of the inspection on September 4,199 The licensee acknowledged the findings presented. No proprietary information was identifie Enclosure 2

. ._ _ .

.

PARTIAL LIST OF PERSONS CONTACTED ,

Licensee Birch M., Safety Assurance Manager Boyle M. Radiation Protection Manager Glover. R.. Operations Superintendent

,

'

forbes. J., Engineering Manager Jones. R., Station Manager

.11tholson, L., Compliance Specialist Kitlan. M., Regulatory Compliance Manager Deterson. G., Catawba Site Vice-President Propst, k.. Chemistry Manager

&

.

$

.

Enclosure 2

- . _ . _ .- _ _ . _ . _ . . _ _

_ _ . _ _ , . . _ . . . _ _ . _ . . _

. . _ . - - _ _

i

.

28 ,

IPSPECTION PROCEDURES USED IP 37550: Engineering IP 37551: Onsite Engineering IP 62700: Maintenance Implementation IP 61726: Surveillance IP 61700: Surveillance Procedures and Records IP 62707: Maintenance Observation IP 71707: Plant Operations IP 71750: Plant Support Activities IP 92901: Followup - Operations IP 92902: Followup - Maintenance IP 92903: Followp Engineering IP 92904: Followup Plant Support ITEMS OPENED, CLOSED, AND DISCUSSED Doened 50 413/97-11-01 NCV Failure to follow Control Rod Movement Testing Procedure (Section 01.2)

50 414/97-11-02 VIO Failure to Correctly Develop a D01 Replacement Plan and Perform D01 Replacement Activities in Accordarice With the Controlling Procedure (Section 01.3)

50-414/97-11-03 URI Failure of 2B EDG Turbocharger Mounting Bolt (Section M1,3)

50 413,414/97-11 04 URI Use of Aluminum HEPA Filter Separators Inside Containment (Section E2.3)

50 413.414/97-11-05 VIO Failure to Identify Conditions Adverse to Quality and Take Corrective Actions During Review in Accordance with GL 96 01 (Section M7.1)

'

Closed 50-413,414/95-18-02 DEV Control Room Habitability Discrepancies (Section P8.1)

50 413,414/97-05-01 URI Non Conservative SG PORV Technical Specification (Section E8,1)

50-413,414/95 20-01 VIO Inadequate Incorporation of Vendor Information Into Diesel Generator Maintenance Instructions (Section M8.1)

Enclosure 2

__

- - -- - __ _ . _ . . - . .

29 50 413/95 16 01 VIO Inadequate Operating Procedure for Re-establishing Normal letdown Resulting in a Water Hammer (Section 08.1)

50 413,414/94 17 04 URI Overpressure Protection for RN Pumps (Section M8.2)

4 LIST OF ACRONYMS USED AFW -

Auxiliary Feedwater CACFU - Containment Auxiliary Carbon Filter Unit CFR -

Code of Federal Regulations DEV -

Deviation D01 - Digital Optical Isolator DPC -

Duke Power Company EDG -

Emergency Diesel Generator

EP -

Environmental Protection EPRI -

Electric Power Research Institute ESF -

Engineered Safety Feature FlP -

Failure Investigation Process FSAR -

Final Safety Analysis Report HEPA -

High Efficiency Particulate Air IAE -

Instrument and Electrical IFI -

Inspector Followup Item INPO -

Institute for Nuclear Power Operations IR -

Inspection Report

KC -

Component Cooling Water KD -

Emergency Diesel Generator Engine Cooling Water System

. LER -

Licensee Event Report MNS -

McGuire Nuclear Station MSIV -

Main Steam Isolation Valve NCV -

Non Cited Violation NSD -

Nuclear Site Directive NSW -

Nuclear Service Water 0ATC - Operator at the Controls PIP -

Problem Investigation Process PORC -

Plant Oversight Review Committee PORV -

Power Operated Relief Valve um -

aarts per million

)RT -

2ressurizer Relief Tank 0A -

Quality Assurance RCS -

Reactor Coolant System RG - Regulatory Guide RP&C -

Radiological Protection and Chemistry RPS -

Reactor Protection System SG -

Steam Generator SR0 - Senior Reactor Operator TS -

Technical Specifications UFSAR - Updated Final Safety Analysis Report URI -

Unresolved item V10 -

Violation VV -

Containment Ventilation Enclosure 2

. _ - - .

. _ , - -- ,_. .-

i

,

i 30 .

WO -

Work Order YC -

Control Area Chilled Water li l

!

>

>

S t

-

,

Enclosure 2 i

. - . .. , . . . _ , . -

- . . . . - . -. . , , - -. --.