ML20199H516

Insp Rept 50-333/97-11 on 971215-19 & 980107.Noncited Violations Identified.Major Areas Inspected:Review Issue Associated W/Upgrade Replacement of Original General Electric TIP W/Siemens Digital Sys

ML20199H516
Person / Time
Site:	FitzPatrick
Issue date:	01/28/1998
From:	Ruland W NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I)
To:
Shared Package
ML20199H465	List: ML20199H462 ML20199H516
References
50-333-97-11, NUDOCS 9802050070
Download: ML20199H516 (9)

See also: IR 05000333/1997011

Text

.

I

.

U.S. NUCLEAR REGl.lLATORY COMMISSION

REGION I

Docket No:

50 333

License No:

DPR 59

Report No:

50 333/97 11

Licensee:

New York Power Authority

Facility:

James A. FitzPatrick Nuclear Power Plant

Location:

Post Office Box 41

Scriba, New York 13093

Dates:

December 1519,1997 and January 7,1998

Inspector:

Leonard Cheung, Senior Reactor Engineer

Gordon Hunegs, Sr. Resident inspector

Approved by:

William Rutand, Chief

Electrical Engineering Branch

Division of Reactor Safety

_-

-

__

_

=

9802050070 980120

'

POR

ADOCK 05000333

G

PDR

m

-

l

.

.

'

EXECUTIVE SUMMARY

James A. FitzPatrick Nuclear Power Plant

NRC Inspection Report No. 50 333/97-11

This inspection v.as conducted to review the issue associated with the upgrade

(replacement) of the original General Electric traversing incore probe (TIP) with a Siemens

digital system. This report covered the result of a one week onsite inspection by one

region based inspector and one resident inspector.

Enoineerino

The licensee's replacement in 1991 of the original GE TIP system with a Selmens

e

digital TIP system involved an USQ and was in violation of 10 CFR 50.59 (a) (1)

and (2). (E8.1)

The three primary containment isolation ball valves in the TIP system were

inoperable from 1991 to 1996 in that these valves could be opened (by an

unintended design function) when containment isolation was required. This was not

in ecmpliance with FitzPatrick Technical Specifications Section 3.7.D.1. (E8.1)

The licensee's disposition of a Deviation / Event Report for replacement of a

nonsafety-related 24 Vdc power supply in the TIP system was weak. (E8.1)

One unresolved item is closed. (E8.1)

.

Ooeratigna

Operators showed a weakness in recognizing a pl6nt condition that required

technical specification actions to be taken in response to the TIP ball valves failing

opan. (04.1)

Plant was placed in cold shutdown in accordance with technical specification

requirements. (04.1)

i

_ = _

=_

11

_ _ _ _ _ . _ _ _ _

- _ . _ , , _ _ -.__ _ __ _ . _ _ _ _ _ _. _ ___ _

!-

l

-

l

1

!

Reoort Details

Summary of Plant Status

,

This was a special engineering inspection to review the upgrade (replacementl of the

>

original General Electric traversing incore probe (TIP) with a Siemens digital system. During

the inspection, FitzPatrick was restarting from a forced outage to repair the safety relief

i

valves, and was at fu't power at the end of the inspection.

!

E8

Miscellaneous Engineering issue (92903)

'

4

I

E8.1

(Closed) Unresolved item (96 06 04): Traversina incore Probe (TIP) Svstem Ball

Valve Control Failure

,

,

a.

Insoection Scoos

,-

,

i

On September 16,1996, a human error caused a momentary loss of power to the

1

TIP control system and the activation cf the containment isolation signal. When the

.

power was restored, the licensee found that the three primary containment isolation

,

ball valves in the TIP system opened with a valid Group 2 containment isolation

- signal present. ' At that time, the TIPS were not beir.g inserted or withdrawn. The

-

licensee attributed the cause of the valve opening to a power supply failure in the

,

torque control unit of the TIP systen, The inspector reviewed pertinent licensee

documents to determine licensee's compliance with safety and regulatory .

'

requirements.

b.

Observations and Findinas

,

Backaround

in February 1991, the licensee replaced the originti General Flectric (GE) TIP system'

with a Siemens digital system (Modification F188-253). The licensee stated that-

the Siemens system wa.; the only one installed in the United States, therefore, there

r

would be no generic concern for this issue.

,

1The original GE TIP system consisted of three subsystems; each consisted of motor

= control and drive mechanism, and a guide tube that penetrated the primary

,

containment, and an indexer that was inside the primary containment. _ Each guide

i;

tube (3/8" outside diameter) had two primary containment isolation valves (one ball

valve'and one shear valve) located outside the containment wall. The indexer vos

inside the primary containment. The GE design criteria for the containment isolation

as indicated in Section 5.3.2 of GE document NEDC-22253, "BWR Owners' Group

Evaluation of Cuntainment Isolation Concerns," dated October 1982 states that:

-

,

'

"Under normal verating conditions, the TIP system guide tubes do not

>

4>

communicate with the containment atmosphere because purge air supplied

to the box surrounding the indexing mechanism eff sctively precludes such

"

,

communication. However, following a LOCA (loss of coolant accident) or

containment pressurizing event, a check valve on the box will open, resulting

,

In direct communication between the containment atmosphere and the TIP

.

guide tubes. Consequently, GDC (General Design Criterion) 56 is the

applicabie NRC requirement for TlP system isolation design."

,

d

.-

.

.

- . - .

.

.-.-.. . - - - - - - -

,

-

. - - . - ,

- . . -

_

.

.

2

The Siemens system Iso coniisted of three subsystems similar to the GE system,

except that the guide tube size was 10 mm instead of 3/8". The Siemens system

also contained three new indexers. As documented in Section 6.2 of a HtzPatrick

report (JAF RPT NMS 01511, Revision 1, dated December 6,1996), the provision

for t..e indexer design was as follows:

"Under normal operating conditions, the TIP system guide tubes do not

communicate with the containment atmosphere because the indexing

mechanism is sealed against containment atmosphere and effectively

precludes such communication. However, following a LOCA (loss of coolant

accident) or a guide tube break inside containment, direct communication

between the containment atmosphere and the TIP guide tubes is established.

Consequently, GDC 56 is the applicable NRC requirement for TIP system

isolation design."

The Siemens system used four programmable logic controllers (PLC); one in the

central control bait in the main control room, and one in each subsystem. This

system also used a common torque control unit, which had a 24 Vdc power supply

(115 Vac input).

Following the September 16,1996, event, the licensee found that the power supply

-

<

for the torque control unit had failed. The licensee's evaluation dt.termined that this

power supply f ailure had caused the TIP position indication to go to zero.

l

Subsequently, the computer software (as designed) opened all three primary

'

containment isolation ball valves on a TIP position indication of ze o, in spite of a

valid Group 2 containment isolation signal. The licensee also determined that all

PLCs functioned as designed. The inspector found that this design function had not

been analyzed by the licensee during the 1991 design change (TIP system

replacement) process.

The licensee had issued a Nuclear Safety Evaluation (JAF 8E-89146, Traversing

Incora Probe Upgrade, dated September 12,1990) to sup#t the 1991 TIP system

design change. The inspector reviewed this safety evaluation and found that it did

not nalyze the failure mode of the system components (nonsafety related) that

could cause a malfunction in the safety related containment isolation valves. The

safety evaluation also did not discuss tha hardware or the software (or firmware) of

_

the four PLCs, nor the verification and validation of the software.

Desian Basis and Technical Soecifications Reauiremants

The inspector reviewed FitzPatrick Final Safety Analysis Report (FSAR), and found

that the design basis for the three primary containment isolation ball valves

(0*/SOV-104A,B, Clin the TIP system was specified in the FSAR as follows:

1) Notes 16 to FSAR Table 7.3-1 for these three primary containment isolation

valves states, "During normal power operation the TIP ball valves are closed except

during LPRM (local power range monitor) calibration operations when the TIP is

inserted or withdrawn to measure reactor power at various locations within the

reactor core"; and 2) FSAR section 7.5.9.2 (Revirion 4, dated July 1993) states

that (for each TIP subsystem) a guide tube (pr! mary containment isolation) ball valve

opens only when the TIP is being inserted.

,

.

,

3

10 CFR 50.59(a)(1) states that a licensee may make changes in the facility as

described in the safety analys:s report, without prior NRC approval, unlesp the

proposed change involves an unreviewed safety question (USO). 10 CFR 50.59(a)(2) states that a proposed change is deemed to involve a USO if a

possibility for a malfunction of a ddlerent type than any evaluated previously in the

safety arialysis report may be created.

However, in 1991, the licensee made a change to the TIP system that in folved a

USO without prior NRC approval. Specifically, the licensee's replacement in 1991

of the original General Electric TIP syste.n with a Siemens digital TIP system

involved a USO in that the Siemens digital system contained an unintended function

which caused, on September 16,1996, a malfunction of a different type for the

three primary containment isolation ball valves in the TIP system. The three ball

valves opened in the presence of a valid Group 2 containment isolation signal and

the TIPS were not being inserted or withdrawn. The ball valves could not be closed

'

until the next day when a 24 Vdc power source was temporary connected to the

torque control unit of the TIP system. This incident occurred when the power

j

supply to the TIP system was lost and subsequently restored. The unintended

l

function in the Siemens digital system was not previously evaluated in the FSAR nor

in the safety evaluation (JAF SE 89146, dated September 12,1990) e ssociated

with TIP system replacement. This is a violac (eel 50 333/97-11-01)

The inspector reviewed Licensoe Event Report (LER) 96-010-01 and found that the

inadvertent opening of the three primary containment isolation ball valves with a

valid containment isolation signal present was discussed briefly in this LER.

However, the condition that FitzPatrick potentially operated the three ball valves

outside the design basis was not reported.

The inspector's review of FitzPatrick Technical Specifications (TS) indicated that

section 3.7.A.2 requires that primary containment integrity shall be maintained at all

times when the reactor is critical or the reactor water temperature is above 212'F,

and fuel is in the reactor vessel, except while performing low power physics tests at

atmospheric prescure at power levels not to exceed 5 MWt. TS section 3.7.D.1

requires that whenever primary containment integrity is required, containment

isolation valves shall be operable. TS section 4.7.D identified the containment

isolation valves as those listed in FSAR section 7.3, which included the three

primary containment isolation ball valves (07SOV 104A,B,C)ln the TIP system.

However, from 1991 to September 1996, when primary containment integrity was

required to be maintained, the three primary containment isolation ball valves in the

TIP system were inoperable, in that these valves could be opened by an unintenced

design function in the TIP system in the presence of a valid containment isolation

signal. The inspector determined that the 1991 replacement of the TIP system also

violated FitzPatrick TS as discussed above. (eel 50-333/97 11-02)

_ _ . _ _ . _ _ _ . . _ _ _ _ _ _ _ . _ . _ . _ _ . _ _ . . _

_ _ _ _ . _ _

.

a

. - -

i

4

Licensee's Corrective Actions :

[

l

Following the September 16,1996, event, the licensee made a change to the

'

control of the three ball valves in the TIP system. The hcensee placed the circuit

breakers that control power to these valves into normally "open" position (except

during TIP operation). The licensee also completed a revised version of Nuclear

Safety Evaluation JAF SE-89146 on December 6,1996, and revised the affected -

'

surveillance procedures to reflect the change. Implementing the above change'

would preclude the ball valves from being opened by errant digital signals. The

inspector's review of these documents found them generally acceptable. However,

the revised safety evaluation still did not discuss the " validation and verification" of

- the PLC software, and did not recognize that the 1991 TIP system replacement

involved an USO.

The licensee also completed an evaluation (Report No. JAF RPT-NMS 02511,

_

Assessment of the TIP System Containment Isoletion Design, Revision 1) on

December 6,1996. Thic document referred to a FitzPatrick report (JAF RPT-NMS-

02511, Revision 1, dated December 5,1996), which again refer.to another

FitzPatrick document, JAF CALC RAD-00057, entitled " Potential Radiological

Consequences of the TIP System Failure to isolate Under Normal and Accident

Conditions." The results of these analyses Indicated that the maximum bounding

radiological release was within the 10 CFR 100 limit.-

The other three associated containment isolation valves (explosive operated shear

valves) are normally open and do not automatically clo:e when containment

isolation signal is initiated. They are to be manually closed by the operators.

During this inspection, thh licen'see still had not yet determined the cause of the

power supply failure in the torque control unit. Following the September 16,-1996,

event, the licensee ordered two power supplies (all nensafety related); one for

replacement of the failed unit and one for spare, from the vendor. The licensee

re<:eived the first unit about four days later, and_without understanding the

installation instructions, which was in German, the licensee promptly installed the

unit. The licensee failed 4 ) recognize that this power supply was a dual input

voltage (115/230 Vac) mt and that the shipped unit was configured for 230 Vac

installation. On December 4,1996,the licensee measured the output voltage of the

spare power supply and found that the maximum output voltage that could be

adjusted was 20.16 Vde, much lower than the specified output of 24 Vdc. The

.. licenses stated that the TIP system was still operable even though the torque

. control unit power supply was configured incorrectly.1The inspector's review of

Siemens specifications confirmed the licensee's statement, indicating the acceptable

input voltage to the torque control unit ranged from 13 - 33 Vdc.

During a November 14,1997, telephone call, to the vendor, the licensee was

informed that for 115 Vac input voltage, jumper.BR.1 must be installed on the

output of the full wave bridge in the power supply unit, as indicated in the Germen

installation instruction. Subsequently, the licensee issued a Deviation / Event Report

(DER Nc. 97-1558)to address this issue. The inspector eviewed this DER, which

was closed on November 26,1997, and found the disposition to be week, even for

a nonsafety-related item, whose failure had already caused an inadverte- opening

.-

1

,

5

of the containment isolation valves. For the corrective actions, the resolution was:

WR 96 06511-05had been initiated to replace existing power supply with a

correctly configured power supply. There were no actions taken to ensure that

personnel performing installation must read and understand installation instructions

before starting installation. During the December 19,1997, exit meeting, the

licensee management stated that additional follow up was needed for this DER,

c.

Conclusion

The inspector concluded that the licensee's replacement in 1991 of the original GE

TIP system with a Selmens digital TIP system involved an USQ and was in violation

of 10 CFR 50.59(a)(1) and (2). The inspector also concluded that the three primary

containment isolation ball valves in the TIP system were inoperable from 1991 to

1996 in that these valves could be opened (by an unintended design function) when

containment isolation was required. This constituted a vlotation of FitzPatrick

Technical Specifications Section 3.7 D.1.

The licensee's disposition of DER 971658 for replacement of a nonsafety related

24 Vdc pwoer supply in the TIP system was weak.

The original unresolved item is closed.

04

Operator Knowledge and Performance

04.1 Operator Actions Associated With Spurious Opening of Traverse incore Probe (TIP)

System Valves During September 16,1996 Event

a.

Insoection Scone

The inspector reviewed post trip logs, operator logs, work requests and deficiency

and event reports associated with the TIP system. Additionally, discussions were

held with operation department staff members,

b.

Observations and Finoinas

The following describes the sequence of events associated with the TIP ball valves:

On September 16,1996, at 1:04 p.m. the TIP ball valves received a group 11

Isolation signal due to low reactor pressure vessel water level and initia!!y closed.

Operators verified and logged that all Group ll containment isclation valves were

closed. At 1:35 p.m., the TIP ball valves inadvertently opened following

energization of a electrical bus. At approximately 2:00 p.m., operators recognized

.

that the TIP ball valves were open. Operations shift management discussed the

need to isolate the TIP penetrations using the shear valven and decided that the

action was not necessary. Plant design provides for a shear valve that can cut

through the TIP cable and isolate the penetration if the ball valves are unable to

close because of a TIP cable fouling the valves. The decision to not close the shear

valve was based, in part, on plant conditions, which showed that the conditions in

containment did not indicate a need for isolation, and that the _TIP system had not

been in use prior to the event. However, the TIP ball valves were not declared

i

.

.

6

inoperable, nor was a limiting condition for operation (LCO) of Technical

Specificatinns Section 3.7.D.2, Primary Containment Isolation Valves, entered at

i

that time. Shift management requested assistance to get the TIP ball valves shut.

l

Reactor engineering was unable to shut the valves and at approximately 9:00 p.m.

l

the operations support center dispatched en emergency repair team which identified

a power sJpply problem in the TIP controllogic system. On September 17, at

6:00 a.m., the reactor was in cold shutdown. Subsequently, the TIP ball valves

were closed af ter a 24 Vdc power source was temporary connected to the torque

l

control unit of the TIP system. At 6:00 p.m., the TIP ball valves were declared

inoperable and the LCO was entered.

Technical Specifications Section 3.7.D.2 requires that, with one or more of the

containment isolation valves inoperable, actions be taken within 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> to restore

'

the valvo, or isolate the oenetration, if this can not be met, the reactor shall be in

cold shutd wn within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. Since the shear valves were not closed witnin 4

hours then the plant was required to proceed to cold shutdown.

c.

Conclusions

Operators showed weakness in recognizing a plant condition that required technical

cpecification actions be taken in response to the TIP ball valves fai'ing open. The

reactor was placed in cold shutdown within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> while operators made

attempts to shut the TIP ball valves. Although operators did not declare the valves

inoperable upon discovering they were open and could not be shut, nor did they use

the shear valves to isolate the penetration, the plant was placed in cold shutdown in

accordance with technical specification requirements.

E9

FSAR Reviews

A recent discovery of a licensee operating their facility in a manner contrary to the

updated final safety analysis (UFSAR) description highlighted the need for a special,

focused review that compares plant practices, procedures and/or parameters to the

UFSAR descriptions.

While performing the inspections discum:.a in thic report, the inspectors reviewed

the applicable portions of the UFSAR that related to the areas inspected, including

FSAR Table 7.3-1 and Section 7.5.9.2, that pertained to the design besis for the

three primary containment isolation ball valves in the TIP system. The inspector

verified that other reviewed sections of the FSAR wording were consistent with the

observed p' ant practices, procedures and/or parameters.

l

.

.

..

.

,

.

.

.

..

.

_ _ _

_.

.

_

..

.

i

a

e

7

XI

Exit Meeting

i

The inspector met with the licensee personnel at the conclusion of the site inspection on

December 19,1997, and summarized the scope of the inspection and the inspection

+

results. No proprietary information was knowingly included in this report from those

documents. The licensee acknowledged the inspection find!ngs at that meeting.

The inspector amended the exit meeting in a January 7,1998, telephone call to

Messrs. A. Zarenba and M. Abramski of the New York Power Authority. The inspector

stated that the unintended function in the Siemens digital system had also caused the three

primary containment isolation ball valves in the TIP system inoperable from February 1991

to 1996 as discussed in Section E8.1.b of this report.

PARTIAL LIST OF PERSONS CONTACTED

M. Abramski, Licensing

W. Bennett, Design Engineer

W. Berzins, Manager of Communicatlan

M. Burnstein, l&C Lead Engineer

M. Colomb, Site Executive Officer

R. Converse, Technical Assessment Coordinator

S. Kohr, Supervisor, Mechanicol Design Engineering

D. Lindsey, Administration

D. Ruddy, Director, Design Engineering

R. Steingerwalt, Licensing Engineer

D. Toplay, Administration

D. Vandermark, Quality Assurance Manager

A. Zarenba, Licensing Manager

NaC

G. Hunegs, Senior Resident inspector

ITEMS OPENED AND CLOSED

Ooened

50-333/97 11-01

eel

USO for TIP system replacement

50 333/97 11-02

eel

TIP system containment isolation valves inoperable

Closed

50 333/96-06 04

URI

TIP system ball valve control failure

'

s

-,_

. - ,

.- _ - .

-