ML20211K507

Submits Concerns Re 990624 Event Involving Automatic Reactor Shutdown.More than 5 Failures Were Identified in Event Number 35857

ML20211K507
Person / Time
Site:	Nine Mile Point
Issue date:	07/06/1999
From:	Tom Gurdziel AFFILIATION NOT ASSIGNED
To:	Dicus G, The Chairman NRC COMMISSION (OCM)
Shared Package
ML20211K506	List: ML20211K503 ML20211K507
References
NUDOCS 9909080007
Download: ML20211K507 (3)

Text

..y }

y ..

p 9 Twin Orchard Drive Oswego, NY 13126 July 6,1999 Chairman Greta Joy Dicus US Nuclear Regulatory Commission One White Flint North 11555 Rockville Pike Rockville,-MD 20852-2738-

Dear Chairman Dieus:

I think that a well run nuclear powerhouse does not have 5 (or more) failures during one event. The purpose of this letter is to bring to yoar attention Event Number 35857, where I feel there are more than 5 failures. They are:

1. One Feedwater Flow Controller fails low.

2. Failure of operators to either substitute another Feedwater Flow Controller or reduce reactor power to where the mass of steam out of the reactor does not exceed the mass of feedwater going in. (This is two failures).

3; Failure to demonstrate control of the reactor by anticipating an automatic scram on low reactor water level and providing a manual scam first.

'4. One offsite power source (identified as Line #5), did not transfer. (This is probably two failures: failure to fast transfer automatically, and failure of an intentionally delayed automatic transfer that would allow for voltage decay.)

5. ~ Failure to provide alternate power to, and restore, the offgas system so that condenser vacuum was not lost to the level that main steam isolation would occur.

6. Use of an inoperable system (RCIC), to control reactor water level.

Well, that is more than five.

I am beginning to feel uncomfortable with this plant's operation, Compared with the

industry average, are two automatic scrams in two months good? Is loss of offsite

- power two times in two months good? Have neighboring plants reported any loss of offsite power this year? Is the failure e f RCIC two times in two months good?

I would feel better if you would tell me that the NRC is satisfied that, at Nine Mile Point 2:-

L All feedwater control does not receive power from just one UPS.

9909080007 990830 PDR ADOCK 05000220 0 PDR I: .

_ -- -- a

I ., g )

.1,  ; t- j

, .g i

2. Adequate preventative maintenance is now being performed on all UPSs. (This includes confidence that the presently installed UPSs have manuals and i

drawings that are' consistent with their manufacture, all "little" batteries are J being replaced on a schedule that is appropriate to any applied float voltage l and ambient panel temperature, the emergency diesel generators are the default power supply to all UPSs, and all UPS capacitors are replaced based on age  !

and temperature requirements. I

3. If there are any pieces of the feedwater system that lock up on loss of power, reactor operators and their supervisors are trained in how to reset them.

I

4. There are no "open" items associated with either IN 91-64, Supp #1, or the Public Meeting of October 18,1991.

]

)

i

5. Probable root causes associated with Event Number 35627 were determined before the plant went back on line.

6. When circuit breakers to offsite power are open, the resulting phase difference I is not more that that allowed by control permissives.

7.. Any instrumentation / control cabinets provided with heavy steel doors to reduce the possibility of electromagnetic interference are operated with their doors closed.

Thank >,

gg "

1 Thomas Gurdziel l

l

. .*e OFFICE OF THE SECRETARY.

CORRESPONDENCE CONTROL TICKET

. PAPER NUMBER: CRC-99-0621 LOGGING DATE: Jul 10 99 ACTION OFFICE: EDO AUTHOR: -THOMAS GURDZIEL AFFILIATION: NEW YORK ADDRESSEE: CHAIRMAN GRETA DICUS LETTER DATE: Jul. 6 99 FILE CODE: IDR 5 NMP

SUBJECT:

ALLEGATIONS RE NINE MILE POINT,

' ACTION: Direct. Reply ,

DISTRIBUTION: COMRS SPECIAL HANDLING: SECY TO ACK CONSTITUENT:

NOTES: OCM #3812 DATE DUE: Jul 23 99 SIGNATURE: . DATE SIGNED:

AFFILIATION:

I EDO --Gl9990345

a

f. pa ar c4 t r . UNITED STATES -

{ .g NUCLEAR REGULATORY COMMISSION 5 4j REGION 1 -

g-q g-g 475 ALLENDALE ROAD KING OF PRUSSIA, PENNSYLVANIA 19406 1415 e...+

August 30, 1999 Mr. Thomas Gurdziel 9 Twin Orchard Drive Oswego, NY 13126

Dear Mr. Gurdziel:

I am responding to your letter to Chairman Dicus, dated July 6,1993, in which you expressed '

concerns involving the June 24,1999, Nine Mile Point Unit 2 (Unit 2) automatic reactor shutdown. The NRC staff shares your concern for the proper operation of Unit 2 and we have pursued the answers to many of the same questions you raised in your July 6 letter. The NRC l resident inspectors were r.ugmented by other NRC Region I inspectors to evaluate the Unit 2 equipment issues and to assess the overall performance of the Unit 2 staff and management during and following this event. The inspectors' findings and conclusions are currently under review and will be documented in NRC Inspection Report No. 50-220 and 50-410/99-06, by mid-September 1999. A copy of this report will be forwarded to you, when available.

Prior to Unit 2 restart and return to electrical generation, the NRC staff reviewed and discussed with Niagara Mohawk Power Corporation (NMPC) the various equipment failures, safety systems responses, and operator actions associated with the June 24 automatic shutdown.

These discussions with NMPC were to ensure a common understanding of all of the issues and to convey our expectation that corrective actions be thorough and complete prior to unit restart.

In general, the NRC staff found that the Unit 2 operators responded adequately to the failures and challenges encountered during the June 24 automatic shutdown. The two automatic shutdowns (June 24 and April 24,1999) within two months are above the current industry

. average (slightly less than one scram per unit per year) and were caused by unrelated

. equipment failures or malfunctions. While it is true that both automatic shutdowns involved offsite power problems, there does not appear to be any significant correlation either between -

those two events or with the performance of other plants. The NRC staff is not aware of the neighboring nuclear plants of Nine Mile Point Unit 1, James A FitzPatrick, or Ginna being subject

~

to any loss of offsite power events since the beginning of the year. We have been monitoring' the Unit'2 reactor core isolation cooling (RCIC) system performance in recent months, as documented in inspection Reports 50-220 and 50-410/99-04 and 99-05. Copies of these two reports with our assessments of licensee performance and copies of the NMPC licensee event reports involving the two automatic shutdowns are enclosed for your information.

l I

I i

l- .tl ,

Thomas Gurdziel 2 l ~- The items for which you requested a specific response in your July 6 letter have also been i ' addressed in an enclosure to this letter. I hope that my staff and I have been responsive to your request. if you have any further detailed questions or concems regarding the performance of Unit 2, please contact Michele G. Evans, Chief, Branch 1, Division of Reactor Projects Region I, i (610-337-5224) responsible for direct inspection oversight of the Nine Mile Point facilities.

i Sincerely, I

I l

l

) )

bortJ. il r Regional Administrator Docket No. 50-410

)

Enclosures:

As stated f l.

I o

I l

1

)

1.-.

e

? s y Thomas Gurdziel - 3-Distribution w/encis:

, ' D. Holody, ORA L l - R. Crienpak, DRP -

l W. Lanning, DRS

.B. Holian, DRS l- M. Evans, DRP -

W. Cook, DRP

. M. Oprendsk, DRP .

!- G19990345 L , CRC-00-0821 OCM #3812 l PUBLIC-l Region i Public Document Room (w/ concurrences) .

Nuclear Safety information Center (NSIC)

Distribution w/encis: (VIA E-MAIL) -

W. Travers, OEDO M. Knapp, OEDO '

F. Miraglia, OEDO '

P. Norry, OEDO S. Shirley, EDO -

. M. Bridgers, EDO l S. Bums, OGC .

S. Collins, NRR:

R. Borchardt, OE E. Adensam, NRR D. Hood, NRR G. Hunegs, SRl, Nine Mile Point 1 & 2 M. Fudge, ORAL  ;

I i

i DOCUMENT NAME: G:\BRANCHi\GURDZIELRESPONSE.WPD  !

- To receive a copy of this document, indicate in the box: "C" = Copy without .

attachment / enclosure = "E" = Copy with attachment /enclosura "N" = No copy '

OFFICE - Rl/DRP l- NRR l Rl/DRP n l Rl/RA m il NAME MEvans @ DHood ptaf-m.) RCrienjak A/f J' HMiller Mld v--

DATE 08/4/99 080W99 08/#99 i. 08/fr99 '

L OFFICIAL RECORD COPY i

.e .-

'O >3 M C Uy

.kg UNITED STATES NUCLEAR REGULATORY COMMISSION

[%

5 j REGloN 1 ,

o, g 475 ALLENDALE ROAD t

g ,e KING OF PRUSSIA, PENNSYLVANIA ',9406-1415 June 21, 1999 Mr. John H. Mueller

. Chief Nuclear Officer Niagara Mohawk Power Corporation Nine Mile Point Nuclear Station Operations Building,2nd Floor P.O. Box 63 Lycoming, NY 13093

SUBJECT:

NRC INTEGRATED INSPECTION REPORT NOS. 50-220/99-04 AND 50-410/99-04

Dear Mr. Mueller:

This report transmits the findings of safety inspections conducted by NRC inspectors at the Nine Mile Point Nuclear Station, Units 1 and 2, from March 28, through May 8,1999. At the

~

conclusion of the inspection, the findings were discussed with members of your staff.

During the six-week inspection period covered by this report, operation of the Nine Mile Point Nuclear Station reflected an acceptable safety focus. The Unit 1 outage was well managed with appropriate emphasis on shutdown risk. At Unit 2, a few performance shortcomings in the areas of maintenance and engineering surfaced as a result of the automatic reactor shutdown which occurred on April 24. For example, maintenance on the reactor core isolation cooling system resulted in its failure to operate on demand and an inadequate design review of a modification to the uninterruptible power supply system contributed to it's failure. Additionally, we observed that your staff's troubleshooting and analysis of the Unit 2 equipment problems were not methodical or well coordinated. However, we noted that you and your staff recognized these performance shortcomings and were developing actions to improve.

Effective programs were maintained for radioactive material waste management and transportation of radioactive materials. Radiological controls for the Nine Mile Point Unit i refuel outage were effectively planned and implemented and were focused on jobs with elevated exposure estimates, high dose rates, and radiologically complex work.

Based on the results of this inspection, the NRC has determined that six Severity Level IV violations of NRC requirements occurred. These violations are being treated as Non-Cited Violations (NCVs), consistent with Appendix C of the Enforcement Policy. These NCVs are described in the subject inspection report. If you contest the violation or severity level of these NCVs, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with copies to the Regional Administrator, Region 1; and the Director, Office of Enforcement, Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Senior Resident inspector at the Nine Mile Point facility.

.pf6=/o/002 V ffl'

E ,

4 .

o John H. Mueller 2 In accordance with 10CFR2.790 of the NRC's " Rules of Practice," a copy of this letter and its enclosures will be placed in the NRC Public Document Room.

Sincerely,

' W he Michele G. Evans, Chief Projects Branch 1 Division of Reactor Projects Docket Nos. 50-220,50-410 l License Nos. DPR-83, NPF-69

Enclosure:

NRC Inspection Report Nos. 50-220/99-04 and 50-410/99-04 I cc w/ encl:

G. Wilson, Esquire M. Wettertiahn, Winston and Strawn J. Rettberg, New York State Electric and Gas Corporation P. Eddy, Electric Division, Department of Public Service, State of New York C. Donaldson, Esquire, Assistant Attomey General, New York Department of Law J. Vinquist, MATS, Inc.

F. Valentino, President, New York State Energy Research and Development Authority J. Spath, Program Director, New York State Energy Research and Development Authority i

S 9

I

,. 1 ..

., s.

U.S. NUCLEAR REGULATORY COMMISSION REGION I Docket / Report Nos.: '50-220/99-04' 50-410/99-04 License Nos.:.. DPR-63 l NPF-69 Licensee: Niagara Mohawk Power Corporation P. O. Box 63 Lycoming, NY 13093 l 1

Facility: Nine Mile Point, Units 1 and 2 f

Location: Scriba, New York Dates: March 28,1999 - May 8,1999 Inspectors: G. K. Hunegs, Senior Resident inspector R. A. Fernandes, Resident inspector R. A. Skokowski, Resident inspector F. J. Amer, Ill, Reactor Engineer A. L. Della Greca, Senior Reactor Engineer i K. S. Kolaczyk, Operations Engineer G. W. Morris, Se ior Reactor Engineer R. C. Ragland, Radiation Specialist Approved by: Michele G. Evans, Chief Projects Branch 1 Division of Reactor Projects l

t I-( Nyv767F5Es f&f . o

P ;f .

V.

L-

• EXECUTIVE

SUMMARY

.. Nine Mile Point Units 1 and 2 50-220/99-04 & 50 410/99-04 March 28,1999 - May 8,1999 l This inspection report included aspects of licensee operations, engineering, maintenance, and

( plant support. The report covered a six-week period of resident inspection. The results of an l occupational radiation exposure and radwaste management and transportation inspection from l - Aprii 7 - 16, an inservice inspection program review from April 19 - 23, and an enginwring I

inspection from April 12 - 16 were also included in this inspection report.

Operations l The April 24 Unit 2 automatic reactor shutdown from 100 percent power was characterized by l the licensee and NRC staffs as a risk significant transient. The cause was determined to be a

generator protection circuit relay failure which also resulted in a residual (slower) transfer to off-site power. The slow transfer caused large motor loads such as reactor feedwater pumps, l reactor recirculation pumps, and condensate booster pumps to trip. Operator performance with l respect to procedure use, communications, and control of plant equipment was good. Senior

! management oversight of scram recovery efforts was appropriate. Major equipment failures l- included the reactor core isolation cooling system and a partial loss of the uninterruptible power I

supply system. These equipment failures and other minor equipment problems did not significantly impact recovery efforts. (01.2)

The Unit 1 outage stiutdown risk program was wellimplemented. The communication of plant protected equipment and safety system status was good. (01.3)

Overall, NMPC's approach to identifying and resolving equipment performance problems L following the April 24 Unit 2 reactor scram was acceptable. Positive aspects of NMPC's post-scram evaluation process included the establishment of multi-discipline teams to review i equipment performance, the conduct of periodic status briefs, and the use of vendor services.

Senior management effectively challenged their staff's post-scram analysis which contributed to a more rigorous evaluation and the re-creation of the event using the plant simulator. However, a few performance shortcomings related to the scram evaluation process were apparent.

Although the overall process was thorough, equipment troubleshooting and failure analysis were not methodical. NMPC management recognized these shortcomings and was developing l methods to improve its staff's problem solving skills. (O7.1) '

l l

Between March 5 and March 12,1999, Unit 2 experienced two events where the automatic depressurization system nitrogen storage tanks had excessive leakage. NMPC failed to recognize that the leakage exceeded the allowed limit, and therefore, did not take the required  ;

limiting condition of operation actions. This was a non-cited violation of Unit 2 Technical Specification 3.5.1. (08.2) l l

ii

Executive Summary (cont'd)

Maintenance The Unit 1 fuel off load was well controlled. Communications between the operators on the refuel bridge, as well as between the refuel bridge and the control room were observed to be good. (M1.2)

The installation of the emergency core cooling system torus suction strainers was well controlled. The work environment was clean, organized and good foreign material exclusion controls were in place. (M1.3)

During the Unit 2 scram, the reactor core isolation cooling (RCIC) system failed to operate as i required and was manually tripped. This RCIC system failure was attributed to an inadequate maintenance procedure and the licensee's over-reliance on vendor support for a 1998 RCIC

{

j turbine trip throttle valve rebuild. The failure to ensure an adequate maintenance procedure was prepared and used to perform work on the RCIC system was a non-cited violation. Based on recent operating history, the RCIC system has exceeded its Maintenance Rule performance criteria. (M2.1)

On April 24, a Unit 2 generator protection circuit relay failed which caused a reactor scram.

NMPC effectively evaluated the cause and consequences of the relay failure and implemented acceptable corrective action. (M2.2)

Non-destructive examination personnel were qualified, and adhered to procedures while performing examinations. The core shroud and reactor vessel weld inspection plans were in accordance with the requisite NRC safety evaluation. Deficiencies identified during inspection activities were properly documented. A new surveillance program provided enhanced oversight of vendor activities. (M3.1) 1 Enoineerina During the Unit 2 reactor scram transient, one of the two reactor protection system uninterruptible power supplies (UPS) failed. Excessiva currents, which caused the inverter DC power supply fuse to blow, were the result of a UPS design deficiency involving a newly installed maintenance bypass switch. NMPC identified that this vendor supplied UPS design change received an inadequate engineering design review. (E1.1)

Unit i design changes that were reviewed, correctly addressed the concems for which the modifications and been developed. Typically, the analyses accurately described the purpose of the modification and the intended results; the calculation and safety evaluations satisfactorily supported the design changes; and the design change process was acceptably implemented.

(E1.2)

In the case of the Emergency Core Cooling System (ECCS) strainer modification, the technical scope of the design change was comprehensive, but the licensee's original review of an ECCS

. pump air ingestion calculation lacked thoroughness regarding a small break loss of coolant l

iii l

w  :.

a. ,

e w

s.

Executive Summary (cont'd)__

accident (LOCA) scenario and required a more detailed analysis and a revision of the supporting calculation. Additionally, the safety evaluation required revision to provide stronger bases for the conclusions contained therein regarding a large break LOCA. The licensee's

' review of air ingestion phenomona associated with the large break LOCA resulted in the L conservative decision to declare inoperable, in the future, any ECCS pump placed in operation for surveillance testing or torus cooling. (E2.1)

From March 20 to March 23,1999, Unit 1 operated with a maximum average planar heat generation rate (APLHGR) exceeding the limits specified by the technical specifications. This technical specification violation was non-cited. NMPC determined that the cause was the inadvertent processing of traverse in-core probe (TIP) data, due to inadequate computer system security on the 3D-Monicore system. Specifically, TIP data could be processed without authorization or operator knowledge from uncontrolled locations. Additionally, the oversight by .

l station personnel with regards to reactivity management and core performance monitoring was poor, in that this discrepancy was not recognized for three days. (E4.1) l The failure to conduct the required ASME Code inservice inspections of the reactor recirculation pump seal housing bolts and flange surfaces during the first and second ten-year inspection intervals was non-cited. (E8.1)

NMPC self-identified and promptly corrected a condition which could have adversely affected the ability of the unit to achieve safe-shutdown, involving the Unit 2 service water intake de-icing heater control circuits which were not protected against a control room fire. This violation of Ucense Condition 2.G was non-cited. (E8.3) l Plant Support Radioactive material / waste management and transportation programs were effectively implemented as evidenced by use of up-to-date regulations and facility licenses, appropriately '

trained personnel, proper procedural guidance and adequate maintenance of procedures, appropriate use of scaling factors to estimate isotopic content of radioactive material / waste ,

packages, and proper shipping records. (R1.1)

Radiological controls for the Unit i refuel outage were effectively planned and implemented and focused on jobs with elevated exposure estimates, high dose rates, and radiologically complex work. (R1.2)

Radiological posting practices for access to radiation areas, high radiation areas, and airbome radioactivity areas were effective as evidenced by well defined boundaries and clear radiological postings. Some or portunities to enhance informational postings on the refuel floor that required

" health physics not fication prior to entry" beneath the drywell dome and reactor head insulation were identified. (R1.2)

Contamination monitoring requirements for access to the Turbine Building 305' Green Area (clean area within the radiologically controlled area (RCA]) did not include an entire whole body IV

~.

Executive Summary (cont'd) frisk similar to the requirements for RCA exit. However, they wore adequee to minimize the risk I for the spread and ingestion of significant amounts of radioactite contamination based on use of detailed procedures, restrictions on personnel that could use fne facility, and close heaiih  !

physics oversight. (R1.2)

Effective high radiation area controls were implemented as evidenced by clear radiological postings, use of locked doors when required, use of " Alarming" dosimetry, use of radiation work permits (RWPs), use of remote door alarms, requirements for a minimum available exposure for access, and increased health physics oversight and monitoring for high radiation area entry.

(R1.2)

Material conditions were good and housekeeping practices were effective as evidenced by clear aisles and walkways, neatly stored tools and equipment, and painted floor and wall surfaces.

(R2.1) -

Self-assessments, audits, and the deficiency / event reporting system were effectively used to identify, evaluate, and resolve radiological control issues as evidenced by the conduct of muniple self-assessments and audits to satisfy the radiation protection program review requirements in 10CFR20.1101(c) and use of the DER system to implement appropriate corrective' actions and controls to prevent unplanned exposures. (R7.1)

One non-cited violation was identified associated with the failure to maintain access restrictions in the upper elevation of the drywell during movement of an irradiated core component on March 15,1997. (R7.1) l l

l l

V

.r: .

s

- TABLE OF CONTENTS page EXECUTIVE SUM MARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

' TABLE OF CONTENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Summary of Plant Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1. Ope rations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 01 Conduct of 0perations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

' 01.1 ~ General Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 01.2 Automatic Reactor Shutdown (Unit 2) . . . . . . . . . . . . . . . . . . . . . . . . . . 1 01.3 Outage Shutdown Safety (Unit 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 07 Quality Assurance in Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

-07.1 Assessment of Post-Scram Troubleshooting Efforts (Unit 2) . . . . . . . . 4 08 Miscellaneous Operations issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 08.1 (Closed) Licensee Event Report (LER) 50-410/98-26 . . . . . . . . . . . . .. 5 08.2 (Closed) LER 50410/99-03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 l l . M aintena nce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 M1 Conduct of Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 M1.1 General Comments (61726, 62707) . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 M1.2 Fuel Off-load Activities (Unit 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 M1.3 Installation of Core and Containment Spray Strainer Assemblies (Unit 1)

........................................................ 8 M2 Maintenance and Material Condition of Facilities and Equipment . . . . . . . . . . 9

~M2.1 Reactor Core Isolation Cooling (RCIC) System Failure During Reactor Scram Transient (Unit 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 M2.2 Generator Protection Relay Failure Resulted in Scram (Unit 2) . . . . . 10 M3 Maintenance Procedures and Documentation . . . . . . . . . . . . . . . . . . . . . . . . 11 M3.1 ' Inservice inspection (Unit 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 .

M8 Miscellaneous Maintenance issues . . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . 13 M8.1 (Closed) LER 50-410/99-02 . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . . . 13 j I

M8.2 (Closed) VIO 50-410/98-05-02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 M8.3 (Closed)VIO 50-220/98-02-05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Ill . Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 E1 Conduct of Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 E1.1 Uninterruptible Power Supply (UPS) Failure During Reactor Scram Transient (Unit 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 E2 Engineering Support of Facilities and Equipment . . . . . . . . . . . . . . . . . . . . . . 15 E2.1 Design Modifications (Unit 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 E4 Engineering Staff Knowledge and Performance . . . . . . . . . . . . . . . . . . . . . . . 17 E4.1 (Closed) LER 50-220/99-03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 E8 Miscellaneous Engineering issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1

1 vi l l

1 i

l

.. Table of Contents (cont'd)

E8.1. (Closed) LER 50-220/98-19 . . . . . . .. . . . . . . . . .. . . . . . . . . . . . . . . . 19 E8.2 (Closed) VIO 50-410/98-05-03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 j E8.3 (Closed) LER 50-410/99-04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 j E8.4 - (Closed) IFl 50-410/98-19-03: Leakage of contaminated water following a scram reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 E8.5 (Closed) Violation 50-220/98-18-02 . . . . . . . . . . . . . . . . . . . . . . . . . . 22 IV. Pient S upport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 R1 Radiological Protection & Chemistry Controls . . . . . . . . . . . . . . . . . . . . . . . . . 23 R1.1 Unit 1 and Unit 2 Solid Radioactive Waste Management and Transportation of Radioactive Materials . . . . . . . . . . . . . . . . . . . . . . . 23 R1.2 Refuel Outage Radiological Controls (Unit 1) . . . . . . . . . . . . . . . . . . . 24 R2 Status of RP&C Facilities and Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 R2.1 Radiological Housekeeping (Units 1 and 2) . . . . . . . . . . . . . . . . . . . . 27 R7. - Quality Assurance in RP&C Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 R7.1 Radiological Control Program Corrective Actions (Units 1 and 2) . . . 28 V. M anagement Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 X1 Exit Meeting Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 ATTACHMENTS Attachment 1- Partial List of NMPC Persons Contacted

-Inspection Procedures Used

- Items Opened, Closed, and Updated

- List of Acronyms Used P

vii .,

I

c

3. . ,

l. <-

ReDort Details  ;

Summarv of Plant Status ~

' Nine Mile Point Unit 1 (Unit 1) began the inspection period at 100 percent reactor power. On April 11, Unit 1 was shutdown to begin a scheduled refueling outage (RFO15) and remained shutdown through the end of the inspection period. Significant outage activities included inspection of core shroud vertical welds and reactor vessel longitudinal welds, and installation of new emergency core cooling system suction strainers.

Nine Mile Point Unit 2 (Unit 2) began the inspection period at 100 percent reactor power. Unit 2 automatically shutdown on April 24 due to the malfunction of an electrical relay associated with the generator protection circuit. Maintenance activities completed during the shutdown included:

reactor core isolation cooling system troubleshooting and repair; uninterruptible power supply system modifications; and recirculation system flow control valve (FCV) maintenance. Following the forced maintenance outage, Unit 2 was returned to service on May 4. Due to recirculation system flow control valve fluctuations, Unit 2 was placed in single-loop operation on May 6.

After FCV adjustments were made, Unit 2 was retumed to two-loop operation on May 9 and was retumed to 100 percent power on May 11.

l. Operations 01, Conduct of Operations 1' 01.1 GeneralComments (71707)

Using NRC Inspection Procedure 71707, the resident inspectors conducted frequent reviews of ongoing plant operations. The reviews included tours of accessible areas of both units, verification of engineered safeguards features (ESF) system operability, verification of adequate control room and shift staffing, verification that the units were operated in conformance with Technical Specifications (TSs), and verification that logs and records accurately identified equipment status or deficiencies. In general, the conduct of operations was professional and safety-conscious.

01.2 Automatic Reactor Shutdown (Unit 2) .

a. Inspection Scope f71707)

On April 24, at 4:19 a.m., Unit 2 experienced an automatic reactor shutdown (scram) from 100 percent power. The inspectors responded to the site and observed portions of the scram recovery process. The inspectors also reviewed the operator logs, post-scram review documentation, and the sequence of events. Additionally, the event was discussed with Unit 2 operations and management personnel.

' Topical heedings such as o1, MB, etc., are used in accordance with the NRC standardized reacto.r inspection report outline.

Individual reports are not expected to address all outline topics. The NRC inspection manual procedure or temporary instruction that was used as inspection guidance is listed for each applicable report section.

L . '.

2

b. Observations and Findinas The cause of the reactor shutdown was the failure of a relay in the generator protection circuitry (see section M2.2). The relay failure caused a turbine trip and subsequent automatic reactor shutdown. Because of the particular relay failure and design of the generator protection circuit, instead of a fast transfer of electrical loads to off-site power, a residual (slower) transfer occurred which caused all feedwater, condensate booster, and recirculation pumps to trip. All control rods fully inserted on the automatic shutdown and vessel level control was maintained by automatic initiation and injection of the high pressure core spray (HPCS) system. The reactor core isolation cooling (RCIC) system also initiated, but failed to come up to speed and was tripped by the control room operators (see section M2.1). All five turbine bypass valves opened to control reactor pressure below the main steam safety relief valve setpoint. Plant cool-down was commenced on natural circulation using the turbine bypass valves. By late evening on April 24, the plant was in cold shutdown.

Coincident with the reactor shutdown, there was a trip of the uninterruptible power supply (UPS) which provides a portion of the power to the reactor protection system (see section E1.1). The partialloss of UPS resulted in several primary containment isolation valve group isolmions. The partial loss of UPS had negligible impact on scram recovery efforts.

The Updated Safety Analysis Report (USAR) classifies this event as an incident of moderate frequency. Notwithstanding the equipment problems, plant response was in accordance with the USAR. Because of the residual transfer of power, and notable

]

1 equipment failures, the NRC staff performed an initiating event risk assessment. The NRC staff's assessment showed that the event was risk significant, in that, the total conditional core damage probability had increased and exceeded the accident sequence precursor threshold value used by the NRC staff for assessing significance. The licensee's risk assessment of this event was consistent with the NRC staff's assessment.

The inspectors evaluated operator performance with respect to emergency operating procedure use, emergency plan use, communications and control of the plant. Based on interviews of operators and operations management and review of operator logs and plant sequence of event information, the inspectors determined that operators responded appropriately to the event. Reactor vessel level and pressure were well controlled.

Subsequent re-creation of the event on the simulator also showed that operator response was appropriate. The inspectors noted that Niagara Mohawk Power Corporation (NMPC) senior managers responded to the site and provided appropriate

- oversight. Additional operators were made available to provide assistance, as necessary. The inspectors observed good communications and good procedure use by  ;

operators during the post-scram recovery period. With the exception of the RCIC and UPS system failures, other major equipment operated as designed. A few minor system discrepancies were appropriately documented in the corrective action program for follow- )

up evaluation and repair.'

i i

l

.2 -

3

c. Conclusions

' The April 24 Unit 2 automatic reactor shutdown from 100 percent power was characterized by the licensee and NRC staffs as a risk significant transient. The cause was determined to be a generator protection circuit relay failure which also resulted in a residual (slower) transfer to off-site power. The slow transfer caused large motor loads such as reactor feedwater pumps, reactor recirculation pumps, and condensate booster pumps to trip. Operator performance with respect to procedure use, communications, and control of plant equipment was good. Senior management oversight of scram recovery efforts was appropriate. Major equipment failures included the reactor core isolation cooling system and a partial loss of the uninterruptible power supply system.

These equipment failures and other minor equipment problems did not significantly impact recovery efforts.

01.3 Qutaae Shutdown Safety (Unit 1)

a. Inspection Scope (71707)

The inspectors reviewed the methods used by Unit 1 personnel to monitor shutdown safety as outlined in station procedures.

b. Observations and Findinas Procedure N1-ODG-11 " Shutdown Operations Protection' Guideline," is used to monitor plant status during shutdown conditions. NMPC uses an attachment to th9 procedure for tracking the status of plant equipment important to shutdown safety associated with decay heat removal, inventory control, electrical power availability, secondary containment, and reactivity control. The attachment is updated each shift by control room operators and is used to brief station personnel at various meetings throughout the day. The inspector observed several briefings and noted good communication from station personnel with respect to emphasizing safety system status and protected components. The inspector noted that visual aides were used on control panels and in equipment rooms to wam personnel of the protected status of safety significant equipment.

c. Conclusions The Unit 1 outage shutdown risk program was wellimplemented. The communication of plant protected equipment and safety system status was good. l l

1 l

l I

r .

d 4

07~ Quality Assurance in Operations 07.1 Assessment of Post-Scram Troubleshootina Efforts (Unit 2)

a. Inspection Scope (71707)

NMPC appointed a post-scram review team to investigate the cause of the Unit 2 scram (see section 01.2). The inspectors attended post-scram review team and site operations review committee (SORC) meetings, observed NMPC troubleshooting efforts and discussed the scram evaluation activities with several members of the Unit 2 management staff. Additionally, the inspectors reviewed the post-scram review

_ procedure. The inspectors assessed NMPC's overall performance in identifying the causes of the equipment deficiencies.

b. Observations and Findinas '

A post-scram review team was appointed to investigate the cause of the scram and to determine corrective action. The technical services department had the lead on '

identifying the causes and equipment performance issues and were assisted by design {

engineering. Several teams were formed to focus on individual equipment performance l problems including the RCIC, UPS, and generator protection relay failure. The teams I used Procedure N2-REP-6, " Post-Scram Review," which provided an overall approach to evaluate the causes of a reactor scram and to review plant equipment performance. The teams held periodic briefs and formal SORC reviews were conducted. Senior management challenged the preliminary and apparent causes of equipment malfunctions during the review process.

The inspector noted that the operators that were involved in the transient prepared critique sheets listing their recollection of the event and actions taken, but a formal method to evaluate operator performance was not evident. Station practice has been to rely on the self-evident nature of operator errors to determine if a more rigorous review is warranted. In the case of this transient, the operations manager determined that it would be beneficial to re-create the event on the simulator to more thoroughly evaluate operator performance and leam from the event. As a result, some minor simulator fidelity issues were identified and additional operator performance insights were gained.

The evaluation of the cause of the scram and equipment performance was particularly challenging because of the complexity of the transient. Additionally, data was not available from the transient analysis recorder following the event, since it had not been properly aligned to automatically trigger the recording of data because of an earlier operator error. NMPC documented this problem in deviation / event report (DER) 2 1260. Based on the inspectors' observations, NMPC's approach to the troubleshooting and analysis of the transient and equipment problems was not methodical. For example, the troubleshooting efforts to identify the cause of the RCIC trip were extensive and drawn out. The eventual identification that tolerances for the trip and throttle valves were incorrect was the result of additional trip throttle valve agitation, late in the troubleshooting process, rather than the conduct of a formal root cause analysis. NMPC

.' l * \

+

5 management attributed their staff's problem resolution weaknesses to the absence of necessary skills and training. The inspectors leamed that NMPC was developing methods and training to improve performance in this area, including increased formal root cause analysis training.

c. Conclusions Overall, NMPC's approach to identifying and resolving wuipment performance problems following the April 24 Unit 2 reactor scram was acceptable. Positive aspects of NMPC's post-scram evaluation process included the establishment of multi-discipline teams to review equipment performance, the conduct of periodic status briefs, and the use of vendor services. Senior management effectively challenged their staff's post-scram analysis which contributed to a more rigorous evaluation and the re-creation of the event using the plant simulator. However, a few performance shortcomings related to the scram evaluation process were apparent. Although the overall process was thorough, equipment troubleshooting and failure analysis were not methodical. NMPC management recognized these shortcomings and was developing methods to improve its staff's problem solving skills.

08 Miscellaneous Operations issues (92700) 08.1 (Closed) Licensee Event Report (LER) 50-410/98-26: Seismic Monitor inoperable for More Than Thirty Days and Special Report Not Submitted. On October 9,1998, NMPC discovered that a reactor building triaxial response spectrum recorder (TRSR) was not properly oriented which rendered the seismic monitor inoperable. The seismic monitoring instrumentation is installed to monitor and record data in the event of an earthquake. This data would be used following an earthquake to verify-that the event was bounded by the analytical model provided in the USAR. NMPC determined that the monitor had been inoperable since at least May 1997, when the equipment was last tested, and was potentially inoperable since initial installation. On October 23,1998, NMPC corrected the orientation of the TRSR and verified proper orientation of the other accessible seismic monitoring equipment.

NMPC concluded that station personnel who developed and revised the seismic monitor surveillance procedures were not aware of the importance and precise tolerances-required for the orientation of the seismic monitor instruments. Consequently, the instruments were improperly positioned and rendered inoperable for greater than thirty

~ days. NMPC's failure to maintain the seismic monitors in an operable status and submit a Special Report, as required by TS 3.3.7.2, constitutes a violation of minor cignificance and is not subject to formal enforcement action.

The inspectors comaieted an on-site review of the LER and verified that it was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the' event, as contained in the LER, were consistent with the inspectors' understanding of the event. The root cause and corrective and preventive actions, as described in the LER, were reasonable. This LER is closed.

l

1. .

6 l

- 08.2i (Closed) LER 50-410/99-03: Automatic Depressurization System (ADS) Nitrogen Leakage in Excess of Unit 2 Technical Specifications Surveillance Limits

a. Insoection Scooe (92700)

Between March 5 and March 12,1999, Unit 2 experienced two events where the leakage from the ADS nitrogen storage tanks exceeded the design basis leak rate. The

~

' inspectors' reviewed the associated DERs, attended pertinent SORC meetings, reviewed

. the subsequent LER, and discussed related issues with NMPC personnel,

b. Observations and Findinos The nitrogen system supplies high pressure nitrogen to the ADS valves. The system consists of high pressure storage tanks located outside the reactor building, which supply nitrogen to tank Nos. 4 and 5, located within the reactor building, which in tum supply nitrogen to the ADS accumulators. Tank No. 4 supplies three ADS valves, while i tank No. 5 supplies four ADS valves. The tanks are normally isolated from the outside storage tanks and are periodically re-pressurized to make-up for normal leakage.

Event 1 l On March 5, Unit 2 completed a surveillance test on the ADS nitrogen system. The test j exercised various' valves within the system and required operators to remove the blank {

flange on the nitrogen emergency fill connection and attach a test assembly. Following the test, operators removed the test assembly, re-installed the flange and verified acceptable tank pressures. Over the next day, operators re-pressurized tank No. 5 several times. Subsequently, operators identified a leak at the blank flange. The gasket was replaced and the pressure in the tank was stabilized. DER 2-1999-0682 was written to evaluate the event, and the subsequent review showed that the leak rate from tank No. 5 exceeded the TS allowed valuef Event 2 '

On March 9,1999, operators responded to a low nitrogen pressure alarm on tank No. 4 and manually re-pressurized the tank. ' From March 9 to March 12, operatoas re-pressurized the tank five more times while searching for leaks. Initially, the station shift supervisor (SSS) considered the leakage to be of a similar magnitude as past leaks and concluded that the TS leakage limit was not exceeded. However, on March 12, the SSS determined that Unit 2 may have exceeded the TS limit and initiated DER 2-1999-0749.

Later that day, NMPC identified and repaired a few small nitrogen system leaks. The leak rate decreased to below the TS limit. Subsequently, NMPC found the normally closed valve 2GSN*V73A slightly open, and difficult to operate. NMPC concluded that J since the first re-pressurization of tank No. 4, on March 9, that valve 2GSN*V73A had

. permitted leakage.

1 I

i 1

,e .

p-7 in addition to the two events described in the LER, NMPC believes that excessive

- nitrogen leakage occurred at other times and it was not recognized that the TS limit or the design basis was exceeded.

As described in the LER, NMPC determined that, based on the maximum observed leakage during these two events, approximately 1.68 and 1.86 days of nitrogen for tank Nos. 4 and 5 would have been available. Although, this was less than the design basis of five days, NMPC concluded that 1.5 days was sufficient to allow for a nitrogen truck to arrive on site to resupply the nitrogen tanks. The inspectors considered this to be reasonable. Nonetheless, the failure to take the actions required by TS 3.5.1.e.2 during the period when the ADS tanks leakage rate exceeded the TS allowed limits is a violation. This Severity Level IV violation is being treated as a Non-Cited Violation, consistent with Appendix C of the NRC Enforcement Policy (NCV 50-410/99-04 01).

This violation is in the licensee's corrective action program as DERs 2-1999-0682 and 2-1999-0749.

The inspectors completed an on-site review of the LER and verified that it was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the event, as contained in the LER, were consistent with the inspectors' understanding of the event. The root cause and corrective and preventive actions as described in the LER were reasonable. This LER is closed.

c. Conclusion Between March 5 and March 12,1999, Unit 2 experienced two events where the automatic depressurization system nitrogen storage tanks had excessive leakage.

NMPC failed to recognize that the leakage exceeded the allowed limit, and therefore, did not take the required limiting condition of operation actions. This was a non-cited violation of Unit 2 Technical Specification 3.5.1.

II. Maintenance M1 Conduct of Maintenance M1.1 General Comments (61726. 62707)

Using NRC Inspection Procedures 61726 and 62707, the resident inspectors periodically observed various maintenance activities and surveillance tests. As part of the observations, the inspectors evaluated the activities with respect to the requirements of the Maintenance Rule, as detailed in 10CFR50.65. In general, maintenance and -

surveillance activities were conducted professionally, with the work orders (WOs) and necessary procedures in use at the work site, and with the appropriate focus on safety.

Specific activities and noteworthy observations are detailed in the inspection report. The inspectors reviewed procedures and observed all or portions of the following maintenance /s%t 'eillance activities:

L -

I.

8

{

1 e WO 99-08109, Hydraulic control unit post maintenance testing.

e WO 99-06450, Uninterruptible power supply inverter cleaning and inspection.

]

1

-e N2-OSP-lCS-R002, Reactor core isolation cooling.

e WO 98-03424, Feedwater heater replacement.

• RFMSHRD30, Electric discharge machine shroud weld V9 and V10 activities.

M1.2 Fuel Off-load Activities (Unit 11 a.- 'Insoection Seone (60710)

The inspectors observed portions of Unit 1 fuel off-load activities using the guidance provided in NRC inspection Procedure 60710, " Refueling Activities."

b; Observations and Findinas The inspectors observed fuel off-load activities from the control room and from the refueling bridge. The off-load was pec formed in accordance with approved procedures, and was well controlled. The inspectors considered the communications by the operators on the refuel bridge and between the refuel bridge and the control room operators to have been good. The inspectors independently verified installation of the refueling interlock jumper and that a sample of the fuel moves were correct.

c. Conclusions The Unit 1 fuel off load was well controlled. Communications between the operators on the refuel bridge, as well as between the refuel bridge and the control

' room were observed to be good.

M1.3 ' inste!!etion of Core and Containment Sorav Strainer Assemblies (Unit 1)

a. Insoection Scooe The inspector reviewed work order packages for installing the emergency core cooling system (ECCS) torus strainers to ensure the installation was being conducted in accordance with station drawings and work instructions.

b.' Observations and Fmdings The inspector utilized work order packages 98-03314-05,06,07, and 08 to verify that the J core spray and containment spray systems' strainer assemblies were properly installed.

The packages included quality assurance hold points, foreign material exclusion signatures, as well as signatures for verification of component fit-up. The inspector verified that welders were utHizing approved welding procedures and welding material.

Additionally, the inspector verified the welder's qualifications were current and compatible for the weld procedure and process being utilized on the modification.

V

,l .

9 The inspector toured the torus and work site and observed that the work areas were clean and organized and that waste materials were kept to a minimum to reduce the l challenges to foreign material exclusion controls. In addition, the inspector noted that quality assurance personnel were assigned to the project and were conducting routine surveillance activities.

c. Conclusions The installation of the emergency core cooling system torus suction strainers was well controlled. The work environment was clean and organized and good foreign material exclusion controls were in effect.

M2 Maintenance and Material Condition of Facilities and Equipment I l

M2.1 Reactor Co'e Isolation Coolina (RCIC) System Failure Durina Reactor Scram Transient (Unit 2)

a. Inspection Scope (62707. 37551) 1 During the April 24 reactor scram transient, the RCIC system failed (see section O1.2).

The inspectors reviewed applicable sections of the USAR, RCIC operating procedures, and the DER disposition. The inspectors walked down portions of the system, observed system troubleshooting efforts, and intervimved the operator who was responsible for the operation of the RCIC system at the time of its failure.

b. Observations and Findinas During the reactor scram, reactor low water level was reached and the RCIC system received an automatic start signal. The RCIC injection valve opened and the trip throttle valve' indicated that it was open. However, the maximun. RCIC turbine speed observed by operators was 200 rpm with zero discharge flow indicated. Based on these control room indications, the control room operator manually secured the RCIC turbine. Based on the observed system operating parameters, the inspector concluded the operator's ,

j action to trip the RCIC turbine was appropriate. I l

i '

l Subsequent troubleshooting showed that the RCIC system had received a valid initiation signal and that the steam admission and outboard injection valves had opened. Data l recorders confirmed that the RCIC turbine speed had increased to 200 rpm, at which l point the turbine trip valve was tripped.

NMPC conducted extensive troubleshooting and determined that tha latching ,

mechanism for the trip throttle valve was not sufficiently engaged. NMPC determined that the set-up of the overspeed trip linkage and associated valve components was not correct. The inspectors determined that the RCIC turbine trip throttle valve had been disassembled and rebuilt during the 1998 outage.. NMPC obtained vendor assistance to complete the work and had relied upon the vendor's expertise. Licensee review determined that the overspeed trip linkage tolerances were not described in the work t-

, 1 10 package and consequently the linkage was re-assembled with incorrect tolerances. The failure to provide an adequate work procedure is a violation of 10 CFR 50 Appendix B Criterion V, " Instruction, Procedures, and Drawings " This severity level IV violation is being treated as a Non-Cited Violation, consistent with Appendix C of the NRC Enforcement Policy (NCV 50-410/99-04 02). This procedural adequacy violation is in the licensee's corrective action program as Deficiency Event Report (DER) 1099-1254.

Corrective actions included revising the maintenance procedure and providing additional training for maintenance personnel. The licensee also determined that additional industry operating experience was available, but not used, conceming RCIC system trip throttle valve maintenance.

The Maintenance Rule performance criteria for the RCIC system is two functional failures over a two-year period. This failure was classified as a maintenance preventable functional failure and actual performance shows three functional failures during the previous two-year period. At the end of the inspection period, NMPC was evaluating the RCIC system for classification in Maintenance Rule category (a)(1).

c. Conclusions During the Unit 2 scram, the reactor core isolation cooling (RCIC) system failed to operate as required and was manually tripped. This RCIC system failure was attributed to an inadequate maintenance procedure and the licensee's over-reliance on vendor support for a 1998 RCIC tud>ine trip throttle valve rebuild. The failure to ensure an adequate maintenance procedure was prepared and used to perform work on the RCIC system was a non-cited violation. Based on recent operating history, the RCIC system has exceeded its Maintenance Rule performance criteria.

M2.2 Generator Protection Relav Failure Resulted in Scram (Unit 2)

a. Inspection Scooe The Unit 2 reactor scram was caused by the failure of a generator protection circuit relay.

The inspector observed and reviewed NMPC's troubleshooting and evaluation methods used to determine the cause of the relay failure.

b. Observations and Findinos NMPC troubleshooting effort showed that the volts / hertz relay associated with the generator protection circuit had failed. The circuit design was such that the relay failure caused the turbine trip (and reactor scram) and caused a residual (slower) transfer of electrical loads to offsite power sources.

No apparent cause for the relay failure was identified. Inspection of the relay did not reveal any physical characteristics for the failure mode. Bench testing showed that the relay was defective' and that the malfunction would provide a spurious trip signal with an outcome the same as the event that was experienced. To obtain more specific information concoming the failure mode, the failed relay was shipped to an independent

y ..

m. .

11 i

laboratory for failure analysis. The relay was replaced and calibrated satisfactorily.

Other similar relays in use were recalibrated and tested satisfactorily.

The inspector reviewed the failed volt / hertz relay work history which showed that this relay was replaced during outages in 1996 and 1998. The failed relay and similar relays used in the generator protection circuit fall under the Unit 2 preventive maintenance program and are calibrated every refuel outage.; These relays were included within the scope of the Maintenance Rule and this event was classified by NMPC as a functional

! failure.

r f c. Conclusions l

On April 24, a Unit 2 generator protection circuit relay failed which caused a reactor scram. NMPC effectively evaluated the cause and consequences of the relay failure and implemented acceptable corrective action.

I M3 Maintenance Procedures and Documentation i )

M3.1 Inservice inspection (Unit 1) )

a. Inspection Scope (73753)

The inspectors reviewed the inservice inspection (ISI) activities that were part of Refueling Outage (RFO) 15. The review involved performing a walkdown of portions of the core spray system piping and verifying piping welds were reflected in the ISI program 1 manual and system isometric drawings. Non-destructive examination (NDE) activities j were observed, and the qualifications of NDE personnel verified.' Additionally, the inspectors assessed NMPC's oversight of contractor NDE activities,

b. Observations and Findings

'ISI Proaram Manual No deficiencies were noted in the ISI program manual during the field walkdown of the core spray system. The list and location of core spray system piping welds contained in the manual, matched the as-built system configuration. . However, during the field walkdown the inspectors identified errors in the core spray system weld map isometric drawing F-45183 C. Specifically, the ISI program mariual indicated welds 81-WD-128 and 81-WD-183-A were located downstream of core spray pumps 11 and 12. The inspectors confirmed the welds were located in the correct location on the core spray piping. However, they were not shown on the corresponding weld map drawing. NMPC documented this drawing error in DER 1-99-1225. i Both ~ welds were located in ASME Code Class 2 piping and were not among the population of welds that NMPC had selected for NDE activities. This approach was in accordance with ASME Code requirements, which indicate only 25% of the

12 applicable welds'in ASME Code Class 2 piping need be examined over the 10-year inspection interval. Accordingly, the weld map error did not result in an ASME Code violation and this ISI Program administration oversight was of minor safety consequence not subject to formal enforcement action.

Observation of NDE Activities The inspectors witnessed several NDE field inspections, including an ultrasonic (UT) examination performed by NMPC personnel on a recirculation system piping weld, and a visual examination of a service water system piping hanger conducted by a contractor. The individuals who performed the examinations met the training and experience requirements outlined in procedure SNT-TC-1 A " Recommended Practice, Personnel Qualification and Certification of Non-Destructive Testing."

While observing the UT examination, the inspectors verified the UT test equipment was calibrated in accordance with industry and NMPs. standards. Further, the inspectors verified that deficiencies uncovered during the visual and UT examinations were documented as required in DERs.

Core Shroud and Beltline Weld insoection Activities NMPC had made arrangements with two vendors, General Electric and Framatone, to perform NDE activities on the horizontal welds in the reactor vessel and core shroud, I respectively. By review of vendor inspection plans and interviews with NDE personnel, the inspectors verified the inspection scope for the shroud and reactor vessel welds were .

In accordance with the NRC approved inspection plans described in NRC correspondence to NMPC, dated March 24 and April 7,1999, respectively. To minimize the possibility that relevant indications would be overlooked, both vendors had at least two individuals who independently review the NDE data, pversiaht of NDE Activities During this outage, NMPC changed its philosophy regarding oversight of contracted NDE i activities. Prior to the change, NMPC NDE personnel provided little formal oversight of I contracted NDE activities. Instead, oversight was provided on an informal basis, whereby NMPC personnel would observe contractor activities on a time-available basis.

Formal oversight was limited to yearly audits of the ISI program.

During this outage, NMPC developed a formalized ISl surveillance schedule that outlined which NDE activities would be monitored. Most monitoring was conducted by NMPC NDE personnel. However, contract personnel where scheduled to oversee some NDE activities where NMPC did not have the necessary in-house experience to adequately

. observe and evaluate. Surveillance plan observations were to be documented and forwarded to management for review.

The inspectors did not have the opportunity to review any completed surveillance reports or observe performance of surveillance in the field, so it was not possible to comment on

] * .

j.- ..

13 the quality of the effort. . However, the new surveillance program does provide NMPC additional assurance that contracted activities will be properly conducted,

c. Conclusions Non-destructive examination personnel were qualified, and adhered to procedures while performing examinations. The core shroud and reactor vessel weld inspection plans were in accordance with the requisite NRC safety evaluation.~ Deficiencies identified during inspection activities were properly documented. A new surveillance program provided enhanced oversight of vendor activities.

l M8 Miscellaneous Maintenance issues (37551, 92700, 90712, 92903)

M8.1 (Closed) LER 50-410/99-02: Missed Technical Specification Channel Functional Test of the Recirculation Flow Upscale Rod Block. The technical details associated with this

. LER were discussed in NRC Inspection Report (IR) 50410/99-03, Section M1.2. Th' inspectors completed an in-office review of the LER and verified that it was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the event, as contained in the LER, were consistent with the inspectors' uriderstanding of the event. The root cause and corrective and preventive actions as ,

' described in the LER were reasonable. This LER is closed. 1 l

M8.2 (Closed) VIO 50410/98-05-02: Failure to conduct surveillance test on batteries.  ;

Specifically, during Refueling Outages (RFOs) 4 and 5, credit was inappropriately taken for the battery performance test, in lieu of the battery service test for the Division I 125 voit battery. Subsequently, NMPC issued LER 50410/98-09 " Missed Battery TSSR

[ technical specification surveillance report] Due to inappropriate interpretation." This ]

J LER was reviewed and closed in NRC IR 50410/9645. Based on the review of the LER, which provided the root cause and corrective actions regarding the event, NMPC was not required to provide a separate response to the violation. The inspectors verified implementation of the corrective actions associated with this event. Violation 50410/98-05-02 is closed.

l M8.3 (Closed) VIO 50-220/98-02-05: Inadequate plant impact in Work Order (WO) package.

Specifically, during the development of a troubleshooting WO associated with a control room chilled water temperature control valve, the impact of removing two leads was not adequately evaluated. As a result, removing these leads caused an unanticipated opening of the control room ventilation outside air and return air dampers. The inspectors confirmed the completion of the corrective actions associated with the event as described in NMFC's June 26,1998, response to the violation. Violatica 50-220/98-02-05 is closed

l 14 111. Ennineerina E1 Conduct of Engineering E1.1 Uninterruotible Power Supolv (UPS) Failure Durina Reactor Scram Transient (Unit 2)

a. Inspection Scope (37551)

During the reactor scram transient, one of the two reactor protectien system uninterruptible power supplies (2VBB-UPS3B), failed. The inspector reviewed applicable sections of the USAR, UPS operating procedures, and the DER disposition,

b. Observations and Findinas The power supply for the reactor protection system (RPS) consists of two UPS systems.

Each UPS has three power sources: the preferred attemating r 'ent (AC) source, the direct current (DC) source, and the maintenance AC sourc , pon loss of the preferred AC source, the UPS automatically switches to the DC soura. Each UPS is connected to its RPS through two redundant electrical protection assemblies (EPAs). The EPAs are designed to protect the RPS circuits from voltage or frequency deviations.

During the event, power was lost to the preferred UPS power supply causing the UPS to transfer to the battery. Upon re-energizing the AC switchgear, the UPS DC source fuse blew. Also, the voltage had lowered sufficiently enough to cause the EPAs to trip on undervoltage, resulting in a loss of power to the RPS. The impact of the loss was limited because the plant was already shutdown.

NMPC performed a formal event and causal factor analysis to determine the root cause of the UPS failure. It was determined that, excess current caused the DC power supply fuse to blow. The excess current was the result of a design deficiency with the control circuit board for the maintenance bypass switch. The specific design deficiency was incorrect grounding of a control circuit board. NMPC determined that the design deficiency was created during the installation of the UPS maintenance bypass switch.

The maintenance bypass switch for the failed system was a vendor supplied modification I

which was installed during the 1998 refueling outage and for the other train of UPS, in 1996. During the modification review process, NMPC did not recognize that the grounding circuit for the maintenance bypass switch was incorrect. Contributing to 3 NMPC's oversight was the absence of appropriate vendor supplied design change drawings. As documented in DER 2-1999-1707, corrective actions included a review of  ;

the engineering design change process. In addition, NMPC's interim corrective action  !

l included the installation of a temporary modification to remove the motor-operated feature of the maintenance bypass switch, effectively removing the circuit card design deficiency.

NMPC class;fied the UPS failure as a Maintenance Rule functional failure. The inspector determined that this was the only functional failure for the UPS system during the previous two-year period.

__ U

]

• l*

15

c. Conclusions During the Unit 2 reactor scram transient, one of the two reactor protection system uninterruptible power supplies (UPS) failed. Excessive currents, which caused the inverter DC power supply fuse to blow, were the result of a UPS design deficiency involving a newly installed maintenance bypass switch. NMPC identified that this vendor supplied UPS design change received an inadequate engineering design review.

E2 Engineering Support of Facilities and Equipment E2.1. Design Modifications (Unit 1) a inspection Scope (37700)

The inspectors reviewed selected Unit 1 design change packages (DCPs) to assess the quality of engineering analyses and to verify that the design change process complied with plant administrative procedures and regulatory requirements. The inspection addressed temporary and permanent design changes and included a review of the background information, applicable analyses, calculations, safety evaluations, intemal review process, and post-modification testing activities. The inspectors also conducted walkdowns of selected installations to verify their conformance with applicable documents.

b. Observations and Findinas The inspectors determined that the reviewed system and component changes correctly addressed the concems for which the design modifications had been prepared and that the applicable analyses accurately described the purpose of the modification and the intended results. Except as described below, the analyses, calculation:; and safety evaluations were detailed, supported the design changes, and had been appropriately reviewed. The inspectors identified no concems with the installed equipment or post-modification testing performed.

ECCS Suction Strainer Reolacement This modification pertained to the installation of new horizontal stacked disc strainers in the torus and was initiated to address strainer plugging concems raised by NRC Bulletin

. g6-03, " Potential Plugging of Emergency Core Cooling Suction Strainers by Debris in Boiling-Water Reactors." Besides the installation of the strainers, the modification package addressed other needed changes, including: (1) the addition of new spectacle flanges and a strainer between the condensate storage tanks and the core spray pumps; and (2) the removal of retired-in-place hydrogen-oxygen monitoring tubing in the torus to eliminate a potential direct debris source. The inspectors found the technical scope of the design changes to be comprehensive and the licensee's review of the procedures requiring revision due to the design changes to be thorough.

, i 16 I

in vendor calculation No. S14STRAINERM002, "ECCS System Strainer Air / Steam Ingestion Analysis," the licensee evaluated whether the air bubbles that formed in the Unit 1 torus during a loss of coolant accident (LOCA) or a safety-relief valve (SRV) discharge presented a challenge to the operation of the core spray and containment spray systems. The inspectors found that, although the licensee had performed the required independent review of the calculation, the review lacked thoroughness in certain areas. For instance, based on the results of the calculation, the licensee had concluded that an SRV actuation while the containment spray pumps were operating, such as in response to a small break LOCA, would result in air being ingested into the containment spray pump strainers and cause degradation of the pumps. However, the licensee did not address this potential condition in the unreviewed safety question determination section of the safety evaluation. As a result of the inspectors' questions in this area, the licensee asked the vendor to perform a more detailed analysis of this specific scenario and subsequently determined that air ingestion and, hence, pump degradation would not 3 occur.

During a large break LOCA, the vendor calculation indicted that the time lapse between the onset of the LOCA and the start of the pumps would prevent air ingestion into the  ;

new ECCS suction strainers and would not challenge the operability of the pumps. In the calculation, the vendor concluded that a pump, which was already running at the onset of a large break LOCA may be momentarily degraded due to air ingestion; however, the pumps would still be able to achieve the flow rates assumed in the accident analysis.

The licensee accepted the calculation results, but conservatively decided that, whenever a core or containment spray pump was placed in operation for testing or torus cooling, they would declare that loop of the system not operational and follow the TS requirements regarding the limiting condition for operation (LCO) of that system. As in the case of the small break LOCA, the licensee had not specifically addressed in the safety evaluation the potential degraded condition of the pump for which the LCO was necessary. The licensee explained that the potential for air ingestion by the pumps was a pre-existing unrecognized condition that was being alleviated by the new strainers and that the LCO was a conservative measure to assure the reliability of the affected system.

The inspectors determined that the two conditions described above should have been included in the licensee's unreviewed safety question review.' Upon further review of the issue, the licensee determined that the safety evaluation should be revised to provide stronger bases for the conclusions reached.' Accordingly, the licensee initiated Deviation / Event Report (DER) 1-1999-1480.

c. Conclusions Unit i design changes that were reviewed, correctly addressed the concems for which the modifications had been developed Typically, the analyses accurately described the purposa of the modification and the intended results; the calculation and safety evaluations satisfactorily supported the design changes; and the design change process was acceptably implemented.

In the case of the Emergency Core Cooling System (ECCS) strainer modification, the technical scope of the design change was comprehensive, but the licensee's original l

,y .

17 review of an ECCS pump air ingestion calculation lacked thoroughness regarding a small break loss of coolant accident (LOCA) scenario and required a more detailed analysis and a revision of the supporting calculation. Additionally, the safety evaluation required revision to provide stronger bases for the conclusions contained therein regarding a large break LOCA. The licensee's review of air ingestion phenomena associated with the it rge break LOCA resulted in the conservative decision to declars inoperable, in the future, any ECCS pump placed in operation for surveillance testing or torus cooling.

E4- Engineering Staff Knowledge and Performance E4.1 (Closed) LER 50-220/99-03: NMP1 Thermal Limit Exceeded the Requirements of Technical Specifications

a. Insoection Scooe (37551. 92700)

On February 19,1999, the Unit 13D-Monicore system was mistakenly updated with a traverse in-core probe (TIP) power distribution from February 4,1999. This action resulted in the plant being operated with the average planar heat generation rate -

(APLHGR) exceeding the limits specified in the Technical Specifications. The inspector reviewed the LER and the documentation associated with NMPC's corrective action program.  ;

b. Observations and Findinas l l

3D-Monicore is a system of computer programs designed to monitor and predict important core parameters. The programs calculate reactor power, moderator void and flow distributions in the core. This information is used to determine other core  ;

parameters such as margins to thermal limits, and fuel exposure. The programs are designed to track current reactor parameters automatically (usually once per hour) or on I demand. The computer program accuracy is enhanced by making use of in-core neutron flux measurements. NMPC utilizes the TIP system for in-core flux measurements and inputs this data to the 3D-Monicore. On March 23, NMPC completed a routine TIP data collection run and subsequently transferred the new data to the 3D- ,

Monicore sys, tem. The new TIP data correctly updated the computer's calculated core i power distribut;on and following the printout of the core parameters the operators determined that the APLHGR value for one s'rea of the core was 2.2 percent above TS limits. Operators immediately reduced power to restore the APLHGR within TS limits.

NMPC's investigation into the event determined that an inadvertent " Process TIPS" command was entered into the 3D-Monicore system. This action essentially put old in-core flux measurements into the program. During the investigation, NMPC determined that the action had a discernable effect on the computer printout of the core parameters, but was not identified by the operators or the reactor engineering group at the time of the error. NMPC's investigation also determined that the error did not cause any adverse effects until after a control rod pattern adjustment was made on March 20,1999. Had the proper TIP case been in the computer program at that time, there is a high probability that the particular rod adjustment would not have been done. Evaluation by NMPC

18 determined that an APLHGR of 2.2 % above the limit wou.o not have exceeded any of the 10 CFR 50.46 licensing criteria, and therefore, had minimal safety consequences.

NMPC determined that the cause of inadvertent processing of TIP data was inadequate computer system security on the 3D-Monicore system. The system was not protected, in that the design allowed TIP data to be processed without authorization and without waming from uncontrolled locations. The inspectors determined that additional information and follow-up was required in the area of computer security (IF1-50-220/99-04-03). NMPC determined that the reactor engineering group failed to recognize the corruption of the system due to insufficient analysis of daily 3D-Monicore data, in that the discrepancy was not recognized for three days. Corrective actions in the LER included disabling the ability to process TIPS from uncontrolled locations. The NMPC root cause evaluation identified several other corrective actions including developing tools to aid in monitoring the accuracy of 3D-Monicore and tracking key core thermal limit parameters for trending and analysis purposes.

As discussed above, NMPC determined that the actual impact of exceeding the thermal limit was small. Nonetheless, the failure to maintain core thermal limits as required by TS 3.1.7.a is a violation. This severity level IV violation is being treated as a Non-Cited Violation (NCV), consistent with Appendix C of the NRC Enforcement Policy (NCV 50-220/99-04-04). This violation is in the licensee's corrective action program as DER 1-1999-0837.

The inspectors completed an on-site review of the LER and verified that it was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the event, as contained in the LER, were consistent with the inspectors' understanding of the event. The root cause and corrective and preventive actions as described in the LER were reasonable. This LER is closed.

c. Conclusiong From March 20 to March 23,1999, Unit 1 operated with a maximum average planar heat generation rate (APLHGR) exceeding the limits specified by the technical specifications. 1 This technical specification violation was non-cited. NMPC determined that the cause was the inadvertent processing of traverse in-core probe (TIP). data, due to inadequate computer system security on the 3D-Monicore system. Specifically, TIP data could be processed without authorization or operator knowledge from uncontrolled locations.

Additionally, the oversight by station personnel with regards to reactivity management and core performance monitoring was poor, in that this-discrepancy was not recognized for three days.

l l

1

s ;r 19 E8 ' Miscellaneous Engineering lasues (92700)

E8.1 -(Closed) LER 50-220/98-19: Missed ASME Section XI Inservice inspection Due to Cognitive Error,

s. Inspection Scope On November 13,1998, while reviewing an issue associated with a contingency work package, NMPC discovered certain visual inspections had not been performed on plant components as required by the ASME Code. The missed inspections concemed the

- failure to perform visual examinations of the reactor recirculation pump seal housing

~ bolts and flange surfaces, during the first and second ten year inspection intervals.

b. Observations and Findinas NMPC determined the inspections were missed since the reactor recirculation pumps bolts had not been classified as pressure retaining components in design documents.

The recirculation pump flanges were not examined, because ISI personnel overlooked an ASME Code requirement that stated, if any of the five recirculation pdmps are disassembled during an interval, one pump flange surface inspection must be performed. During the first and second ten year intervals, several recirculation pumps were disassembled.

Niagara Mohawk corrective action included revising the ISI inspecticn plan to incorporate the required hspections. A review of the ISI plan was conducted to

. ensure ASME Code and ret ulatory requirements were identified in the plan. No other missed inspections were identified. Finally, an ' inspection of the recirculation pump seal housing bolts was scheduled for completion during refuel outage (RFO) 15.

The inspectors reviewed the ISI program plan and self assessment reports, and verified the plan had been modified. The failure to conduct the required ASME code inspections is a severity level IV violation and is being treated as a Non-Cited Violation, consistent with Appendix C of the NRC Enforcement Policy (NCV 50-220/99-04-05).' .This violation is in the licensee's corrective action program as LER 50-220/98-19.

The inspectors completed an on-site review of the LER and verified that it was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the event, as contained in the LER, were consistent with the inspectors' understanding of the event. The root cause and corrective and preventive actions as described in the LER were reasonable. This LER is closed.

20 i

c. Conclusion The failure to conduct the required.ASME Code inservice inspections of the reactor recirculation pump seal housing bolts and flange surfaces during the first and second ten-year inspection intervals was non-cited.

I E8.2 (Closed) VIO 50-410/98-05-03: Failure to perform adequate design for emergency diesel generator (EDG) modification on fuelline. Specifically, the installation of a 1993 modification to the EDG fuel lines failed to include a rubber grommet at the piping support to compensate for system vibration. As a result, vibration of the fuel line pipe caused fretting of the pipe at the location of the pipe support and on April 14,1998, during surveillance testing of the Division ll EDG, a fuelleak developed. NMPC issued DER 2-98-0891 to address this issue. The root cause and corrective actions associated with this DER were reviewed in NRC IR 50-410/98-05. Based on this review, NMPC was not required to provide a separate response to the violation. The inspectors verified completion of the corrective actions associated with this event. Violation 50-410/98 03 is closed. ,

l E8.3 (Closed) LER 50-410/99-04: NMP2 Service Water intake De-Icing Heater Control Circuits do not Meet Fire Protection Program Requirements

a. Insoection Scope (92700)

On March 18,1999, during a review of the Safe Shutdown Analysis for a control room fire, Unit 2 personnel determined that the service water intake de-icing heater control circuits were not included in the analysis. The inspectors reviewed the associated DER, attended pertinent SORC meetings, reviewed the subsequent LER, and discussed related issues with NMPC personnel.

b. Observations and Findinas The Unit 2 service water intake structure openings are equipped with bar rack heaters to eliminate the potential for frazil ice adhesion. Frazil ice formation can occur when the intake structure temperature drops near freezing. Therefore, TS require the heaters to be operable whenever the intake tunnel water temperature is below 39 degrees F. As part of the corrective actions for LER 50-410/99-01, "NMP2 Outside the Design Basis Due to Safe Shutdown Service Water Pump Bay Unit Coolers Being Out-of-Service," l Unit 2 determined that a control room / relay room fire that renders the service water l intake de-icing heaters inoperable, coinci.l3nt with service water temperatures that approach freezing, could lead to a complete loss of service water. This condition was  !

not in accordance with the Safe Shutdown Analysis as described in the Unit 2 UFSAR, and this condition has existed since the initial operation of the plant.

Upon identification, NMPC established a fire waten for the control room / relay room fire area, which will be in place until a design change to correct the deficiency is implemented. NMPC expects to complete the design change by November 30,1999, in i

I I

L

.?.

21 addition, NMPC is continuing their review of the safe-shutdown capability as described in LER 50-410/99-01. The inspectors considered this actions to be appropriate.

NMPC Licensee Conditior u.G requires implementation of the fire protection program-described in the UFSAR. . The fire protection program includes an analysis of the ability to achieve safe-shutdown of the unit in the case of a control room fire. The failure to ensure that the service water intake de-icing capability is available during a control room fire, could impair the ability to achieve safe-shutdown, and is a violation of this licensee condition. This_ Severity Level IV violation is being treated as a Non-Cited Violation, consistent with Appendix C of the NRC Enforcement Policy (NCV 50 410/99-04-06).

This violation is in the licensee's corrective action program as LER 50-410/99-04.

The inspectors completed an on-site review of the LER and verified that it was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the event, as contained in the LER, were consistent with the inspectors' understanding of the event. The root cause and corrective and preventive actions as described in the LER ware reasonable. This LER is closed.

c. Conclusion NMPC self-identified and promptly corrected a condition which could have adversely affected the ability of the unit to achieve safe-shutdown, involving the Unit 2 service water intake de-icing heater control circuits which were not protected against a control room fire. This violation of License Condition 2.G was non-cited.

E8.4 (Closed) IFl 50-410/98-19-03: Leakage of contaminated water following a scram reset.

Draining evolutions of the scram discharge volume (SDV) have resulted in the contamination of a small ~ a rea of the reactor building floor. The contamination is caused by steam condensing into and leaking from a ventilation duct to which the equipment drain cooler and the SDV vent piping are connected. NMPC compensated for this by ensuring that applicable ~ procedures incorporate a plant announcement to stand clear of the area during SDV venting and draining. The NRC originally became aware of this issue in 1997 (NRC inspection report 97-11). Later, the NRC opened the IFl to conduct further review of the issue. The purpose of this inspection was to review the system design and to evaluate the safety implications.

The potential for the leakage was created in the 1994-1995 time frame, after the licensee implemented a design change that relocated the reactor building equipment drain header piping. Before the implementation of this change, the hot, pressurized drain lines from the reactor core isolation cooling and the residual heat removal steam condensing systems shared a common header with the cool, gravity equipment drain lines. This common header provided a path for the fluid from the pressurized sources to flash into steam, through the gravity drains, into the reactor building. The relocation of the piping to separate the high pressure drain header from the gravity drain header was successful in eliminating steam from the gravity drains. However, an associated change that tied the SDV vent piping into the equipment drain cooler vent line, effectively created a path

22 of less resistance to a connecteo antilation o ;ct allowing leaks from the duct onto the floor during scram reset.

The inspectors' renow of the design implications determined that the small amount of contaminated water emitted from the ventilation duct was controlled and contained in accordance with approved site radiation protection procedures. The inspectors also determined that within the affected area there was no safety equipment which could be adversely impacted by the water spillage. Based on this review, the inspectors did not consider the small amount of water spillage to be safety significant but noted that, although the licensee had implemented several design changes in the equipment drain system during the last five years, they had not implemented an acceptable piping configuration that resolved the original system interaction and the venting contamination concems. The water spillage also constituted an operator work-around in that it required additional operator actions during the drain evolution of the SDV.

NMPC had proposed and was evaluating attematives to eliminating the inadvertent contamination experienced during SDV draining. The proposed actions included the rerouting of the SDV vent line and the use of a check valve. Based on the licensee actively pursuing the resolution of this low safety significant issue, this item is closed.

E8.5 (Closed) Violation 50-220/98-16-0-2: Failure to identify and promptly correct a condition adverse to quality. On August 20,1998, NMPC determined that they had failed to recognize, in 1996 and 1997, a low cooling water flow to the motor bearing of core spray pump No.122. The licensee reported the finding in LER 98-16. At the time of the finding, the NRC reviewed the issue and was satisfied that the licensee had developed a comprehensive corrective action plan to resolve the discrepancy. Nonetheless, the licensee's failure to evaluate and correct their finding was a violation of the Appendix B, corrective action program.

During the current follow up review, the inspectors confirmed that the actions described in the LER and the subsequent engineering evaluation had been satisfactorily completed. Specifically, the inspectors verified that: (1) the repair of the affected pump had been satisfactorily completed; (2) the extent of condition had been addressed; (3) i' the applicable maintenance procedures had been revised; and (4) the lesson learned from the event had been discussed with responsible personnel from engineering, operations, and maintenance. This item is closed.

I I

m a 3

.i

• 23 )

\

IV.' Plant Suonort R1 Radiological Protection & Chemistry Controls R1,1 Unit 1 and Unit 2 Solid Radioactive Waste Manaaement and Transoortation of Radioactive Materials

a. Inspection Scope (86750)

A selective review was performed to evaluate NMPC's basis for certifying that radioactive materials and wastes intanded for disposal were properly classified, described, packaged, marked and labeled and that radioactive material / waste shipments were made in accordance with applicable shipping regulations. Information was gathered by a review of the following: possession or access to applicable federal regulations; maintenance of licenses for facilities that radioactive materials or wastes were shipped; training records; procedural guidance and procedural maintenance; use of scaling factors to infer the concentration of difficult-to-measure radio nuclides; methods used to classify radioactive wastes; shipping records; and through interviews with cognizant personnel and tours through the plant.

b. Observations and Findings A review of records verified that NMPC had ready access to up-to-date copies of federal regulations including 49 CFR Parts 100-179 and 10 CFR Parts 20,61, and 71. A review of selected records verified that up-to-date copies of licenses were maintained for facilities to which radioactive materials and wastes were shipped. A selected review of training records showed that members of the Radwaste organization responsible for preparing radioactive waste shipments had received current training on regulations and computer programs used to prepare shipments. An interview with a training specialist revealed that initial training and periodic training were provided on equipment and processes that generate radioactive waste. A review of selected procedures verified that detailed instructions were provided for personnel involved with the transfer, packaging and transport of radioactive wastes. Procedural guidance was adequately maintained and several procedures were in the process of being revised to incorporate recent changes to the regulations and to make needed improvements.

l The Radweste group primarily used computer programs to classify and prepare radioactive material and waste shipments. ' A selected review of shipping records showed that waste classifications were accurate and no discrepancies were identified.

Shipping records were prepared in accordance with procedural guidance and included appropriate information such as radiation and contamination surveys, emergency ,

response information, and shippers certification of the adequacy of the shipment.

c. Conclusions Radioactive material / waste management and transportation programs were effectively implemented as evidenced by use of up-to-date regulations and facility licenses,

24 appropriately trained personnel, proper procedural guidance and adequate maintenance of procedures, appropriate use of scaling factors to estimate isotopic content of radioactive material / waste packages, and proper shipping records.

R1.2 Refuel Outaae Radioloaical Controls (Unit 1)

a. Insoection Scope (83750)

A review was performed of radiological controls implemented for outage work.

Information was gathered by a review of radiation exposure goals, selected licensee initiatives to maintain radiation exposures as low as is reasonably achievable (ALARA),

through discussions with cognizant personnel, through a review of radiological posting practices, by a review of administrative controls for the Unit 1 Turbine Building " Green Area," a review of high radiation area access controls, and a review of the following documents:

1. ALARA Review 99-12 "Drywell In-Service inspections (ISI), Erosion / Corrosion (E/C)

Exams and Support Work"

2. ALARA Review 99-10, "Dewater and Desludge Torus, Replace ECCS Suction )

Strainers"

3. ' ALARA Review 99-06, " Replace Thirty-Seven Control Rod Drives (CRDs)"

4. ALARA Review 99-11, " Disassembly / Reassembly of Reactor Vessel, Fuel Movements and Decontamination"

5. Al. ARA Review 99-03, "Drywell - Repack Valves - All Elevations"

6. ALARA Review 99-17 "Drywell Floor Drain Sump and Associated Work"

7. GAP-RPP-08, Rev 5, " Control of High, Locked High, and Very High Radiation Areas"

8. S-RAP-RPP-0801, Rev. 8, "High Radiation Area Monitoring and Control"

9. S-RAP-RPP-0103," Posting Radiological Areas" 10.10CFR61.55 data for NMP1 dry active waste (DAW)

11. Safety Evaluation 96-102, " Safety Evaluation for Turbine Building Green Area" )

12. Deviation / Event Report No. 1-1999-1106, " Unlocked Source Storage Locker - Turbine Building 261'- Condenser Bay."

13.N1-RSP-1Q, " Accountability of Calibration and Check Sources at NMP Unit 1, j Rev.O."

b. Observations and Findinas An ALARA exposure goal of 280 person rem was set for RFO15. The majority of dose (more than 200 person-rem) was estimated to be received from drywell work. Significant dose jobs included 70 person-rem for drywell ISI, 25 person-rem for torus desludging and ECCS suction strainer replacement,23 person-rem for CRD exchanges, 20 person-rem for refuel floor activities,18 person-rem for drywell valve repacks, and 17 person-rem for mit,cellaneous drywell inspections. Interviews with cognizant personnel and reviews of documentation revealed that the radiological controls organization was staffed with trained and qualified personnel and there was early involvement in planning for jobs with elevated exposure estimates, high dose rates, and radiologically complex work.

[.,I*

25 ALARA reviews were thorough and provided details of applicable industry events.

Examples of radiological controls and ALARA/ initiatives included the following:

• Significant amounts of temporary shielding were installed in the Unit 1 drywell to reduce general area and job specific dose rates;

• Floor and equipment drain lines were flushed to reduce general area dose rates in the radwaste building, reactor building, and drywell and allowed the drywell to be down i posted from a locked high radiation area to a high radiation area; l

• Detailed component and equipment locations were included on drywell briefing maps;  !

1

- The torus was desludged with a back-flushable filter skid which reduced the need for i filter handling and disposal; and

• Cameras were stationed in key work areas for remote monitoring of work.

2 RFO15 Handbook An outage handbook was distributed to plant personnel to communicate important outage information such as responsible contacts and telephone numbers, meeting times, and plant maps. However, the inspector noted that the handbook included information regarding the expected response to an alarming dosimeter that was inconsistent with procedural guidance and general employee training. The handbook stated that if an individual received a dose rate alarm on their alarming dosimetry they should move to a lower dose rate area. Procedural guidance and general employee training instructed personnel that if an electronic dosimeter alarm occurs, personnel shall leave the work area and report to radiological protection. NMPC issued a DER and distributed a correction to the RFO15 Handbook. No known examples of improper response to alarming dosimetry occurred as a result of this temporary inconsistency and no violations of NRC requirements were identified.

Radioloaical Boundaries Overall, radiological boundaries were clearly defined and posted. However, several opportunities for improving radiological postings were identified on the Unit i refueling floor. For example, during reactor vessel disassembly, the drywell dome and the reactor vessel head insulation were stored on the refueling floor. General contamination beneath these components was approximately 10,000 - 60,000 dpm/100 cm2 and areas on the bottom of the insulation package had levels up to 24 mrad /hr/100 cm2. Access to these components was restricted with several signs indicating " Contact Health Physics Prior to Entry." During a tour of the refuel floor, several locations that provided access to areas beneath the drywell dome and reactor head insulation did not have readily observable radiological postings. Upon notification by the inspector, health physics staff members immediately posted the identified areas. The improvements in radiological posting were considered an enhancement to existing postings and no violations of NRC requirements were identified.

. x

's I

26 Administrative Controls For the Unit 1 Turbine Buildina " Green Area" A clean area (" Green Area") was setup within the radiologically controlled area (RCA) on Unit 1 Turbine Building 305 foot elevation to allow turbine building workers to eat, drink, and use sanitation facilities. A safety evaluation was used as the basis for establishing the area and administrative controls for setup, use, and health physics oversight of the

. facility were included in procedure S-RAP-RPP-0103, Rev. 9, " Posting Radiological Area." Controls for " Green Area" access included continuous health physics oversight; requirements and provisions for contamination monitoring of hands, feet, and face prior to entry; frequent contamination and radiation surveys; and special provisions for the transport of food and other clean items within the RCA.

The radiation protection manager acknowledged that personal contamination surveys for

" Green Area" access did not include a whole-body frisk. However, he pointed out that the risk for the spread of contamination and potential ingestion of significant amounts of contamination were minimized by limiting use of the facility to individuals that were briefed on use of the facility and who worked in areas with re!stively low contamination levels and lower risk of hot particles (i.e., turbine building workers); contamination monitoring was performed for body parts with the highest probability of contamination (hands, feet, and face); and the health physics staff maintained close oversight and monitoring of the facility. Additionally, all RCA workers were required to receive a whole-body frisk prior to exiting the RCA and experience gained during the previous outage demonstrated that radiological controls implemented for the ' Green Area" were effective in controlling contamination.

Hioh Radiation Area Access Controls Controls for high radiation area access included detailed procedural guidance; radiological postings; frequent use of locked doors and required use of locked access controls for areas that could result in an individual receiving a dose equivalent in excess of 1000 mrem per hour at 30 centimeters; radiation work permit (RWP) controls; use of

" alarming" dosimetry; requirements for a minimum available exposure for access; use of remote door alarms; and increased health physics oversight and monitoring. Two health physics technicians and two health physics supervisors selected for interviews demonstrated thorough knowledge of high radiation area controls. Tours of the plant confirmed that high radiation and locked high radiation areas were appropriately posted and doors that were required to be locked were found locked or appropriately controlled i

by health physics staff. A review of DERs for the last year showed no negative trends in high radiation area postings or high radiation area access controls.

DER 1-1999-1106 dated April 15,1999, was written to document and investigate the l

' discovery of an unlocked radioactive source cabinet by a health physics supervisor during a routine tour. The source cabinet was used to store radioactive sources used for instrument source checks and was posted as a high radiation area. The maximum dose rate found in the cabinet was 80 mrem per hour at 30 cm from a source. Upon identification, a source inventory was conducted which showed that all sources were accounted for and the cabinet was locked. Actions taken to identify and correct the

..,m,,,.__..._.., . .

___..,,,m, , . , . , __. . , , . .,.

-x .

s'

• 27 deficient condition and to place the issue into the corrective action system were appropriate. The failure to maintain the posted high radiation area storage cabinet locked was contrary to radioactive source storage requirements in N1-RSP-1Q,

- Accountability of Calibration and Check Sources at NMP Unit 1," Rev. O. This violation is minor in nature and is not subject to formal enforcement action.

c. Conclusions Radiological controls for the Unit 1 refuel outage were effectively planned and implemented and focused on jobs with elevated exposure estimates, high dose rates, and radiologically complex work.

Radiological posting practices for access to radiation areas, high radiation areas, and airbome radioactivity areas were effective as evidenced by well defined boundaries and clear radiological postings. Some opportunities to enhance informational postings on the refuel floor that required " health physics notification prior to entry" beneath the drywell dome and reactor head insulation were identified.

Contamination monitoring requirements for access to the turbine building " Green Area" (clean area within the radiologically controlled area {RCA)) did not include an entire whole body frisk similar to the requirements for RCA exit. However, they were acceptable to minimize the risk for the spread and ingestion of significant amounts of radioactive contamination based on use of detailed procedures, restrictions on the personnel that could use the facility, and close health physics oversight.

Effective high radiation area controls were implemented as evidenced by clear radiological postings, use of locked doors when required, use of " Alarming" dosimetry, use of radiation work permits, use of remote door alarms, requirements for a minimum available exposure for access, and increased health physics oversight and monitoring for high radiation area entry.

R2 Status of RP&C Facilities and Equipment R2.1 Radioloaical Housekeepina (Units 1 and 2)

a. Insoection Scooe (83750 and 86750).

Plant tours were conducted to evaluate housekeeping and cleanliness and material conditions. Information was gathered through tours of Unit 1 and Unit 2 radwaste buildings and through the Unit i drywell, reactor and turbine building.

b. Observations and Findinos Housekeeping practices were effective as evidenced by clear isles and walkways, neatly stored tools and equipment, and well illuminated work areas. Material condition for the reactor and radwaste buildings were generally very good with painted floor and wall surfaces.

28

c. ' Conclusion Material conditions were good and housekeeping practices were effective as evidenced by clear aisles and walkwa.vs, neatly stored tools and equipment, and painted floor and wall surfaces.

R7 Quality Assurance in RP&C Activities i

' R7.1 Radiolooical Control Proaram Corrective Actions (Units 1 and 2)

a. Inspection Scope (86750)

A review was performed of the use of audits, appraisals, and DERs for the identification and resolution of deficiencies in the area of radiological controls and radwaste management and transportation. Information was gathered through discussions with cognizant personnel and selected reviews of quarterly self-assessments, audits, and DERs.

b. Observations and Findinos A combination of self-assessments and quality assurance audits were performed to meet the requirements for an annual review of radiation protection program content and implementation as required by 10CFR20.1101(c). Quarterly self-assessments were performed at each unit to identify trends in program areas. Self-assessments included a review of radiation exposure, radiological safety indicato.s, and radiation worker performance. Semi-annual self-assessments in radiological controls were performed to review common elements of Unit 1 and Unit 2 radiological controls programs. Quality assurance audits of radiological controls and radwaste management were often performed with the assistance of industry peers. Deficiencies arising from self-assessments and audits were addressed through the DER system and opportunities for improvement were maintained on an " Action Item List" for evaluation and review.

Two significant deficiencies were selected to evaluate the effectiveness of corrective actions:

DER 1-97-0762 dated March 16,1997, was written during the previous refuel outage at NMP1 (RFOi4) after an irradiated core component (tie-rod mid-support piece) was placed on the reactor cavity seal plate which resulted in elevated dose rates and an audible indication of increased dose rates on the upper drywell radiation monitor system.

Two individuals who were or may have been in the upper drywell at the time of the event were alerted by the audible indication of increased dose rates (increased chirping rate) and exited the drywell without receiving a significant exposure. Dose rates on the mid-support piece were subsequently measured to be 200 rem per hour on contact.

Immediate corrective actions included restriction of personnel access to the upper elevations of the drywell; relocation of the mid-support piece to the equipment pit; and issuance of a stop work order on the fuel floor. A root cause analysis identified multiple l barrier failures which resulted in the event including the failure to recognize and l

D,; .

_3 s E

,1 a;

29

communicate that the mid-support piece was an irradiated core component.

Preventative actions included revision of multiple procedures to include appropriate communications and radiological controls for restriction of personal access during movement of the mid-support piece and other irradiated core components.

Radiological control barriers that were in place at the drywell included 1) use of electronic alarming dosimetry,2) use of a remote radiation monitoring system with detector probes in the upper drywell, and 3) the health physics staff had a policy to evacuate the upper drywell if elevated radiation readings / alarms were received by the upper drywell radiation monitoring system. Total dose to the individuals electronic dosimetry for the drywell entry was 10.7 mrem and 29.5 mrem and the maximum dose rates measured by the worker's electronic dosimetry was 145.9 mrem per hour and 174.9 mrem per hour. These readings were typical for routine drywell entries. Maximum dose rates mearures by the radiation monitoring system averaged 159.3 mrem per hour.

Conservative exposure calculations estimated that the maximum dose rate to the head of an individual standing in the upper drywell would have been 978 mrem per hour.

A selected review of procedures verified that appropriate revisions had been made and interviews with cognizant personnel confirmed that personnel were knowledgeable of the ev'ent and actions to take for the movement of irradiated core components.

Technical Specification 6.11 " Radiation Protection Program" states that " procedures for personnel radiation protection shall be prepared consistent with the requirements of 10 CFR 20 and shall be approved, maintained and adhered to for all operations involving personnel radiation exposure." Procedure S-RAP-RPP-0801, *High Radiation Area Monitoring and Control" step 3.8.1 required the drywell 259 foot elevation ladder to be in place and locked to control access to the upper elevations of the drywell during the movement of an irradiated core component. Contrary to this requirement access-

- restrictions to the upper elevations of the drywell were not maintained on March 15,1997 during movement of a tie rod mid-support piece. This Severity Level IV violation is being treated as a Non-Cited Violation, consistent with Appendix C of the NRC Enforcement Policy (NCV 50-220/99-04-07). This violation was in the licensee's corrective action program as DER 1-97-0762.

DER 1-97-1346, dated April 29,1997, was also written during the previous refuel outage at Unit 1 after irradiated fuel was placed in the 198 cell rack adjacent to the spent fuel pool gate, resulting in unexpected elevated dose rates after drain down of 70 Rem per hour at contact with the bottom of the gate and 700 mrom per hour immediately above the spent fuel pool gates. Access to areas with elevated dose rates were appropriately controlled prior to and after the discovery of the elevated dose rates, no uncontrolled exposures resulted from the event, corrective and preventative actions were determined to be appropriate and no violations of NRC requirements were identified. An apparent cause evaluation determined that this event occurred because of a mis-commt'nication

. or understanding between engineering and reactor engineering personnel regarding precautions for moving spent fuel within six feet of the spent fuel gate. Additionally, there were. no procedures which specifically prevented the storage of fresh spent fuel adjacent to the spent fuel gate, immediate corrective actions included restriction of access to the i

r . .

30 area, establishing the area above the spent fuel gate as a high radiation area, and flooding the area between the spent fuel pool gates which dropped dose rates to 7 mrem per hour. Preventive actions included revision of procedures to require health physics notification prior to movement of the lower spent fuel gate shield plug (No. 9); to require shield plug No. 9 to be in-place for cavity work or to control the area as a locked high radiation area when the shield plug was not in-place; and a revision of fuel handling procedures to limit the storage of fuel in the 198 cell to only new fuel or spent fuel greater than one year old. A selected review of procedures during the inspection verified that appropriate revisions had been made and interviews confirmed that health physics personnel assigned to the refuel floor were knowledgeable of the event and actions to take when the lower spent fuel gate was moved.

c. Conclusions Self-assessments, audits, and the deficiency / event reporting system were effectively used to !dentify, evaluate, and resolve radiological control issues as evidenced by the conduct of multiple self-assessments and audits to satisfy the radiation protection program review requirements in 10CFR20.1101(c). ' Appropriate corrective actions and controls to prevent unplanned exposures were implemented as a result of previous deficiencies.

One non-cited violation was identified associated with the failure to maintain access restrictions to the upper elevation of the drywell during movement of an irradiated core component on March 15,1997.

V. Management Meetings X1 Exit Meeting Summary The inspectors presented the inspection results to members of the licensee management at the conclusion of the inspection on May 27,1999. The licensee acknowledged the findings presented.

1

)

g#

l, .

ATTACHMENT 1 PARTIAL LIST OF PERSONS CONTACTED Nianara Mohawk Power Corporation D. Bosnic Manager, Operations, Unit Two

' S. Doty Manager, Maintenance, Unit One N. Paleologos Plant Manager, Unit Two -

F. Fox Acting Manager, Maintenance, Unit Two R. Smith _ Plant Manager, Unit One N. Rademacher Manager, Quality Assurance

= D. Topley Manager, Operations, Unit One INSPECTION PROCEDURES USED IP 37550 Engineering IP 37551_. On-Site Engineering

- IP 61726 Surveillance Observations IP 62707 Maintenance Observations IP 71707 Plant Operations IP 71750' Plant Support IP 73753 Inservice Inspection IP 83750 Occupational Radiation Exposure IP 86750- Solid Radweste Management and Transportation of Materials IP 90712 - In-Office Review of Written Reports of Non-Routine Events at Power Reactor Facilities

- IP 92700 Onsite Follow-up of Written Reports of Non-Routine Events at Power

. Reactor Facilities IP 92904 Followup - Plant Support -

ITEMS OPENED. CLOSED. AND UPDATED OPENED 50-410/99-04-01 NCV Automatic Depressurization System Nitrogen Leakage in Excess of NMP2 Technical Specifications Surveillance Limits.

50-410/99-04-02 NCV Unit 2 Reactor Core Isolation Cooling System Failure During Reactor Scram Transient.

50-220/99-04-03 IFl Computer Security for 3D Monicore.

50-220/99-04-04 NCV Failure to Maintain Core Thermal Limits as Required by Technical Specifications.

. 7 Attachment 1 2 50-220/99-04-05 NCV Failure to Conduct the Required ASME Code Inspections.

50-410/99-04-06 NCV Failure to Ensure that the Service Water intake De-Icing Capability is Available During a Control Room Fire.

50-220/99-04-07 NCV Radiological Control Program Correction Actions.

CLOSED i

50-410/99-04-01 NCV Automatic Depressurization System Nitrogen Leakage in Excess of NMP2 Technical Specifications Surveillance Limits.

50-410/99-04-02 NCV Unit 2 Reactor Core Isolation Cooling System Failure During Reactor Scram Transient.

50-410/98-05-02 VIO Failure to Conduct Surveillance Test on Batteries.

50-220/98-02-05 VIO Inadequate Plant Impact in Work Order Package.

50-220/99-04-04 NCV Failure to Maintain Core Thermal Limits as Required by Technical Specifications.

50-220/99-04-05 NCV Failure to Conduct the Required ASME Code inspections.

50-410/98-05-03 - VIO Failure to Perform Adequate Design for Emergency Diesel Generator Modification on Fuel Line. ,

50-410/99-04-06 NCV Failure to Ensure that the Service Water intake De-icing Capability is Available During a Control Room Fire.

50-410/98-19-03 IFl Leakage of Contaminated Water Following a Scram Reset.

50-220/98-16-02 VIO Failure to identify and Promptly Correct a Condition Adverse to Quality.

50-220/99-04-07 NCV Radiological Control Program Correction Actions.

50-410/98-26 LER Seismic Monitor Inoperable for More than Thirty Days and Special Reports Not Submitted.'

50-410/99-03 LER ADS Nitrogen Leakage in Excess of Unit 2 Technical Specifications Surveillance Limits.

50-220/99-03 LER NMP1 Thermal Limit Exceeded the Requirements of Technical Specifications.

4 Attachment 1 3 50-220/98-19 LER Missed ASME Section XI Inservice Inspection Due to Cognitive Error, 50-410/99-02 LER Missed Technical Specification Channel Functional Test of the Recirculation Flow Upscale Rod Block.

50-410/99-04 LER NMP2 Service Water intake De-Icing Heater Contro' Circuits do Not Meet Fire Protection Program Requirements.

1 l

i

L ., l

.l\ .

Attachment 1 .4. l LIST OF ACRONYMS USED AC Alternating Current ADS- Automatic Depressurization System ALARA As Low As is Reasonably Achievable APLHGR Average Planar Heat Generation Rate I

-APRM Average Power Range Monitor ASME American Society of Mechanical Engineers BWRVIP Boiling Water Reactor Vessel and Intemals Project CDF Core Damage Frequency CFR Code of Federal Regulations CRD - Control Rod Drives DAW Dry Active Wastes DC Direct Current DCP Design Change Package DER Deviation / Event Report DOT- Department of Transportation

dpm Disintegration Per Minute EC Emergency Condenser E/C ' Erosion / Corrosion ECCS Emergency Core Cooling System ECS Emergency Cooling System EDG Emergency Diesel Generators EP Emergency Preparedness EPA . Electric Protection Assemblies 1 ESA Engineering Supporting Analysis ESF Engineered Safeguards Feature ESL Equipment Status Log FCV Flow Control Valve -

FWBP Feedwater Booster Pump GAP Generation Administration Procedure HPCI .High Pressure Core injection HPCS High Pressure Core Spray IFl- _ inspector Followup item IGSCC Intergranular Stress Corrosion Cracking ,

IR inspection Report

' ISEG - Independent Safety Engineering Group ISI In-Service inspection LCO' Umiting Condition for Operation LER Licensee Event Report LOCA Loss of Coolant Accident LPRM Local Power Range Monitor LSFT Logic System Functional Test APRM Local Power Range NCV Non Cited Violation -

-NDE Nondestructive Examination NMPC- Nine Mile Point Corporation p

33 3.* ~

, av LAttachment 1 5 NRC Nuclear Regulatory Commission PRA, Probability Risk Analysis PRNM Power Range Neutron Monitor -

QA- Quality Assurance RCA- Radiological Controlled Area .

RCIC Resctor Core Isolation Cooling RF015- Refusiing Outage RPS Reactor Protection System RWP

• Radiation work Permii SDV Scram Discharge Valve SORC. Station Operating Review Committee SRV - Safety Relief Valve SSS Station Shift Supervisor TIP Traverse incore Probe TS Technical Specification USAR Updated Safety Analysis Report Unit 1 Nine Mile Point Unit 1 Unit 2 Nine Mile Point Unit 2 UPS Uninterruptible Power Supply

-UT Ultrasonig.

WO - Work Order W

4 e

U _

qe_ -e 1 g>R 40uq

• UN:TED STATES j t

[e r,,

g NUCLEAR REGULATORY COMMISSION AEGION I

]'

"l 475 ALLENDALE ROAD g,4 KING OF PRUSSIA. PENNSYLVANIA 19406-1415

• • ..+ l August 3, 1999 j l

'Mr. John H. Mueller Chief Nuclear Officer Niagara Mohawk Power Corporation l

' Nine Mile Point Nuclear Station Operations Building,2nd Floor P.O. Box 63 Lycoming, NY 13093

SUBJECT:

NRC INTEGRATED INSPECTION REPORT NOS. 50-220/99-05 AND 50-410/99-05

Dear Mr. Mueller:

This report transmits the findings of safety inspections conducted by NRC inspectors at the Nine Mile Point Nuclear Station, Units 1 and 2, from May 9, through June 19,1999. At the conclusion of the inspection, the findings were discussed with members of your staff.

During the six-week inspection period covered by this report, operation of the Nine Mile Point Nuclear Station reflected an acceptable safety focus. Emergent work during the Unit 1 outage, including the core shroud weld and tie rod repairs and recirculation piping weld examinations were well controlled from a safety perspective. At Unit 2, additional reactor core isolation cooling (RCIC) system performance deficiencies were identified by your staff during on-line maintenance. We are concemed with the quality of work performed by the maintenance, engineering and operations staffs which contributed to the RCIC system degradation. Although the specific issues were resolved, the underlying causes wanant additional attention.

In accordance with 10CFR2.790 of the NRC's " Rules of Practice," a copy of this letter and its enclosures will be placed in the NRC Public Document Room.

Sincerely, NJte 4 som i Michele G. Evans, Chief Projects Branch 1 Division of Reactor Projects l

Docket Nos. 50-220,50 410 i I

l License Nos. DPR-63, NPF-69 Enclosurei NRC Inspection Report Nos. 50-220/99-05 and 50-410/99-05 4

4

g%

John H. Mueller --- 2- .

cc w/ encl:

G Wilson, Senior Attorney -

M. Wetterhahn, Winston and Strawn J. Rettberg, New York State Electric and' Gas Corporation -

P. Eddy, Electric Division, Department of Public Service, State of New York C Donaldson, Esquire, Assistant Attorney General, New York Department of Law J. Vinquist, MATS, Inc.

F. Valentino, President, New York State Energy Research and Development Authority J. Spath, Program Director, New York State Energy Research ,

and Development Authority l

1

)

l

y ..

7..

E John H. Mueller 3.

Distribution w/ encl:

H. Miller, RA/J. Wiggins, DRA (1) '

' M. Evans, DRP W. Cook, DRP R. Junod, DRP Region I Docket Room (with concurrences)

Nuclear Safety information Center (NSIC)

PUBLIC

. NRC Resident inspector Distribution w/ encl: (VIA E-MAIL)

M..Tschiltz, RI EDO Coordinator E. Adensam, NRR S. Bajwa, NRR D. Hood, NRR l M. Campion, RI l Inspection Program Branch (IPAS) l R. Correia, NRR DOCDESK-l I

1 DOCUMENT NAME: G:\ BRANCH 1\NMPSTUFF\NMP9905.WPD To receive a copy of this document, Indicale in the box: "C" = Copy without attachment / enclosure T = Copy with attachment / enclosure "Sf"= No copy OFFICE - Rl/DRP lE Rl/DRP lE NAME GHunegs MA(, MEvans /tudb L DATE 07/29/99 08/9/99 l OFFICIAL RECORD COPY l

l l

L

t .

x,,

U.S. NUCLEAR REGULATORY COMMISSION REGION I Docket / Report Nos.: 50-220/99-05 50-410/99-05 License Nos.: DPR-63 NPF-69 Licensee: Niagara Mohawk Power Corporation P. O. Box 63 Lycorning, NY 13093 Facility: Nine Mile Point, Units 1 and 2 Location: Scriba, New York Dates: May 9,1999 to June 19,1999 Inspectors: G. K. Hunegs, Senior Resident inspector R. A. Fernandes, Resident inspector R. A. Skokowski, Resident inspector J. E. Carrasco, Reactor Engineer Approved by: Michele G. Evans, Chief Projects Branch 1 Division of Reactor Projects Qg I} 0&0-,.

a p .. l EXECUTIVE

SUMMARY

Nine Mile Point Units 1 and 2 50-220/99-05 & 50-410/99-05 May 9,1999 - June 19,1999 This integrated inspection report includes aspects of licensee operations, engineering, maintenance, and plant support. The report covered a six-week period of resident inspection and the results of an inservice inspection review.

Qoerations Unit 1 core reload was well performed with good communications, independent verification, and procedure use noted. (01.2)

The reactor restart from the Unit i refueling outage was conducted in a conservative, well controlled manner. Effective supervision and oversight was provided by senior management.

(01.3)

Maintenance The reactor core isolation cooling (RCIC) system trip encountered during surveillance testing was the result of a poorly developed system flushing methodology. The subsequent on-line RCIC system maintenance outage was not effectively and efficiently executed to ensure the system unavailability time was minimized. NMPC's root cause determination for the RCIC turbine trip was reasonable and the corrective actions appropriately implemented and documented in the associated deficiency event reports. (M1.2)

Acceptable control of the technical details and appropriate oversight of the contractor performing the non-destructive examinations (NDE) of the core shroud at Unit 1 was noted. The contractor ,

used state-of-the-art ultrasonic technology to detect and size weld indications and cracks. The contractor used acceptable means for the interpretation of the NDE data and the NDE personnel were determined to have been properly certified. (M2.1)

During the refue!!ng outage for Unit 1, appropriate reviews of the indications detected in the recirculation piping safe-end to elbow and nozzle to safe-end welds were performed. (M2.2)

During the Unit i reactor vessel hydrostatic test, a leak developed in the reactor vessel bottom head drain line. The cause was determined to be thermal stress induced fatigue which was caused by a system valve packing leak onto the adjacent downstre.m piping. The inspectors noted that the valve packing leakage was a long-standing material condition problem, the consequence of which was not fully recognized until the crack was identified, analyzed, and repaired. NMPC's corrective actions were acceptable. (M2.3) ii i

s, Executive Summary (cont'd) ,

Enaineerina inspection of core shroud vertical and horizontal weld inspections at Unit i showed that required structural margins were satisfied. However, inspection results for the V10 weld showed some crack depth change. NMPC decided to pre-emptively repair the V9 and V10 welds using a contingency repair which was previously approved by the NRC. The installation of the repair clamp was well controlled. (E1.1)

A core shroud tie rod upper spring assembly repair at Unit 1 was well conducted. A team approach to develop a repair plan, good utilization of mock-up training land good radiological controls practices were noted by the inspectors. (E1.2)

On May 18, while performing work on the Unit i refuel floor, the reactor building hoist trolley connection failed. The apparent cause of the failure was fatigue of the threaded rod connection.

Previously conducted crane inspections were not sufficient to identify the equipment degradation and long-term corrective actions from a February 1988 failure had not been effective. (E1.3)

Plant Sucoort Radiological controls during the Unit 1 outage were good. Protective clothing, dosimetry and radiological posting requirements and radiation protection technician oversight were effective in minimizing personnel exposure. (R1.1) l iii l

f . . -

1 l

J o ' TABLE OF CONTENTS ~

Page EXECUTIVE

SUMMARY

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i1 TABLE OF CONTE NTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Summary of Plant Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 l

I . Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 i O1 . Cenduct of 0perations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 01.1 General Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 O1.2 . Core Reload Activities (Unit 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 01.3 ' Post Outage Startup (Unit 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

'08 Miscellaneous Operations issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 l 08.1 (Closed) Licensee Event Report (LER) 50-410/99-05 . . . . . . . . . . . . . 2 1 i

l l . M aintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 l M1 Conduct of Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 M1.1 General Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 ,

M1.2 Reactor Core Isolation Cooling (RCIC) System Maintenance (Unit 2) . 3 l M2 Maintenance and Material Condition of Facilities and Equipment . . . . . . . . . . 5 )

. M2.1_ ~ Inspection of Core Shroud Vertical Welds (Unit 1) . . . . . . . . . . . . . . . . 5 '

M2.2 Recirculation Piping Weld Examinations (Unit 1) . . . . . . . . . . . . . . . . . 7

- M2.3 Reactor Vessel Bottom Head Drain Line Leak (Unit 1) . . . . . . . . . . . . . 8 ,

i l l i . Engineering . _. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 l E1 Conduct of Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 I E1.1 Core Shroud Vertical Weld Repair (Unit 1) . . . . . . . . . . . . . . . . . . . . . . 9  ;

E1.2 Core Shroud Tie Rod Upper Spring Assembly Repair (Unit 1), . . . . . 10  ;

E1.3 - Reactor Building Crane Auxiliary Hoist (Unit 1). . . . . . . . . . . . . . . . . 11 E8 Miscellaneous Engineering issues . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . 12  ;

E8.1 (Closed) LER 50-410/99-01 Supplement 1 . . . . . . . . . . . . . . . . . . . . . 12 l E8.2 Review of Year 2000 Program and implementation . . . . . . . . . . . . . 12 '

IV. Plant Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 12 R1 Radiological Protection & Chemistry Controls . . . . . . . . . . . . . . . . . . . . . . . . . 12 R1.1 Refuel Outage Radiological Controls (Unit 1) . . . . . . . . . . . . . . . . . . . 12

. V. Management Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 X1 Exit Meeting Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 iv

s, Table of Contents (cont'd) ' .

ATTACHMENTS .

Attachment 1 - Partial List of NMPC Persons Contacted

-Inspection Procedures Used

- Items Opened, Closed, and Updated

- List of Acronyms Used 5'

V

,/ .

ReDort Details Summary of Plant Status l Nine Mile Point Unit 1 (Unit 1) began the inspection period in cold shutdown in a scheduled refueling outage. Unit i restarted on June 14. The plant was at 80% power by the end of the  ;

inspection period. Major outage activities, in addition to refueling, included the repairs of the core shroud vertical welds and core shroud tie rod, inspection of the reactor vessel beltline, l replacement of two feedwater heaters, and modification of the emergency core cooling system suction strainers.

Nine Mile Point Unit 2 (Unit 2) began the period at 65 percent power following a forced outage and subsequent single recirculation loop operation. On May 9, Unit 2 was returned to two loop operation and reached 100 percent power on May 11. The unit remained at 100 percent power through the remainder of the inspection period,

l. Operations 01 Conduct of Operations '

01.1 GeneralComments (71707)

Using NRC Inspection Procedure 71707, the resident inspectors conducted frequent reviews of ongoing plant operations. The reviews included tours of accessible areas of both units, verification of engineered safeguards features (ESF) system operability,

. verification of adequate control room and shift staffing, verification that the units were operated in conformance with Technical Specifications (TSs), and verification that logs and records accurately reflected equipment status. In general, the conduct of operations was professional and safety-conscious.

01.2 Core Reload Achvities (Unit 1)

' a. Inspection Scooe (71707)

The inspectors observed portions of the core reload to verify that fuel movements were done in accordance with station procedures and Technical Specifications.

b. Observations and Findings The core reload was performed in accordance with fuel handling procedures N1-FHP-278, Whole Core Reload, and N1-FHP-25, General Description of Fuel Moves. The inspector observed fuel handling operations from the refuel floor, as well as, the control room. The operators utilized good three-way communications and independent verification during the process of reloading the core. Verification of fuel moves was

• Topical headings such as o1, M8, etc., are used in accordance with the NRC standardized reactor inspection report outline.

Individual reports are not expected to address all outline topics. The NRC inspection manual procedure or temporary instruction that was used as inspection guidance is listed for each applicable report section.

g 2

independently performed on the refuel bridge, as well as, step verification from the control rcom.

During the core reload, operators noted that one of the two refueling mast cables was in a degraded condition and ceased fuel moves. The cabling and cable handling j equipment was inspected and subsequently repaired. The inspector reviewed the work i order and post work testing and found them to be acceptable. The 'nspector noted that l the discovery of the degraded cable was good and the repair was completed satisfactorily,

c. Conclusions Unit 1 core reload was well performed with good communications, independent verification, and procedure use noted.

01.3 Post Outaae Startuo (Unit 1)

a. Inspection Scope (71707)

The inspectors observed reactor startup activities following the refueling outage. This review included the conduct of operations, resolution of plant problems, and management oversight.

b. Observations and Findings The reactor startup was conducted in a conservative, well controlled manner. Pre-evolution briefs were thorough and a safety focus was emphasized. Operators were aware of the status of testing and properly addressed identified deficiencies. During the approach to criticality, the reactor went critical on a control rod double notch. Operators responded appropriately by inserting the control rod and changing the pull sheet to continue the startup. Throughout the reactor restart evolution, senior Niagara Mohawk Power Corporation (NMPC) managers provided oversight of activities.

c. Conclusions The reactor restart from the Unit 1 refueling outage was conducted in a conservative, well controlled manner. Effective supervision and oversight was provided by senior management.

08 Miscellaneous Operations issues (92700) 08.1 (Closed) Licensee Event Report (LER) 50-410/99-05: Reactor Trip Due to a Main Generator Protection Volts / Hertz Relay Failure. The technical issues associated with this LER were described in NRC inspection report 50-410/99-04, Sections 01.2, M2.2,-

and E1.1. The inspectors completed an on site review of the LER and verified that the report was completed in accordance with the requirements of 10CFR50.73. Specifically, the description and analysis of the event as documented in the LER were consistent with

j .

,r 3.

the inspectors' understanding of the event. The root cause and corrective and preventive actions as described in the LER were reasonable.

The inspector noted that the reactor core isolation cooling (RCIC) system post-maintenance and surveillance testing following the 1998 outage did not identify that the mechanical linkage for the turbine trip and throttle valve was misadjusted. Plant staff

. troubleshooting revealed that the trip throttle valve overspeed trip mechanism was improperly set-up to ensure proper long-term engagement of the trip hook and latch lever (reference Non-Cited Violation 50-410/99-04-02). Because of the misadjustment, the trip latch was only nominally engaged, but satisfactorily functioned during testing.

However, during the event the excessive engagement tolerance coupled with normal system operating vibration caused the trip throttle valve to unlatch and close. NMPC has revised the periodic RCIC test procedure to include the oroper trip mechanism tolerances and verification of proper trip latch engagement. This LER is closed,

11. Maintenance M1 Conduct of Maintenance M1.1 General Comments (61726. 62707)

Using NRC Inspection Procedures 61726 and 62707, the resident inspectors periodically i observed various maintenance activities and surveillance tests. As part of the )

observations, the inspectors evaluated the activities with respect to the requirements of the Maintenance Rule, as detailed in 10CFR50.65. In general, maintenance and surveillance testing activities were conducted professionally, with the work orders (WOs) and necessary procedures in use at the work site, and with the appropriate focus on safety. Specific activities and noteworthy observations are detailed in the inspection report. The inspectors reviewed procedures and observed all or portions of the following maintenance / surveillance activities:

• WO 99-08931, Gas Treatment System o Surveillance Test (ST) Q27, Reactor Building' Closed Loop Cooling Check Valve Operability Test e ST M3, Suppression Pool Drywell Relief Valve Exercise e N1-PM-V7, Turbine Trip Test

• N2-OSP-ICS-QOO2, Reactor Core isolation Cooling Test M1.2 Reactor Core Isolation Coolina (RCIC) System Maintenance (Unit 2)

a. Inspection Scope (61726)

During routine testing to support retuming the RCIC system to service following an on-line maintenance outage, the turbine driven pump tripped on low suction pressure following the changing of the pump's water supply. NMPC remained in the fourteen-day limiting condition for operation (LCO) outage to evaluate, troubleshoot, and make any necessary repairs to address this issue. The inspectors reviewed NMPC's activities to

4 evaluate the effectiveness of corrective actions and to ensure that the system was being tested and operated consistent with station procedures.

b. Observations and Findenas The portion of the test that was being performed at the time of the RCIC system trip involved the swapping of the water supply (pump suction) from the suppression pool back to the condensate storage tank (CST). The system suction is normally aligned to the CST and low water level in the CS T causes the system to automatically swap-over to

' the suppression pool. This function was tested successfully. However, to flush the system with clean water, the test procedure directed the operators to re-align the suction back to the CST. Shortly after opening the suction valve the RCIC pump tripped on low suction pressure.

NMPC assembled a few teams to investigate the issues surrounding this RCIC system trip. Troubleshooting included: instrument venting and calibration; installation of system performance monitoring equipment; test procedure changes and additional testing requirements; and inspection of several system check valves. NMPC's investigation determined that because of system configuration, voids formed in the suction piping from the CST while the RCIC pump was aligned and drawing water from the suppression pool. Upon suction swap-over back to the CST, the voids collapsed and caused a rapid pressure transient. This pressure transient dropped low enough to cause the pump to trip on low suction pressure. NMPC determined that the pressure transient was further amplified by the unsatisfactory performance of a check valve in the suction piping of the keep-fill pump. The inspectors concluded that NMPC's root cause determination was reasonable and that the associated Deficiency Event Reports (DERs) properly documented the results and corrective actions. However, it appeared that the licensee introduced this RCIC system problem via a pooriy researched and reviewed surveillance test procedure change for flushing the system piping using the CST water.

The inspectors noted that the control room operators made a 10 CFR 50.72 notification (Event No. 35706) on May 12,1999, identifying a preliminary determination that the RCIC system was inoperable because of the system trip on suction swap-over during testing. The licensee subsequently determined that a successful suction swap-over from the suppression pool back to the CST was not a system design requirement. Changes were made to the surveillance procedure to perform an attemate method of flushing system piping after pumping water from the suppression pool. Consequently, the licensee concluded that the RCIC system was not inoperable as a result of the trip on suction swap-over from the suppression pool to the CST. On June 9,1999, NMPC retracted their May 12,1999, event notification. The inspectors reviewed the basis for the retraction and found it acceptable.

The inspectors observed that the operators experienced difficulty in performing post-work testing after system restoration from the intemals inspections of the RC!C system check valves. During the post-work test, the RCIC pump lost flow and was manually tripped from the control room. Subsequent review and investigation by NMPC determined that the system piping was not adequately filled and vented. Licensee

y .

/

5 investigation identified that the system operating procedure did not provide adequate direction for filling the system following an extensive system breach. The inspectors i

concluded that, in addition to the procedural inadequacies, the work control process could have been more thorough with regards to system restoration following this type of intrusive maintenance.

The inspectors noted that the licensee used 12 days of the available 14-day LCO action statement to complete the necessary maintenance and restore the RCIC system to an operable status. The licensee's intemal guidance recommends on-line maintenance be limited to 50 percent of the available LCO time, to account for any unforeseen contingencies. Although the RCIC system suction swap-over trip and subsequent check valve intemals inspections contributed to the lengthening of the outage, these events occurred early in the LCO outage window and the 50 percent target was not achieved.

The inadequate system refill and venting contributed to this delay. Accordingly, the licensee's processes for efficiently and effectively resolving these RCIC system problems appeared to have been challenged, and thus adversely impacted the availability of a system important to safety. The plant management acknowledged this observation and shared the inspectors' concem for safety system availability.

Subsequent to this inspection period, additional issues were identified with the RCIC system following a June 24,1999 automatic reactor shutdown. NRC review of these issues will be documented in NRC IR 50-220 & 50-410/99-06.

c. Conclusions The reactor core isolation cooling (RCIC) system trip encountered during surveillance testing was the result of a poorly developed system flushing methodology. The subsequent on-line RCIC system maintenance outage was not effectively and efficiently i executed to ensure the system unavailability time was minimized. NMPC's root cause )

determination for the RCIC turbine trip was reasonable and the corrective actions appropriately implemented and documented in the associated deficiency event reports.

M2 Maintenance and Material Condition of Facilities and Equipment M2.1 Insoection of Core Shroud Vertical Welds (Unit 1)

a. Insoection Scope and Backaround (73753)

The inspector reviewed and assessed the adequacy of the in-Service Inspection (ISI) examinations of the vertical welds of the core shroud during refueling outage 15 (RF015).

The core shroud is a stainless steel cylinder that surrounds the active core and provides a barrier to separate the upward flow of coolant through the core from the downcomer feedwater inlet and recirculation flow. A loss of structuralintegrity of the core shroud could potentially result in the loss of core geometry and inability to maintain proper alignment of the fuel. The event that could trigger this consequence is a main steam line ;

6 bfeak accident and the complete failure of shroud horizontal welds H4 and H5 and vertical welds Vg and V10. This event coupled with a seismic event could potentially cause a deflection of the fuel rods, which may prevent rod insertion.

b. Observations and Findinas At Unit 1, the core shroud horizontal and vertical welds have been inspected and determined to have intergranular stress corrosion cracking (IGSCC) in and near the heat-affected zone (HAZ)'of the welds. To address the horizontal weld cracking, the NRC staff reviewed and approved the licensee's attemative repair method involving the installation of core shroud stabilizer assemblies (tie-rods). With the tie-rods installed, the licensee is no longer obligated to examine the horizontal welds per their ISI program.

The ISI examination interval for the vertical welds was established based on NMPC engineering analysis of the existing cracks and consideration for potential crack growth.

The inspector verified that the licensee had completed the scanning of all the vertical welds and the pre-selected intersections between the vertical and the horizontal welds.

The inspector observed some of the data interpretation performed by the contractor.

The inspector also reviewed the results of the ultrasonic (UT) examination and the comparison of these results to the UT examination results of the previous refueling outage (RF014). During RFO14, the contractor (GE) used the Smart 2000 computerized data acquisition and imaging system, and a multiple probe in a single housing that utilized a 45 degree shear, a 60 degree longitudinal, and a creeping wave. During RFO15, the contractor (Framatome Technologies) used the Accusonex computerized data acquisition system and probes consisting of 45 degree,60 degree, and 80 degree.

During RFO14, the shroud ring vertical welds (outside surfaces) were inspected using enhanced visual techniques (EVT1). Because the vertical welds were machine flashed, some of the welds were not located and consequently not inspected (i.e., welds V15 and V16). However, NMPC did commit to the NRC staff to develop a technique to locate these welds and inspect them during RFO15. Of the welds inspected in RFO14 using EVT1 methodology only, no cracks were identified. During RFO15, NMPC satisfied their commitment. Using ultrasonic testing (UT) methodology, examination data of the accessible segments of shroud ring vertical welds V1, V2, VS, V6, V13, V14, V15, and V16 showed no cracks.

The inspector observed that during RF014, vertical welds V7 and V8 were UT inspected with a coverage of about 50% of the weld length and the results showed that there were no indications. Re-examination during RF015 identified no cracking. Vertical welds V3 i and V4 were UT inspected in RFO14 with an inside diameter crack identified in weld V4.  !

This V4 crack was analyzed and dispositioned as acceptable per Boiling Water Reactor Vessel Inspection Program (BWRVIP) criteria. Vertical weld V3 examination results showed a few smallindications that were dispositioned as acceptable. During RFO15, re-examination of welds V3 and V4 with better UT coverage identified acceptable results, i Vertical welds Vg and V10 were UT and EVT1 examined (both inside and outside  !

diameter) in RFO14. The cracks identified in these welds were analyzed and determined 1

1

.-[

/

7 to be acceptable for one cycle of operation. During RFO15, these welds were re-examined via UT and V9 had only minor changes, compared to RFO14. However, weld V10 demonstrated a significant change in depth. The average crack growth of V10 was determined to be 0.25 inch. This translated into a crack growth rate of 1.72E-5 inch /hr, which was less than the specified 2.2E-5 inch /hr NMPC acceptance criterion. While a crack growth rate of 1.72E-5 inch /hr would have been acceptable for one more cycle of operation, NMPC conservatively decided to repair vertical welds V9 and V10.

The inspector noted that the licensee used UT to examine intersections between horizontal and vertical welds for welds V9, V10, V3, and V4. In addition, 6 to10 inches of the base metal was examined to ensure the quality of the base metal on each side of the verticalwe!ds inspected.

c. Conclusions .

1 Acceptable control of the technical details and appropriate oversight of the contractor performing the non-destructive examinations (NDE) of the core shroud at Unit 1 was noted. The contractor used state-of-the-art ultrasonic technology to detect and size weld indications and cracks. The contractor used acceptable means for the interpretation of the NDE data and the NDE personnel were determined to have been properly certified.

V2.2 Recirculation Pioina Wald Examinations (Unit 1)

a. Insoection Scoce and Backaround (73753)

In 1983, the recirculation piping was replaced at Unit 1 due to extensive intergranular .

stress corrosion cracking (IGSCC) in pipe welds and safe-ends.. The cause of cracking j was determined to have been an aggressive water chemistry environment along with weld and fumace sensitized stainless steel components and weld residual stresses.

During this inspection, the inspector assessed the RFO15 ultrasonic inspections performed on reactor recirculation system (RRS) pipe welds. The inspector reviewed the pertinent drawings and records and conducted interviews with ISI and engineering personnel engaged in the NDE of the reactor recirculation piping welds,

b. Observations and Findinas The inspector noted that two safe-end to elbow welds (32-WD046, loop 12 and 32-WD086, loop 13) were identified with circumferential indications near the weld root that exceeded the acceptance criteria in the American Society of Mechanical Engineers (ASME) Code,Section XI, paragraph IWB-3514.3. As required by the ASME Code, the licensee performed expanded scope inspections of RRS pipe welds and identified rejectable indications in two additional welds (32-WD126, loop 14 and 32-WD168, loop 15). The inspector verified that these rejectable weld indications were properly reported in Deviation / Event Report (DER) No.1-1999-1255, dated May 13,1999. The inspector determined that the disposition of this DF.R also addressed welds 126 and 168, which were reported under DERs 1-1999-1411 and 1-1999-1559, respectively.

I

I; *

• u' \

)

i i

8 Following the identification of these rejectable indications, NMPC performed a review of the weld inspection history. As documented in DER 1-1999-1255, the 1983 replacement fabrication records were examined to determine the extent and location of repairs in these welds. Based on this records examination and comparison with the new UT data, NMPC concluded that these indications were lack of fusion from prior repairs and none were indication of IGSCC. Alternatively, these indications were characterized as

]

construction induced, not service induced. Accordingly, these rejectable indications {

were evaluated and determined

• accept-as-is," in accordance with the criteria contained in the ASME Code,Section XI, Subsection lWB 3600. The inspector confirmed that NMPC plans to submit to the NRC the results of the analysis associated with the acceptability of the safe-end to elbow indications, in accordance with the reporting requirements of ASME Code,Section XI, Subsection IWB-3600.

During RFO15, NMPC performed UT examinations of the safe end to nozzle welds and identified indications on one RRC pipe suction nozzle. NMPC dispositioned these indications as " acceptable" per ASME Code,Section XI, Paragraph IWB 3500.

c. Conclusions During the refueling outage for Unit 1, appropriate reviews of the indications detected in the recirculation piping safe-end to elbow and nozzle to safe-end welds were performed.

M2.3 Reactor Vessel Bottom Head Drain Line Leak (Unit 1)

a. Inspection Scope During the performance of the reactor vessel hydrostatic test, a leak was identifed in the reactor vessel drain line. The inspector performed a partial system walkdown, discussed the leakage with NMPC personnel and reviewed the corrective actions.

b. ObservaV201_and Findinas On June 6, during the vessel hydrostatic test, a leak was identified in the reactor vessel bottom head drain line downstream of the manualisolation valve. The leak was from a crack located on the top of the pipe approximately one inch from the pipe to valve socket weld. The vessel hydrostatic test was secured and the plant was depressurized. NMPC installed freeze seals to facilitate removal and replacement of the affected section of pipe.

A vendor laboratory analysis showed that the crack was typical of fatigue cracking. The cracking was concentrated on the outside diameter surface on the top of the piping. In addition, it was determined that poor weld fit up contributed to high stress at the weld.

The cracking was caused by the direct surface exposure of the pipe to leakage from the adjacent manual isolation valve packing. Review of operational history identified that the manual isolation valve had exhibited packing leakage during several operational cycles.

The long-time leakage onto the pipe was evidenced by the discoloration and deposits

v  ;. .

,/

9 built-up on the pipe surface. In hindsight, this valve packing leak had not been appropriately addressed.

NMPC documented their corrective actions in DER 99-1907. A walkdown was performed of the remaining sections of the drain line piping and no discrepancies were identified. A temporary modification was installed to shield the new piping from possible future packing leakage from the adjacent valve. From a risk perspective, the NRC staff concluded that a catastrophic break in the drain line (at power) would be significant. In ,

particular, any efforts to isolate the postulated pipe break would be difficult, if at all j possible, due to the only isolation valve upstream of the postulated break being manually operated. Absent a means to isolate this postulated pipe break, long-term reactor water inventory control may have to be achieved via containment flood-up.

c. Conclusions During the Unit i reactor vessel hydrostatic test, a leak developed in the reactor vessel bottom head drain line. The cause was determined to be thermal stress induced fatigue which was caused by a system valve packing leak onto the adjacent downstream piping.

The inspectors noted that the valve packing leakage was a long-standing material condition problem, the consequence of which was not fully recognized until the crack was identified, analyzed, and repaired. NMPC's corrective actions were acceptable.

Ill. Enaineerina E1 Conduct of Engineering E1.1 Core Shroud Vertical Weld Reoair (Unit 1)

a. Insoection Scope (37551)

The inspector reviewed the safety evaluation related to the attemative repair of the core shroud vertical welds. Portions of the electric discharge machining (EDM) process and installation of the clamp were observed and the post repair inspection plan and results were reviewed.

b. Observations and Findinas During the 1997 refueling outage, NMPC identified that some vertical welds joining sections of the cylindrical stainless steel reactor core shroud were cracked. Core shroud weld inspections which were conducted this outage showed that vertical weld V9 remained essentially unchanged from the previous outage and some crack growth was evident for weld V10. NMPC concluded that the crack growth rate was consistent with ,

their previous analyses and that tie reactor core shroud continued to be structurally

{

sound. (See Section M2.1) i l

1

10 Based on the results of the examination of the reactor core shroud and analysis, NMPC determined that shroud vertical weld repairs were warranted. Contingency shroud vertical weld repair plans were submitted to and approved by the NRC in a letter dated April 30,1999. The repair is a clamp assembly consisting of a plate with attached pins that are inserted into holes, machined in the shroud by an EDM process on both sides of the vertical weld. The clamps bndge across the flawed vertical weld. Two clamps each were used for the V9 and V10 welds. ' Procedures, quality assurance oversight and controls were sufficient to support proper installation of each repair clamp.

c. Conclusions inspection of core shroud vertical and horizontal weld inspections at Unit 1 showed that required structural margins were satisfied. . However, inspection results for the V10 weld showed some crack depth change. NMPC decided to pre emptively repair the V9 and V10 welds using a contingency repair which was previously approved by the NRC. The installation of the repair clamp was well controlled.

E1.2 Core Shroud Tie Rod Uooer Sorina Assembly Reoair (Unit 1)

a. . Insoection Scone (37551)

During routine inspection of the core shroud tie rod assemblies NMPC discovered that a fastener had become dislodged from one of the four assemblies. The inspector reviewed NMPC's corrective actions and root cause evaluation for the failure of the fastener, b.' Observations and Findinos The fastener was a socket head cap screw located i.n the upper spring assembly.

NMPC's preliminary investigation determined that the most likely failure mechanism was j stress corrosion failure under high stress (thermal induced) resulting in part from the j different materials used. The inspector observed the staging of a mock-up fixture on the ,

refuel floor and subsequent repair work. The inspector noted good radiological and quality assurance support. The repair personnel were utilizing good as-low-as-reasonably-achievable (Al. ARA) and contamination controls in carrying out the task.

Procedures were properly used and mechanics utilized machined fixtures to increase the accuracy of the repairs.

c. Conclusions A core shroud tie rod upper spring assembly repair at Unit 1 was well conducted. A team approach to develop a repair plan, good utilization of mock-up training, and good radiological controls practices were noted by the inspectors.

l

)

? ,

j*

11 E1.3 - Reactor Buildina Crane Auxiliarv Hoist (Unit 1)

a. Inspection Scoos (37551) i On May 18, while performing work on the refuel floor, NMPC personnel observed that one of the four hangers supporting the reactor building crane auxiliary hoist had failed.

The inspector reviewed NMPC's corrective actions and equipment maintenance history.

b. Observations and Findings l

The reactor building crane auxiliary hoist is mounted to the underside of the reactor

. building crane by four threaded rod supports. The load is transmitted from the auxiliary hoist to the reactor building crane by two spherical machined nuts threaded onto the rod, and load bearing on an upper and lower piece of channel iron. In this particular case,

' the second support from the north end of the crane failed. NMPC's immediate corrective actions included stopping work on the refuel floor and processing a temporary l modification to support the auxiliary hoist. The inspector reviewed the temporary modification and concluded that the actions taken to temporarily support the load were acceptable.

The inspector determined that, although, NMPC has a procedure for inspecting the auxiliary hoist, it lacked clarity and did not provide for inspection of the threaded rod supports. The design of the refuel floor is such that portions of the auxiliary hoist cannot be readily inspected without extensive scaffolding. The support hanger that failed had

(, not been inspected.

1 Inspector follow-up determined that one of the supports had failed in February 1988.

The failure occurred following the mis-operation of the reactor building crane when the bridge operator mistakenly went east instead of west with the main trolley, At the time,

' the crane was already near the end of the track and it's movement caused the bridge to strike the rail end stops, with the subsequent failure of the auxiliary hoist trolley support  ;

and some structural welds. NMPC determined the root cause of the February 1988  :

failure to be fatigue as a result of cyclic loading. The apparent cause of the recent failure l

' was also determined to be fatigue. In addition to weld repairs and replacement of the u ~

trolley supports, the recommended corrective actions included structural engineering )

l review of the attachment design and recommendations for a long term modification. The 4 inspector concludeo, that, based on the recent failure that the long term corrective f actions were ineffective. j I

l c. Conclusions ,

1 On May 18, while performing work on the Unit i refuel floor, the reactor building hoist l trolley connection failed. The apparent cause of the failure was fatigue of the threaded 1 rod connection. Previously conducted crane inspections were not sufficient to identify  !

L the equipment degradation and long-term corrective actions from a February 1988 failure had not been effective. j l

I I

1 L l

s, 12-E8 Miscellaneous Engineering issues (92712)

' E8.1 (Closed) LER 50-410/99-01 Supolement 1: Unit'2'Outside ' Design Basis Due to Safe Shutdown Service Water Pump Bay Unit Coolers Being Out-of-Service.

The technical issues associated with this LER were described in NRC inspection report 50-410/99-03, Section E1.3. Supplement 1 provided additionalinformation regarding, NMPC's corrective actions. The inspectors completed an in-office review of the additional information provided in the LER and found it to be acceptable. This LER is closed.

E8.2 Review of Year 2000 Proeiem and imolementation During this inspection period, a review was conducted of Nine Mile Point's year 2000 (Y2K) activities and documentation using Temporary Instruction (TI) 2515/141, " Review

- of Year 2000 Readiness of Computer Systems at Nuclear Power Plants." The review

- addressed aspects of Y2K management planning, documentation, implementation planning, initial assessment, detailed assessment, remediation activities, Y2K testing and validation, notification activities, and contingency planning. The reviewers used NEl/NUSMG 97-07, " Nuclear Utility Year 2000 Readiness," and NEl/NUSMG 98-07, ,

• Nuclear Utility Year 2000 Readiness Contingency Planning," as the primary references for this review.

1 The results of this review will be combined with similar reviews of Y2K programs at other U.S. commercial nuclear power plants and summarized in a report to be issued by the NRC staff by July 31,1999.

IV. Plant Support R1 Radiological Protection & Chemistry Controls i R1.1 Refuel Outaae Radioloaical Controls (Unit 1)

a. Insoection Scone (71750)

The inspectors observed radiological work practices and contr' ols during the Unit 1 l

- refueling cutage including protective clothing and personal dosimeter use, and radiological postings.

b. Observations and Findinos During the outage, the inspectors noted that good radiation protection controls were in effect. The inspectors noted that protective clothing was property used and dosimetry was properly worn Radiological boundaries were clearly defined and posted. Radiation protection technicians were actively providing oversight to help minimize personnel exposure. The inspectors noted that an ALARA goal of 280 person rem was set for RFO

,4, ..

13

15. (See NRC IR 99-04). Actual outage exposure was 330 person rem with the increase due to emergent work.

c. Conclusions

' Radiological controls during the Unit 1 outage were good. Protective clothing, dosimetry and radiological posting requiremen ' and radiation protection technician oversight were effective in minimizing personnel exposure.

V. Management Meetings X1 Exit Meeting Summary The inspectors presented the inspection results to members of licensee management at the conclusion of the inspection on July 15,1999. The licensee acknowledged the -

findings presented.

i

3 ATTACHMENT 1

- PARTIAL LIST OF PERSONS CONTACTED Niaaara Mohawk Power Corporation D. Bosnic Manager, Operations, Unit Two S. Doty Manager, Maintenance, Unit One N. Paleologos Plant Manager, Unit Two F. Fox Manager, Maintenance, Unit Two R. Smith Plant Manager, Unit One N. Rademacher - Manager, Quality Assurance D. Topley . Manager, Operations, Unit One

)

INSPECTION PROCEDURES USED ]

I I

IP 37551 On-Site Engineering )

Surveillance Observations IP 61726 IP 62707 Maintenance Observations IP 71707 Plant Operations IP 71750 Plant Support IP 73753 Inservice inspection IP 92700 Onsite Follow-up of Written Reports of Non-Routine Events at Power Reactor Facilities IP 92712 In-office Review of Written Reports of Non-Routine Events at Power Reactor Facilities l

l ITEMS OPENED, CLOSED, AND UPDATED l CLOSED l l

l 50-410/99 LER Reactor Trip Due to a Main Generator Protection Volts / Hertz Relay Failure 50-410/99-01, Sup 1 LER Unit 2 Outside Design Basis Due to Safe Shutdown Service Water Pump Bay Unit Coolers Being Out-of-Service

P , , , . -

1 f:.

' Attachment 1 2.

. LIST OF ACRONYMS USED ALARA- As Low As Reasonab!y Achievable

-ASME American Society of Mechanical Engineers CST: Condensate Storage Tank DER- Deviation / Event Report EDM Electric Discharge Machining ESF Engineered Safeguards Feature EVT1 ~ Enhanced Visual Techniques GE General Electric HAZ Heat-Affected Zone IGSCC.~ Intergranular Stress Corrosion Cracking

.lR . Inspection Report ISI- In-Service inspection LCO Limiting Condition for Operation LER Licensee Event Report-NCV Non Cited Violation -

NDE Nondestructive Examination ,

NMPC Niagara Mohawk Power Corporation NRC. Nuclear Regulatory Commission RCIC Reactor Core isolation Cooling System .

l RFO14 Refueling Outage Number Fourteen RF015 Refueling Outage Number Fifteen RRS Reactor Recirculation System TI Temporary Instruction TS Technical Specification  ;

Unit 1 Nine Mile Point Unit 1 Unit 2 Nine Mile Point Unit 2 UT Ultrasonic WO ~ Work Order Y2K Year 2000 i

I

Niagara (jMohawk' l

l I

May 24,1999 NMP2L 1869 l l

l l

United States Nuclear Regulatory Commission j Attn: Document Control Desk Washington, DC 20555 RE: Docket No. 50-410 .

LER 99-05 ,

i Gentlemen:

In accordance with 10CFR50.73(a)(2)(i')(B),10CFR50.73(a)(2)(iv),10CFR50.73(a)(2)(vii), and Tecluilcal Specification 3.5.1.f, we are submitting LER 99-05, " Reactor Trip Due to a Main Generator Protection Volts / Hertz Relay Failure."

Very truly yours, M ]N Nick Paleologos i Plant Manager -NMP2 l l

NCP/CES/ kap Attachment xc: Mr'. H. J. Miller, Regional Administrator, Region I Mr. G. K. Hunegs, NRC Senior Resident Inspector Records Management Nme Mile Pomt Nuclear station. RO. Box 63. Lycommg. New York 130934063 a 315.343.2110

• www.nimo.com

~ 9 9 6) G e 5 0 2 2- %

meCconM w v.s. wucuAR ROGUMTC&Y CoWWustoM AMOVE oM H 24me

^

ESWATED N PQ RENSE To CowNH DG5 INFOWN CONN LICENSEE EVENT REPORT (LER) " " '

"sconos'Amo EonNEsSSo'EM""'r*s"mE*5 IE.!)Nu"cuAm EEY a t Elm""n7o"iEcis "MEEsEEosIEE."w^AswINIEoNEE""

FACERY M AME 03 DOCKET NUM8ER (D PAGE ol Nine Mile Point Unit 2 05000410 01 OF 10 Trris se Reedor Trip Due to a Main Generator Prasar*w= Voks/ Hertz Relay Faikwe avsKr DATs as iam wuumEm se mEroa7 DATE(F) OTHER FACs.rTES DdVOLVE m hoowfM DAY TEAR YEAR d MONTH DAY YEAR FACERY MAMES DoCEET NUMBERf5) 04 24 99 99 05 00 05 24 99 N/A N/A oremATeso wooEc1 3 nas aspoRT M SUBMitTsD PURSUAKr To THE RBQUELEMENTS oF le CFR 5: Swet er er mm grerfussmeng) 01 O 20.220t(s) 0 20.22o3<.x2xv) a so.73<.x2xi> 0 so.73<.x2xviii) 100% O 20.2203<.) 0 20.22o3<.x3xi> 0 so.73<.x2)<ii) O so.73(.x2xx)

O 20.2203<.x2><i) O 20.22o3<.>oxii) O so.73<.x2)<iii) O 73.7 ia Mrs CA - 0 20.2203<.x2xii) O 20.2203(.x4) a so.73<.x2)<iv> a crrusa EQMs ' M "y. O 20.2203<.x2)(iii) O so.36<cxt> 0 so.73<.x2xv) MA" ~ ""

y; Q, y V O 2c.2203<.x2xiv> 0 so.36(ex2) a so.73<.x2 xvii) Special Report uCensaCowrAcreaanss umum mAME TsuenowE wuMesa Don P. Bosaic - Operations Manager 315-349-7952 comim owe tmE rom sAcu comonsur rAnxmE -==a se nes mapouT Um cAusE emsu comanner uAnurAc. masonTAmx #Nih cAus svmu l cowaneer uAnurAC. me,onTAsis Tunsa to spot i Mm Tunsa to spec o as scv aiss YEs $ni x x

sA V Bo45 YES a EF FU 0187 YES h@ EL RLY 0082 YES SUpri2MsNTAL Report EKDECTsD 04) sXfsCTED WoNTH DAY YEAR sUBMussoN O YBs tsas, @ EXPECTED 3031859a# A4TI) a peo AaSTRACTame=umw ,u, w w ,.=we emm w;Ue On April 24,1999, Nine Mile Point Unit 2 experienced an automatic reactor scram from 100 percent power.

~ The cause of *he reactor scram was a fast closure of the main turbine stop and control valves. Operators responding to the event were challenged by several equipment concerns including: failure of the reactor core isolation cooling system to achieve rated flow, the trip of two electrical protection assemblies, an air leak on an auxiliary boiler vr.lve, and the need to restart numerous pieces of plant equipment due to the loss of electrical power during the residual (slow) transfer of the non-vital 13.8 kV buses.

The cause of the fast closure of the turbine stop and control valves was determined to be a failure of a main generator protection volts / hertz relay.

Corrective actions included: stabilizing the plant, determining and correcting the cause of the equipment failures, revising procedures, training personnel and establishing longer term corrective actions.

Technical Specification 3.5.1.f requires a special report when an emergency core cooling system injects wa'er into the reactor coolant system. During this event the high pressure core spray system injected water into the reactor vessel.

go.o u u Ucm. uurm e, - ,o....,,-.

, excimu m

LICENSEE EVENT REPORT (LER)

" C*" U^*"

m,s,Un,,cgn, a u, couy mya y n a","." '":."Mo^c2",,"";r"e"C2' ",a"nna?

[ oimemi, omes or u4maosuswr ano nuocsT. wAseaworow, oc met FACILJrY M AME(i) ooCKE,NUWSERQ) tJit MUM 9ER ($ PAGEQ) vsaa sooUmwn41. arvision e MuussR NUMBER Nine Mile Point Unit 2 05000410 99 - 05 - 00 02 OF 10 mr ra m em e ea. m - men nsewn I. DESCRIPTION OF EVENT On April 24,1999, Nine Mile Point Unit 2 automatically tripped from 100 percent power. The cause of the reactor trip was a fast closure of the turbine stop and control valves. There were no maintenance or testing tetivities in progress at the time of the event. j Niagara Mohawk Power Corporation (NMPC) determined that the cause of the fast closure of the turbine stop j and control valves was a volts / hertz relay failure, which actuated a generator lockout relay. The plant '  !

information recorders indicated that the first alarm point was the generator lockout relay, followed by the turbine trip signal. Actuation of either the volts / hertz relay or the neutral over-current relay could have caused  ;

the generator lockout relay to actuate'. The neutral over-current relay tested satisfactory. However, the l

l volts / hertz relay did not test estisfactorily, and this confirmed that it was the event initiator. The volts / hertz relay was replaced with a new rcy less than one year prior to its failure.

When the generator lockout relay actuated, it initiated an automatic residual (slow) transfer of the 13.8 kV non-safety related busses to off-site power sources. The electrical protection scheme tripped large electrical loads l

) from the 13.8 kV busses as designed to prevent damaging plant equipment during the reenergization of l l switchgear. These large loads included the reactor feedwater pumps, reactor recirculation pumps, and condensate booster pumps.

During the residual transfer, electrical power was momentarily lost to some of the normal lighting. The lighting effected was in portions of the turbine, reactor, normal switchgear, screenwell, and control buildings. Also, some of the security perimeter lighting was momentarily interrupted. The transient had no effect on essential or cmergency lighting. The lighting system responded to the transient as designed and did not effect the operator's j response to the transient, i Coincident with the residual transfer, was a loss of output voltage from an uninterruptible power supply which provided a portion of the logic power to the reactor protection system. The most probable cause of the l uninterruptable pcwer supply malfunction was a design deficiency in the maintenance bypass transfer switch I which caused the inverter input fuse to blow. The maintenance bypass transfer switch, not part of the uninterruptible power supply, was installed to feed loads from another alternating current source while performing maintenance on the uninterruptible power supply. The uninterruptable power supply blown fuse was most probably caused by electrical noise generated at the maintenance bypass transfer switch neutral terminal i which is connected as a reference point to the inverter control circuits. High circulating current between the inverter and maintenance bypass power output was most probably caused when the plant 13.8 kV system voltage

! decayed and the inverter operated off the direct current input. The circulating current burned the circuit board trace, which is in close proximity of the neutral terminal. The trace arced, thereby inducing electro-magnetic interference / radio frequency interference noise into the neutral terminal and impacting the operation of the uninterruptible power supply silicon controlled rectifiers firing mechanism which caused two silicon controlled rectifiers to fire simultaneously. This blew the inverter input direct current fuse, resulting in the loss of citernating current output voltage, and initiated the timers on the electrical protection assemblies.

\

c ,0 . UuUci oUmovcO - A Om g .>im.

LICENSEE EVENT REPORT (LER) " ",'g,$*&'0g" ,70 C0"g" "8 E0*g" *y" g g ,

ec smurms -

,.,,- g,g ,ngA,,,g,,ny,,g g:,gu g.;g 01340104 OFFICE OF MAN AGEME,tT AMD BUDGET. WA5HINGTON, DC 3903.

FACEJTY MAME 0) DOCKET NWBER Q) 12R NUMBER 46) . AGE 0)

"^^ '

W ,

M Nine Mile Toint Unit 2 05000410 99 - 05 - 00 03 OF 10 mTa n a, u.-.au.aacr- m van L DESCRIPTION OF EVENT (Cont'd)

The timers in the electrical protection assemblies timed out, causing the two electrical protection assemblies to trip before maintenance supply power was restored to the uninterruptible power supply.

During the reactor trip, reactor water level reached a minimum of 105 inches (119.4 inches above the top of active fuel) and a maximum of 202 inches. Primary containment isolation occurred either due to reactor w:ter level falling below isolation setpoints of 159.3 (Ievel III) and 108.8 (level II) inches or the loss of two electrical protection assemblies. NMPC could not determine which signal came in first. The signals consisted of the following groups:

Group 2 _

Isolation signals to the reactor water outboard sample line isolation valves.

Group 3 Isolation signal to the nitrogen purge isolation valve to the transversing in-core probe.

Group 4 ' Isolation signal to the residual heat removal system sample line valves.

Group 5 Isolation signal to the residual heat removal system shutdown cooling suction valves.

'Gre.y6 Isolation signal to the reactor water cleanup outboard isolation valve.

Group 7 Isolation signal to the reactor water cleanup inboard isolation valve.

Group 8 Isolation signal 'to the reactor bu'ilding closed loop cooling water, drywell fire protection, automatic depressurization system air lines, instrument air, containment leakage monitoring, and reactor recirculation hydraulic power unit lines.

Group 9 Iselation signal to primary containment purge system isolation valves.

The plant response to the various containment isolation signals was as expected for the plant design.

I Several high drywell temperatures alarms were received because of the isolation of drywell cooling. The maximum average drywell temperature reached was 118 degrees Fahrenheit. Operators took appropriate i actions to restore drywell cooling, resulting in the temperature returning to normal. NMPC evaluated the drywell transient and peak temperature data and determined that the increased temperatures were bounded by ,

the equipment qualification.

• Ihe high pressure core spray system automatically initiated and performed as designed to initially control reactor water level. After operators started a condensate booster pump and feedwater pump, operators

' controlled reactor water level with the condensate and feedwater system.

,,,cy.- u ,.mm emmm am. o, 4, gg ,,-

ucsusss svsur aseoar(tsa) =g g g g y _'o =,,g =a= g= o= g=g ,i p -

e conmUAMN m,,m,,,4_Moym,,n,ogagllllnu,

• mggmggmaE[mE ssiu Ot304HH), Of71CE OF M ANAGEMDfT AND BUDGET. W ASHINGTON. DC 2H03.

FACILJrY MAME(1) DOCIET NUMass Q) RER MUMBER te PAGE 01 Nine Mile Point Unit 2 05000410 W Ue$

99 - 05 00 04 OF 10 verr a;--.,- = , n. - wer.-me w n L DESCRIPTION OF EVENT (Cont'd)

The reactor core isolation cooling system initiated, as a result of reactor water level descending below initiation setpoint of 108.8 (Level II), but failed to achieve rated speed. Control room operators observed that the automatic initiation sequence started, and the steam inlet valve and the injection valve opened, but the trip throttle valve indicated intermediate position. The operators confirmed that the high pressure core spray system was controlling reactor water level, then manually tripped the turbine as an equipment protection measure in accorder.cc with operating procedures. After the manual trip, the trip throttle valve position continued to indicate intermediate and the trip alarm was not received for reactor core isolation cooling. The operators then manually closed the trip throttle valve and the steam inlet valve and declared the reactor core isolation cooling system inoperable.

Troubleshooting efforts revealed two problenW The first problem was that the trip throttle valve was in the closed position, but the closed limit switch was not actuated. His limit switch prevented the trip alarm from annunciating, and resulted in intermediate position indication. The second problem was that the trip throttle valve cwW trip mechanism was incorrectly adjusted for adequate engagement of the trip hook and latch up lever during Refueling Outage 6. Inadequate engagement coupled with vibration from the steam admission on the initiation signal caused the trip throttle valve to unlatch and close. During troubleshooting with the trip throttle valve open, a mechanic touched the clevis and pin at the connecting rod to the mechanical trip leakage slightly, and this caused the trip throttle valve to unlatch and close. NMPC reviewed the Refueling Outage 6 work order for the overspeed trip mechanism and determined that the work order and Procedures N2-MMP-ICS-244, " Maintenance of Reactor Core Isolation Cooling Turbine Trip and Throttle Valve", and N2-MPM-ICS-V452, " Reactor Core Isolation Cooling Turbine and Accessories", did not provide proper instructions for the adjustment of the overspeed trip rod, the trip hook and latch up lever engagement, and the spring tension. A review of the vendor manual identified that the manual did include setup information for the cmW trip mechanism. NMPC determined that the reactor core isolation cooling system was potentially inoperable since the last quarterly surveillance performed on February 20,1999, when steem flowed thrcugh the trip throttle valve and the valve demonstrated proper operation. This is based on the fact that when the reactor core isolation cooling system initiated, the reactor pressure was approximately normal operating pressure. Therefore, NMPC has concluded that the reactor core isolation cooling system may not have responded as designed at any time sobsequent to the last successful surveillance test.

The maximum reactor pressure recorded during U asient was 1088 psig. This is 15 psig below the lowest actuation setpoint for the main steam safety relier a. The safety relief valves did not open. The main steam isolation valves remained open throughout the event, and all five turbine bypass valves opened to control reactor pressure by directing steam to the main condenser.

De plant transient analysis recorder failed to trip and record reactor scram data because the trip setpoints were improperly set following troubleshooting earlier in the shift. This complicated data gathering for

e,. U1,m,. =Uuro.m-

-og -..

M 2

} '

UCENSEE EVENT REPORT (LER) g%',";Dy PER R,ESFONE gp O MM 2,M,u,MN Mo .

W NMUAMN M wyg,ANDHM

• AN M 4 H M MM m M P,TM o 01364104L OFFICE OF M ANAGEMENI AND SUDGET. WA38CNGTON DC 3M03.

FACsJIY MAME(1) DOOCET NtM8ER 0) 12R NUMBER d6) PAGE0)

UM Nine Mile Point Unit 2 05000410 99 - 05 - 00 05 OF 10 ruxv e.-,-a,, ,w. . maw seew semw ae L DESCRIPTION OF EVENT (Cont'd) the investigation of the transient. Deviation / Event Report 2-1999-1260 was initiated to determine the cause and develop corrective actions.

Later in the event, operators started a condensate booster pump and determined that the pump could not be used to maintain reactor water level with reactor pressure at 600 psig. This deviated from plant response modeled in the simulator which suggested that the condensate booster pump could begin injecting water at approximately 650 psig. NMPC will review the training scenarios and the transient and determine if the simulator model needs to be revised (Deviation / Event Report 2-1999-1287). Upon recognition that the condensate booster pump was ineffective in raising reactor water level, the operators promptly started a feedwater pump in accordance with operating procedures.

'Ihere were a number of problems that required operators to take additional action to recover from the transient. 'Ihe operators had to start various pieces of equipment due to load shedding as a result of the loss of electrical power during the residual transfer (for example, feedwater pump, condensate pump, instrument air compressors, turbine lift pumps and ventilation systems). While these equipment issues added some complexity to the operators' actions, operators effectively managed the event and placed the plant in cold shutdown. Additionally:

• An auxiliary boiler valve had an air leak, which resulted in the operators stopping the attempt to start one auxiliary boiler and starting the other auxiliary boiler. Work Order 99-08040-00 was written to repair the valve. {

l e The Main Steam Line B and D radiation monitors indicated greater than 4,000 MREM on the safety )

parameter display system, with the actual reading on the radiation monitor panel reading normal- )

(1 MREM). NMPC determined that the cause was a loss of power due to the tripping of the electrical protection assembly. Deviation / Event Report 2-1999-1270 was written to incorporate this problem into the corrective action program.

II. CAUSE OF EVENT The cause of the reactor trip was determined to be a failure of a volts / hertz relay. The relay has been sent out for further failure analysis.

The most probable cause of the Uninterruptible Power Supply 2VBB-UPS3B trip was a design deficiency in the maintenance bypass transfer switch logic controls. As a result of the design deficiency, noise was generated on the maintenance bypass transfer switch neutral terminal which subsequently caused the uninterruptible power supply inverter fuse to blow.

i

f n.o - o . .uCm. v Co mm>,. A, . . ,, .e

, numu; UCENSEE EVENT REPORT (LER) QE",*gg,",88'*"g,,*,,',n"E c """uEE,4N"" **E' ""

N NMUAMN u c,a,, g D,ug ot nug yg7 gag,sgumAR Olmesess. OFTICE OF MANAGEMENT AMD BUDGET. WA36SNGTON, DC 2503.

FACEJfYMAME(O DOCKET NUHSER Q) (EJL NUMBER 44 PAGE On Nine Mile Point Unit 2 05000410 W c M 99 -

05 - 00 06 OF 10 i

mT e~ sa w u umcr.,,,,nu un II. CAUSE OF EVENT (Cont'd)  !

J De cause of the reactor core isolation cooling system failure was determined to be an inadvertent trip of the trip throttle valve. The overspeed trip mechanism was incorrectly aligned. The work order and reactor core isolation cooling system procedures performed during Refueling Outage 6 did not sufficiently address the setup of the overspeed trip mechanism consistent with the vendor manual.

III, . ANALYSIS OF EVENT

• This event is considered reportable under 10CFR50.73(a)(2)(i)(B),10CFR50.73(a)(2)(iv),

10CFR50.73(a)(2)(vii), and Technical Specification 3.5.1.f. 10CFR50.73(a)(2)(i)(B) requires a report for any operation or condition prohibited by the Technical Specification. Technical Specification 3.7.4 was potentially not met for the reactor core isolation cooling system due to inadequate maintenance performed on the trip throttle valve ovwW trip mechanism. 10CFR50.73(a)(2)(iv) requires a report for any event or condition that resulted in manual or automatic actuation of any engineered safety features, including the reactor protection system. 10CFR50.73(a)(2)(vii) requires a report.when any event caused at least one iPaat train to become inoperable. Technical Specifications 3.5.1.f requires a special report when an emergency core cooling system injects water into the reactor coolant system. The high pressure core spray system injected water into the reactor vessel. The total accumulated initiation cycles is 9, and the current usage factor value remains less than 0.70.

The reactor trip is the design response for fast closure of the main turbine stop and control valves. All control rods fully inserted after the reactor trip signal. The reactor core isolation cooling system failed to achieve rated flow, and therefore was not used for level control. The high pressure core spray system initiated and maintained reactor water level as designed. De automatic depressurization s'ystem and low ,

pressure emergency core cooling systems were operable throughout this event. After the residual transfer l was completed, the operators restarted and used a condensate booster pump and feedwater pump to maintain l reactor water level. '

The conditional core damage probability for this event was calculated to be 8.6E-6. The conditional core  !

damage probability was determined based on the loss of feedwater as the initiating event with possible recovery. The reactor core isolation cooling system was conservatively assumed to be lost for the event.

The plant response was in accordance with the Updated Safety Analysis Report transient analysis for a ,

generator load reject with bypass valve operation with the exception of the reactor core isolation cooling system and uninterruptible power supply failures.

Based on the above analysis, there were no adverse safety consequences as a result of this event. The reactor trip posed no threat to the health and safety of the general public or plant personnel.

c .o.M , A u mmA. .EoUmfo.T com. o,,

ow ,, 0 o.M m u

• UCENSEE EVENT REPORT (LER) g*4Tg"gg8C0"ygi'as 4 ,

ion MC NUNUADON gg;gogggMANAogEx;u,, An,ggjwgn,mmggTg o 0120104), OFFICE OF MANAGEMENT AND DVDGET, WAstuMUToN DC 20$01.

FAcEJTY MAME(1) DOOTT NUMBER Q) 12. NUMBER te PAGE0)

E E Nine Mile Point Unit 2 05000410 99 - 05 - 00 07 OF 10 TsxT q -- a w. .nas d sc h = me w n a j IV. CORRECTIVE ACTIONS

1. Operators performed scram recovery actions, and placed the plant in a stable condition.

2. The volts / hertz relay was replaced, and a failure analysis will be performed on the failed relay.

3. The uninterruptible power supply blown fuse was replaced, the uninterruptible power supply was inspected for internal damage, and the motor operated feature of the maintenance bypass transfer switches which contained the design deficiency was disabled.

4. Deviation / Event Report 2-1999-1707 was written to determine the cause of the design deficiency of the maintenance bypass transfer switch and to determine appropriate long-term preventive actions.

5. The overspeed trip mechanism for the reactor core isolation cooling trip throttle valve was adjusted using Work Order 99-08055-05 and was tested at 150 psig reactor pressure and at rated reactor pressure.

6. The reactor core isolation cooling trip throttle valve closed limit switch was replaced and tested satisfactorily.

7. Maintenance reviewed all preventative and corrective maintenance on the reactor core isolation cooling turbine, governor valve, the overspeed trip mechanism, and lube oil system during and subsequent to Refueling Outage 6 to ensure that work was performed to the design criteria, that procedure requirements coincided with vendor manual requirements, and that the steps performed met the acceptance criteria. No other problems were identified during this review.

8. Operators were given training and procedural guidance to ensure the reactor core isolation cooling trip throttle valve is properly latched following activities that unlatch the trip throttle valve.

9. Applicable maintenance procedures for the reactor core isolation cooling system will be revised to provide specific instructions to align the overspeed trip mechanism by July 31,1999.

10. Deviation / Event Report 2-1999-1723 was written to determine the cause of the inadequate work instructions for the reactor core isolation cooling overspeed trip mechanism and to initiate corrective actions.

Nac U Nucun asxxuTonycoMMissioN ArraovEnguy.nmem LICENSEE EVENT REPORT (LER) Q m a y , g aEspog E co,M wrm 7 g ok gug N TEXT CONTINUATION nsconos ANo soonT8 MM AGEMENT RRMsCH (P SMn. UJ. NUCMAR REOUuTORY

$ E o*iv$c TEMS"AGEMF M BD .'W GON?D%C FACRJTY NAME (1) DOOCET NUMBER C) D2 NUMBER (6) PAGE01 YEAR SEQUDrTIAL REVISION NUMBER NUMBER Nine Mile Point Unit 2 05000410 99 -

05 - 00 08 OF 10 mT m.m. .umcr-,.ua w>

V. ADDITIONAL INFORMATION A. Failed components:

The volts / hertz relay failed on April 24,1999, which was the cause of the transient. The volts / hertz relay was considered operable based on the relay performance prior to its failure.

The reactor core isolation cooling trip throttle valve was potentially inoperable since the last quarterly surveillance performed on February 20,1999, when steam flowed through the trip throttle valve and the valve demonstrated proper operation.

The uninterruptible power supply fuse failed at the time of the residual transfer of the 13.8 kV bus on April 24,1999.

=

The auxiliary boiler was inoperable when operators attempted to place tlw boiler in service because of a valve air leak. NMPC could not determine when the air leak developed.

B. Previous similar events:

Nine Mile Point Unit 2 has had a number ofinstances where engineered safety feature actuations occurred (Licensee Event Reports 98-13,98-06,98-05,97-04, and 9644). The root causes of the licensee event reports were different than the root cause for this event. Therefore, the corrective actions from these licensee event reports would not have prevented this engineered safety feature actuation from occurring. j l

There have been no recent similar failures of the reactor core isolation cooling system.

C. Identification of components referred to in this LER:

ir y J ,

QoMhNT h ' '

iEEE'803A FUNCT10N IEEE 805 SYSTEMIDS Volts / Hertz Relay RLY EL Generator 12xkout Relay RLY EL Neutral Over-Curnra Relay RLY EL Reactor Cors Isolation Cooling Turbine TRB BN

r une roam mea us. wuctzAR Raou4Teay CouMunON AmOvsD ous wo. umoios EXPNLES:

EXMMATED DURDEN PER rem *2 To compt.Y Wmt fles SWoRMATION CoumenON LICENSEE EVENT REPORT (LER) R$QtEIT: ELS 658. FORWARD COMMENT: REGARDpeo StRDEN ESTS4 ATE TO THE

• TEXT CONTINUATION RecoRos ao RasoRT: MmAosMEm mRecu esmo us. wuctzAR RsouuToRv CoMMisse0M, WA&MDioTON, DC 2551, AND TO T1E PAPERWORK REDUCTION PROJECT oimoiou,omesor MAnAosMswT mDsupost WAssemon Deman.

FACEJrY MAME 0) D002T MUMBER (2 tJiR NUMBER 4G PAoE On vzAR -

saoussenAt. Reva:On NLAdSER NUMBER Nine Mile Point Unit 2 05000410 99 - 05 - 00 09 OF 10 Tswrc---s,w- aan wmes ssuwas J V. ADDITIONAL INFORMATION (Cont'd)

I I

mis < r e ~r we m.mp .- ..r . . .a , , r s' . . . a4 a1 e m . . . ..

$m# M9@in5WiSCOMPONENT ii La%s!isiiFi!!d

' UIEEE" ' 303A FUNC'I1ONU iIEEE 805 SYSTEMID%

I Ranciar Core teoission Coolier Trio Throule Vdve SCV BN Reactor Core teolanos Coolina steam tales Valve SHV BN Reecear Core ionistics Coolina laiection Vdve INV BN Primary Ceessiamses leolatios Vdves ISV NH +

Reactor Waser Cleanup leoission Vdve ISV CE am Weser Samole Line Isolation Velve ISV CE Nstresse Purne leolation Valve ISV LK Rs abel Heat Removal Shutdows Cooliam Valve ISV BO Raoulmal Heo Removal isolation Vdve ISV BO Fire Presection leolaues Valve ISV KP Ranceor Buildies closed Leap leolation Valve ISV CC lastrument Air leolation Valve ISV LD l

Costaimowas Leakene Monitorier leolatios Velve ISV U Primary Containsneer Purae Isoletion Vdve ISV LK Rasctor Racircesistion Hydraulic Power Unit laalasaan ISV AD I Vdve i Ramesor Racirculation Pump P AD Hink Pressure Core Spray Notrjes NZL BG Unialerruptible Power Supply UJX EF Fuse FU EF Elecincel Switchaear SWGR EA Mais hrbine TRB TA hrbine Stop and Cosuel Valve SHV TA habise lift Pumps P TA habene Dyness Valve V JI Safety Relief Valves RV SB Maie Steen leolatios Valves ISV SB 1.Jait Switch 33 BN Rancear Feedweser Pumps P SI Ceedessene Boosier Pumps P SD

e .

.i . .

E PORM 3a64

1. UJ. MUCtIAR RIiDULATORY COMMRISION APPROVED OMS NO. SIMl04 e' EXPWLE3:

LICENSEE EVENT REPORT (LER) Er! NAM BWOf PG RIMH48E TO COMEWm Ties DeFORMAMN COMmM TEXT CONTINUATION REQUENTr 30.0 00Ls. FORWARD CUtsidDfr5 REGARDB80 BURDEN EXTNATE TO THE RECORDS AND REPORTS MMAGEMDff BRMcH (P-SM U1 NUCtIAR REGUI.ATORY COMMassa0N. WA36ENGTON. DC 3a515. Ah'D TO THE PAPERWORK REDUtmON P9t 01804504). OFTICE OF MAN AGD4DrT AND SUDOET. WA8MB80 TON. DC 3bS03.

FACILJTY NAME (t) DOCKET NUMBER Q)

LIR m24SE146) PAGE0)

YEAR $$QUENTW, REVRMON MuMess Muuna Nine Mile Point Unit 2 05000410 99 - 05 -

00 10 OF 10 sacs c.~ wow.e..ane tsecr sauwon V. ADDITIONAL INFORMATION (Cont'd) s,a. 1 x .: n- C o t- 5 o- -

. . + . -.

wy:ms ,1 COMPONENT & '"W '

4 , - - >-1 ~ . - ,

riIEEE 803A' FUNC110N : M IEEE 805 SYSTEM ID L Electrical Pro (eetion Assembly N/A EF silienn carmWi.a n.,..ir.,re SCR EF Mant Lightirw NIA N/A Instrument Air Compressors CMP LD Radiation Monitor MON IL Auxifiery Boiler Valve V SA Maat Trenaient Aanlysia Recorder XR IQ l

l l

1

F- 4 e 1 t

f ,

)

/ e Niagara hMohawk' l July 26,1999 ,

NMP2L 1880 I 1

l l

United States Nuclear Regulatory Commission .

Attn: Document Control Desk i Washington, DC 20555 1

RE: Docket No. 50-410 LER 99-10 i i

Gentlemen:

l In accordance with 10 CFR 50.73(a)(2)(iv) and 10 CFR 50.73(a)(2)(v), we are submitting

. Licensee Event Report 99-10. " Unit 2 Reactor Trip due to a Feedwater Master Controller Failure."

Very truly yours, l

Nick Paleologos

}%

Plant Manager - NMP2 NCP/CES/ kap Attachment xc: Mr. H. J. Miller, Regional Administrator, Region I Mr. G. K. Hunegs, NRC Senior Resident Inspector Records Management 1

e ese n,- e a . . . e. . , me n.. e9 4.... .. S t . . . w .4 g ieng eM S . 9 4 C 9 4 9 91 t h o en m., niana enen

s s.

APPROVED OMB NO. 3430 Olos g NpC 3G.M 366 U $ NUCLEAR REGULATORY COMMisslON f~ txriRts 4

ESTIMATED BURDEN PER RESPONSE TO COMPLY WITH THis INFORMATION COLLECTION LICENSEE EVENT REPORT (LER) REQUEST: 30 0 HRS. FORW ARD COMMENTS REGARDING BURDEN ESTIM AT E TO THE RECORDS AND REPORT 5 MANAGEMEl(T BRANCH (P-330). U.S. NUCLEAR REGULATORY COMMISSION. WASHINGTON. DC 20$35. AND TO THE PAPERWORK REDUCTION PROJECT (31304 0s). OFFICE OF M ANAGEMENT AND BUDGET. WASHINGTON. DC 20303 DOCKET NUMBER (2) PAGE (3)

FACILITY NAME (0 -

Nine Mile Point Unit 2. 05000410 01 OF 07 mui4 Unit 2 Reactor Trip due to a Feedwater Master Controller Failure REPORT DATE(7) OTHER FACILITIES INVLYED(8)

EVENT DATE(S) LER NUMBER (6)

REvlstON MONTH DAY YEAR FACIUTY NAMES DOCKET NUMBER (5)

MONTH DAY YEAR YEAR SEQUENTIAL NUMBER NUMBER 010 00 07 26 99 N/A 06 24 99 99 N/A THl3 REPORT 15 $UBMITTED PUR5UANT TO THE REQUIREMENTS OF 80 CFR l: tOhnt met er merr gahepensang) O O OPERATING MODE (9) 1

''v5'"

O20.220i(b) 0 20.2203(.x2xv) O so.73<.x2xi> 0 so.73<.x2xviii)

~ 100 % O 20.2203<.xi> 0 20.2203< )(3xi> 0 so.73<.x2xii) O so.73<.x2xx) 0 20.2203<.x2xi) - 0 20.2203< x3xii) O so.73<.x2xiii) O 73.7i O 20.22o3<.x2xii) 0 20.2203<.x4) 8 so.73(.x2xiv> 0 oTHER O 20.2203(ex2xiii) _ O so.36(cxi) e50.73(a)(2)(v) Ug"'"' ***- w a r =

0 20.22o3(.x2xiv) O so.36(ex2) 050.73<.x2 xvii) 8.JCENSEE CONTACT FOR THIS LER 62)

TELEPHONE NUMBER NAME Don Bosnic -Operations Manager (315) 349-7952 CX)MPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRIBED IN THIS REIORT 03)

MANUFAC- REPORTABLE CAUSE SYSTEM COMPONENT MANUFAC. REPORTABli CAUSE SYSTEM COMPONENT TO E. PIX TURER TO EPIX TURER ECBD B045 Yes X EL RLY A500 Yes X SJ EXPECTED MONTH DAY YEAR

' SUPPLEMENTAL REPORT EXPECTED 04)

SUBMISSION O YEs(n a Rs EXnerEosUnwssionoare) 8 NO ABMACr amm se Je W. m w#een marw typewrwen n=s)06)

On June 24,1999, at 3:41 p.m., Nine Mile Point Unit 2 automatically tripped from 100 percent power. The cause of the transient was a low reactor water level due to a failure of the feedwater master controller.

Additiomily, there was an unexpected partial loss of offsite power (Line 5) and the reactor core isolation cooling system failed to perform correctly in the automatic mode of operation.

The cause of the reactor trip was failure of a manual-tracking card in the feedwater master controller due to aging. The cause of the loss of Line 5 was failure of one of the main generator output breaker individual fault relays, The primary cause of the reactor core isolation cociing system flow oscillations was air found in the flow transmitter, with a contributing cause of a miscalibrated flow controller.

Corrective actions included: stabilizing the plant, replacing the feedwater manual-tracking card, replacing the main generator output breaker individual fault relay, calibrating the flow controller, and venting the reactor core isolation cooling system transmitter.

r , ,

, s F

U.S. NUCLEAR REGULATORY COMMIS$10N APPROVED OMB NO 3150 0104 i NRC H)RM 32A EX PtRE5 I'

ESWATED BURDEN PER RESPONSE TO COMPLY WITH THl3 (NFORMATION COLLECrlON l LICENSEE EVENT REPORT (LER) REQUEST: So 0 HRS, FORWARD COMMENTS REGARDING BURDEN ESTIMATE TO THE TEXT CONTINUATION RECORDS ANo RtroRTs MAN AGEuENT .RANCN (R.33el U s. NUCtEAR REcutATORY S""EOirS"ON "4G AN M'YT EI'iN sIfEEw^A H N TYIE"NEEP""

t 3 DOCKET NUMBER (2) LER NUMBER (66 PAGE (3)

F ACILITY NAME (0 YEAR SEQUENTIAL REVISION NUMBER NUMBER Nine Mile Point Unit 2 05000410 99 - 010 - 00 02 OF - 07 l'

TEXT g e ,,,,s. - momner m ua I. DESCRIPTION OF EVENT l

On June 24,1999, at 3:41 p.m., Nine Mile Point Unit 2 automatically tripped from 100 percent power. The cause of the transient was a low reactor water level due to a failure of the feedwater master controller.

Maintenance technicians were preparing to flush the feedwater flow instrument lines in accordance with a work order package. To support the work order package, operators prepared to shift the feedwater level control system from three element to single element control by shifting the master controller to manual.

Immediately after this step was performed, the controller output dropped to zero and the feedwater level control valves started to close. The licensed operator noted that the level control valves were closing and attempted to manually open the valves. After verifying the valves did not open, feedwater flow was low, and reactor water level was decreasing, the operator returned the feedwater master controller to automatic. The valves began reopening to slow the reactor water level decrease. Seconds later, a reactor trip signal at Level

.III (159.3 inches) was' received. Reactor water level started to increase until an offsite power source (Line 5) was de-energized resulting in tripping the feedwater and condensate booster pumps supplied from this electrical source. The subsequent condensate transient caused the remaining condensate booster and feedwater pumps to trip on low suction pressure.

l The reactor trip resulted in a main turbine trip on reverse power as designed. The turbine trip caused a fast transfer of both 13.8 kV buses to offsite power sources. The fast transfer was completed with one 13.8 kV bus transferring to Line 5 and the other transferring to Line 6. Shortly, after the fast transfer of the 13.8 kV

' buses was completed, Line 5 breakers tripped unexpectedly. Division I and III lost electrical power and, as designed, both diesel generators automatically started and energized their respective buses. Prior to the event, part of the electrical system was in an off-normal condition to support planned circuit breaker maintenance. The off-normal electrical line-up resulted in the loss of power to all of the turbine electrohydraulic control system pumps and the offgas system. With the loss of electrohydraulic control system pumps and the offgas system, the condenser was eventually unavailable.

During the reactor trip, reactor water level reached a minimum of 115 inches (129.4 inches above the top of active fuel) and a maximum of 205 inches. Primary Containment Isolation Groups 4 (residual heat removal radwaste discharge and sampling valves) and 5 (residual heat removal shutdown cooling valves and other system valves) isolated due to reactor water level falling below the isolation setpoint of 159.3 (Level III).

The Primary Containment Isolation Groups 4 and 5 valves were in their normal, closed position; therefore, l

the valves did not change position.

l The operators initiated the reactor core isolation cooling system to maintain reactor vessel level following the loss of the feedwater and condensate booster pumps, and.noted flow oscillations while the flow controller was l

l in automatic. The operators placed the flow controller in manual and the oscillations stopped. Operators then used the reactor core isolation cooling system to restore and maintain reactor water level. Oscillations 1

i

e - .

U.$ NUCLEAR RFCULATC2Y COMMIS$10N APPROVED OMB NO 3130 0504 g NRC ICM 3e6A ' EXPIRES ESM ATED BURDEN PER RESPONSE TO COMPLY WITH TH15 lNFORMATION COI.LECTION LICENSEE EVENT REPORT (LER) REQUEST: $0.0 HRS. FORWARD COMMENTS REGARDING BURDEN ESTIMATE TO THE TEXT CONTINUATION RECORDS AND REPORTS MANAGEMENT sRANCH <P-330i. u.s. NUCLEAR REGULATORY j

l

- COMMISSION, W ASHINGTON. DC 20333. AND TO THE PAPERWO*.K REDUCTION PROJECT (315401041. OFFICE OF MANAGEMENT AND BUDGET, WA$HINGTON DC 20$03. )

LI.R NUMBER (6) PAGE (3)

FACitJTY NAME (1) DOCKET NUMBER (2)

YEAR SEQUENTIAL REvtsf0N NUMBER NUMBER 05000410 99 - 010 - 00 03 OF 07 Nine Mile Point Unit 2 TEXT gene pare es recured, est asWueonal#NCIstm JewsJ ft7)

1. DESCRWrION OF EVENT (Cont'd) were observed during each of three occasions in automatic and stopped with the flow controller in manual.

The maximum reactor pressure recorded during the transient was 1019 psig. The operators closed the outboard main steam isolation valves to minimize the cooldown rate and to isolate the condenser, which was losing vacuum as a result of the loss of electrical power to the offgas system. The main steam system safety relief valves were manually cycled to control reactor pressure by directing steam to the suppression pool.

II. CAUSE OF EVENT The cause of the reactor trip was determined to be a failure of the feedwater master controller. Specifically, the manual-tracking card failed to provide an output signal when the feedwater master controller was switched from automatic to manual mode of operation. The manual-tracking card functions to track the feedwater level control valve in the automatic mode of operation and to maintain valve position in the manual mode of operation. The manual-tracking card failed due to aging.

Line 5 was de-energized because the backup protection scheme for the main generator output breakers tripped open all 345 kV breakers adjacent to Breaker R-230. This de-energized the 345 kV bus that powered Line 5.

The cause of the backup protection scheme initiating was the failure of one individual fault relay on the main generator output breakers.

The cause of the reactor core isolation cooling system failure to operate in automatic control was determined to be air found in the flow transmitter sensing lines. The air had accumulated in the flow transmitter from the process stream. A contributing cause was a miscalibrated flow controller. The derivative setting on the flow controller was improperly set.

III. ANALYSIS OF EVENT This event is considered reportable under 10 CFR 50.73(a)(2)(iv) and 10 CFR 50.73(a)(2)(v). 10 CFR 50.73(a)(2)(iv) requires a report when any event or condition resulted in manual or automatic actuation of any engineered safety features, including the reactor protection system. 10 CFR 50.73(a)(2)(v) requires a report

. when any event could have prevented the fulfillment of the safety function of a system to remove residual heat.

a o.

> * .t e*

bec R)RM M6A U.S. NUCLEAR REGUl>.TC Y COMMISSION APPROVEDOMB NO. 3150 4104 ESTIMATED BURDfN PER RESPON$E TO COMPLY WITH THIS INFORM ATION COLLECTION REQUEST 30.0 HR$ FORw ARD COMMENTS REGARDING BURDEN ESTIMATE TO THE

~ LICENSEE EVENT REPORT (LER) RECORDS AND REPORTS M AN AGEMENT BRANCH (P 530L U.S. NUCLEAR REGULATORY TEXT CONTINUATION cOM.ussiON. WASHINGTON. DC 20$$$, AND TO THE PAPERWORK REDUCHON PRO,ECT (31$04104). OFFICE OF M ANAGEMENT AND BUDGET, WASHINGTON. DC 20$03.

FACILITY NAME (t) DOCKET NUMBER (2) LER NUMBER (6) PAGE (3)

I YEAR SEQUENTIAL REVISION

?

NUMBER NUMBER Nine Mile Point Unit 2 05000410 99 - 010 - 00 04 OF 07 TEXT (/asev pay is regasud. not entreamsf MIC Ferar Jews; (17)

III. ANALYSIS OF EVENT (Cont'd)

The reactor trip was the design response to a low reactor water level. All control rods fully inserted in response to the reactor trip signal. The operators manually initiated the reactor core isolation cooling system.

Although automatic control of the reactor core isolation cooling system did not function properly, operators were able to use the manual control to maintain reactor water level. The high pressure core spray system was operable at the time of the event and is designed to initiate on a level 11 signal (108.8 inches). The automatic depressurization system and the low pressure emergency core cooling systems were operable throughout this event.

The' conditional core damage ~ probability for this event has been analyzed using Nine Mile Point Unit 2  ;

probabilistic risk assessment model. The analysis included de-energizing Line 5 and the unavailability of the  !

. feedwater system and the condenser. The analysis does recognize the potent i al for recoveryfh o t eht ree systems. The analysis considered the reactor core isolation cooling system available because the system functioned to maintain reactor water level. Based on the analysis, the conditional core dimage probability is 3.0E-06. i The plant response was in accordance with the Updated Safety Analysis Report transient analysis for a loss of feedwater flow, with the exception of reactor core isolation cooling system flow oscillations in the automatic mode of operation.

Based on the above analysis, there were no adverse safety consequences as a result of this event. The reactor

~

trip posed no threat to the health and safety of the general public or plant personnel.

IV. CORRECTIVE ACTIONS

1. . Operators performed scram recovery actions, and placed the plant in a stable condition. i

2. Maintenance personnel replaced the feedwater manual-tracking card with a new card.

3. Based on discussions with the vendor and the industry, Technical Support personnel will develop reconunendations on improving the reliability of the feedwater manual-tracking card by August 31,

.1999.

4. Maintenance personnel replaced the faulty relay on the main generator output breaker.

5. Nine Mile Point Unit 2 will perform a failure analysis of the failed relay and develop additional corrective actions based on the results of this evaluation, if necessary, by November 1,1999.

F-4, 1

U.S NUCLEAR REGULATG2Y COMMISSION APPROVED OMB NO. 3 50410e NRC PC'M

. 3e6A EXPtRES ESTIMATED BURDEN PER RESPONSE 1D COMPLY WITH THl3 INFORM ATION COLT 1CTION LICENSEE EVENT REPORT (LER) REQUEST: $0 0 HR3 FORWARD COMMENT $ REGARDING BURDEN ESTIMATE TO THE

. TEXT CONTINUATION . RECORDS AND RE, orts uANAoEMENT RANCH (P.S)O).U.S. NUCLEAR REGULATORY (X)MM15$40N, W ASHINGTON, DC 20$35, AND TO THE PAPERWORK REDUCTION PROJECT (31$40t04). OFFICE OF MANACLMENT AND BUDGET, W ASHINGTON DC 20503.

DOCKET NUMBER (2) LER NUMBER (6) PAGE D)

FACluTY NAME(1)

YEAR SEQUENTIAL REVi$lON NUMBER NUMBER Nine Mile Point Unit 2 05000410 99 - 010 - 00 05 OF 07 TEXT g merr gere m segwat, ma admuenst ##C Form Je64 's) (17J IV. CORRECTIVE ACTIONS (Cont'd)

I

6. Maintenance personnel bench calibrated the reactor core isolation cooling system flow contro!!er, checked the flow transmitter for noise and grounds, vented transmitter sensing lines, and veriGed dynamic tuning of the flow controller.  ;

1

7. Procedure N2-OSP-ICS-R002, "RCIC [ Reactor Core Isolation Cooling] System flow Test," was revised to include criteria for early prediction of flow / pressure oscillations and to incorporate the use of the plant computer system parameters for trending data against a baseline. The revised procedure was performed during plant startup.

,8. Procedure N2-OSP-ICS-Q@002, "RCIC [ Reactor Core Isolation Cooling] Pump and Valve Operability Test and System Integrity Test and ASME [American Society of Mechanical Engineers] .

XI Functional Test," was revised to include a step to detect precursors to flow oscillations and to include a step to have maintenance perform system tuning if required. The pump and flow controller portions of the revised procedure were performed during plant startup.

9. Maintenance personnel are reviewing, verifying, and improving procedures to ensure proper performance and documentation of all required reactor core isolation cooling system tuning and calibration activities by August 31,1999.

10. Trending of transmitter sensing line venting results is being used to determine the frequency required to ensure the reactor core isolation cooling system flow transmitter is free of air.

11. An electronic dampening circuit modification for the reactor core isolation cooling flow transmitter will be completed by the end of Refueling Outage 7.

l 1

- V. . ^ ADDITIONAL INFORMATION A. Failed components:

e The feedwater manual-tracking card failed on June 24,1999, which was the cause of the transient.

e An individual fault relay on the main generator output breaker failed on June 24,1999, which was the cause of de-energizing Line 5.

49 d

. '* #4 r

I NRC FC as3e6A U.S. NUCLEAR REGULATC2Y COMMISSION APPROVED OMB NO.3130 4104 EXPIRES EstiMAm BURDEN rER REmNsE TO COMety wiTH THis :NFORM ATiON COLLECriON LICENSEE EVENT REPORT (LER) REQUEST: 50 0 HR$. FORWARD COMMENTS REGARDING BURDEN ESTIMATE TO THE TEXT CONTINUATION . RECORDS AND REPORTS M ANAGEMENT mRANCH (P-330L U.S. NUCLEAR REGULATORY COMMIS56ON. WA5HINGTON DC 2uS55. AND TO THE PAPERWORK REDUCTION PROJECT (313041041. OFFICE OF M ANAGEMENT AND BUDGET WASHINGTON. DC 20$03 FACILITY NAME(t) DOCKET NUMBER (2) LER NUMBER (6) PAGE (3)

YEAR SEQUENTIAL REVislON NUMBER NUMBER Nine Mile Point Unit 2 05000410 99 - 010 - 00 06 OF 07 TEXT (Tawe apart as avguered. mw edersonsf MIC Form Je64 'sJ (17)

V. ADDITIONAL INFORMATION (Cont'd)

B. Previous similar events:

Nine Mile Point Unit 2 has had a number of instances where engineered safety feature actuations occurred (License Event Reports 97-04, 96-04,98-05,98-06,98-13, and 99-05). The root causes of these licensee event reports were different than the root cause for this event. Therefore, the corrective actions from these licensee event reports would not have prevented this engineered safety feature actuation from occurring.

I Licensee Event Reports 95-10 and 98-06 document partiallosses of offsite power. Both of these instances, the breaker backup protection scheme functioned as designed. The root causes of these licensee event reports were different than the root cause for this event. Therefore, the corrective

- actions from these two licensee event reports would not have prevented this partial loss of offsite power.

Licensee Event Report 99-05 documented a failure of the reactor core isolation cooling system. The root cause was determined to be that the overspeed trip mechanism on the trip throttle valve was incorrectly aligned. Again the root cause was different; therefore, the corrective actions from Licensee Event Report 99-05 would not have prevented this reactor core isolation cooling system failure.

C. Identification of components referred to in this licensee event report:

Components IEEE 803A Function IEEE 805 System ID Reactor Core Isolation Cooling System' N/A BN Reactor Core Isolation Cooling Flow FC BN Controller Reactor ' Core Isolation Cooling Flow FT BN Transmitter Residual Heat Removal Shutdown Cooling ISV BO Valve Residual Heat Removal isolation Valve ISV BO i Electrical Bus BU EA and EB r l

U.S. NUCLEAR REGULATORY COMMISSION APPROVED OMB NO 3150 4104 1

NRC FCGM 366A EXPlRES I

ESUMATE BURDEN PER RESP NSE TO COMPLY wtTH TH15 INFORMATION COLLECTION LICENSEE EVENT REPORT (LER) REQUEST: SO O HR$. FORW ARD COMMENTS REGARDING BURDEN ESTIM ATE TO THE REGM AND REPORTS M AN AGEMENT BRANCH (P.2), U.S. NUC1DR REGULATORY TEXT CONTINUATION COMMISSION, W ASHINGTON. t)C 20535. AND TO THE PAPERWORK REDUCTION PROJECT 01504106). OFFICE OF M ANAGEMENT AND BUDGET. WASHINGTON. DC 20503 DOCKET NUMBER (2) LER NUMBER (6) PAGE (3)

F ACluTY MAME(Il YEAR 5EOUENTIAL REVISION NUMBER NUMBER Nine Mile Point Unit 2 05000410 99 -

010 -

00 07 OF 07 TEXT (ysaw armv a severaf, mer adeteoast NitC Fonn JoM 's) # 7)

V. ADDITIONAL INFORMATION (Cont'd) 1 C. Identification of components referred to in this licensee event report (Cont'd):

l l

Electrical Breakers BKR FK Electric Relay RLY EL 1 Main Turbine TRB TA Turbine Electrohydraulic Control Pump P JJ Safety Relief Valves RV SB Main Steam Isolation Valves ISV SB Reactor Feedwater Pumps P SJ Reactor Feedwater Master Controller LC SJ Reactor Feedwater Manual-Tracking Card ECBD SJ Reactor Feedwater Level Control Valve LCV SJ Condensate Booster Pumps P SD Condenser COND SG Offgas System N/A WF High Pressure Core Spray N/A BG Diesel Generator DG EK Suppression Pool N/A NH i

l l

l l

l l