IR 05000341/1990007

Insp Rept 50-341/90-07 on 900331-0525.Violations Noted.Major Areas Inspected:Action on Previous Insp Findings,Operational Safety,Maint,Surveillance,Followup of Events,Ler & Review of Maint on East Mainstream Bypass Valve & Reactor Scram

ML20055D858
Person / Time
Site:	Fermi
Issue date:	06/27/1990
From:	Defayette R NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III)
To:
Shared Package
ML20055D837	List: IR 05000341/1990007 ML20055D836 ML20055D848
References
50-341-90-07, 50-341-90-7, NUDOCS 9007100086
Download: ML20055D858 (28)

Text

- - - - -.. -,,. -., -.

.

'

f

.,.

.-

'

-

,

,

,

U. S. NUCLEAR REGULATORY COMMISSION

!

REGION !!!

Report No. 50-341/90007(DRP)

Docket No. 50-341 Operating License No. NPF-43 Licensee:

Detroit Edison Company 2000 Second Avenue Detroit, MI 48226 fccility Nanie; Fermi 2 Inspection At: Fermi Site, Newport, Michigan inspection Conducted: March 31 through May 25, 1990 Inspectors:

W. G. Rogers S. Stasek f. Brush M. Farber a

.

,

J O N 2 'l N

'

'

Approved By:

1. W. Defayette, '11ef Reactor Projects Section 2B Date Inspection Summary Inspection on March 31 through May 25,1990(ReportNo. 50-341/90007(DRP))

Areas Inspected: Action on previous inspection findings; operational safety; maintenance; surveillance; followu) of events; LER followup; review of maintenance on the east mainsteam >ypass valve; reactor scram followup; self-assessment capability; fitness for duty TI followup; and TMI followup.

Results: Overall the licensee exhibited many of the same weaknesses discussed

~

in the most recent SALP report.- Maintenance activities were deficient with

<

a significant management control breakdown identified in turbine maintenance contractor control during the first refuelino outage (RF01).

Integrated teamwork on plant problems continued to be lacking. This resulted in equipment trending results not translating into effective maintenance actiont vid inadequate troubleshooting of equipment. Weak design practices / post modification testing resulted in a reactor scram. Quality programs deficiencies were noted in the inability of the accountability action plan to identify the J.OOU S$bbk O

f)

O

-

ii i ii mmmmm-imammis---imi--mii

- - -- - -mm--i-p su-

_ _ _ _ _. _ _ - _ _ _ _

I f.** "

. d'.

'

,

%:

-

'

' l

T

- 3 l

east bypass valve deficiency and the lack of e'ffective corrective actions to

'

preclude the 62,000 gallon spill during the; April 10th scram recovery.

These j

e deficiencies resulted in two apparent Severity Level IV violations. Al so,- a j

noncited severity Level V violation was given for a violation of TS requirements

{

for logging reactor pressure and temperature within the required time period during cooldown.

Occasional deficiencies in the implementation of operations

-

administrative controls continued to be evident due to lack of attention to i

detail. Operator off-normal event response continued to be satisfactory.

.However, there were procedural inconsistencies in the post scram recovery of

- '

,

April 30th.

Finally, some technical support deficiencies to emergency actions i

'

were' apparent as evidenced by inadequate emergency operating procedure training

,

in the use of reactor core isolation cooling and a poor full flow test line

design hampering scram recovery efforts of April 10th.' Two violations were j

identified (Paragraph B). involving a notice of violation and one violation

.

(Paragraph 9.j) not involving a notice of violation. Two open items were.

!

,

identified (Paragraphs 5.b and 9.c).

. !

!

l

,

. '

!

!'

P

e

i

i I

t

!

'

t 1.

,

' l

,

i f

i

i y

rT+

w

mulquiin usii u

-

.,?

'

'

.

-

,.

,

'

DETAILS 1.

Persons Contacted a.

Detroit Edison Company P. Anthony, Licensing

• S. Catola,- Vice President, Nuclear Engineering and Services

• P. Conroy, General Manager, Power Generation, Westinghouse

• G. Cranston, General Director, Nuclear Engineering

• P. Fessler, Superintendent Technical

• D. Gipson, Assistant Vice-President, Nuclear Operations L. Goodman, Director of Licensing

• A. Kowalczuk, Superintendent, Maintenance / Modifications

.

• M. Main, Manager, QA, Power Generation, Westinghouse

• P. Mescher, Project Operations, Westinghouse

,

• R. McKeon, Superintendent, Operations

• W. Miller, Director, Plant Safety

• J. Mohler, Staff Assistant, QA and Plant Safety G. Ohlemacher, Principal Engineer, Licensing W. Orser, Senior Vice President, Nuclear Operations J. Pendergast, Compliance Engineer J. Plona, Operations Engineer E. Preston, Director,' Nuclear Training

• T. Riley, Supervisor, Compitance B. Sheffel, Nuclear Production, Technical Engineering ISI

• B. Siemasz, Compliance Engineer, Licensing F. Svetkovich, Operations Support Engineer

• R. Stafford, Director, Quality Assurance W. Tucker, Assistant to the Vice President

• J. Walker, General Supervisor, Nuclear Engineering b.

U.S. Nuclear Regulatory Commission

_

• W. Rogers, Senior Resident Inspector

• S. Stasek, Resident Inspector F. Brush, Technical Support Inspector M. Farber, Project Inspector

• M. Jordan, Chief, Operator Licensing, Section 1

• Denetes those attending the exit meeting on June 1, 1990.

'The inspectors also interviewed others of the licensee's staff during this inspection.

2.

Action on Previous Inspection Findings (92701)

e a.

(Closed) Violation (341/88012-10(DRP)):

Failure to perform independent second verifications. To prevent recurrence, the licensee prov'ded training to maintenance personnel on equipment return to service as well as independent verification requirements.

In addition, required reading was issued to both operations and maintenance departments.

,

i p

..Y

. '. '

'

.

.

.

i

.

b.

(Closed) Violation (341/89011-06(DRP)):

Inadequate fire watch

i qualification training.

The inspectors completed review of the licensee's response to the violation on April 4, 1990.

The fire watch training material was revised to include the applicable

plant's. technical specification requirements.

Plant personnel

were also trained using the new material.

,

c.

(Closed) Open Item (341/89021-07(DRP)):

Implementation of the Measuring and Test Equipment (M&TE) program.

The licensee subsequently revised administrative procedure NPP-MT1-01, " Measuring l

and Test Equipment Program" to address issuance of test equipment l

during backshifts.

The question into timeliness of investigations of out-of-calibration instruments was subsequently resolved by the licensee via streamlining of the investigative process as well as placement of a coordinator to facilitate the required reviews.

Timeframes to conduct the investigations thereafter substantially improved. A concern into the amount of breakage observed upon M&TE

returned to the issue cage was determined to be about average when

-

the licensee polled other utilities on this matter.

The inspector had no further questions and this item is considered closed.

This area will continue to be periodically reviewed as part of the routine inspection program, d.

(Closed) Open Item (341/90002-04(DRP)):

Monitoring of control center HVAC. The licensee has determined that additional monitoring and stiffening is not necessary.

However, fan teardown and inspection will be done every refueling outage, (Closed)UnresolvedItem(341/89018-03(DRP)): Operability of e.

residual heat removal mechanical draft cooling tower brakes.

The licensee did not maintain the nitrogen pressure for the brakes due to a lack of understanding of the design bases for the brakes and

,

that the, do have a direct bearing on ultimate heat sink operability.

This is the same root cause as violation 341/88014-01 and, as such, is only an additional example.

Therefore, no. violation is warranted

'

and the corrective actions to violation 341/88014-01 should address this matter generically.

!

f.

(Closed) Unresolved Item (341/89018-05(DRP)):

Failure to maintain a noninterruptible instrument air system drain pot operable.

The licensee did not recognize that removing the drain pot from service for maintenance rendered that NIAS division inoperable due to a lack of understanding of the design bases for the drain pots. This is

,

the same root cause as violation 341/88014-01 and, as such, is only an additional example.

Therefore, no violation is warranted and the.

t corrective actions to violation 341/88014-01 should address this

!

matter generically, s

g.

(0 pen)OpenItem(341/90005-05(DRP)):

Residual heat removal (RHR)

,

suction line failure analysis.

During the inspection period the engineering research department completed the RHR analysis and determined that the suction tap failure was a high vibration point.

Furthermore, other taps were identified as high vibration points, i

'

_ __

d J,'

'

.

,

,

I Numerous conference calls, written correspondence, and a meeting

on April 23, 1990, occurred on this vibration matter.

During these

.

dialogues another instrument tap, on the discharge of the RHR pumps, cracked.

The licensee reduced the stress on these targeted taps

'

through shortening the taps and/or improving the welds.

A more

, complete inspection by a regional inspector in this area occurred

'

and more detailed observations / findings will be documented in

inspection report 341/90008, i

h.

(0 pen) Violation (341/90005-06(DRP)):

Failure to maintain position control over a NIAS valve. During this inspection period a number of other equipment was found out of position.

This included another valve in NIAS and two room temperature controllers for safety-related equipment. Presently, these three abnormal conditions are being r

reviewed by the licensee's security department for any related i

attributes. The inspector will withhold any further regulatory'

'

action or, these events and withhold judgement on the adequacy of the 341/90005-06 violation response until the security review

-

is completed and discussed with the inspector.

!

,

l

'

1.

(0 pen) Open Item (341/86039-01) Valve serviceability for

,

217 safety-related valves:

The inspector examined the Deviation

Event Reports (DER), selected Engineering Design Packages (EDP),

Licensing Action Notices, Regulatory Action Commitment Tracking i

'

System (RACTS) sheets, and correspondence associated with this i

issue.

The licensee's commitment was to complete 50% of the serviceability modifications by the first refueling outage, 75%

.

by the end of the second refueling, and completion by the end of

'

the third refueling.

Present status shows that following the completion of the first refueling outage, 82.5% of the required modifications had been completed which is significantly ahead

,

of the commitment schedule.

The inspector examined ten of the modifications, selected randomly, to verify installation.

Those installations had been completed.

,

The inspector was concerned that although less than 20% of the

'

modifications remain to be completed, the licensee nas chosen not i

to schedule any work on this project during the second refueling

'

outage.

This item will remain open until the modifications are

'

,

completed.

j.

Closed (0 pen Item) 341/89011-07 Lubricatior, programi The inspector reviewed the recently approved lubrication procedure and reviewed selected issues from the Plant Manager's memorandum NP-89-0041. The_

inspector determined that the issues had been satisfactorily addressed and that significant improvements had been made in the lubrication-program since the NRC's concern about the lubrication program was

-

i Q raised.

Lubrication during maintenance on HPCI and RHR Complex l

ventilation dampers examined by the inspector and no problems were noted.

The' inspector met with the engineer in charge of the lubrication program and discussed actions to be taken to ensure continuity of the program in the event of personnel changes.

The

!

I

$

!

'

j;.,.l

\\

.

'

-

.

!

,

'

actions were determined to be adequate to ensure the continuity of the program.

The inspector has no further questions in this matter and considers the item closed, k.

Closed (Violation) 341/89036-02 Failure to complete Limiting i

Condition for Operation 3.3.2.b as required:

The inspector

-reviewed the licensee's corrective actions as discussed in the response to the Notice of Violation and the Licensee Event Report through document raview and interviews. The inspector specifically examined the actions taken with regard to the failure to properly place the instrument channel in the tripped condition after the

problem was identified.

The inspector determined that the licensee's

'

corrective actions were adequate and had been satisfactorily completed.

<

This item is considered closed.

1.

Closed (Violation) 341/89036-01 Inadequate and. incorrect installation and testing procedures for transmitter B21N0810:

The inspector reviewed the licensee's response to the Notice of Violation. The inspector specifically examined the actions taken

with regard to:

(1) the maintenance personnel improperly installing the transmitter; (2) failure of the Quality Control inspection activities to identify the installation error; and (3) the technical error in the post-modification surveillance procedure; and verified that corrective actions were adequate and completed as required.

With respect to the adequacy of the surveillance procedure, some controversy arose relating to the licensee's position on post-modification testing. This is more fully discussed in Paragraph d., the closure of Licensee Event Report 89-030.

The

inspector has no further questions on this matter and considers

'

it closed.

3.

Operationa s Safety Verification (71707)

The inspectors observed control room operations, reviewed applicable logs and conducted discussions with control room operators during the period from March 31 to May 14, 1990. The inspectors verified the operability of selected emergency systems, reviewed tagout records and verified i

proper return to service of affected components.

Tours of the reactor building and turbine building were conducted to observe plant equipment-

>

conditions, including potential fire hazards, fluid leaks, and excessive i

vibrations and to verify that maintenance requests had been initiated for equipment in need of maintenance.

The inspectors, by observation and direct interview, verified that the physical security plan was being implemented in accordance with the station security plan, The inspectors observed plant housekeeping / cleanliness conditions and i

verified implementation of radiation protection controls. During the inspection, the inspectors walked down the accessible portions of the

following systems to verify operability by comparing system lineup with

'

.,

+

,

i;

,

L.

.

,,

!

'

plant drawings, as-built configuration or present valve lineup lists;

,

observing equipment conditions that could degrade performance; and l

verified that instrumentation was properly valved, functioning, and calibrated.

,

Core Spray System - Division I l

'

Noninterruptible Air System - Division II

Thermal Recombiner System - Divisions I and II i

Emergency Diesel Generator No. 14

'

Electric and Diesel Fire Pumps

!

The inspectors also witnessed portions of the radioactive waste system controls associated with radwaste shipments and barre 11ng..

These reviews and observations were conducted to verify that facility

operations were in conformance with the requirements established under technical specifications,10 CFR, and administrative procedures, a.

During the walkdown of the Thermal Recombiner System, the inspector noted the Division I Emergency Equipment Cooling Water (EECW)_ room cooler return isolation valve (P44-F062A) was in a throttled position

'

and was not secured by a lockwire.

The same valve in Division II was also throttlec but was secured with a lockwire. Both valves were found to be consistant with the system operating procedure (SOP 23.409)

lineup sheets.

However, P44-F062A was not accurately reflected in p

the functional operating sketch (FOS), was not included in surveillance WPP-27.000.01, " Locked Valve Lineup Verification," nor administrative procedure NPP-OPI-09, " Locked Valve Guidelines," and

-

was not included in the master list of locked valves maintained as Design Calculation (DC) 4959.

'.

The reason for the inconsistency was determined to be a decision to lock P44-F062A by the operations department after the initial list of required locked valves was developed by engineering.

The valve was subsequently lockwired in the field and the SOP updated to reflect the change. However, no communication was made to engineering

'

so that the additional valve could be addressed under the drawing

-

program, as well as to enable updating of the associated design

calculation DC-4959. Additionally, surveillance NPP-27.000.01 was

'

not updated at the time of the change.

In response to this matter, the licensee is updating the FOS, design

calculation, and the associated procedures to include the subject valve.

-

b.

During control room walkdowns conducted during the inspection period, discrepancies were noted with the implementation of the control room

'

information system (CRIS).

The discrepancies included:

(1) not updating the CRIS log when a dot was posted on an MSIV leakage control valve, and (2) not properly deleting from the CRIS log a dot removed

-.

from the P808 panel (turbine building HVAC fan) which, consequently, resulted in conflicting entries for the same dot when it was subsequently reposted on the P603 panel (Division EECW Radiation Monitor).

,

.--.,_.

. -

.

. - - - - - - -

-. - - - -

-

t

.._ _-.

' '

!

,s

... -

.-

.

,

'

-

c

'

c.

During the inspection period the licensee informed the inspector

that management directives to the operators may have directed operators to perform actions less conservative than delineated in

the Technical Specifications (TS).

In 1989 the licensee recognized that some of their TS submittals were more conservative than the existing TS.

Therefore, management generated a memorandum that was-

'

placed in the TS notebook directing that these more conservative

actions be taken in lieu of the TS actions.

However, some of those suppositive conservative actions were not.

The licensee cancelled the memorandum and transferred the truly conservative actions into controlled administrative mechanisms and reviewed previous operator

,

actions with rsspect to the non-conservative actions. The review

"

revea'ed th&t no TS action was ever violated.

No violations or deviations were identified in this area.

4.

Monthly Maintenance Observation (62703)

Station maintenance activities on safety-related systems and components listed below were observed to ascertain that they were conducted in accordance with approved procedures, regulatory guides and industry codes or standards and in conformance with technical specifications.

The following items were cor,sidered during this review:

the limiting i

conditions for operation were met while components or systems were

-

renoved from service; approvals were obtained prior to initiating the work; activities were accomplished using approved procedures and were inspected as applicable; functional testing and/or calibrations were performed prior to returning components or systems to service; nuality control records were maintained; activities were accomplished by qualified personnel; parts and materials used were properly certified; radiological controls were implemented; and fire prevention centrols were implemented.

Work requests were reviewed to determine the status of outstanding jobs and to assure that priority is assigned to safety-related equipment maintenance which may affect system performance.

The following maintenance activities were observed / reviewed:

'

018C891101 EDG 14 Lube Oil Sump Sightglass Replacement I

W843900126 EDG 14 Quarterly and Semiannual PMs

006D900427 RHR Pump D P44 Drain Valve Repair

0090900410 Repair of Vacuum Breaker T22 F400F Indication

W839891215 EDG Quarterly and Semi-annual Preventive Maintenance

W419891019 EDG Service Water Flow Loop Calibration Following completion of maintenance on Emergency Diesel Generator No.14, the inspector verified that the system had been returned to service properly.

No violations or deviations were identified in this area.

.

m

,s

...

,

c

-

,

.

.

-

,

,

I

5.

Monthly Surveillance Observation (61726)

The inspectors observed surveillance testing required by Technical Specifications and verified that: testing was performed in accordance.

,

with: adequate procedures; test instrumentation was calibrated; limiting conditions for operation were met; removal and restoration of the

'

affected components were accomplished; test results conformed with i

Technical Specifications and procedure requirements and were reviewed by personnel other than the individual directing the test; and any.

deficiencies identified during the testing were properly reviewed and resolved by appropriate management personnel.

<

,

The inspectors witnessed the following test activi_ ties:

'

24.202.01 SRV Vacuum Breaker Valve Operability Test

'

24.206.01 RCIC System pump and Valve Operability Test

24.402.01 Drywell and Suppression Chamber Vacuum Breaker Operability Test

'

24.404.04 Division II SGTS Filter and Secundary Containment Isolation Damper Operability Test

43.401.388 Local Leakage Rate Testing For Penetration X-7A t

a.

Regarding surveillance 24.402.01, the inspector noted that I

containment isolation valves (CIV) T48-F416 through T48-F427, used

.

to supply nitrogen to the vacuum breaker valve actuators, during-l testing, were all simultaneously opened during the test.

The control

'

switches for these CIVs are located on instrument racks in the reactor building in two different quadrants (half of the valves are operated i

from each) and the operator typically moves between the two areas and opens all twelve valves per the procedure.

Consequently, the

!

inspector questioned the prudence of opening all 12 containment

,

penetrations simultaneously to perform the test.

The test could just as well be performed by cycling each containment isolation

,

valve one at a time, testing the associated vacuum breaker and

.

restoring the CIV to its normal closed configuration prior to proceeding to the next valve.

Currently, the licensee is evaluating alternate means to conduct the required testing, b.

Regarding 24.206,01, the inspector noted that upon initiation of a reactor core isolation cooling (RCIC) turbine trip as part of the surveillance test, pump minimum flow valve E51-F019 began cycling.

The valve was then deenergized and the RCIC system declared inoperable and troubleshooting of the problem ir.itiated. The cause was subsequently determined to be a plugged snubber in an associated pressure instrument line. This pluggr.d line caused the valve logic to-sense a low RCIC flow /high discharge pressure condition which opened the valve.

Once the valve was open, the valve logic subsequently sensed the RCIC turbine trip and the valve reclosed.

This process continued until the pres = pre in the instrument line decayed to less than 125 psig (approx Nate). The snubber was repa1 %d/ replaced and the situation corrected.

_

'

r

!

-

ma.

m p v1

,,,

t. p.

- ;., i =

-

>

-

h-

.,

O

\\

d

However, this appears to be a repeat of a problem witnessed by the-q inspector during a surveillance on February 2, 1990, where the same:

_

symptoms'were evidenced.

A' work request was initiated atithat time

to troubleshoot the problem (reference WR 0050900202). The root-l t

cause: at' that time may.not have been appropriately evaluated and_

'

the condition not corrected until the second occurrence.

Pending

'

-completion of the inspector's review, this is considered an open

,

item (341/90007-Ol(DRP)).

The inspectors performed a record review of completed surveillance tests.

.

.The review was to determine that the test was accomplished within the required Technical' Specification time interval,' procedural steps were L

,

L properly initiated, the procedure accetance criteria were met,. independent verifications were. accomplished by people other than those performing the i

test,'and the tests were signed in and out of the control room surveillance

.

log book. The surveillance tests reviewed were:

o

'

,

-

.

'

24.000.02 Attachments 2 and 3; Shif tly, Daily, Weekly 'and

~l Situation Required Surveillances

'

• .

24.138.06 Jet Pump Operability Test 24.606.01 Traversing In-Core-Probe System Valve Operability

'

Test

24.714.01 Post Accident Sampling System Valve Operability.

and Timing Test

44.010.116 IRM A Channel Calibration

,

54.000.07 Core Performance Parameter Check Regarding_ procedure 24.000.02, Attachment 2, the inspector noted that the procedure' included erroneous references to 24.000.02, Attachment 3 and

vice versa.

Further-questioning revealed that 24.000.02 had recently-

'

been revised and had been in actual use approximately two weeks.

The

revision _had been purely administrative in nature, therefore,' performance L

of the surveillances had been technically adequate.

However, the procedure had been prepared with erroneous. references included, had gone through'the

,

,

' review process (which specifically included an administrative review)

without'~the-~ errors being -identified prior to issuance of the revision,

and the procedure had been in use'for a period of time prior to the

' errors being identified by the inspector,

!

.No violations or deviations were identified'in this area, j

6.

Followup'of Events (93702)-

During'the inspection period, the licensee experienced several events,

.some of: which required prompt notification of the NRC pursuant to u

10:CFR 50.72.. The inspectors pursued the events onsite with licensee and/or?other NRC officials.

In each case, the inspectors verified that

_'the notification was correct and timely, if appropriate, that the licensee

!

'

was taking prompt and appropriate actions, that activities were conducted

!

'

.

within regulatory requirements and that corrective actions would prevent future recurrence. The specific events are as follows:

<

--..- --

- - -

-- - -

-

c:

,

,

,

h g.f.,.

'!

'

.

r

~

'

,

April-10,.1990 Reactor Scram Due to Loss of, Division I-

-*

Drywell Pneumatic System (This event is (1 discussed in detail in Section 9)

i

>

,

..

RWCU Trip on Differential Flow While Placing i

'-

April.11, 1990 -

System Into Service (Subsequent. licensee p

review determined that this isolation was

, -[

expected and addressed in current

'

procedures)

.

.

'

':

April 28, 1990 NIAS Aftercooler Drain Valve P50-F206B Discovered Open During Surveillance

April 28,1 1990 HPCI Room Cooler Temperature Controller Found Mispositioned

May'1, 1980 ENS Inoperable

.

May 3, 1990'

North Reactor Feedpump Trip No violations or deviations were identified in this area.

7.

Licensee Event Reports ' Followup (92700)

Through direct-observations, discussions with licensee personnel, and

>

,

review of records', the following event reports were reviewed to determine.

r that reportability requirements were fulfilled, immediate corrective action

'was. accomplished, and corrective action to prevent recurrence had been i

,

accomplished in accordance with technical specifications.

'

s a.

(Closed) LER.89-032, ESF Actuations due to a Blown Fuse, l

b.-

In' addition to the review of the LER, the inspector reviewed a DER'with

,

the.following observations:

DER.90-0208, During this reporting period an inspector reviewed

,

DER 90-0208 and.has a concern that the licensee's Technical

' Specification LC0 on steam dome pressure is 1040 psig while the Y

cycle 2 reload report analysis uses 1005 psig. The DER also references the actual. steam dome pressure, for this ' cycle as being i

i 1025-1030 psig. 'This actual pressure was read from the-process computer, however, the strip chart r_ecorder and gage on panel 603 were indicating approximately 1005 psig. This discrepancy between

.

'

g process computer wide' range reactor: pressure and panel wide range-_

!

~

reactor pressure has.been seen at another BWR4. The possible root

<

>

W cause is an error in calculating the curve used to convert the

!

' millivolt-input to:the process computer to psig.. The calculation

'

,

doesn't include the. correct resistance of the circuit used to develop

!

the-millivoit signal from the current loop of the pressure transmitter, t

'

The licensee is investigating the apparent discrepancy between Technical Specifications' and the cycle 2 reload report.

,

,

L

.

y c.

(Closed) LER 89029:

Engineered Safety Feature Actuations Due to

-

Loss of Reactor Protection System Motor Generator Set "B" Power, i

The~ inspector reviewed the LER, DER 89-1233, and Potential Design

'J.

Change-(PDC) 10972.

The inspector also verified the completion of the assigned reading by the operators and the installation of a

the shackle on the Motor Generator Set circuit breaker as specified by the PDC.

The inspector has no further questions in this matter and considers it closed.

w,

!

i

,,

$ 9...'.

..

.

N(.1

..

,

L'

'

'

,4

d.

(Closed)'LER89030:

Improperly ' Installed Reactor Vessel Water -

t Transmitter.

The' inspector reviewed the LER, DER 89-1437 Humani Performance Evaluation System (HPES) report 90-001, and' the licensee's-

response to the Notice of Violation. accompanying Inspection Report

{

.

-

,^

-No. 50-341/89036(DRP) -From the documents included with the DER,

'

the inspector noted that there is a viewpoint within the organization,,

as expressed.in Nuclear Generation Memorandum NE-PJ-90-0017, that Post-Modification Testing requirements need only confirm the -

'

objectives-of the modification'and critical design characteristics.

'It_ was the inspector's _ view that to imply that the entire

- post-modification test process is acceptable when in fact it failed

.to identify correct installation, appeared to be' erroneous.

In a

.

subsequent conference call with the licensee staff, the intent of-

'

the response.was clarified and the inspector had no further concerns in that regard. The inspector verified through document review,

.and interview'that the corrective actions outlined in the LER had

.been completed.

The matter is considered closed, o

No violations or deviations were identified in this area.

8.

Review of Maintenance on the East Mainsteam Bypass Valve i

v>

.

.

During the inspection. period, the inspector performed.a review of maintenance work package WR N139890526. The review was promulgated by a Deviation Event Report (DER-90-0264) which documented a deficiency

that had been identified on' April 11, 1990, while the unit was shutdown following a scram.

Specifically, the DER addressed a-condition where the main steam east-bypass valve (E BPV) was determined to be incap6ble of fully

j stroking.

Although'the valve itself is not designated as safety related,

,

,

L it' performs a function-important to safety in that consideration is given i

to the power level below which a reactor scram on a turbine trip may be

{

bypassed -(assning a given amount of mainsteam bypass capability).

This

is reflected in the technical specifications as a MODE consideration.

J (i.e., not. requiring scram 'on turbine trip in MODES other than MODE '1).

I Additionally bypass capacity is factored'into thermal hydraulic limits

imposedonreactoroperationviaminimumcriticalpowerratio(MCPR)

values.

As such, the licensee designated this equipment'as QlM, which administratively was to be treated as equivalent to safety related' under j

'

the licensee's work control. program, al The-inspector reviewed work package N139890525 and interviewed members of

the operations,' maintenance, technical, and nuclear engineering staffs j

,

and.. determined the following:

-p

,

a.

The valve was rebuilt during'the refuel outage that occurred l

'

'

during the September-December 1990 timeframe and the stroking i

problem occurred as a direct result of that work.

During the

'i rebuild effort the valve stem was replaced. The replacement j

was procured through English Electric, LTD (the main turbine

vendor) and met all required vendor specifications.

However,

the replacement stem was slightly longer and therefore certain j

i J

n

s

. - -

. -.

- - - -

-

-.

-

-

. -

.

. -

- -

- -. - -

- - - - -

-

.-

.1

-

'.

.,.

,

.

,

,

g, e..

!

,v-

,

,

'

i (l steps for setup were required to properly = mate the valve to its l

,

f associated valve actuator. This setup was not properly performed-l

,

by the-contractor (Westinghouse) doing the job and as a result, l

,

'

-linkages in the-actuator were skewed out of = their normal-positions.

n

-

-Thereafter, when the valve was stroked in the open direction,..

,

,

.

the linkages moved such that they contacted a mechanical stop-

""..

'at approximately 88 percent open.

,

b.

'The root cause for the inadequate maintenance was a breakdown!

l

,of-nearly' all aspects of the work control process associated'

~

.with this activity.

,

,

(1) The-work instructions (procedures) were developed.cnsite

'

by. Westinghouse just prior to the beginning of the refuel y

outage and submitted to DECO personnel for review / approval,

Because Westinghouse had contracted to do all the main..

j

,

turbine generator work during the outage and'that work was-

~

y assumed to be all non safety related,. Production Quality

I

' Assurance (PQA) did not become involved in the certification i

of the contractor's procedures. The technical engineering

.!

group did review the procedures and approved them for' usage, j

However,-the inspector subsequently determined the t

Westinghouse procedures to be-. inadequate.. This was based on the. fact that the instructions'were generic in certain

,

areas, did not reference tha actuator-to-valve setup

.

instructions or include

, e.ular drawing which provided

the setup: instructions, number of steps erroneously <

,

referenced DECO mainter.

'uction NPP-35.109.02 as the

-

.

procedure to follow at A

. points. = NPP-35.109.02 was not the maintenance instre on for bypass valve work but

!

rather directed control valve-work.

Additionally. the steps-l referenced from the procedure were not related in either l

- procedure to the particular activity being ~ accomplished at-

.

that point.

l (2) Plant personnel indicated that a subject expert from the:

l turbine vendor (English Electric) was in attendance'during :

'

..the rebuild activities, and provided an oversight function

.'

as an extra level of control during critical' steps to ensure

the valve was properly worked on.

It was unknown why the'

t vendor representative did not identify the deficiencies-

?

' relating to the setup activities,

,

(3)

'A PQA holdpoint established for replacement of-any parts-was inadvertently missed.

The apparent cause of this was -

that.the holdpoint was specified in the main work package.

,

However, the main work package included a number of sub packages only one of which was the rebuilding of the E BPV, and the parts replacement holdpoint had not been

'

transferred to the sub package which the workers had at the job site.

!

,

.?

.

.

.

.

.

.

.

.

.

.

..

E

',.

>

y,:. n

,-

[o,

.<

u V

_,

-(4)~ Nopostmaintenancetesting(PMT)wasspecifiedin-the work package. Typically, if PMT is required, the administrative controls established to ensure the proper-m

,

sequence, authorizations and review is implemen_ted is via a PMT sheet which would include those specific requirements.

.

Neither the' inspector nor licensee personnel could find a-PMT sheet in this work package.- In fact. only one PMT could be found. for. all the' turbine generator packages.

The:

,

. inspector requested the licensee determine if other means.

of reconstructing whether PMT (traceable to the work package) was viable..No alternate' documentation of PMT

.

<

'

could be found.

The inspector has to infer from this that w

no PMT was_ formally specified or performed at the time ofs

<

return-to-service.

'

(5) ; Review of' the completed' work packages for. technical adequacy _

during the final closure cycle was inadequate. The licensee

.d

,.

-informed the inspector that only a sample of the completed -

j packages related to the main turbine generator work received; a technical review. The others received what was termed a.

)

" sanity" check which seemed to be an administrative--review -

to ensure all required documentation was included in the-q ackage.

The work package for the E: BPV received this-i

.p' sanity" check, i

The following four parts of 10 CFR 50 apply for the above-

. described conditions for the. bypass valve.

i y

10 CFR 50, Appendix B, Criterion V " Instructions, j

Procedures,:andDrawings,"statesIn.part" Activities o

affecting quality shall:be prescribed by documented

,

instructions.... of a itypeLappropriate' to the

..

circumstances..,. ' Instructions ~... shall include-

.

appropriate quantitative or qualitative acceptance criteria-j for determining that important' activities have'been

!

u satisfactorily accomplished."

_,,

10 CFR 50, Appendix B, Criterion VI, " Document Control,"

j states in part " Measures shall be established to control-the j

issuance of.... instructions... which prescribe all.

.

activities affecting quality. These measures shall assure

'l that documents... are reviewed for adequacy.. _.."

]

10 CFR 50, Appendix B, Criterion X, " Inspection," states in.

!

part "A program for inspection of activities-affecting quality shall be established 'and executed....

<

Examination... shall be performed for each work

!

operation where necessary to assure quality '...."

"

10 CFR 50, Appendix B, Criterion XI, " Test Control," states in part "A test program shall be established to assure that j

all testing required to demonstrate that structures, systems,

>

and components will preform satisfactorily in service is i

'

,,

.q 1 -

'

i

,_f__

_

p

.[.

I

.

,

..

m =

';._

,

. identified and~ performed in accordance with written test

'

procedures which incorporate the requirements and acceptance.

limits. contained in applicable design documents."

'

T-Contrary to the above, while rebuilding of the Important-to. Safety-east mainsteam bypass valve during the fall 1989 refueling outage the licensee:.

'

,

a Failed to prescribe adequate documented instructions-

!,m

'

'

including the lack of appropriate quantitative'or-

'

qualitative acceptance criteria:for' determining that

the east mainsteam bypass valve rebuilding effort was.

,

satisfactorily accomplished in'that the instructions ~

did not provide appropriate quantitative acceptance measurements for the length.of valve stroke.

.

.'

- *

Failed to establish adequate measures for~ reviewing

the' documents associated with the rebuilding-of the east mainsteam bypass valve.in~that the review process o

did not identify that' it was impossible to properly

~

r6u11d the valve with the instructions provided.

,

,

Failed-to establish / execute a program for inspection

>

" ~

of the east mainsteam bypass valve in that there were no designated inspection points or work observations prescribed.

..I Failed to establish testing required to demonstrate:

-

,

that the east mainsteam bypass valve.would preform satisfactorily in service-in'that no post maintenance?

'

'

-testing was prescribed.

'

g

.

Because of the numerous deficiencies that occurred relative to n.aintenance-I.

.on the E BPV which resulted in the valve not functioning l properly, this

. breakdown is considered a violation:of 10 CFR 50, ' Appendix; D Criteria V,. - VI, X and XI, " Quality Assurance Program," (341/90007-02(DRP . q .. ' '

In preparation for reactor startup, the operations department-conducted.

g surveillance 24.109.002, " Turbine _ Bypass Yalve Operability Test" on

November 23, 1989. During the test, it was noted that the E BPV was !

indicating a 0-88 percent stroke. The surveillance performance form

(SPF) was annotated of the fact and included information'that preventive-

'j > > maintenance activity PH K733890822 would address the problem. -The.I&C-department conducted the PM the following day without the restricted-

stroke problem being identified. The span of the position indicating

l loop was readjusted to reflect the new valve. stroke length (i.e., 0-88

' u percent valve stroke equated to 0-100 percent position indication).

] Surveillance 24.109.002 was then reperformed and all acceptance criteria j met.

Since the root-cause of the deficiency identified during the first j surveillance was not adequately determined, so that corrective actions

could be implemented, the reactor was operated at power with the mainsteam ci 't i ! 's

i l l, j , - - .

ao fje .;o - . x p, , 's I bypass system in a' degraded condition.

This is considered a violation ' of 10 CFR 50, Appendix B, Criterion XVI, " Corrective Action," , L .(341/90007-03(DRP)), e On December 15, 1989, the operating crew noted-that the E BPV was not giving.a dual indication until the valve stroked 40 percent open.

No ' operability concern was evidenced in'that the'positiou indicating meter.

appeared.to be properly functioning and the valve was responding as it

should. However,-a work request was initiated at that time to address' p the dual indication problem.

The troubleshooting of this problem'under .that work request, on April 11, 1990, resulted'in tho discovery o.f the stroke problem.

Corrective actions were then initiated to return the valve to full stroke capability.. Those actions werr completed ptior to

returning the reactor to power. An engineering eveluation was performed . and determined that the design steam flow would still have been achievable .-through the bypass system with the west BPV capable of fully stroking and ' '

the E BPV limited to 88 percent.

" In response to a number of problems that had been identified during the-November-December 1989 timeframe, the licensee initiated the Accountability.

' Action Plan.(AAP) to determine the scope of any further problems that had: .not been identified and to ensure adequate corrective actions would be taken (reference Inspection Report 341/90002).

Three areas of followup ' that'the'AAP was to address that potentially could have identified this q issue were: j a.

'The Licensee would review all LCOs generated and closed during the I first refuel outage,;to ensure adequate return-to-service testing ! was= performed.

j o b.

All. surveillance test packages generated during or after the outage, j which were completed but with noted problems, were to be reviewed to j

determine whether additional testing required as a-result had been

, properly completed.

'c.

All open work packages for safety related equipment that were filed j complete but awaited paper closure were to be reviewed for any operability concerns.

, Although Item I had been reviewed, the licensee apparently had not ~' ncluded the turbine valve work sub packages under this action.

Item 2 l i - was addressed to the extent that the deficiency identified during the j operations surveillance of the BPV was checked to ensure another-surveillance had been subsequently performed with the acceptance

criteria met.

No technical review was conducted for that item on

any of the surveillances reviewed under the AAP. ~ Item.3 was not

-specifically addressed since the turbine generator work was assumed i to-be all non safety-related. Therefore, with the BPV being designated Q1M (QA Level 1 for maintenance only) it was not included in this portion of the review.

! , f )

< - > ' [

ex - c a

4 .O

^[ . c o-c .. , (o e F

' C .' i _9.

Reactof Scram Followup

. , - a.

Introduction w On April' 10,:1990 the reactor scrammed from 1000 power.. The initiating event was an equipment failure. At 0156 the motor generator set to' reactor protection bus A tripped when a motor protection relay.

failed.

Loss of reactor protection-bus A caused numerous engineering

safety features equipment to' automatically start and numerous ' , containment isolation logics to actuate.

Included in those a containmentL isolation actuations was Division 1, of-the drywell- ' pneumatics system. Approximately six minutes later the. inboard main.

j stean.. solation valves (MSIVs) drifted closed due t'o a loss of motive ' d ? ' force, nitrogen from the drywell pneumatics-system, necessary to keep: ~ the valves open. Closure of the-MSIVs actuated a reactor scram. All , control rods inserted into the reactor core.

Over the next 2-1/2_ hours reactor pressure was controlled through-

use of the safety relief valves discharging to the torus. The use; , of reactor core isolation cooling (RCIC), as a pressure control

.' mechanism,.was' attempted but was not totally successful. During 't this same time period,. reactor vessel level was controlled by standby-feedwater and RCIC.

Torus water temperature was maintained to an i acceptable limit by placing residual heat removal (RHR) subsystems- , into the-torus cooling mode.

Torus water level was maintained to an

E acceptable height by the use of the torus water management system.

'! , " During this post scram recovery period numerous equipment problems ' were encountered and two spills occurred, one of approximately 25 gallons and the other of approximately 62,000 gallons, i Two and:a half hours after the scram, the MSIVs were opened and the d L condenser was established as the' reactor heat sink.

Normal shutdown, ., L operations continued from this point.

! l .b.

' Initial Operator Response } > l .0n shift personnel responded properly to the loss of;RPS ' bus A.

W. hen - p the scram occurred, an improper E0P entry condition, MSIV closure, was , ' selected for the use of the E0Ps. This was a non-critical decision in I ' that within six seconds a true entry condition, low reactor water level, existed. The appropriate immediate scram actions were~ properly j carried out and torus cooling was established prior to the SRVs. O lifting.

Subsequently, the E0P entry conditions were reemphasized

to all-operating. crews, r ' c.

Reactor Pressure Control Review ' The safety relief valves (SRV) performed as designed including.the lo-lo set logic associated with SRVs A and G under seven separate , automatic or manual pressure reduction actuations.

During the manual { use of the SRVs, the same SRV, A, was repeatedly used in lieu'of the SRV cycling matrix.

This was a decision made by the NSS based upon , the knowledge that SRV A was working even with some level of failure l l '

.;

w -; .g} g' f ' , . w . - w .i s: ! of the'drywell pneumatic ~ system, torus cooling was in service assuring adequate; thermal' mixing and'SRV A'would be the first valve actuated by , , the automaticLlo-lo. set. logic. Though this was a noncritical- .. decision it:was-not' consistent with operator training on the t use of SRVs.

, Even though the SRVs functioned properly, the number of SRV' lifts j' ~ could have been significantly less through the use of RCIC in.the-test mode In the test mode, RCIC takes a suction from the condensate '

storage tank (CST) and discharges to the CST.

Op?rators initially' [ ' s attempted to use the CST-to-CST mode.at 0245:without venting discharge piping in the CST test line and with the RCIC turbine still in

' operation. One of the discharge valves, E41F011, would not open.

' and its thermal overloads tripped. Once the:line was" vented,. additional attempts to open E41F011 were unsuccessful resulting in tripping'of:the thermal overloads and loss of the valve motor through: o a winding short.

The use of the CST-to-CST pathway is severely hampered by an ' undersized motor on valve E41F011, which is unable to open under i > RCIC discharge pressure, resulting in the need to' shut down the RCIC turbine and vent the test return line prior to its use.

This takes l considerable timeL and must be' done from outside the: control room.

Also, this valve is kept normally de-energized'at the~ motor control ' center due to Appendix R fire protection requirements... It takes' time-to rack-the breaker in and' install its fuses outside the control room prior to valve usage.

. The inspector pursued whether this means'of pressure control, i CST-to-CST, was approved by the licensee and:1f so, what level of review had been performed on the' use of'this pathway. - The inspector was informed by the licensee, following discussions with General Electric, that the CST-to-CST pathway was considered in ~the' development of the. generic emergency procedures-guidelines. The-inspector ascertained that the training department considered this-an appropriate means to control pressure,Lbut no training had'ever '

been given to' operators on the use of the CST-to-CST pathway under emergency operating conditions. Engineering diJ not evaluate the switches / controls associated with the-test return pathway in the , .' E0P review..EOP verification-and validation actions did not encompass this pathway, sincerit is done' through reference to s S0P 23.206.

Therefore, the design weakness was not rectified or

compensated for in more intensive operator training.

The operating crews were provided training on the E41F011 valve-as to the need to vent. The CST-to-CST operation will be evaluated, .."' verified and validated under the E0P program.' To address any other potential weaknesses in E0P usage the licensee committed to evaluate all E0P evolutions to assure that the critical actions are encompassed by operator training by July 31, 1990. Com I considered an open item (341/90007-04(DRP))pletion of these actions is l .

. , .

, ...... _.. . -.. _.......,....... .... - -.- - -.. " ' .,f a .A , ,, " (, , , 't I In: addition, prior to the April.10th scram, the E41F011 valve was in ' a. degraded condition. -On December 13,'1989, this VDC valve's stroke-p , E time increased from 36 seconds to 52' seconds and placed on increased A '

testing frequency'_ The memorandum informing'the cognizant' system

, . engineer.of the stroke time -increase was not sent until February 20, 1990.

The system engineer was on! maternity leave and the matter was ' left unattended by her supervisor, who was short two of five systems _ engineers during this time frame.

No' corrective action was taken with- -, regards _to this valve until after its failure on April 10th.

> Subsequently, the licensee has provided a higher degree of visibility-to equipment trend problems. Whether these corrective actions are effective.will be monitored in future inspection periods.

'd.

' Reactor Vessel Level Control Review " The standby feedwater system was initially used to maintain between ' Level 2 and Level 8.

level control-was transferred to RCIC after

e approximately five minutes. When the B SBFW pump was shut down, the W

auxiliary oil pump trip light illuminated. _This particular pump was not used any more in the post scram recovery.

RCIC continued to-be used for. level control until the RCIC turbine tripped on' a high reactor vessel. level condition (Level 8) at 0255. 0ver the next. 25 minutes level decreased to.115 inches at which time RCIC was placed back in service increasing vessel level.

Following a vessel swell,

due to lifting SRV A at 0410, a level 8 RCIC turbine trip occurred.

' p_ rior to the need to use RCIC again the condenser was established as J T 'the heat _ sink, reactor pressure reduced and reactor vessel level: , maintained by the heater feed pumps through the startup level control system.

The inspector reviewed the equipment history on the failed auxiliary oil pump and did not identify-a previous equipment problem.

,

e.

Torus Temperature Control Review The torus temperature was controlled by RHR torus cooling which=was initiated very early in the post scram recovery. The only problem noted dealt with the ability of the operators to recognize the.E0P entry _ condition associated with torus temperature. The E0P entry condition is on torus average temperature at 95 degrees and by the "' safety parameter display system (SPDS) readouts, the highest average-temperature was 93 degrees.

However, by another torus temperature , indicating device, the torus temperature recorder, the 95 degree action limit was exceeded for 8 minutes from 0321 to 0329. The operators did not recognize this situation since primary review for torus temperature was via SPDS.

An annunciator on a main control panel, separate from the recorder, should have alarmed alerting the operators to the 95 degree situation.

Even though operators did not respond to this annunciator, the action for high torus temperature had already been taken, initiation of RHR in torus cooling, and thus had no detrimental effect on the outcome of the post scram recovery.

i !

P~ . , ,

,jm,y . . + , m

. c , ' - l < p

, ' lThedesignweaknessof=havingtheannunciatorononepanelandthe ( ' recorder on another had'already been recognized-by the~11censee's ' ' human factor's program and wil1~be rectified during RF02.- s U ~~ < ; Additional operator. training was provided to all the operating crews <

L, .on the.need to use multiple indications when monitoring.the plant.

' , .-During the-post' scram review the inspector noted that the-sequence of events (SOE) alarm' printer did not type the high' torus- ' > . . temperature alarm (8037). After discussion with the licensee, the, , Jalarm/ annunciator was tested resulting in the annunciator illuminating' ' ' but the SOE printer did not type the alarm.: During RF01, this alarm , was to be:added to the SOE printer but was.not, due to poor . instructions in-the engineering design package. -A work request has-.been-initiated to troubleshoot the alarm / printer problem.

,

f.

' Torus Level Control

Operators-appropriately controlled torus level even with~the SRVs discharging reactor steam / water into the torus. At'0310 the E0P' entry condition for high torus level was met and operators - appropriately attempted to place the torus water management-system tinto-service..One of the actions associated with placing.the system- ,into service _is opening a discharge valve to the torus, G5100F606.. The'first attempt'to open this motor operated' valve was unsuccessful- , t and only dual valve position was' indicated. A second attempt opened 'the valve fully. No other problems were' encountered.

The: inspector reviewed the licensee's troubleshooting efforts on.

valve G5100F606. The: original work request directed maintenance

. personnel :to do a complete checkout of the valve. These instructions.

' were modified to only stroke the valve'open and closed. These actions I , were accomplished and the work request closed.

The previous. maintenance history on this valve revealed that the

i valve'had torqued out going open on November 28, 1989. This was-25 days after the LLRT repair post maintenance testing was

'successfully performed.

The open torque switch setting was . increased and the valve stroked satisfactorily.

' " In both corrective maintenance instances, November 1989 and April 1990, root cause evaluation was inadequate and the corrective actions poor. The inspector discussed future maintenance actions.with the- ' electrical maintenance personnel. The licensee concluded that , additional. troubleshooting in the form of MOVATS diagnostic testing was warranted.

Subsequently, maintenance personnel found the opening

thrust to be approximately 500 ft-lbs. less than prescribed by the design calculations. The torque switch setting was increased to

provide'the necessary torque.

In addition, three other valves associated with the torus water management system-(TWMS) will be- , MOVATS tested.

, l- , l l

) . . - .

p- - Q]G*.;<L ^ y - , . ,

. , V

• .

Additional review of maintenance history revealed 'another problem.

n , o This problem dealt with the documentation of_the torque switch- ' , settings in the maintenance history.

There were numerous instances

. where the as-found torque switch setting was' inconsistent with the . , previous as-left setting. This matter.was brought to the licensee's , - attention. The licensee concluded;that movement of a backing plate _ - ! .. on the SMB-000 torque switch could allow:a one-half increment " difference in the setting, coupled with the tight location t

" associated with this. motor. operator switch compartment, caused.

- O the incons.istent reading, j , . . , At the end of the inspection period electrical maintenance management i was evaluating how best to assure consistent readings on the SMB-00')

torque switch.

62,000' Gallon Spill- ! l 'A.few minutes after the scram, the. pressure in the 5-north feedwater= heater increased to the point that the thermal relief; valve lifted allowing-feedwater to spill upon the floor and into_ the: turbine-j drain system. The continuous discharge of fe'edwater through.the.

relief valve.overstressed the relief valve piping causing, total separation of the piping and:some insulation from the'h'ater shell.

e > AtLO430 the operator in. the radwaste control room acknowledged a' _hi-hi-hi turbine building drain sump alarm. turned on ansecond sump ' pump and dispatched an operator to observe the sump, condition.1 aThe.

'!' field operator informed the radwaste control operator that the level was high but was -not overflowing. Contrary to management d'rections,

the radwaste operator did not inform the' control room of the ' unusually ~high_ sump level.

_ , The radwaste operator was hampered in being. alerted and dealing with the spill by a preexisting equipment failure. On March 3, 1989c the i hi-hi sump alarm switch was determined to be out of service, was T CRIS dotted, and had not been repaired by April 10, 1990.. The switch.

serves a dual function, first to alert the radwaste operator of-a highL sump condition and to automatically. start a second sump pump. However,

this second sump pump was manually placed in service when the hi-hi-hi.

" alarm came in later during the spill.

At 0607 the personnel in the turbine building informed the control ' - room of turbine building flooding. The leak was located in the- ., heater room.. By 0930 the heater was isolated stopping the leak-and the contaminated areas in the turbine building were radiologically posted. All of the spilled water, approximately 62,000 gallons, was controlled on site in' the power block until processed by the radwaste system.

Testing of the failed relief valve on the 5 north heater revealed- . that its setpoint was at 835 psig which is equal to the discharge pressure of the heater feed pumps.

Inspection of the sister relief line on the 5 south heater revealed cracks in the relief line piping.

'

, ! - . . . . .

p - - - g9 . l

• '

, E' r m, 4. :

The inspector reviewed: licensee actions to the vibration problem

which;isfurtherdiscussedin,InspectionReportNo.~341/90009(DRS).

0n February 26, 1989, the licensee experienced its first scram from o

100. percent power resulting in the 5 north and-1: north relief valves " lifting. This condition was documented on February 28, 1989, in~ , , DER 89-0296. A night order was written to. operators informing them ,. ^ - of the' potential for these valves lif ting and-the' necessity to check - '! these areas.- Also a design analysis was undertaken to ascertain the ", ( source of.the pressure causing relief valve actuation, j , l Corrective action under the DER was to establish balance of plant.

' ' .' pressure / flow computer trending points, check the relief valve ' O setpoints, replace the; reactor feedpump minimum flow valves to preclude _a potential: pressure transient when! reactor feedpumps ~ trip, and analyze whether these actions. resolved the~ problem.- During-RF01 these actions were taken and the 5 north: relief valve.setpoint m was found to be 935;psig. Also, at some point during 1989, the night '

order was cancelled that alerted operators to the relief valve problem.

' Following evaluation of a low power scram, in December 1989, without: => lifting heater = relief valves, technical staff personnel considered i the problem solved and recommended closure of the DER.

However, the.

Lproblem.had only been observed in scrams from high (95 to-100 percent) , power.

_{ i . Following the April 10th spill, the DER was amended and correctivet

. action included replacement of the 5 north relief valve,-stress- ! , reduction of the: relief valve line for feedwater heater 5: north' j , and south throughLshortening of the line and increasing.the weld j < A size / configuration, evaluatingJ the need for installation of a more - j reliable relief valve and modification ~of the scram recovery procedure j to mandate ' inspection of the feedwater heater rooms.

j

.h.- 25 Gallon Spill-At 0212 operators reset the scram and a low reactor water. level scram signal immediately occurred. At 0227 operators reset the > scram again. At 0232. a health physics technician observed water i.

coming from ventilation ductwork in the nurth RHR heat exchanger

room.

The vent from the. scram discharge volume-(SDV) discharges j into the ventilation ductwork at this point and the licensee determined that.the water came from the SDV vent..The water , was routed to a Corner' room sump and the control room' notified.. - , From discussion-with licensee training personnel, the inspector , < ascertained that resetting the scram with reactor pressure / level unstable was inconsistent with previous training. Though operator' actions were inconsistent with their training the inspector could ' ' o not conclude that the spill was caused by the scram resets. A DER was initiated to evaluate the spill and the design of the SDV vent.

Also, additional training has been provided to the operating crews j s on when to reset a scram.

I i'

o q

p7 - j [; K Q.,o _ n (3 O , .e, , , v p;;

,-

i.

Other Problems Following the Scram , W, The second: floor-reactor building airlock interlock failed at 0233 x !c causing operators and health physics technicians to use the fifth floor cirlock for access to the reactor building. This caused ~an . f, ' additional: delay in venting the RCIC discharge'line to the CST.

A-y , microswitch on' one of the doors to the airlock had stuck causing thel ' ". failure; There have been numerous airlock door failures:in the past.

< ' -When.RPS A-deenergized valve G33f001, the: inboard reactor water .i ' ss.

cleanup containment isolation valve, closed on a containment. r b isolation' signal,but the Group 10 isolation mimic indication did not-

_ . change state as it should have. The. isolation mimic is considered at

' ' part of:the licensee's SPDS. Troubleshooting revealed the; problem-j to be V the actualivalve or the-electrical penetration for the a g valve's circuitry which would require a drywell entry. The work

l L

request was' deactivated until a forced or. planned outage.' _The ' ~1nspector did note'that the work request had been coded as-a < Priority 3-versus a 2 as established by the work request , " administratiae controls.

i

- , The indication for the No. 4 tur.bine stop valve did not indicate' . closed in the expected time frame.

The cause was old hard grease which slowed the indication mechanism. The grease was replaced.. ,a j.

Staff-Compliance with Required Monitoring Activities , 'Following the reactor scram, the required chemistry samples were collected and analyzed within the requisite time frame. The results: i , l showed that' no fuel' damage or activity release occurred due to the: - ,, h.

transient placed-upon the plant.- ]

Upon exiting the emergency operating procedures at 0440 with MSIVs being opened and the plant stable, operators. began cooling the plant-down-for entrance into cold shutdown.

However,. operators didinot.

=begin recording reactor pressure and temperature until'0530 on L procedure NPP-22.000.05 data sheets

Use'of these data sheets is

' j h the mechanism utilized by the licensee to comply with Technical l Specification Surveillance Requirement 4.4.6.1.1'. This section'of

E . the Technical Specifications _ requires the reactor coolant system.

[ ' ' . temperature and pressure be' determined within cooldown limits at , least -once per 30 minutes during system cooldown.. The inspector.

ascertained from computer data that operating personnel did not j exceed the cooldown rate or the cooldown limits but did not record i the requisite information. The inspector _ determined-this to be a i violation (341/90007-05(DRP)) and considered whether a notice of

violation was warranted. The inspector concluded this a Severity i Level V violation because the cooldown limits were not exceeded, _ y t operators were cognizant of the cooldown requirements as determined by interviews, and the matter had minimal safety significance. A , , Notice of Violation will not be issued because this violation meets the criteria of 10 CFR 2, Appendix C, Paragraph V.A.

' l i ys

~. - - - - - -

' ' j W.4 2- > ,1

. . ' . k.

Initiating _ Event Review ., At'the loss of RPS A, an operator observed smoke from the RPS MG set ' cabinet.' Therefore... troubleshooting / inspection centered on.that . cabinet.- This inspection determined the initiating event to be a-

. , failed relay coil.

The coil was sent to the engineering research

' department' for analysis which determined a.high temperature failure of the coil.

'Other coil-failures with great similarity have been observed in.the . industry. These failures occurred after about.10-years of service and-were attributed to the elevated temperature in the MG set' cabinet.

'< caused by a cabinet design which lacks forced cooling.

This particular relay had been' inservice for seven years. Additionally, - -it -is most probable that the already shortened relay life was ~further-shortened by maintenance evolutions on area vantilation-units which increased the ambient temperature to its maximum = allowable.

The relay in the companion MG set has-been replaced and a periodic irelay replacement-schedule,_probably every five years, is being established for these relays.

' l '. oRoot Cause of-the Scram - ' Prior to-RF01, the transversing incore probe (TIP) purge system-nitrogen supply came from the inerting system.

In 1988 indexer B of, the TIP system was replaced in the drywell. When the indexer was- , installed.in'the_drywell, the ten compression fittings made up to - . the indexer leaked. This leakage was considered acceptable, since there were no leakage requirements associated with the compression fittings.

During RF01 the TIP purge system was modified such that the TIP purge gas came;from the drywell pneumatics. system and the purge . pressure was increased. ~The engineering evaluations associated with-the design change only considered SRV' opening capability and.MSIV.

closing capability, not MSIV opening capability.

Indexer B compression fittings were not repaired and no leakage requirements: were-established associated with the integrity of the.TIP system in the design change' post modification testing.

, Following RF01, nitrogen consumption doubled and drywell venting- -increased. Operations personnel determined the leakage rate and-ascertained that it was from the drywell pneumatics system, but ' due to communication weaknesses, with the technical staff never

. received input that the leakage was from indexer B.

.c The preexisting indexer B leakage, coupled with the increased purge pressure, invalidated an input to the simulator on drywell pneumatic-leakage and rendered previous training on loss of a RPS bus inadequate.

Operators had been trained on loss of an RPS bus with no reactor scram for at least 20 minutes.

This had held true in numerous actual losses I

'

1 ';[.! t )$ f

qi

, 4J .,6 '

.

m W

' ... ' ~ . of a.RPS bus-in the facility prior to RF01.

However, the design N change process did not take into account the ramifications of adding another leakage pathway the TIP system to the drywell pneumatics

system.- Also, there never existed an initial or continuing validation Lof the nominal leakage inputted into the simulator for pneumatic <

leakage upon which. operator training was based.

,m.- Inspector Review of Licensee Scram Evaluation- - 'The scram report was completed in accordance with licensee procedures.

The scram was properly classified.

The root cause of the scram was properly determined. Generally, the scram analysis was good.

There, was one weakness dealing with the lack of=a prescribed method for 1.nterviewing the operators upon relief of their duties.

Significant Inspector _0bservations " n.

': '(1) Plant operators were able to successfully carry out the primary functions of the E0Ps for this event even with numerous - equipment failures and failing to execute all E0P actions-in accordance with previous training.

) l (2) -Operators did not fully' establish all the requisite administrative duties upon exiting the E0P umbrella.-

(3) LThere was inadequate periodic testing / validation of the assumed- , simulator drywell pneumatic system leakage.

The ramifications of adding the TIP purge. system to the drywell: pneumatic system - was not adequately evaluated by engineering / technical support personnel and -limited post modification testing was established c_ " for accepting the modified drywell' pneumatic ' system.

(4) The CST return line design-is weak. The evaluation for use . of the CST return line'under the E0Ps was inadequate and E0P

operator training on the use of,RCIC in.the-CST-to-CST moda I ^ was inadequate.

-(5) The corrective-actions to the original heater relief valve.

f problem in February 1989 was inadequate to preclude the ! . 62,000 gallon spill.

< i (6) There was no evidence of interdepartmental-teamwork / communication j in attempting to resolve equipment deficiencies as' evidenced by _l the' lack of timely notification and correction for the increased ' n stroke time of valve E41F011, technical staff failure to inform , operations of the where the N2 leak was coming from following l RF01, and failure to understand that G5100F606 did not open on ! first demand.

I C (7) The present visual observation techniques used by maintenance personnel to ascertain the torque switch setting on a SMB-000 i is not repeatable.

, i

>

,

< i

[ o ',.g

. , p g: * s< _ in

,(

L-10.

Vibratibn-Induced Cracking in RHR Small Bore Connections (37701) ic The status of the licensee's program to eliminate cracking in RHR small bore connections was reviewed by a region based NRC inspector. The weld f ailures forming _ the basis for the investigation-were summarized in the-

licensee's presentations given at NRC Headquarters and Region III in late

'Apri141990.

These consisted of the following failures: 'The 1987 fatigue crack in the "A" pump discharge header drain.

'

• -

-The 1988 fatigue crack in the "D" pump suction sample line.

The 1989 pin-hole leak in the "C" pump suction sample-line.

• The 1990' fatigue crack in the Division I LPCI high point vent.

. Prior to the 1988 failure, the pump on the "D" line had been run-dead-headed.

During the period that the pump was dead-headed, vibration 'in the= system would be expected to increase significantly, thereby increasing.the tendency to cause vulnerable areas to fail.

The'1989 pin hole was ground out and repaired, No root cause was formally established.

< 'The 1990 defect occurred at the base of a fairly long, cantilevered vent line with two valves and a cap at the end.

The crack occurred in the' nipple at the toe of the weld, at a stress riser where the cross section , changes abruptly.

The : sources of the ' vibration to which the small bore failures were ~ attributed:are the RHR pumps. Other failures in the area which tend to reinforce this position include the failures of: (1) a threaded pipe connection supplying emergency. equipment cooling water to RHR pump shaft ~ ,

seals; (2) bolts-holding electrical conduit boxes to two-different RHR pumps; and (3): a mechanically connected. snubber oil reservoir. Attempts to replace the snubber oil reservoir failed until it was moved to a more

' vibration free mounting surface.

The licensee's concern with the vibration in these pumps prior to the plant's operation caused them to call in General Electric, Byron' Jackson, and finally, their consultant,: Structural Dynamics Research Corporation.- - Structural Dynamics, in Report SDRC Project No. 11472 dated October 1982, recommended the installation of additional bracing to the system.. That bracing was installed and is still in position. Additional analysis.on the pumps was performed by the licensee and reported in Engineering .Research-Department Report 850-15-5', dated May 15, 1985.

That report established pump baseline vibration values for use in inservice testing (IST) where results are reported to show no unacceptable increase in vibration from that baseline.

p g- '4 8' ., ee .i- ,- + ..(

s

Different degrees of corrective action were provided by the licensee ini each case.

For example,. the cantilevered LPCI high point vent ~with~ two valves and a_ cap was modified in three ways:

One valve was eliminated, thereby reducing cantilevered mass.

' E The lengths of the nipples were reduced, thereby shortening the , moment arm.

• The welds were reinforced and abrasively contoured to form a

< L continuous, gradual curve to minimize stress concentration at the' intersection of the weld with the nipple.

H.

Other' welds were treated on an individual basis as the licensee's analyses-- dictated. Small bore joints in the RHR and in.the.ECCS cooling water to .

l the RHR pump shaft seals were analyzed for stress. Any joints which

. m h exceeded the stress limit recommended by OM.3-1982, (10 Ksi) were modified i as necessary to bring the final stress down to OMa-1988 limits.(7.69 Ksi).

The basis statement was weld reinforcement and blending to eliminate stress risers.

If-the nipples were longer than necessary, they were reduced in.

uE length.: -Fin' ally, where possible, the supported _ mass was minimized.

In the ? case of the LPCI high point vent, this required elimination of:one redundant valve.

The.NRC inspector reviewed several representative welding procedures and ' discussed;the small bore joint modification philosophy with the responsible engineers.

The welding procedures were acceptable and the modification-

, philosophy was based on sound, well established engineering practice. -The .i ! personnel associated with the corrective actions-were competent and wel1 . . qualified to define the necessary changes, !l

Although the. licensee's investigation into the failure of RHR small bore-L socket weld failures is incomplete, the NRC inspector saw no evidence that' i faulty welds were the root-cause of the problem. The weld defect most-frequently reported'in these welds 'is porosity.

Since socket welds are j l-normally subjected only to surface type examinations, subsurface porosity as'such, would be undetectable.

However, there is no indication that porosity contributed to the ' observed fatigue cracking.

. Incipient vibration-induced cracking occurs on the outside surfaces of the part and preferentially occurs at highly stressed areas in which a sharp l, . change of cross section takes place.

The preliminary evidence reviewed by j ' the NRC inspector indicated that the cracks detected to'date in the RHR- ' small bore piping originated in such areas.

Vibration induced cracking- , would be expected to occur at these-areas even though the welds were of i , acceptable quality, j ' In summary, the.NRC inspector confirmed that the licensee's efforts to I resolve the vibration induced cracking of small bore RHR piping joints is

progressing in a satisfactory manner.

Future progress of the program will .f continue to be monitored by resident and region based inspectors.

.; - J i

,

TT '- ' ' T' - [[.3:7.y - 8.-."

-

' .,$ f 11. 'Self-Assessment Capability (40500) ' - ., [~ On May 10, 1990,' the inspector attended an offsite review committee.

' = meeting. The inspector determined that quorum and composition requirements , . -.were met. The content-of the meeting was consistent with Technical ,; -Specification requirements.

~12.

Followup of: Fitness for Duty temporary Instruction (255104) -(Closed)LTI 2515/104)i Inspection of Initial Fitness-for-Duty Training- ' . Programs. During 'the'. inspection period the inspector attended policy . awareness,' supervisory and escort #itness for duty training.

Upon' attending the classes, the inspector completed the applicable questionaires.

, , ,13.

Followup of TMI' Action Items (NUREG-0737) , 1.. During the inspection period, the following TMI action Item was reviewed: ' ~ , . (Cl o sed) - I. G.'1. 3: Special Low Power Test Program for BWRs. As indicated-iniInspection Re. port 341/90005, the licensee was to reconstruct the training attendance master list for those plant: evolutions' that were required to be observed per this task action item.

That activity was ~ subsequently completed and the inspector reviewed the attendance-list and verified requisite, members of each operating shift in existence at the time of low power testingz had observed the testing. This TMI item is considered closed.

14. -0 pen Items Open. items are matters which have been discussed with the licensee,~which will be reviewed further by the inspector, and which involve some action i on1the part of the NRC or licensee or both. Two open items disclosed r during the inspection are discussed in Paragraphs 5.b and 9.c.

15.

Exit Interview (30703) y The inspectors met with licensee representatives (denoted in paragraph 1)- on June 1, and informally throughout 'the inspection period and summarized the scope and fidings of the inspection activities. The inspectors also discussed the likely informational content of the inspection report-with , ' ' regard to documents or processes ' reviewed by the inspectors during the

' inspection. The licensee did not identify any such documents / processes-as proprietary.

' \\ [

. }}