IR 05000244/1993022

Insp Rept 50-244/93-22 on 931024-1127.Violation Noted.Major Areas Inspected:Operations,Maint,Engineering & Plant Support

ML17263A505
Person / Time
Site:	Ginna
Issue date:	12/20/1993
From:	Lazarus W NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I)
To:
Shared Package
ML17263A504	List: IR 05000244/1993022 ML17263A503
References
50-244-93-22, NUDOCS 9312280143
Download: ML17263A505 (54)

Text

U. S. NUCLEAR REGULATORY COMMISSION

REGION I

Inspection Report 50-244/93-22 License: DPR-18 Facility:

Inspection:

Inspectors:

R. E. Ginna Nuclear Power Plant Rochester Gas and Electric Corporation (RG&E)

October 24 through November 27, 1993 T. A. Moslak, Senior Resident Inspector, Ginna E. C. Knutson, Resident Inspector, Ginna Approved by:

W.

hief, Reactor Projects Section 3B Date INSPECTION SCOPE Resident inspection ofplant activities including operations, maintenance, engineering, and plant support.

9312280143 931221 PDR ADOCK 05000244 Q

PDR,

EXECUTIVE SUMIVIARY Several significant challenges to stable plant operation occurred, including a reactor trip resulting from a failed feedwater regulating valve positioner, and entry into cold shutdown to replace a section of steam generator blowdown piping that developed an unisolable leak.

Operators appropriately responded to off-normal conditions, stabilizing the plant.

Management's conservative decision to place the plant in cold shutdown for blowdown piping replacement demonstrated a strong commitment to plant and personnel safety.

Maintenance Failure of a feedwater regulating valve positioner could have been averted had the maintenance department taken broader actions in response to a recent industry technical bulletin. Two valves in the containment spray system did not have their chains and locks reattached following completion of a surveillance test as a result of technician's not following the guiding procedure.

The valves were found in their required positions and the oversight did not affect system operability.

Management promptly responded by having a comprehensive investigation performed to identify the contributing factors and subsequently took measures to prevent recurrence.

~

~

Engineering Corporate and site engineering staffs effectively coordinated modifications to the AMSAC system, corrections to feedwater regulating valve response, thereby improving ADFCS operation, and replacement of a leaking section of steam generator blowdown piping.

Plant Support Health Physics personnel effectively coordinated in-containment activities to minimize personnel doses during blowdown piping replacement and other outage-related maintenance.

Chemistry technicians conscientiously monitored changes in steam generator blowdown sample conductivity and sulfate concentration during plant startup, alerting operations management when values approached action levels.

Through the use of a helium tracer gas technique, technicians identified leaking main condenser component a Executive Summary Safety Assessment/Quality VeriTication Management utilized various resources to assure that work met acceptance standards and that the underlying causes of plant incidents were identified and expeditiously corrected.

These resources included having comprehensive human performance evaluations (HPE) performed to establish the factors that resulted in a failure to reinstall chains/locks on two valves and why a pre-critical reactor trip occurred, The Quality Assurance organization provided oversight for various surveillance tests and maintenance activities, including verifying that the blowdown piping replacement met applicable code TABLEOF CONTENTS EXECUTIVE SUMMARY lv 1.0 OPERATIONS (71707)

o

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

1.1 Operational Experiences 1.2 Control ofOperations...............................

1.3 Automatic Reactor Trip due to Failed Main Feedwater Regulating Valve Positioner................ ~.....................

1.4 Reactor Shutdown due to Unisolable Leak in "B" Steam Generator Blowdown Pipmg 1.5 Automatic Reactor Trip During Startup due to High Source Range Flux vel t

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

Le

~

~

~

~

2.0 MAINTENANCE(62703, 61726)..

~ ~....

2.1 Corrective Maintenance 2.1.1 Routine Observations.......

2.2 Surveillance Observations 2.2.1 Routine Observations.......

2.2.2 Incomplete System Restoration System Performance Test....

~

~

~

~

~

~

~

~

~

~

~

~

~

Following Containment Spray

~

~

8

10

10 ENGINEERING (71707, 92701)

3.1 AMSAC Design Modification ~..... ~...... ~....

12 PLANT SUPPORT (71707)

4.1 Radiological Controls.......................

4.1.1 Routine Observations,................,

.

4..2 Security 4.2.1 Routine Observations................

~..

4.3 Fire Protection...........................

4.3.1 Routine Observations...................

4.4 Emergency Preparedness...... ~..............

4.4.1 Annual Emergency Preparedness Evaluated Exercise 4.4.2 Prompt Public Notification System Test

12

12

13

'3

13 5.0 SAFETY ASSESSMENT/QUALITY VERIFICATION 5.1 Periodic Reports...................

5.2 Licensee Event Reports...............

13

6.0 ADMINISTRATIVE(71707, 30702, 94600)

6.1 Backshift and Deep Backshift Inspection 612 Meetings

~

~

~

o

~

~

~

~

~

~

~

~

~

~

~

~

o

~

6.2.1 Training Program Meeting 6.2.2 SALP Management Meeting 6.2.3 Exit Meetings

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

~

14

14

15 lv

DETAILS 1.0 OPERATIONS (71707)

1.1 Operational Experiences On November 7, 1993, main generator output ramped down from 505 megawatts (MW) to 461 MW over a period of approximately 30 minutes due to a temporary loss of steam to the main condenser air ejectors.

The problem resulted from a failed valve stem and was quickly corrected.

On November 10, 1993, an automatic reactor trip occurred due to low-low (17 percent) water level in the "A" steam generator.

The low-low level condition was the result of a feedwater transient caused by mechanical failure of the valve positioner for the "A" feedwater regulating valve.

Following repair to the feedwater regulating valve, criticality was achieved on November 11, 1993 and the plant was placed on-line and power escalated on November 12, 1993.

Power was reduced from 86 percent to 48 percent on November 14, 1993 for identifying and plugging main condenser tube leaks to correct off-normal secondary water chemistry.

Following tube plugging, power was increased and stabilized at 98 percent on November 16, 1993.

On November 17, 1993, power was expeditiously reduced and the plant placed in cold shutdown to repair a through-weld pinhole steam leak that developed in an unisolable section of the "B" steam generator blowdown system piping. Affected piping was subsequently replaced and reactor startup began on November 22, 1993.

The plant was placed on-line later that day.

Power was held at 48 percent on November 23, 1993 to investigate the cause of off-normal secondary water chemistry, Following application of a sealant to the 1B2 waterbox tube sheet/condenser shell interface, power was escalated, with full power (98 percent) achieved on November 24, 1993. Full power was maintained through the remainder of the inspection period.

1.2 Control of Operations Overall, the inspectors found the R. E. Ginna Nuclear Power plant to be operated safely.

Control room staffing was as required.

Operators exercised control over access to the control room.

Shift supervisors maintained authority over activities and provided detailed turnover briefings to relief crews.

Operators adhered to approved procedures and were knowledgeable of off-normal plant conditions.

The inspectors reviewed control room log books for activities and trends, observed recorder traces for abnormalities, assessed compliance with technical specifications, and verified equipment availability was consistent with the requirements for existing plant conditions.

During normal work hours and on backshifts, accessible areas of the plant were toured.

No operational inadequacies or concerns were identified.

1.3 Automatic Reactor Trip due to Failed Main Feedwater Regulating Valve Positioner Event Description On November 10, 1993, operators commenced a routine performance test of the "A" motor driven auxiliary feedwater (MDAFW)pump.

The "A"main feedwater regulating valve (FRV)

was operating in manual to control feedwater oscillations, as discussed in inspection report 50-244/93-10.

In response to the expected increase in total feedwater flow caused by operation of

the "A"MDAFWpump, the control room operator reduced main feedwater flow by adjusting the "A"FRV slightly in the closed direction. After initiallyresponding normally, the "A"FRV subsequently began to open.

The control room operator attempted to close the valve, but operation of the main control board positioner had no apparent affect.

Within a period of approximately 20 seconds, the "A" FRV had reached its full open position.

As the "A"FRV was failing open, main feedwater flow to the "A" steam generator (S/G) was increasing; this, in turn, caused a reduction in main feedwater flow to the "B" S/G.

The advanced digital feedwater control system (ADFCS) responded by causing the "B" FRV to open more fully. As the transient progressed, feedwater flow increased beyond the limitfor ADFCS automatic operation, and the system automatically shifted to manual.

In such a transition, the FRVs are left in the "as-is" position.

Consequently, the end condition of the FRVs was with the "A" FRV failed fully open and the "B" FRV opened more fully than was required for normal steady state operation.

Water level in the "A" S/G increased as a result of the "A" FRV failing open.

When level reached 67 percent, feedwater to the "A" S/G was isolated due to actuation of the high S/6 water level engineered safety feature (ESF).

This protective feature operates independently of ADFCS, and functions by venting the associated FRV air operator.

When the "A"FRV closed,

"A" S/G water level rapidly decreased due to continued operation at full power, and the feedwater isolation signal cleared.

With operating air restored, the "A" FRV returned to its failed-open condition.

When the "A" S/6 feedwater isolation occurred, all main feedwater was diverted to the "B" S/G.

Due to the position of the "B" FRV that had been established earlier in the transient, this resulted in a rapid increase in "B" S/G water level. At the same time, level in the "A"S/G was decreasing due to the feedwater isolation, and operators were still attempting to establish control of the "A" FRV.

As a result, no action was taken to control level in the "B" S/G.

The level increase was slowed when main feedwater flow was restored to the "A" S/G, but not in time to prevent a feedwater isolation from occurring on the "B" S/G.

With the "A" FRV again fullyopen, and main feedwater flow to the "B" S/G isolated, level in the "A" S/G again began to rise rapidly.

However, prior to level reaching the main feedwater isolation setpoint, the "A" FRV again failed closed.

"A" S/G level decreased to 17 percent, which produced an automatic turbine/reactor trip on low-low S/G water level; With the prompt decrease in S/G level (shrink) due to loss of steam load, the "A" FRV again failed open.

The "B" S/G feedwater isolation signal had cleared just prior to the reactor trip, and shrink in the "B" S/G provided additional margin to a subsequent feedwater isolation.

This, along with'he failed-open condition of the "A" FRV, caused the addition of main feedwater to both S/Gs followingthe trip to be much greater than normal. As a result, the reactor coolant system (RCS)

cooled down more rapidly than normal. Since ADFCS had shifted to manual, automatic closure of the FRVs as RCS temperature approached no-load temperature did not occur. This produced additional RCS cooldown, with the resultant reduction in RCS pressure.

Operators recognized

the excessive RCS cooldown and secured the main feedwater pumps to reduce the cooldown rate.

When the main steam isolation valves were closed, as directed by procedure, RCS cooldown was arrested and plant conditions were stabilized in hot shutdown.

The cause of the "A" FRV failure was found to be that the linkage between the valve positioner and the valve stem had separated; specifically, the nut/panhead screw that connected the linkage rod to the valve positioner feedback arm had loosened and separated.

When this happened, the valve positioner failed, by design, to a position corresponding to the valve being fully closed.

In response to this erroneous feedback, the valve had fully opened.

Corrective Action/Gutage Maintenance Activities The licensee determined that the cause of the "A" FRV failure was that the nut that had been used on the linkage pivot that failed was inadequately secured, in that it was neither a self-locking nut, nor had a lock washer been used.

Per the manufacturer's parts list, the nuts used for this application are to be elastic stop nuts.

Licensee inspection of air operated valves in the plant determined that 30 other valves, including the "B" FRV, did not have elastic stop nuts installed.

As corrective action, elastic stop nuts were installed in the valve positioner linkages of both FRVs. The existing panhead screw/nut pivot attachments on the remaining air operated valves were verified to be tight, with installation ofelastic stop nuts to be accomplished during the 1994 refueling outage.

The use of non self-locking nuts in these valve actuators was documented by a non-conformance report (NCR 93-248), which also provided justification and authorization for interim use.

The licensee utilized the forced outage to conduct several other maintenance activities.

Diagnostic testing was performed on both FRVs in an attempt to improve ADFCS stability. As a result of this testing, the valve actuator for the "A" FRV was replaced, and both FRVs were repacked.

Additionally, a software modification was made to the ATWS Mitigation System Actuation Circuitry (AMSAC) to install a power level time delay lock-in feature, as discussed in section 3.1 of this report.

Following plant startup on November 11, 1993, power escalation was stopped at approximately 30 percent due to secondary plant water chemistry restrictions.

Earlier routine testing at power had indicated slight leakage ofcirculating water into the 1B2 waterbox.

Therefore, the licensee utilized this period of low power operation to search for leaking main condenser circulating water tubes.

No leaking tubes were identified, and power escalation resumed on November 13, 1993.

As water chemistry failed to improve during the subsequent attempt to return to full power, power was reduced on November 14, 1993 to again support single main condenser operation.

Six leaking tubes were plugged in the 1B1 waterbox, and three were plugged in the 1B2 waterbox.

Additionally, the licensee identified that leakage was occurring at the 1B2

waterbox tube sheet/condenser shell interface.

Repairs were attempted using a sealant compound.

Although these repairs were not fullysuccessful, power escalation resumed, and full power was achieved on November 16, 1993.

Discussion The inspectors arrived in the control room within minutes of the reactor trip. The inspectors observed good operator response and procedural adherence.

Operators verbally repeated each procedural step to control room supervision prior to executing the step.

Transitions between procedures were correctly sequenced and occurred smoothly.

Operators were aware that the RCS pressure reduction due to the cooldown could result in an automatic initiation of the safety injection (SI) system.

Although they were aware that plant conditions did not require SI, their actions were taken in accordance with the emergency procedures rather than in attempts to avert SI initiation. From review of the plant computer post-trip review package, the inspector noted that, at the time that RCS cooldown was terminated, pressure was 11 psi above the automatic SI initiation setpoint of 1750 psig.

The inspectors concluded that the operators had responded appropriately to the reactor trip, and that, despite the unexpectedly large cooldown rate, had promptly stabilized plant conditions in hot shutdown.

The inspector reviewed the licensee's corrective actions and considered that issues related to the reactor trip had been adequately resolved to support restart.

The inspector did, however, note t

several weaknesses in the licensee's corrective actions.

Specifically:

Although the cause of the initiating transient was clearly established to be failure of the

"A" FRV valve positioner linkage, all aspects of the resultant feedwater flow transient were not clearly understood prior to restart.

Specifically, the "A"FRV had cycled from full open to full shut three times during the transient.

The first closure was due to a feedwater ESF isolation signal at 67 percent S/G level; however, the cause for the two subsequent closures was not clearly understood.

The fact that the two closures in question occurred at progressively lower S/G level suggests that they were related to the control operators'ttempts to close the valve; however, this was not positively established.

At the close of the inspection period, the licensee was continuing to investigate the cause of these two FRV closures.

The non-conformance report which justified interim use of the non-self-locking fasteners was weak in that it characterized the "A" FRV failure as a random event rather than a generic concern.

This conclusion was based on inspection results of the remaining air operated valves, which identified no similar impending failures.

The inspector considered that this conclusion was not consistent with the failure mechanism; loosening of non self-locking nuts due to valve cycling and component vibration.

The inspector agreed with the licensee's conclusion that the probability of another such failure prior to rework is minimal, given that the valves were inspected as a result of this event, and that the rework will occur during the March 1994 refueling outag ~ The 10 CFR 50.59 evaluation associated with this non-conformance was weak, in that it indicated that one of the causes of failure of the "A" FRV as being that it was subject to constant cycling.

Although FRV typically are subject to constant cycling, the "A" FRV had been operated in manual (to control feedwater flow oscillations) for nearly the entire operating cycle. Therefore, the valve had been subject to very little cycling since maintenance had last been performed on it during the 1993 refueling outage.

The inspector agreed, however, with the conclusion that interim operation with non-self-locking nuts on the subject safety-related valves was acceptable.

Some advance notice of failures of this type was given by Westinghouse Technical Bulletin 92-09, "Failure of Pressurizer Spray Valve Linkage," dated July 31, 1993, which was written in response to an incident involving an air operated valve.

In that incident, a nut securing a pan-head machine screw which held the feedback linkage arm of one of the two pressurizer spray valves loosened and fell off, thus detaching the linkage from the valve stem connection."

The bulletin addresses the use of elastic stop nuts in this application, as well as providing recommendations for alternate means ofassuring positive locking of nuts. The licensee received this technical bulletin, and had taken action to ensure that torque requirements were specified in procedures.

However, the response could have been broader in scope and perhaps generated a more thorough review of existing use of non-locking fasteners.

In summary, failure of the "A" FRV produced a secondary plant transient that resulted in a reactor trip due to low-low steam generator water level. Operator response to the resultant off-normal post-trip cooldown was in accordance with the applicable procedures.

Post-event review showed that the licensee could have been more effective at implementing vendor recommendations that could have prevented the FRV failure. Corrective actions subsequent to the trip were adequate to support plant restart; however, some aspects of these corrective actions are under further licensee review.

1.4 Reactor Shutdown due to Unisolable Leak in "B" Steam Generator Blowdown Piping On November 17, 1993, the licensee conducted a controlled shutdown to repair an unisolable secondary plant steam leak in containment.

The leak was detected on November 16, 1993, using the containment video monitor.

In a subsequent containment entry, the source of the leak was found to be a through-weld pinhole in a 90-degree elbow in the "B" steam generator (S/G)

blowdown line (schedule 80, 2 inch, original piping). The leakage rate, as determined using the containment water inventory monitoring system, was approximately 0.2 gallons per minute.

RG&E management directed that the plant be placed in cold shutdown to support code repairs, since the "B" S/G must be depressurized and drained to permit cut-out and replacement of affected piping and fittings.

At 6:00 p.m. on November 17, 1993, power was reduced at 20 percent per hour, with the plant going off line at approximately 10:30 p.m.

Upon achieving cold shutdown, conditions on November 18, 1993, the "B" S/G was drained, the affected piping cut-out, and prefabricated replacement piping installe The inspector observed various activities supporting the piping replacement, including plant shutdown, nondestructive examination of the replaced components, and the licensee's investigation into identifying the failure mechanism.

Power reduction and placing the plant into hot shutdown conditions was performed in a controlled, deliberate manner, with the appropriate level of management involved.

Operators adhered verbatim to operations (0) procedure 0-2.1, "Normal Shutdown to Hot Shutdown," and verbally repeated each step to supervision prior to performing. A shortcoming was noted in that operations personnel did not anticipate a blowdown isolation signal upon tripping the turbine, thereby placing a maximum pressure on the leaking component.

However, this oversight did not result in a noticeable change in leak rate; operators promptly opened blowdown isolation valves and the plant cooldown resumed.

In verifying weld quality of the replacement blowdown piping and fittings, the licensee invoked inservice inspection code case N-416 for Section XI, Division 1 of the ASME Boiler and Pressure Vessel Code.

Code case N-416 provides for alternate rules for hydrostatic testing for replacement of Class 2 piping when the piping cannot be isolated by existing valves.

Specifically, the system hydrostatic test that is generally required for such replacement can be deferred until the next regularly scheduled system hydrostatic test provided that the partial penetration welded joints of the replaced components be examined using an acceptable surface examination method, and that, prior to returning the system to service, a visual examination (VT-2) for leakage be conducted during the system functional test.

Code case N-416 is an acceptable testing alternative as identified in Regulatory Guide 1.147, "Inservice Inspection Code Case Acceptability ASME Section XI Division I," Revision 10, July 1993.

On November 19, 1993, the inspector observed the weld surface examination of the prefabricated replacement blowdown piping and confirmed that it was performed by qualified technicians from RG&E's Materials Engineering and Inspection Services Department, in accordance with the licensee's Nondestructive Examination (NDE) procedure NDE-300-1, "Dry Powder Magnetic Particle Exam-Prod, Yoke, Coil, and Direct Methods," Revision 5, dated February 24, 1993.

No weld discontinuities were identified.

Installation welds were examined and accepted using the same procedure.

The VT-2 examination of the installed piping was performed on November 22, 1993, upon maintaining the secondary steam plant at hot shutdown conditions of 1000 psig and 547'F for a four hour test period.

No leakage indications were evident.

Following the initialleak identification, the licensee's maintenance analyst compared radiographs of the affected elbow that were taken during the 1993 refueling outage, as part of the Erosion/Corrosion Program, with current radiographs, in an attempt to identify failure precursors.

No precursors were readily evident.

From examining the failed weld and current radiographs, the licensee concluded that the leak resulted from an erosion/corrosion mechanism that slowly propagated from a minute weld porosity during the 24 years of service of the blowdown piping.

Through observation of the various activities required to place the plant in a condition to support the blowdown piping replacement, review of the work package (Work Order 19369423),

attendance at PORC/work planning meetings, and discussions with licensee personnel, the inspector concluded that the licensee acted in a prompt and conservative manner to provide assurance that the replaced blowdown piping would function reliably.

1.5 Automatic Reactor Trip During Startup due to High Source Range Flux Level Following the completion of repairs to the "B" S/6 blowdown piping, operators commenced a

reactor startup at 5:42 a.m. on November 22, 1993.

Control rods were being withdrawn in accordance with operations instruction 0-1.2, "Plant Startup from Hot Shutdown to Full Load,"

with an estimated critical rod height of 100 steps on the last of four control rod banks to be withdrawn (control bank "D"). At a rod height of about 80 steps on control bank "D", the reactor was still sub-critical, with reactor power high in the source range at about 10'ounts per second (cps).

This power level is also above the minimum sensitivity of the intermediate range nuclear instruments (10" amps).

Once sufficient overlap between the source and intermediate range nuclear instruments has occurred (corresponding to 10" amps on either intermediate range nuclear instrument channel), permissive circuit P-6 allows the high nuclear flux (source range)

trip to be bypassed.

Bypassing this trip requires operator action; an indicating light ("Power Above P-6") energizes to indicate that the permissive circuit has been satisfied and, thus, that the trip can be bypassed.

At 6:42:49 a.m., reactor power as indicated by intermediate range nuclear instrument channel N-36 reached 10" amps; source range nuclear instruments both indicated approximately 10'ps.

Per procedure, the high nuclear flux (source range) trip was to be bypassed when the "Power Above P-6" indicating light energized, and the control room operators were closely watching for receipt ofthis indication. By 6:43:39 a,m., both intermediate range nuclear instrument channels were above 10'" amps, however, the "Power Above P-6" indicating light still had not illuminated.

During this period, reactor power was being increased at between 0.5 and 1.0 decades per minute, which was within procedural limitations.

As power continued to rise and the P-6 indicating light remained out, the Control Room Foreman concluded that there must be a problem with the P-6 permissive circuit, and directed the Head Control Operator to reduce power by driving control rods inward.

Before this action could be taken, at 6:44:49 a.m., a reactor trip occurred when the high nuclear flux (source range) trip setpoint was reached.

The reactor had not achieved criticality, and therefore the trip had no effect on plant conditions.

Subsequent investigation revealed that both of the two light bulbs that illuminate the "Power Above P-6" indicator were burned out.

During the post-trip review, the licensee identified several factors that contributed to the reactor trip. Specifically:

The procedure specified the "Power Above P-6" indicating light energizing as the criteria for defeating the high neutron flux (source range) trip, although other indications that the P-6 permissive has been satisfied are availabl Criticalitywas going to occur in the source/intermediate range nuclear instrument overlap region; with one of the principle indicators of criticality being a positive startup rate while the control rods are stationary, operators were concerned with maintaining a visible startup rate to support an accurate determination of criticality. As a result, very little time had been available to recognize and deal with the P-6 indicating light problem.

From experience on the plant simulator, operators had seen that the "Power Above P-6" indicating light did not energize until Sx10'ps.

This further reduced the time available to recognize and deal with the problem.

As corrective action, procedure 0-1.2 was changed to specify additional indications to be monitored to determine that the P-6 permissive had been satisfied.

The two light bulbs were

~ replaced and proper operation of the "Power Above P-6" indicating light was verified prior to reactor startup.

After these actions had been completed, a reactor startup was successfully conducted on the afternoon of November 22, 1993.

Additional corrective action included simulator training for the operators who had been involved in the trip, and initiation of a human performance evaluation.

Through review of the plant computer pre-and post-trip data, the inspector verified that the reactor startup rate had not been excessive prior to the reactor trip.

However, the inspector considered that the primary cause of the reactor trip had been that, by maintaining a positive startup rate to facilitate making an accurate determination of criticality, operators did not have sufficient time to recognize and successfully deal with the P-6 indicating light failure.

The inspector considered that the licensee's corrective actions were appropriate, and had no additional concerns on this matter.

2,0 MAINTENANCE(62703, 61726)

2.1 Corrective Maintenance 2.1.1 Routine Observations The inspector observed portions of maintenance activities to verify that correct parts and tools were utilized, applicable industry code and technical specification requirements were satisfied, adequate measures were in place to ensure personnel safety and prevent damage to plant structures, systems, and components, and to ensure that equipment operability was verified upon completion.

The following maintenance activities were observed:

~

Work Order 9301464,

"Clean/Replace Gasket on FT-930 (sodium hydroxide flow meter),"

performed in accordance with Maintenance Procedure (M)-37.130,

"Disassembly and Reassembly of Pipe Flange Connections," revision 11, dated August 7, 1992, observed October 25, 1993

~

Disassembly of the flange revealed that the fasteners were normally subject to a significant dead weight load from the downstream piping. When the mechanics removed the final fastener and attempted to lower the downstream piping to its rest position; not having reached it after approximately six inches of downward deflection, they returned the pipe to its assembled position and reinstalled one fastener to hold it in place.

Examination showed that the weight of the downstream piping had flattened threads on all four of the flange fastener studs.

The studs and bolts were replaced.

Although the post-maintenance leak test was satisfactory, leakage again slowly developed.

The licensee is evaluating the adequacy of this piping support.

Work Order 9301413,

"Replace Flex Gasket on "B" Gas Stripper Feed Pump,"

performed in accordance with M-37.130, "Disassembly and Reassembly of Pipe Flange Connections," revision 11, dated August 6, 1992, observed November 4, 1993 Work Order 19369454, "Change Oil and Clean Out Service Water Side ofPAF01B ("B" motor driven AFWpump) Bearing," performed in accordance with M-11.5C, "Auxiliary Feed Water Pump Minor Mechanical Inspection and Maintenance," revision 21, dated June 18, 1992, observed November 20, 1993 This maintenance was performed in response to high temperature that was noted on the outboard bearing housing during normal pump operation.

The oil was found to be slightly cloudy and discolored; a sample was sent off-site for analysis, but results were not available as of the close of the inspection period. The water jacket was cleaned and, although some fouling was present, this did not correct the high bearing temperature condition.

Subsequently, the outboard bearings were replaced.

Work Order 19369457, "Repair PAF01B ("B" motor driven AFW pump) Pump Thrust Bearing," performed in accordance with M-11.5B, "Major Mechanical Inspection of the Auxiliary Feed Pumps - Worthington," revision 16, dated August 7, 1992, observed November 21, 1993 Follow-on corrective action for the high temperature condition discussed above.

The cause was found to be partial obstruction of the oil circulation passages in the bearing casing, caused by bowing ofplastic shims on the inner bearing cover. A sealant material was also found in the oil circulation passages, which further restricted flow. The thrust bearings were replaced, although they still appeared to be serviceable.

Additionally, a new shim pack was installed, and no sealant was used during reassembly.

Work Order No. 19369467,

"Replace N2 regulator 8612A," observed November 23, 1993 Work inside containment, conducted while at power.

The inspector considered that the pre-job ALARAbriefing was adequate for the scope of the maintenance.

No deficiencies were noted during the conduct of maintenanc.2.1 Routine Observations Inspectors observed portions of surveillances to verify proper calibration of test instrumentation, use of approved procedures, performance of work by qualified personnel, conformance to limiting conditions for operation (LCOs), and correct system restoration following testing.

The following surveillances were observed:

Performance Test (PT)-16Q-T, "Auxiliary Feedwater Turbine Pump

- Quarterly,"

revision 8, dated October 21, 1993, observed October 28, 1993 PT-32B, "Reactor Trip Breaker Testing - "B" Train," revision 14, dated March 12, 1993, observed November 2, 1993

~

Instrument Calibration Procedure (CPI)-ACCELEROGRAPH-51,

"Calibration of Kinemetrics Model SSA-2 Strong Motion Accelerograph," revision 1, dated April 15, 1993, observed November 9, 1993

~

PT-12.1,

"Emergency Diesel Generator 1A," revision 72, dated October 26, 1993, observed November 9, 1993

~

PT-9, "Undervoltage and Underfrequency Protection 11A and 11B - 4160 Volt Buses,"

revision 18, dated November 6, 1992, observed November 23, 1993 The inspector determined through observing these tests that operations and test personnel adhered to procedures, test results and equipment operating parameters met acceptance criteria, and redundant equipment was available for emergency operation.

2.2.2 Incomplete System Restoration Following Containment Spray System Performance Test On October 28, 1993, the licensee informed the inspector that an auxiliary operator, while performing his rounds, had found that chains/locks had not been installed, as required, to secure the containment spray eductor supply test line isolation valves (873C/D).

The valves were verified to be in their proper (closed) position, but the chains and locks that were to be attached were found lying on the floor in the vicinity of the valves.

The chains and locks were subsequently installed.

Since the cause was not apparent, operations supervision directed that reactor plant systems line-up procedures S-30.1 through S-30.5 and S-30.7 be immediately completed to verify that other emergency safeguards equipment was properly configured, and that an investigation be undertaken to determine the cause of this oversight.

No other safety-related equipment was found improperly configure From findings of the human performance evaluation investigation (HPES No. 93-11), the licensee concluded that the valves were properly closed but the chains/locks were not installed, following completion of the containment spray system quarterly performance test, PT-3Q.

Chains/locks were not in place from test completion at 2:32 p.m. on October 25, 1993, until discovery at 9:30 a.m. on October 28, 1993.

Many factors were found to have contributed to this oversight, including a change of testing personnel after the system was aligned to its test configuration; improper implementation of the independent verification procedure (A-1404), in which personnel who restored the system to its original configuration also performed the independent verification; and improper procedure adherence, in which the restoration steps of PT-3Q were not completely followed in that the valves were closed but chains/locks not installed as specified in the test procedure.

As a result of the human errors identified, licensee management took prompt disciplinary action against the responsible individuals and reiterated management expectations on procedural compliance to Results and Test Department personnel.

The inspector reviewed the circumstances resulting in the failure to install chains/locks on valves 873C/D following completion of the surveillance, and the licensee's response.

The inspector concluded that this oversight did not compromise the operability 'of the containment spray system, and accordingly, had minor safety significance.

The inspector determined that the oversight resulted when test personnel did not strictly adhere to administrative requirements for procedure compliance and for properly conducting independent verification of the system restoration.

Through review of the HPES investigation, and the resulting corrective actions, the inspector determined that a detailed investigation was conducted, appropriate disciplinary actions and counseling were provided to individuals involved, and management expectations were reiterated on the conduct of independent verifications/procedure adherence.

Based on this review, the inspector determined that the actions of Results and Test personnel resulting in this oversight were contrary to the administrative requirements specified in licensee Administrative (A) procedures A-503, Procedure Adherence, and A-1408, Independent Verification. Compliance with these requirements is mandated by Technical Specification 6.8.1.

The inspector concluded that these procedural violations were the result of an isolated employee action and that present licensee administrative controls were effective to identify and correct the condition in a reasonable time.

Licensee management also aggressively pursued the cause and took appropriate corrective action to prevent a recurrence.

In accordance with the provisions of 10 CFR 2, Appendix C, Section VII.B,a violation was not cited for this incident because the relevant criteria were met.

These criteria include the licensee promptly identified the plant condition, the condition was of minor safety significance, immediate corrective actions were appropriate to prevent recurrence, the condition was evaluated for reportability, and, though not required, was promptly provided to the NRC resident office, and the occurrence was not a repeatable violation that should have been preventable through corrective actions taken on a prior violatio.1 AMSAC Design Modification On November 11, 1993, the licensee utilized the opportunity provided by the forced outage to modify software to the AMSAC system, thereby enabling a power level lock-in feature which was not incorporated into the original AMSAC design.

The work was performed as a station modification (SM) and installed/tested using procedure SM-4230.4, "AMSACPower Level Time Delay Lock-in Feature Modification."

Through discussions with licensee representatives, review of supporting documentation and observations of work in-progress, the inspector concluded that the modification was effectively coordinated between the licensee's corporate engineering, site operations, and site technical support staffs.

The implementing station modification proc'edure (SM-4230.4) was found to contain the appropriate level of detail to control the installation, testing and subsequent turnover of the completed modification to plant operations.

SM-4230.4 was found to have received a detailed PORC review with a requisite safety evaluation prior to implementation.

4.0 PLANT SUPPORT (71707)

4.1 Radiological Controls The inspectors periodically confirmed that radiation work permits were effectively implemented, dosimetry was correctly worn in controlled areas and dosimeter readings were accurately recorded, access to high radiation areas was adequately controlled, survey information was kept current, and postings and labeling were in compliance with regulatory requirements.

Through observations ofongoing activities and discussions with plant personnel, the inspectors concluded that the licensee's radiological controls were generally effective.

4.2 Security 4.2.1 Routine Observations During this inspection period, the inspectors verified that x-ray machines and metal and explosive detectors were operable, protected area and vital area barriers were well maintained, personnel were properly badged for unescorted or escorted access, and compensatory measures were implemented when necessary.

No unacceptable conditions were identifie.3.1 Routine Observations The inspectors periodically verified the adequacy of combustible material controls and storage in safety-related areas of the plant, monitored transient fire loads, verified the operability of fire detection and suppression systems, assessed the condition of fire barriers, and verified the adequacy of required compensatory measures.

No discrepancies were noted.

4.4 Emergency Preparedness 4.4.1 Annual Emergency Preparedness Evaluated Exercise The inspectors participated as members of the NRC inspection team during conduct of the 1993 R. E. Ginna Nuclear Power Plant partial participation emergency preparedness exercise on November 17, 1993.

Other participants included emergency response organizations in Wayne and Monroe counties (resident and adjacent counties, respectively), and the state of New York.

No significant deficiencies were noted; detailed assessment of the exercise is presented in inspection report 50-244/93-18.

4.4.2 Prompt Public Notification System Test On November 10 1993 the licensee conducted a test of the Prompt Public Notification (siren)

System.

The system consists of 96 sirens located within a 10 mile radius of the site.

Only one siren failed to activate during the test.

95 of 96 sirens successfully operating met the 90%

acceptance criteria for system operability.

5.0 SAFETY ASSESSMENT/QUALITY VERIFICATION 5.1 Periodic Reports'eriodic reports submitted by the licensee pursuant to Technical Specification 6.9.1 were reviewed.

Inspectors verified that the reports contained information required by the NRC, that test results and/or supporting information were consistent with design predictions and performance specifications, and that reported information was accurate.

The following report was reviewed:

Monthly Operating Reports for October, 1993 No unacceptable conditions were identifie A licensee event report (LER) submitted to the NRC was reviewed to determine whether details were clearly reported, causes were properly identified, and corrective actions were appropriate.

The inspectors also assessed whether potential safety consequences were properly evaluated, generic implications were indicated, events warranted onsite follow-up, and applicable requirements of 10 CFR 50.72 were met.

The following LER was reviewed (Note: date indicated is event date):

~

93-005, Failure to verify the load shedding capability of safeguards equipment powered from emergency buses by simulating a loss ofoffsite power in conjunction with a safety injection test signal.

This oversight resulted from a

Technical Specification misinterpretation and is addressed in NRC Inspection Report 50-244/93-21

~ (October 11, 1993)

The inspector concluded that the LER was accurate and met regulatory requirements.

No unacceptable conditions were identified.

6.0 ADMINISTRATIVE(71707, 30702, 94600)

6.1 Backshift and Deep Backshift Inspection

~

~

~

~

~

During this inspection period, a backshift inspection was conducted on November 17, 1993.

Deep backshift inspections were conducted on the following dates:

November 6, 11, 13, 20, and 21, 1993.

6.2 Meetings 6.2.1 Training Program Meeting On October,26, 1993, RG&E officials met with NRC Regional staff, in the Region I office, to discuss the weak performance of licensed operators on their requalification examinations.

The utilityprovided the NRC staff with plans for changing its operator training program to improve test performance.

6.2.2 SALP Management Meeting On November 15, 1993, NRC Regional management met with RG&E officials, at the Ginna Training Center, to discuss the results of the Systematic Assessment of Licensee Performance (NRC SALP Report 50-244/92-99).

At this meeting, the licensee identified initiatives that are planned to address program weaknesses identified in the SALP Report.

Copies of handouts and transparencies used by the NRC and RG&E during the course of this meeting are provided as Attachment I and II, respectivel The exit meeting for inspection report 50-244/93-18 (Annual Emergency Preparedness Exercise)

was held on November 18, 1993, by Mr. David Silk.

The exit meeting for inspection report 50-244/93-23 (Snubber Inspection and Steam Generator Replacement Programs)

was held on November 19, 1993, by Mr. Joseph Carrasco.

At periodic intervals and at the conclusion of the inspection, meetings were held with senior station management to discuss the scope and findings of inspections.

The exit meeting for the current resident inspection report 50-244/93-22 was held on December 1, 199 ATTACHMENTI ROCHESTER GAS & ELECTRIC COMPANY SYSTEMATIC ASSESSMENT OF LICENSEE PERFORMANCE PRESENTATION SLIDES NOVEMBER 15, 1993

GINNA STATION SYSTEMATIC APPRAISAL OF LICENSEE PERFORMANCE (SALP)

November 16, 3993

AGENDA

~ Introduction

~ Operations

~ Engineering

~ Maintenance

~ Plant Support

~ Concluding Comments R. E. Smith R. A. Marchionda P. C. Wilkens S. T. Adams J. A. Widay R. C. Mecredy

OPERATIONS

~ Stren ths management support and oversight of operations operations prompt and consistent response to transients and abnormal conditions performance duringinfrequent evolutions outage performance

OPERATIONS

~ Stren ths continued shift turnover and knowledge of off no-rmal plant conditions coordination with other support groups

OPERATIONS

~ 0 ortunities For Im rovement formality attention to detail self-checking troubleshooting response to industry falsification issue

ENGINEERING AND TECHNICAL SUPPORT

~ Stren ths

~

commitment to safety

~

communication efforts

~ station support working relationshi ps improved response time recognition and support of priorities

~ planning outage 8 long range planning groups development of work planning tools

ENGINEERING AND TECHNICAL SUPPORT

~ planning (continued)

configuration management information system

~

steam generator replacement

~

technically capable staff

ENGINEERING AND TECHNICAL SUPPORT

~

ortunities For Im rovement

~ simplified processes

~

systems engineers

~ relief requests

~ LER cause codes

MAINTENANCE

~

Stren ths management oversight and support teamwork effective maintenance programs determination of root causes

Maintenance

~

Qn oin Initiatives maintenance procedure upgrade program reliability centered maintenance implementation configuration management information system implementation

~ 0 ortunities For Im rovement resolution of balance of plant issues integration of prioritization, planning and and scheduling of work

r

Maintenance

~

ortunities For Im rovement continued diagnostics team for predictive maintenance maintenance rule implementation pla PLANT SUPPORT

~ Stren ths excellent program management and commitment to plant security effective emergency response organization ALARAintegration in steam generator replacement project proactive 10CFR20 initiatives

PLANT SUPPORT

~ 0 ortunities for Im rovement person rem-exposure reduction attention to housekeeping continued vigilance of fire protection program implementation of enhancements in radiological environmental monitoring and radioactive effluent control

CONCLUDING COMMENTS

~

On oin Initiatives performance improvements through business planning self assessment process improvements steam generator replacement

~ 0 ortunities For Im rovement managemeri t expectations formality of operations

CONCLUDING COMMENTS

~ 0 ortunities for Im rovement continued organizational alignment

~

Conclusions

ATTACHMENT11 U.S. NUCLEAR REGULATORY COMMISSION

REGION I

SYSTEMATIC ASSESSMENT OF LICENSEE PERFORMANCE PRESENTATION SLIDES NOVEMBER 15, 1993

Ginna SALP Management Meeting

.....:.:::,:.:,.,:...;,, Assessment: Period

'::."..

January 19, 1992 - September 11, 1993

~y,8 Rangy

~e (4O C0

Wg*g+

Presentstion

~ Introduction

~ Report Presenitaihon

,, icensee reset 3 ion

~ Discussion

~ Closing Remarks

T. Martin J. Linvilllle RGB E Co.

T. Martin T. Martin

Revised SALP Process Effective July 14, 1993

~ Changed from 7 areas to 4 incor ora

<<in o eaC area

...

.-EP,, Radiqlogica j..Controls...and,.Security...::,,

combined into "Plant Support"

~ SALP Board Membership consists of 4 senior managers

~ Emphasis on the last 6 months of the period

~ Trends not in6uded in the category rNings

SALP Eunctional Areas ao era ions

'E'ngineering

- '"

"'"-aintenance

~ Plant Support

Performance Category Ratings ae

-

u erior e orrnance

"""'" '~:Chtegor'y"'2'--'Good'PerfornianM'

Category 3 - Acceptable Performance

~

I

~

PALP Category Ratings for the Previous Period Ending 1/18/92

.

~ Plant Operations 8 lOO ICB OA fOS Maintena'rice/Surveillance

~ Emergency Preparedness

~ Security

~ Engineering 8 Tech Support

~ Safety Assessment/

Quality Verification

a SALP Category Ratings for Period Ending 9l11/93

~O

OperatNns:'-" -""'-

'

"- '":: "-:-" 2-:-:":::: - "-

~ Engineering

~ Maintenance

~ Plant Support

Operations Category 2

~ Management exercised good oversight

~ Operators responded well to transients

'

'"C'intel'of'system cohfigora56n'"iiias Viieak""'

Management response 5o AO logkeeping issue was not thorough and self-critical

Engineering Category

~ Management oversight-of major modifications was exceptional

~ Strong support for operations and

-'.-"":maintenance':departrnents---

~ Effective safety assessments and analyses in problem resolution

~ Excellent communications between corporate and site engineering

~ Licensing submittals wet e of goed 5echnica(

quality

Maintenance Category

Aggressive preventive maintenance program Strong management oversight of malor maintenance activities

-::.- ': '85'or@"reotwau.im'deteirninatioa: program:.:.':-

s Inadequate correc5ve action for service wa0er valve deficiencies

~ Strong management commitment to QA

~ Balance of plant challenges 5o safety systems

Plant Support Category 2

~ SignNcant program upgrade and superior performance m plant secunty

-: -:"Effec5ve:eIrnergency'preparedness:progrm-

~ Good performance in radiological controls

~ Slow implementation of 10 CFR Part 20 requirements

~ ALARAprogram uses dose rate vice dose

~ Some problems still exist:iin the RadieileglcaIi Environmental Monitoring Program (REMI )

Overall Conclusion

~ Overall performance continues to improve

~ Good performance in operations and plsnt support

-':: 'Svperior:::performance.in..engineerjng.+mal,,

maintenance

~ Increased management-attention is needled to reduce challenges to plant systems

~ Some weaknesses in revising radiological procedures and implemen5ng the AGAMAand REMP programs Steam generator replausrneet ohs~llenges:lie ahead