IR 05000387/1993019
| ML17158A042 | |
| Person / Time | |
|---|---|
| Site: | Susquehanna  |
| Issue date: | 12/09/1993 |
| From: | Jason White NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML17158A041 | List: |
| References | |
| 50-387-93-19, 50-388-93-19, NUDOCS 9312170031 | |
| Download: ML17158A042 (87) | |
Text
UNITED STATES NUCLEAR REGULATORY COMMISSION
REGION I
Inspection Report Nos.
License Nos.
Licensee:
Facility Name:
Inspection At:
Inspection Conducted:
Illspecioi's:
50-387/93-19; 50-388/93-19 NPF-14; NPF-22 Pennsylvania Power and Light Company 2 North Ninth Street Allentown, Pennsylvania 18101 Susquehanna Steam Electric Station Salem Township, Pennsylvania October 5, 1993 - November 15, 1993 G. S. Barber, Senior Resident Inspector, SSES D. J. Mannai, Resident In r SSES Approved By:
J.
ite, Chief ctor Projects Section No. 2A, Date I~i:1li iR
'
~
'
(during day and backshift hours) of station activities, including: plant operations; radiation protection; surveillance and maintenance; engineering and technical support; and safety assessment/quality verification. Findings and conclusions are summarized in the Executive Summary.
Details are provided in the full inspection report.
t 9312170031 931210 PDR ADOCK 05000387 PDR
EXECUTIVE SUSBfARY Susquehanna Inspection Reports 50-387/93-19; 50-388/93-19 October 5, 1993 - November 15, 1993 Operations (30702, 71707, 71710)
On October 5, 1993, at 6:04 p.m., control rod 14-35 moved from full in (position 00) to position 04 with no operator action.
The licensee determined that the unexpected rod movement was due to a transponder card failure. There have been at least 16 previous cases where control rods have moved independent of a demand signal with at least seven of them due transponder card failures.
For the four most recent cases, the licensee determined that no core thermal limits were violated.
However, there were real reductions in the margins to these limits. The safety related functions of the Rod Sequence Control System and the Rod Worth Minimizer may also be bypassed by these malfunctions since control rods move independent of them being selected.
The final root cause, corrective action, and reportability of these events willremain unresolved pending further review by the licensee and evaluation by the NRC.
Section 2.2.1 pertains.
Maintenance/Surveillance (61726, 62703)
The inspector observed portions of the reactor water level backfill modification tie-in to the lower (actuation and isolation) condensing pots.
This modification required a manual isolation valve be disabled in the open position to preclude inadvertent pressurization of the instrument rack with Control Rod Drive (CRD) system pressure.
The positioning of this valve was questioned by the inspector since inadvertent pressurization could result in false low reactor water level and false high reactor pressure signals.
These signals would result in various ECCS actuations.
The inspector determined the modification and work plan did not require independent verification of valve positioning prior to disabling the valve.
Also the inspector identified some work planning weaknesses related to equipment status control.
Section 3.3.1 pertains.
During the outage, the inspector observed portions of Control Rod Drive (CRD) changeout.
The licensee encountered some problems with CRD handling equipment and mechanism removal.
The licensee's immediate resolution of the problems was effective.
The licensee committed to implement final resolutions prior to the next refueling outage.
However, the inspector identified some inter-departmental and intra-departmental communications weaknesses when work was stopped to resolve identified problems.
Section 3.3.2 pertain I
Safety Assessment/Assurance of Quality (40500, 90712, 92700, 92701)
A Since 1990, crack indications have been identified in the beltline region of core shrouds for various boiling water reactors.
In response to these concerns, GE issued SIL 572, "Core Shroud Cracks", which recommended that in vessel inspections be performed to detect cracking that could affect structural integrity. The licensee performed these inspections on October 20 and 21 and no cracks were identified.
The inspector independently confirmed the licensee's results.
The inspector also questioned the licensee on the scope of their inspection and noted the licensee's inspection plan was initiated before their receipt of SIL 572 Revision 1 and, as such, did not fully address all of the recommended actions in the SIL. Three SIL items were not fully addressed at the conclusion of the inspection.
After detailed discussions with the licensee, the inspector concluded that the licensee's inspection addressed the most susceptible crack locations.
However, no justification or analysis existed to describe why the licensee's deviation from the SIL were acceptable.
The licensee has agreed to provide the necessary justification and willreevaluate the need for further inspections.
Section 6.1 pertain SUMMARYOF OPERATIONS 1.1 Inspection Activities The purpose of this inspection was to assess licensee activities at Susquehanna Steam Electric Station (SSES) as they related to reactor safety and worker radiation protection.
Within each inspection area, the inspectors documented the specific purpose of the area under review, the scope of inspection activities and findings, along with appropriate conclusions.
This assessment is based on actual observation of licensee activities, interviews with licensee personnel, independent calculation, and selective review of applicable documents.
1.2 Susquehanna Unit 1 Suxnmary At the start of the inspection period, Unit 1 was in day 10 of its seventh refueling outage and in cold shutdown (operational condition 5).
Outage activities generally progressed as planned.
A number of fuel handling problems occurred throughout the period.
t An Augmented Inspection Team (AIT) was dispatched on October 29 to review and evaluate the circumstances and safety significance of a series of problems encountered during Unit 1 refueling activities.
Since October 6, the following events occurred: (1) refueling bridge operators grappled and removed the wrong fuel bundle during defueling and inappropriately returned it to its original core position, (2) spurious hoist motor overcurrent lockouts occurred on the refueling bridge with no conclusive cause being identified, which was followed by a sudden drop in the Unit-1 refueling bridge mast due to binding in the mast assembly (from a bent mast), (3) a refueling bridge operator apparently failed to realize that a double blade guide assembly which was being carried on the fuel handling grapple extended beyond the level needed to clear the vessel wall; and consequently the blade guide hit the vessel wall while attached to the grapple, and (4) following the licensee's complete inspection and review of these incidents, and subsequent modification of procedures, another problem involving the refueling bridge crane was encountered when a loud noise and air bubbles were observed to be coming from the mast assembly as the operator was lowering the mast while preparing to pick up a fuel bundle.
Though there was no apparent damage to any fuel bundles, NRC Region I was concerned'with the repetitive nature of these problems.
As a result of these events, an AITwas dispatched and a Confirmatory Action Letter (CAL)
was issued which documented the licensee's agreement to suspend further refueling activities until the AIThas an opportunity to review and evaluate the incidences and the corrective measures.
The results of this inspection willbe described in NRC Inspection Report 50-387/93-8.3 Susquehanna Unit 2 Sunnnary Unit 2 operated at or near full power for the duration of the inspection period.
Operators conducted several routine power reductions during the. period to facilitate control rod pattern adjustments, surveillance testing, and maintenance.
There was one ESF actuation during the inspection period.
The High Pressure Coolant Injection (HPCI) system suction realigned from the condensate storage tank (CST) to the suppression pool, at 9:27 p.m., November 10.
This event was subsequently reported per 10 CFR 50.72.
After investigation, system configuration was returned to normal.
This actuation occurred while I&C technicians were performing a monthly surveillance of the suppression pool high water level channels.
The licensee was still investigating the cause of the event at the end of the inspection period.
The inspector's initial review of this actuation concluded that no immediate safety concern existed.
The inspector willcomplete the review of this activity after the Licensee Event Report (LER) is issued.
2.
OPERATIONS 2.1 Inspection Activities The inspectors verified that the facility was operated safely and in conformance with regulatory requirements.
Pennsylvania Power and Light (PP&L) Company management control was evaluated by direct observation of activities, tours of the facility, interviews and discussions with personnel, independent verification of safety system status and Limiting Conditions for Operation, and review of facility records.
These inspection activities were conducted in accordance with NRC inspection procedure 71707.
The inspectors performed 6.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> of deep backshift inspections during the period.
These deep backshift inspections covered licensee activities during between 10:00 p.m. and 6:00 a.m. on weekdays, and weekends and holidays.
2.2 Inspection Findings and Review of Events 2.2.1 Unexpected Control Rod Movement On October 5, 1993, at 6:04 p.m., control rod 14-35 moved from full in (position 00) to position 04 with no operator action.
The control room operators received several alarms in the control room, including a "rod drift" alarm.
The licensee determined no control rod manipulations were being performed at the time.
Operators, in response to the alarms, initiated a computer check of control rod positions (OD-7). The initial OD-7 showed rod 14-35 position at -99 which caused the rod drive control system (RDCS) to lock up.
This lock up prevented further rod movement.
After
RDCS was reset, OD-7 showed that rod 14-35 was at position 04 and the alarm screen (CRT) showed movement from position -1 to position 4. In order to reinsert the rod, the licensee used a test card to clear the rod select block to allow its reinsertion to position 00.
The rod was successfully repositioned and all other control rod drives (CRDs) were hydraulically isolated and normally blocked to ensure no further rod motion was possible.
Immediate investigation by instrumentation & control (1&C) personnel determined that when rod 14-35 was being self-tested by the reactor manual control system (RMCS), a transponder card inappropriately acknowledged a withdraw signal to its respective supply and exhaust, directional control valves.
The acknowledge signal should only have been sent to one valve.
The licensee also determined that no fuel was in the cell, and thus, no fuel limits were challenged.
The licensee concluded that this event was not reportable, and documented it in SOOR 93-320.
In response to this event, the licensee formed a task team to investigate this and other problems with the RDCS and the RMCS.
The team determined that there were three potential causes for outward "rod drift". They were:
Mechanical binding of the collet fingers in the release position.
This could occur during normal control rod withdrawal allowing the rod to travel to the full out position (position 48).
Failure of a transponder card associated with a control rod that had been inserted past the full in position (position 00).
A failed transponder card concurrent with a leaky directional control valve.
After troubleshooting, the team determined that the transponder card for rod 14-35 failed intermittently. The most likely cause of this failure was a faulty diode in the transponder card.
The Reactor Manual Control System (RMCS) uses a diode bridge network and a control transistor for each solenoid valve on each Hydraulic Control Unit (HCU). Allof the components for a given HCU are located on a transponder card.
The licensee's assessment indicated that a failure of any diode in the bridge network willcause the AC power for the associated solenoid valve to bypass the control transistor for one half of each AC cycle. If this failure is in the withdraw solenoid, the associated withdraw solenoid valve willattempt to cycle (i.e., open and close) 60 times a second, which allows control rod drive flow into the withdraw side of the affected control rod drive piston.
The exhaust valve is not affected by this failure and willnot open to vent the other side of the CRD drive piston.
However, the other side of the CRD piston willslowly vent through the seals into the Reactor Vessel allowing the rod to drift slowly outward.
In this condition, the affected solenoid valves will chatter until the condition clears.
This unexpected rod withdrawal was similar to two events that occurred in July and November 1992.
(SOORs 2-92-065 and 1-92-351).
In response to the failure, the licensee implemented or willimplement the following five corrective actions:
Replaced the failed transponder card for control rod 14-35.
This action was complete on October 7, 1993.
(WA S-37144)
Replaced all four directional control valves for rod 14-35.
This action was complete on October 24, 1993.
(WA S-37173)
Perform a stroke and differential pressure test of rod 14-35.
Perform a full out test of rod 14-35.
Revised operating procedures to ensure that any control rods in the over-travel position willbe returned to position 00 shortly after a scram.
The licensee's task team concluded that the root cause of the control rod drift was a failure of rod's transponder card concurrent with the rod being beyond position 00.
They also concluded that the RMCS functioned as designed because it stopped rod movement when it locked up.
However, this function is non-safety related and is provided to limitthe effects of faults.
The licensee added procedure steps to require the repositioning of all control rods post scram or post shutdown to position 00.
Inspector Conclusion The inspector interviewed personnel, reviewed information, and observed activities to assess the licensee's actions for the unexpected control rod movement.
The inspector noted that the licensee's immediate actions were prompt and effective.
Their action to hydraulically isolate the other HCUs was prudent.
I&C was quick to have the control rod repositioned to 00 and to diagnosis the problem as a failed transponder card for 14-35.
Licensee actions immediately following the event were conservative and directed at ensuring safe operations.
The inspector reviewed the operating history to determine the frequency of unexpected rod movements and found that there have been at least 16 prior occurrences.
The three most recent occurrences involved one rod (18-43) that unexpectedly withdrew from position 24 to position 30 during a November 1992 reactor startup (SOOR 1-92-351) and two rods (34-07 and 14-55) that unexpectedly inserted from position 48 to 00 during power operation in July 1992 (SOOR 2-92-065).
For all three of these cases, a "rod drift" alarm was received and operator intervention was unsuccessful at stopping rod motion. In the most recent case (rod 18-43) no specific cause of the unexpected rod withdrawal was determined.
For the two previous cases, (rods 34-07 and 14-55), the licensee identified the root cause of the unexpected insertions as a failure of diodes on the transponder cards.
The SOOR (2-92-065)
that describes these failures also specifically identifies that seven of the previous failures were due to transponder card failures.
For the November 1992 rod withdrawal event, the
licensee reanalyzed this event assuming the rod withdrew from position 00 to position 48 (full-out) and determined that the change in critical power ratio (CPR) was bounded by the maximum allowable change in CPR for the continuous rod withdrawal accident analysis.
Thus, a minimal reduction in safety margin actually occurred.
However, this reduction was bounded by existing safety analysis.
The occurrence and frequency of unexpected rod movement events is a concern to the NRC.
10 CFR 50, Appendix A, general design criteria (GDC) 25 requires that the protection system be designed to prevent exceeding fuel design limits in the event of any single malfunction, such as, accidental withdrawal of control rods.
This design feature is safety related.
The licensee did not consider rod withdrawal in the case of the November 1992 event, or post scram rod insertion past position 00 as a single failure event, though the transponder card failures challenged GDC 25.
The inspector also noted that Final Safety Analysis Report (FSAR) 15.4.1.2 assumed that operator error was the only cause of continuous rod withdrawal during reactor startup accident analysis and that protection was provided by the rod sequence control system (RSCS).
The rod worth minimizer (RWM)
although not credited in the accident analysis, may also 'terminate this event.
The inspector noted that a control rod must be selected for these systems to function properly.
Accordingly, the safety functions of rod sequence control system (RSCS) and the rod worth minimizer (RWM) would be bypassed by these transponder card failures since they occur independent of what rod is selected.
For the November 1992 and October 1993 rod withdrawals, some protection was provided by the RDCS since it halted rod motion.
However, this feature is not safety-related and is not credited in the accident analysis.
These malfunctions have occurred and may continue to occur based on the current operating history.
In response, the licensee has initiated action to perform additional evaluations to determine whether specific component level weaknesses were the source of the transponder card failures or whether other factors, such as, debris or contaminants prevented directional control valve closure or collet piston seating.
In addition, the licensee is reviewing the known failure history to determine ifthese events are reportable, and to determine if additional corrective action is necessary to prevent or further limitunexpected rod motion events.
The final root cause, corrective action, and reportability of these events willremain unresolved pending further review by the licensee and evaluation by the NRC.
(URI 50-387/93-19-01)
3.
MAINTENANCE/SURVEILLANCE 3.1 Maintenance Inspection Activity
On a sampling basis, the inspector observed and reviewed selected maintenance activities to ensure that specific programmatic elements described below were being met.
Details of this review are documented in the following section.2 Maintenance Observations The inspector observed and/or reviewed selected maintenance activities to determine that the work was conducted in accordance with approved procedures, regulatory guides, Technical Specifications, and industry codes or standards.
The following items were considered, as applicable, during this review: Limiting Conditions for Operation were met while components or systems were removed from service; required administrative approvals were obtained prior to initiating the work; activities were accomplished using approved procedures and quality control hold points were established where required; functional testing was performed prior to declaring the involved component(s) operable; activities were accomplished by qualified personnel; radiological controls were implemented; fire protection controls were implemented; and the equipment was verified to be properly returned to service.
These observations and/or reviews included:
WA 20342, Change Out CRD Control Rod Drive Mechanism 22-11, dated October 5.
WA 33683, Install Tie-in From New Reactor Water Level Back Fill System to XY-B21-1DOOYACondensing Chamber Reference Leg and Disable Valve 142002A, dated October 18.
WA 24802, HPCI Turbine Control System Calibration, dated November 8.
WA 20331, Remove/Reinstall 719'levation Wall Panels, dated November 8.
WA 33099, Recharge Automatic Depressurization System (ADS) Bottles With Nitrogen, dated November 8.
WA S34882, Removal/Repair/Reinstall Mast on Unit 1 Refuel Platform, dated November 9 and 10.
Inspection Report 50-387/93-80 pertains.
3.3 Inspection Findings The inspector reviewed the listed maintenance activities.
The review noted that work was properly released before its commencement; that systems and components were properly tested before being returned to service and that maintenance activities were conducted properly by qualified personnel.
Where questionable issues arose, the inspector verified that the licensee took the appropriate action before system/component operability was declared.
Except for the Refueling Mast repair (WA S34882), and as otherwise noted below, the inspectors had no further questions on the listed activitie p
~
3.3.1 Reactor Water Level Modification The inspector observed portions of the Reactor Water Level Backfill modification tie-in to the condensing pots on October 18.
The modification was being installed in response to NRC Bulletin 93-03, "Resolution of Issues Related to Reactor Vessel Water Instrumentation in BWR's". The modification package required a manual isolation (normally locked open)
valve to be disabled in the open position.
The licensee determined this measure was necessary to avoid pressurizing the instrument rack with CRD pressure should the valve be closed inadvertently.
This would result in false indication of high reactor pressure (1300 psig) and low reactor water level (below TAF). These false signals would produce a number of Emergency Core Cooling System (ECCS) initiations and primary containment isolations.
The licensee's work plan required the valve be verified open by the Work Group Foreman.
During a job site interview, the foreman stated that he opened the valve, removed the handwheel, and the chain locking device.
The inspector observed the mechanic tack welding a cap over the valve stem and around the stem in the valve yoke area to prevent operation of the valve.
The inspector noted that the work plan did not require documentation of the valve being verified open.
The licensee also did not require independent verification of the valve's position prior to disabling the valve.
In order to perform the work, maintenance personnel positioned valves within the blocking boundaries as allowed by station work procedures.
However, the valve positionings were not recorded on NDAP-QA-502-5 Form "Equipment Status Changes Within Blocking Points" nor was the form in the work package as required by NDAP-QA-502, "Work Authorization System".
The valve positioning was, however, documented in the action taken form in the work package.
The NDAP-QA-502-5 form was completed later that day.
The inspector informed maintenance supervision of the conditions identified.
The inspector concluded that the work plan did not include the appropriate instructions in the work package to properly document, and independently verify, valve position prior to disabling the valve.
The inspector reviewed the installation strategy noted in the Installation KickoffMeeting (IKM)minutes for this modification package.
The inspector concluded that post-modification testing would have verified the valve in the proper position.
This testing would provide an acceptable means of position verification, however, this method was not intentionally chosen to verify valve position.
Since the work plan was developed from the IKMminutes, the work plan did not require any independent verification of valve position.
In response to inspector concerns, the licensee has changed the IKMinstallation strategy for the Unit 2 modification to have the valves'osition confirmed and independently verified by operations department prior to disabling it. The inspector considered the licensee's course of action appropriat The inspector concluded that although the NDAP-QA-502-5 form was filled out for the work activity after the evolution, the form was not included in the work package prior to work.
The omission was not detected by the work planner, or by any subsequent reviews by either Planning Group Supervision or Work Group Supervision.
The inspector subsequently reviewed the work package planning and review checklist and noted that it was not reviewed by either the planning or work group foreman.
The block for checking verification provisions was marked not applicable (N/A). Based on a review of licensee procedures, the inspector determined that the checklist use is required but that no formal proceduralized implementation instructions existed.
The inspector considered this a weakness in the planning process.
The inspector determined since no valves were mispositioned, neither instance was safety significant.
However, the need for greater licensee awareness for the proper administrative control of valves during maintenance and modification activities was indicated for both planning and field implementation activities.
The licensee committed to have the work package planning and review checklist use proceduralized by January 1, 1994.
3.3.2 CRD Mechanism Removal Problems During Unit 1 Refueling Outage Problem Description On October 5, while removing the first Control Rod Drive (CRD) to be exchanged (22-11)
per MT-055-015, the Nuclear Energy Services (NES) hydraulic liftcylinder was unable to support the weight of the mechanism.
The condition allowed the CRD to slowly drift out of the vessel until the weight of the CRD was supported by the elevator.
This condition was unexpected.
The licensee successfully removed CRD 22-11 and transferred the mechanism to the CRD rebuild room.
On October 6, at 12:10 p.m., CRD 58-39 was being lowered from the reactor vessel to perform the 10-6 uncoupling check.
The detection of excessive leakage during this check indicates that the rod may still be coupled.
The mechanism was lowered 10 inches and could not be raised the required 6 inches, which prevented the uncoupling check in accordance with the procedure.
The licensee continued efforts to remove the CRD without resolving this discrepancy.
Outward motion stopped at 12 foot 2 inches, and the drive could not be either raised or lowered from that position.
Approximately 10 to 20 gpm leakage existed at the time.
Maintenance contacted General Electric (GE) to assess the problem.
GE indicated that the rod must still be coupled as evidenced by the 12 foot 2 inch distance.
Consequently, the licensee successfully reinserted the CRD at 10:30 p.m., in accordance with contingency instructions 'contained in the procedure.
The licensee documented both occurrences on Significant Operating Occurrence Report (SOOR) 93-32 Licensee Response For the October 5 problem, the licensee identified that the NES liftcylinder was leaking hydraulic fluid. After replacing the device, it was tested with a dynamometer to ensure satisfactory operation and performance.
The licensee subsequently initiated a revision to the procedure to include a dynamometer check with established acceptance criteria.
Previously, the licensee only checked the liftcylinder in the horizontal position when it was unloaded.
For the October 6 problem, the licensee determined that the uncoupling tool, used to determine and confirm control rod drive detachment, erroneously indicated that the CRD was uncoupled from the control rod.
Maintenance, along with Nuclear System Engineering (NSE), determined that the reed switch on the particular uncoupling tool was highly sensitive and provided false indication that the rod was uncoupled.
Consequently, the licensee has revised the acceptance criteria for the uncoupling tool functional test to specify the distance the reed switch should pick up and drop out to indicate actual control rod drive detachment.
The licensee performed the uncoupling check using an uncoupling tool tested in accordance with the new acceptance criteria.
The uncoupling check determined that two drives were still coupled.
Subsequently, the licensee successfully uncoupled the CRDs in accordance with the procedure.
The licensee determined that the reason the mechanism could not be reinserted during the uncoupling check on October 6 was that insufficient air pressure was supplied to the NES hydraulic liftcylinder.
The licensee determined that the air line to the NES hydraulic lift also diverted air pressure to the CRD rebuild room via a tee connection.
Such air pressure diversion resulted in the insufficient supply pressure to the NES machine.
The inspector reviewed the Installation and Operating Manual gOM) for the NES machine which specified a supply pressure of 90-125 psi and noted that the licensee's arrangement used a 125 psi service air line with no supply pressure gage.
The licensee subsequently provided a dedicated line for the remainder of the CRD exchanges and initiated action to install a supply line gage with a specified minimum supply pressure for operation.
Inspector Conclusion The inspector determined that the licensee's immediate resolution was effective for the problems encountered.
However, the inspector noted that the information flow within and between the Maintenance and Operating organizations, relative to the problems involving the CRD removal activities, was not timely or clear.
For example, Operations personnel were not made aware of the October 5 problem until the next day.
Relative to the October 6 problem, while Maintenance notified the control room of the stuck CRD, the Shift Supervisor was not notified of the stuck CRD by the Unit Supervisor.
In view of the potential for significant reactor vessel leakage imposed by CRD removal operations, clear and timely communications are essentia Additionally, the inspector noted that though it was known that the uncoupling check could not be performed in accordance with the procedure, Maintenance personnel still continued efforts to remove the CRD mechanism until it became stuck.
Maintenance personnel made the decision to continue lowering the mechanism without consulting maintenance management or notifying operations shift supervision.
The inspector reviewed the procedure and determined that the directions were unclear relative to the identification and resolution of problems that could be encountered when performing uncoupling checks.
However, since the CRD could not be confirmed to be uncoupled and station management was not notified of the circumstance, the decision to continue lowering the device was imprudent.
The licensee agreed this was an error in judgment and committed to revising the procedure.
The SOOR resolution willalso address the importance of resolving unexpected or anomalous conditions when encountered, and the need to promptly notify station management of such occurrences.
The final SOOR resolution and procedural revisions willbe implemented prior to the Unit 2 March 1994 refueling outage.
The inspector willcontinue to follow licensee resolution of these issues.
4.
ENGINEERING/TECHNICALSUPPORT 4.1 Inspection Activity The inspector periodically reviewed engineering and technical support activities during this inspection period.
The on-site Nuclear Systems Engineering (NSE) organization, along with Nuclear Technology in Allentown, provided engineering resolution for problems during the inspection period.
NSE generally addressed the short term resolution of engineering problems; and interfaced with the Nuclear Modifications organization to schedule modifications and design changes, as appropriate, to provide long term corrective action.
The inspector verified that problem resolutions were thorough and directed at preventing recurrence.
In addition, the inspector reviewed short term actions to ensure that they provided reasonable assurance that safe operation could be maintained.
Licensee actions were acceptable.
5.
PLANT SUPPORT 5.1 Radiological Controls PP&L's compliance with the radiological protection program was verified on a periodic basis.
These inspection activities were conducted in accordance with NRC inspection procedure 71707.
Observations of radiological controls during maintenance activities and plant tours indicated that workers generally obeyed postings and Radiation Work Permit requirements.
No significant observations were mad C
0
5.2 Emergency Preparedness The inspector reviewed licensee event notifications and reporting requirements for events that could have required entry into the emergency plan.
No events were identified that required emergency plan entry.
5.3 Security PP&L's implementation of the physical security program was verified on a periodic basis, including the adequacy of staffing, entry control, alarm stations, and physical boundaries.
These inspection activities were conducted in accordance with NRC inspection procedure 71707.
The inspector reviewed access and egress controls throughout the period.
No significant observations were made.
6.
SAFETY ASSESSMENT/QUALITY VFAUFICATION 6.1 Core Shroud Inspection - Unit 1 Background In 1990, crack indications were identified in the beltline region of a core shroud for an overseas reactor which had operated for approximately 16 years.
In response to these indications, General Electric (GE) issued Rapid Information Communication Services Information Level (RICSIL) 054, "Core Support Shroud Crack Indications," on October 3, 1990 to all GE BWR owners.
This RICSIL recommended visual inspection of high carbon 304 stainless steel (SS) core shroud seam welds and heat affected zone (HAZ) around the welds.
During recent inspections, Brunswick Unit 1 (BWR-4 reactor) identified circumferential cracks and axial cracks in various horizontal seam wields.
In the HAZ of weld H-3, a 360 circumferential crack was identified in the core shroud.
This weld fuses the top guide support ring to the lower shroud.
Brunswick Unit 1 is planning on repairing this crack prior to startup.
Licensee Followup In response to these concerns, GE issued SIL 572, "Core Shroud Cracks", which was subsequently revised (Revision 1), on October 4, 1993, to specify inspections at the next scheduled refueling outage.
This SIL recommended that in vessel inspections be performed to detect cracking that could affect structural integrity. The recommended method was an enhanced visual inspection (VT-1) that was capable of resolving a one mil wire on the inspection surface.
The licensee performed their core shroud inspection using the enhanced
VT-1 methodology on October 20 and 21.
Activities to perform this inspection were initially implemented prior to the revised SIL and in response to the July indications observed at Brunswick and recent indications at Peach Bottom.
The licensee inspected two vertical welds and approximately 120'f three horizontal welds (CF, DB and DC) which were equivalent to the H-3, H-4, and H-5 welds identified in the SIL. No cracks were identified.
Inspector Conclusion The inspector independently observed portions of the videotape used by PP&L inspectors to identify cracking in the core shroud.
None were identified.
The inspector also questioned the licensee on the scope of their inspection and noted the licensee's inspection plan was initiated before their receipt of SIL 572 Revision 1 and, as such, did not fully address all of the recommended actions in the SIL. Three SIL items were not fully addressed at the conclusion of the inspection.
They were:
The need to inspect welds H-1 through H-7, inclusive.
The lack of cleaning of all weld surfaces and the acceptability of hydrolazing.
The availability of the core shroud fabrication history.
~fter detailed discussions with the licensee, the inspector concluded that the licensee's inspection addressed the most susceptible crack locations.
However, no justification or analysis existed to describe why the licensee's deviation from the SIL were acceptable.
The licensee has agreed to provide the necessary justification and willreevaluate the need for further inspections.
Based on the above, the inspector has no further questions.
7.
MANAGIMENTAND EXITMEETINGS 7.1 Resident Exit and Periodic Meetings The inspector discussed the findings of this inspection with station management throughout and at the conclusion of the inspection period.
Based on NRC Region I review of this report and discussions held with licensee representatives, it was determined that this report does not contain information subject to 10 CFR 2.790 restrictions.
7.2 Unit 1 Turbine Failure Management Meeting A Management Meeting was held on November 4 between NRC and PP&L to discuss the licensee's root cause analyses for the Unit Turbine Failure that occurred on July 12, 1993.
PP&L presented their findings and provided an opportunity for NRC management to better understand the failure and its potential generic implication /
Attachment 1 provides a copy of PP&L's presentation.
Attachment 2 is a list of meeting attendees.
7.3 Inspections Conducted By Region Based Inspectors
~ln ggg~in gggg~l'gg
~Re ~rtin ~
~n~gr 10/12 - 10/14/93 11/08 - 11/12/93 ISI 93-17 Occupational Exposure 93-21 R. McBrearty J. Noggle
ATTACHMENT 1 NRC MANAGEMENTMEETING UNIT 1 TURBINE BLADE FAILURE EVENT INTRODUCTION AGENDA George T. Jones - VP - Nuclear Engineering PRESENTATIONS
~ Turbine Design/Event Robert A. Saccone Acting Manager - Nuclear Systems Engineering
~ Root Cause Investigation Glenn D. Miller-Manager - Nuclear Technology CONCLUDINC REMARKS George T. Jones
UNIT 1 TURBINE BLADE FAILURE EVENT TURBINE DESIGN/EVENT R. A. Saccone
- Acting Manager Nuclear Systems Engineering Turbine/Generator Design Turbine Blade Failure Event
+
Event Time Line Damage Assessment Repairs
+
Modifications -and Enhancements Unit 1 Start-Up
+
Unit 1 7RIO Inspections
+
LP-B Rotor Repairs
UNIT 1 TURBINE/CENERATOR OVERVIEW TURBINE
~ Ceneral Electric 1800 rpm, tandem compound, six flow, non-reheat, steam turbine, designed for 1084 MWE at initial steam conditions of 965 psia and 0.25% moisture.
~ Consists of one HP, and three LP turbines
~ 38 inch last stage buckets (LSB)
~ Has multipressure condenser at 2.99, 3.56, and 4.43 in.
Hg Abs.
~ Extraction steam for normal five stage feedwater heating and three feed pump turbine drives.
~ LP rotors of monoblock design (both Units)
UNIT 1 TURBINE/GENERATOR OVERVIEW (Cont'd)
GENERATOR
~ General Electric 1280 MVA, 1800 rpm, direct connected, 4 pole, 60 Hz, 24,000 V.
~ Liquid cooled stator, hydrogen cooled rotor, synchronous generator rated at 0.90 power factor.
~ Generator is sized to accept the gross output of the turbine.
PROTECTIVE FEATURES
~ There are 17 emergency trip functions.
~ The High Turbine Vibration trip automatically shut down Unit 1 during transient.
NAC MANAGEMENTMEETING - 11-04-93
E
UNIT 1 TURBINE/CENERATOR OVERVIEW (Cont'd)
TURBINE SAFETY ANALYSIS
~ Unit 1 bucket failure was bounded by FSAR analysis.
~ FSAR Section 3.5.1.3, Turbine Missiles, bounds the failure experienced.
- Analyzed failure assumed entire wheel burst along with all buckets.
- Mass of Unit 1 failure (2 buckets) was about 1% of analyzed mass failure.
~ Failed buckets did not -damage or penetrate the turbine shel EVENT TIME LINE 7/'12/93.-
SSES Unit 1 scrammed from 100% power due to 1635 main turbine control valve fast closure as a result of high turbine vibration.
ALL EQUIPMENT OPERATED PER DESIGN.
NO UNUSUAL OPERATOR ACTIONS WERE REQUIRED TO PLACE THE UNIT IN A STABL'E CONDITION
~ Both reactor recirc pumps tripped via EOC-RPT logic as designed.
~ Main generator primary lockouts tripped on the primary anti-motoring relay followingturbine trip.
~ Auxiliary busses fast transferred from the main auxiliary transformer to the startup bus.
~ All RPS trip signals were generated in a timely fashion and all control rods fully inserted.
~ All Reactor pressure, level parameters responded in accordance with design.
NRC MANAGEMENTMEETING - 11-04-93
EVENT TIME LINE 7/12/93 (Cont'd)
~ Condensate demineralizer inletconductivity (CD I)
rose sharply following the event due to damaged condenser tubes.
~ High offgas flows were present following the transient due to condenser tube damage and subsequent water box isolation and venting.
NRC MANAGEMENTMEETING - 11-04-93
I g
c
EVENT TIME LINE (Cont'd)
't 235
-
ENTERED COLD SHUTDOWN.
SPING, main steamline and offgas radiation levels both trended with power.
There were no indications of fuel damage.
Primary Containment integrity was maintained throughout the transient.
The only challenge to containment was the momentary lifting of the
'D'nd
'E'RVs for three and nine seconds respectively.
NRC MANAGEMENTMEETING - 11~.93
DAMACE ASSESSMENT
~ Inspection manways for the outer hood of the A, B, and C Low Pressure (LP) rotors were removed.
~ Last stage (L-0) bucket damage was observed on the turbine end of the LP-C rotor.
~ Upon removal of LP-C rotor, L-0 and L-1 buckets, visual inspection revealed:
On the L-1 rotating row of the LP-C turbine end, buckets ¹11 8 ¹89 separated from their fingers at the upper dowel pin hole. (There are six fingers/bucket.)
The two ejected buckets were approximately 120 degrees apart.
The L-0 rotating row of the LP-C turbine end had extensive consequential damage including stellite strip tears.
- The L-0 stationary row of the LP-C turbine end upper and lower halves had consequential damage to all airfoil tips.
- On the L-1 rotating row of the LP-C generator end, bucket ¹1 had four of six fingers broken.
NRC MANAGEMENTMEETING - 11~.93
~
P F
t P,
DAMAGE ASSESSMENT (Cont'd)
v
- The L-1 stationary row of the LP-C turbine end lower half was damaged.
- -The L-1 rotating row of the LP-C turbine end spill strips were destroyed.
This was the only sign of rubs on any rotating/stationary row other than a
cover rub on bucket 01 of L-I generator end.
- There was no damage to either bearings 47 or 48.
~ Disassembly of LP-C rotor, L-1 buckets, both ends showed the following bucket finger damage:
- 32 completely broken fingers
- 31 fingers with visible cracks Most cracks initiated at the centerline of the top pin holes (3 rows, 2 pins per row)
~ Mag Particle examination, L-I and L-0 wheel areas, (both ends of rotor), was satisfactory.
~ Mag Particle examination of L-0 buckets, both ends, showed no cracked finger I
~ e
~,
~
DAMACE ASSESSMENT (Cont'd)
~ Based on LP-C inspections and available C.E.
model analysis, it was decided to visually inspect LP-B rotor (in place).
LP-B rotor assembly was inspected 360'hile in place.
No visible damage/cracks were observed on the L-l bucket covers and tenons, or the L-0 buckets.
A lift check of the LP-B L-1 buckets was performed and results were normal.
No damage was observed on any other LP-8 components.
Based on the LP-B inspection results, combined with rotor
"de-tuning" it was determined the rotor was acceptable for safe operation until the Ul 7RIO.
~ MAIN CONDENSER INSPECTION
- All damage was confined to the LP-C condenser.
(One tube bundle)
12 tubes were sheared/punctured.
- Approximately 91 tubes were slightly dented.
(All at periphery of the bundle)
- Several pieces of turbine material (different sizes) were found at various locations on the tubes and trays.
- Silt from circulating water entering the condenser through the sheared/punctured tubes was found on the tubes and trays.
NRC MANAGEMENTMEETING - 11.04.93
MODE
CALCULATED FREQUENCY
LP CI Genef"odor
REPAIRS
~ LP-C ROTOR
- Installed new L-1 buckets on both ends of rotor.
- Installed new I-0 buckets on turbine end.
- Repaired damaged. diaphragms and spill strips.
- Performed all routine refurbishment activities normally done during turbine overhaul.
- Performed low speed balance of rotor on test stand prior to installation.
NRC MANAGEMENTMEETlNG - 11~93
'l
~
~ f REPAIRS (Cont'd)
- fngineering assessment and oversight of recovery included:
hands on inspection flushing of condensate demin bypass line
+
changeout of all condensate demin resin
+
hand cleaned, where practical, loose debris hydrolyzed tubes and trays
+
de-mucked hotwell to remove debris/silt plugged 91 tubes which had signs of damage cut out and removed sections of sheared tubes
- The above ensured no debris reached the reactor during the transient nor would the reactor be affected during unit startup.'
Checked condenser tube integrity during initial vacuum pull with mechanical vacuum pump (total of 4 additional tubes plugged at this time). Total of 107 tubes plugged.
NRC MANAGEMENTMEETING - 11-04.93
I
'
l
MODIFICATIONSAND ENHANCEMENTS
~ Modification and Safety Evaluation installed 4394 lb.
MASS RING on the generator coupling, turbine end, to de-tune rotor.
~ Modification and Safety Evaluation installed 3 "Torque Collars" on turbine bearings 4, 6, and 8 to measure rotor torsional response during turbine startup and synchronization.
NRC MANAGEMENTMEEllNG - 1'3<4.93
UNIT STARTUP
~ Nuclear Systems Engineering developed and implemented (through the
"Special, Infrequent or Complex Test/Evolution" program) technical procedure TP-193-027, "Ul Main Turbine Torsional Vibration Testing",
to confirm adequacy of the mass ring modification.
~ TORSIONAL TEST Required temporary modification of generator excitation to permit a controlled application of negative phase sequence current.
+
Involved a
s Iow acceleration of the turbine/generator through a
band of speeds bracketing the normal'running speed of 1800 rpm.
Out of phase synchronization not performe a
UNIT STARTUP (Cont'd)
~ Testing showed the mass ring modification successfully detuned the 20th mode harmonic from a calculated 120.4 hz to measured 118.1 hz.
~ Unit 1 was successfully synchronized to the grid at 0258 hrs on 8/31/93. The turbine performed without problems through the start of U1 7RIO on 9/25/93.
NRC MANAGEMENTMEETING - 11-04-93
~
~
U1 7RIO INSPECTIONS
~ As part of planned outage activities, removed LP-8 rotor.
- Mag particle testing of L-0 and L-1 buckets (turbine end) and L-I buckets (generator end) revealed a total of 4 indications on L-l, turbine end buckets.
- Mag particle testing of LP-B rotor at above bucket areas showed NO indications.
- Based on these inspections together with available C.E. torsional analysis, Engineering concluded no further NDE inspections were required.
NRC MANAGEMENTMEETlNG - 11%4.93
LP-B ROTOR REPAIRS
~ Replaced all i-1 buckets, turbine end.
~ Performed all routine turbine overhaul inspections and necessary refurbishing.
~ Performed low speed balance of rotor on test stand prior to installation.
NRC MANAGEMENTMEHlNG - 1144-93
T P
ur ine aiure na sis NRC Management Meeting 11/04/93
roac:
na sis BUse ec Problem Statement: "On July 12, 1993 at 1635 the Unit 1 LP 'C'urbine end ¹11 and ¹89 buckets failed causing an extended forced outage and loss of generation."
Effect Cause NRC Management Meeting 11/04/93
o en ia auses Flow induced vibration Harmonic excitation 3. Water induction Material strength deterioration Manufacturing and assembly issues 6. Material issues 7. Torsional resonance NRC Management Meeting 11/04/93
ow n UCe i ra iOn
~ Several types of flow induced vibration
~ Bucket rows other than L-1 would be affected
~ Failures of covers, tenons and bucket tips would occur before dovetail finger failure
~ Evidence shows this to be very unlikely NRC Management Meeting
armonic xci a ion
~ Occurs when the natural frequency or mode of a component equals a multiple of the frequency of the turbine system
~ Failure would tend to occur only in susceptible components
~ All bucket and bucket group frequencies are within design acceptance criteria
~ SSES bucket tests shows all buckets within acceptance criteria
~ Actual cracking was observed in both 5 and 6 bucket groups NRC Management Meeting
acr n Ucion
~ Occurs either as:
a large column of water over a short time, or a lesser quantity over a long period of time
~ No evidence of increased erosion
~ No feedwater heater high level alarms prior to event
~ No blockage of steam extraction lines observed
~ This failure mode would produce major damage to the entire stage of the turbine NRC Management Meeting
t
a eria ren e eriora ion
~ Material strength and hardness tests were within design ranges
~ Susquehanna SES water chemistry has been acceptable
~ No detrimental contaminants have been found during metallographic inspections
~ No intergranular stress corrosion cracking was found
~ Radiation doses are not significant enough to alter material properties NRC Management Meeting
anu ac urin an ssem ssues
-
~ No unusual dimensional tolerances
~ As-built L-1 bucket frequency and moment weight acceptable.
~ As-found bucket dovetail hole sizes within allowable values
~ Bucket dovetail overspeed capability is calculated at 73%, design criterion is 50%
~ No unusual difficultywith bucket material procurement, design or manufacture
~ Bucket reassembly documented as routine
~ The reaming process used is standard for buckets of this type.
NRC Management Meeting
a eria ssues
~ Material specifications are standard for buckets of this service.
~ Extensive metallurgical and chemical tests have shown materials are within design specification
~ All dovetail pins were easily removed NRC Management Meeting
orsiona orsiona esonance i ra ion
~ Twisting of shaft about its axis
~ The mass and geometry of the rotating element define its natural frequency
~ Torsional mode = combination of individual component natural frequencies
~ When rotor speed equals a mode frequency, the rotor system speed and natural frequency are in resonance
~ When resonance occurs, energy transfer into the components is possible NRC Management Meeting
I
~
orsiona esonance i ra ion imuus
~ Ifthe electrical distribution system were perfectly balanced, each of the three phases would have an identical load
~ Not all loads are three-phase
~ Load unbalance creates current flow in the stator opposite in rotation to the main current flow (referred to as Negative Phase Sequence Current)
~ Interaction with rotating electrical field creates alternating torque at a frequency twice the operating frequency
~ This torque transfers energy to the shaft in the torsional direction
~ Energy is transferred from the generator to the
~
orsiona ai Ure esonance 0 e
~ GE calculation after failure placed the rotor natural frequency of the 20th mode at 120.4HZ + 2.1 HZ
~ This creates the potential for excitation by the generator negative phase sequence currents
~ Sufficient stress can be created in the L-1 bucket dovetails to drive high cycle fatigue.
~ The as.-found general magnitude and location by row and turbine section of dovetail finger cracking matches calculation.
~ A comparison of the SSES and Maanshan bucket failures shows many similarities.
~ Subsequent torsional testing showed very high response for 20th mode NRC Management Meeting
~-
MODE
CALCULATED FREQUENCY
=
LP CI Genei" odor
ea ur ica vaua ion
~ Bucket failure was not a continuous process Failure would have occurred in <245 days at stress levels high enough to propagate cracks
~ Initiation of cracks was not a factor in the fatigue life since cracks were already present in the holes due to reaming
~ Some very high stress excursions as well as some lower stress excursions occurred at various times in the running history of the turbine that continued to propagate cracks NRC Management Meeting
e a ur ica oncusions
~ Variable amplitude, high cycle, low stress fatigue was the cause of bucket fractures
~ Crack growth direction was tangential to the bucket row supporting the torsional vibration cause
~ Initiation of cracks occurred in the work hardened surface of the reamed pin holes
~ Crack growth occurred over the entire operating history of the turbine and was not a single event that occurred just before failure
~ Torsional vibration caused by negative phase sequence currents providing a stimulus at the generator that acted in resonance with the rotor's 20th torsional mode of vibration.
~ The torsional vibration led to high cycle fatigue at the top dovetail pin holes which caused two L-1 buckets to fail.
NRC Management Meeting
~
en ssues
~ Impact on Unit 2
~ Future Turbine Design Cbanges
~ Future Inspection Frequency and Metbods NRC Management Meeting
m acson ni
~ Calculated results same for both units
~ Testing of other dual units shows differences in vibrational modes
~ Operating times differ
~ Less electrical transients on Unit 2 NRC Management Meeting
1
e aive inimum e uence ummar a ues Negative Sequence High
I Unit 1 Blades 6 Unit 2 Blades M
Note: Unit 2 Blade events in the minimum value group include several while installed on Unit 1. Specifically, 6 lows and 4 mediums.
0
20
Number of Events
50 NRC Management Meeting
11/04/93
I
e uence a ues e aive aximum ummar Negative Sequence High:;;...~2
II Unit 1 Blades Kl Unit 2 Blades M
<4%~@ "i~ 7
Note: Unit 2 Blade events in the maximum value group iriclude several while installed on Unit 1. Specifically, 6 lows, 3 mediums and i high.
0
20
Number of Events
50 NRC Management Meeting
11/04/93
'C
> ~
C
~ Torsional test planned prior to Unit 2 6 RIO
~ Torque collar & instrumentation Material available for installation Modification package issued
~ Mass ring Material ordered Modification in progress
~ Inspect blades during U2-6RIO
~ Install new blades as needed NRC Management Meeting
NRC MANAGEMENTMEETING UNIT t TURBINE BLADE FAILURE EVENT CONCLUDINC REMARKS - C. T. Jones 1.
ACCEPTANCE OF ROOT CAUSE 2.
CORRECTIVE ACTIONS 3.
UNIT 2 IMPACTS 4.
CENERIC IMPLICATIONS NRC MANAGEMENTMEETING - 11-04-93
AITACHMENT2 NAME TURBINE MEETING ATTENDANCE TITLE Nuclear Regulatory Commission Wayne Hodges Charles Miller Wayne Lanning Jacque Durr John R. White Harold Gray David J. Mannai Hal Ornstein John Tsao A. Lohmeier P. Patnaik Director, DRS Acting Deputy Director, DRS Deputy Director, DRP Chief, Engineering Branch Section Chief, DRP Mobile Lab - NDE Chief Resident Inspector Senior Reactor Systems Engineer - AEOD NRR/EMCB Materials Engineer Material Section Material Section Pennsylvania Power and Light George T. Jones Robert Saccone Glenn Miller James M. Kenny Michael B. Detamore L.E. Willertz J.P. Felock Philip W. Brady Rick Wehry Robert Kichline Matthew Hober, Jr.
VP - Engineering Manager Nuclear Systems Engineering Manager - Nuclear Technology Licensing Supervisor Project Manager Maintenance Technology - Metals BOP Supervisor, Nuclear Systems Engineer System Analysis Electrical Site Compliance Licensing Nuclear Technology Others Dave Ney Bob Maiers Paul Schott Anthony L. Moffa PA DER BRP BRP Nuclear Engineer Liberty Technologies Nuclear Program Liberty Technologies Nuclear Program