IR 05000361/2012010

IR 05000361-12-010 and 05000362-12-010, 08/20/2012 - 09/28/2012, San Onofre Nuclear Generating Station, Augmented Inspection Team Follow-up Inspection

ML12318A342
Person / Time
Site:	San Onofre
Issue date:	11/09/2012
From:	Blount T Division of Reactor Safety IV
To:	Peter Dietrich Southern California Edison Co
References
IR-12-010
Download: ML12318A342 (37)

Text

UNITE D S TATE S NUC LEAR RE GULATOR Y C OMMI S SI ON ber 9, 2012

SUBJECT:

SAN ONOFRE NUCLEAR GENERATING STATION - NRC AUGMENTED INSPECTION TEAM FOLLOW-UP REPORT 05000361/2012010 AND 05000362/2012010

Dear Mr. Dietrich On September 28, 2012, the U.S. Nuclear Regulatory Commission (NRC) conducted a follow-up inspection of 9 of the 10 unresolved issues identified by the Augmented Inspection Team in Inspection Report 05000361/2012007 and 05000362/2012007 (ML12188A748). The inspectors did not review Unresolved Item 2012007-04, Evaluation of Changes in Dimensional Controls during the Fabrication of Unit 2 and Unit 3 Replacement Steam Generators, because your staff had not completed their evaluation and corrective actions for this item. The Augmented Inspection Team reviewed the circumstances surrounding the Unit 3 steam generator tube leak that occurred on January 31, 2012. The enclosed report documents the inspection results discussed with Mr. D. Bauder, Vice President and Station Manager, and other members of your staff on September 28, 2012.

The inspectors examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.

The inspectors reviewed selected procedures, documents, and records and interviewed personnel.

The NRC closed 8 of the 10 unresolved items. The two remaining unresolved items are related to the mechanistic cause of the tube-to-tube vibration resulting from fluid-elastic instability. The combination of higher than predicted thermal-hydraulic conditions and lack of sufficient anti-vibration bar-to-tube support led to the fluid-elastic instability. The NRC will conduct subsequent inspections and reviews to determine the complete sequence of events and regulatory actions, as applicable, for these two unresolved items.

P. Dietrich -2-In addition to the other inspections and reviews, the NRC plans to conduct a Category 1 public meeting in November 2012 at which time the NRC staff will seek to understand the technical basis for your response to the NRC Confirmatory Action Letter, dated March 27, 2012 (ML12087A323). The next planned inspection activity is the Confirmatory Action Letter Inspection scheduled for the week of December 3. After completion of the Confirmatory Action Letter Inspection and the Office of Nuclear Reactor Regulation technical review associated with your Confirmatory Action Letter Response for Unit 2, dated October 3, 2012 (ML12285A263), a Category 1 public exit meeting will be held to discuss the results of the inspection and technical review. Following that meeting, the NRC will conduct internal management discussions to further assess and evaluate the results of those efforts and develop NRC conclusions. The inspection and technical review reports will be made publically available within 45 days of the public exit meeting. The NRC conclusions associated with Unit 2 will also be made publicly available following the publication of the inspection and technical evaluation reports.

Two NRC-identified violations of very low safety significance (Green) were identified during this inspection. Further, a licensee-identified violation, which was determined to be of very low safety significance, is listed in this report. The NRC is treating these violations as non-cited violations (NCVs) consistent with Section 2.3.2 of the Enforcement Policy.

If you contest these non-cited violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, Region IV; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at San Onofre Nuclear Generating Station.

If you disagree with a cross-cutting aspect assignment in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the Regional Administrator, Region IV; and the NRC Resident Inspector at the San Onofre Nuclear Generating Station.

In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs Agencywide Document Access and Management System (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Thomas B. Blount, Acting Director Division of Reactor Safety

P. Dietrich -3-Docket Nos.: 05000361, 05000362 License Nos: NPF-10, NPF-15 Enclosure:

Inspection Report 05000361/2012010 and 05000362/2012010 Attachment:

Supplemental Information cc w/ encl: Electronic Distribution

ML12318A342 SUNSI Rev Compl. Yes No ADAMS Yes No Reviewer Initials GEW Publicly Avail. Yes No Sensitive Yes No Sens. Type Initials GEW Team Team Team C:DRS/PSB2/ C:DRS/PSB2/

C:DRP/D Member/RIV Member/RII Member/RIV AIT Team Lead AIT Team Lead JReynoso JRivera-Ortiz BHagar GWerner RLantz GWerner via email via email via email /RA/ /RA/ /RA/

10/29/12 10/25/12 10/24/12 10/29/12 11/2/12 11/2/12 D:DRP D:DRS KKennedy TBlount

/RA/ /RA/

11/7/12 11/9/12

Table of Contents

SUMMARY OF FINDINGS

............................................................................................................ 3

Summary of Plant Status .............................................................................................................. 5 1. REACTOR SAFETY .............................................................................................................. 5 4OA3 Follow-up of Events and Notices of Enforcement Discretion ............................................ 5 4OA5 Other Activities .................................................................................................................. 6

.1 (Closed) Unresolved Item 05000362/2012007-01, Adequacy of the Trip/Transient and Event Response Procedure ............................................................................................. 6

.2 (Closed) Unresolved Item 05000362/2012007-02, Evaluation of Unit 3 Vibration and Loose Parts Monitoring System Alarms ........................................................................... 8

.3 (Closed) Unresolved Item 05000362/2012007-03, "Evaluation of Retainer Bars Vibration during the Original Design of the Replacement Steam Generators ............................... 10

.4 (Closed) Unresolved Item 05000362/2012007-05, Shipping Requirements not in Accordance with Industry Standards ............................................................................. 14

.5 (Closed) Unresolved Item 05000362/2012007-06, Shipping Requirements not in Accordance with Design and Fabrication Specifications................................................ 17

.6 (Closed) Unresolved Item 05000362/2012007-07, Unit 3 Steam Generator 3E0-88 Stresses Related to Handling ........................................................................................ 18

.7 (Discussed) Unresolved Item 05000362/2012007-08, Non-Conservative Thermal-Hydraulic Model Results ................................................................................................ 19

.8 (Closed) Unresolved Item 05000362/2012007-09, Evaluation of the Effects of Divider Plate Weld Repairs in Unit 3 Replacement Steam Generators...................................... 21

.9 (Closed) Unresolved Item 05000362/2012007-10, Evaluation of Departure of Method of Evaluation for 10 CFR 50.59 Processes ........................................................................ 22 4OA6 Meetings .......................................................................................................................... 26 Exit Meeting Summary .......................................................................................................... 26 4OA

Licensee Identified Violations

.......................................................................................... 26 LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED (Attachment)....................................... 1 LIST OF DOCUMENTS REVIEWED (Attachment)....................................................................... 2

U.S. NUCLEAR REGULATORY COMMISSION REGION IV Docket: 05000361, 05000362 License: NPF-10, NPF-15 Report: 05000361/2012010, and 05000362/2012010 Licensee: Southern California Edison Company Facility: San Onofre Nuclear Generating Station, Units 2 and 3 Location: 5000 S. Pacific Coast Highway San Clemente, California Dates: August 20 through September 28, 2012 Team Lead: G. Werner, RIV, Chief, Plant Support Branch 2 Inspectors: B. Hagar, RIV, Senior Project Engineer J. Reynoso, RIV, Resident Inspector J. Rivera-Ortiz, RII, Senior Reactor Inspector Approved By: G. Werner, Chief, Plant Support Branch 2 Division of Reactor Safety

SUMMARY OF FINDINGS IR 05000361/2012010 and 05000362/2012010; 08/20/2012 - 09/28/2012; San Onofre Nuclear Generating Station; Augmented Inspection Team Follow-up Inspection.

An Augmented Inspection Team was dispatched to the site on March 19, 2012, to assess the facts and circumstances surrounding a steam generator tube leak at Unit 3 on January 31, 2012. The Augmented Inspection Team was established in accordance with NRC Management Directive 8.3, NRC Incident Investigation Program, and implemented using Inspection Procedure 93800, Augmented Inspection Team. This report documents the follow-up inspection to review nine of the unresolved items that were opened during the Augmented Inspection Team inspection documented in NRC Inspection Report 05000361/2012007 and 05000362/2012007. The follow-up team was comprised of resident and region-based inspectors. Three Green non-cited violations of very low safety significance were identified.

The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, Significance Determination Process. The cross-cutting aspect is determined using Inspection Manual Chapter 0310, Components Within the Cross-Cutting Areas. Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 4, dated December 2006.

Cornerstone: Initiating Events

Green.

The inspectors identified a non-cited violation of 10 CFR 50, Appendix B,

Criterion V, Instructions, Procedures and Drawings, because Operations Procedure SO123-0-A8, "Trip/Transient and Event Review," Revision 5, was not adequate in that it did not define what unplanned reactor trip meant, and the operators did not complete the procedure as required. In response, the licensee revised the procedure to describe unplanned reactor trips as explained in industry guidance. This issue was entered into the licensees corrective action program as Nuclear Notifications NN 201915602 and NN 202161665.

This finding is more than minor because if left uncorrected the performance deficiency could be viewed as a precursor to a significant event. Using Inspection Manual Chapter 0609.04, Phase 1 - Initial Screening and Characterization of Findings, and Inspection Manual Chapter 0609, Appendix A, The Significance Determination Process (SDP) for Findings At-Power, the inspectors determined the finding to be of very low safety significance because a reactor trip was initiated with no loss of mitigating equipment or other associated initiators. This finding does not have a cross-cutting aspect because the associated procedure change occurred in 2003 and was not representative of current performance. (Section 4OA5.1)

Green.

The inspectors identified a non-cited violation of 10 CFR 50, Appendix B,

Criterion XIII, "Handling, Storage, and Shipping," involving the licensees failure to take appropriate measures to control preservation of safety-related equipment during shipping, specifically the protective environment provided for the Unit 3 steam replacement generators was not appropriately specified or monitored. The licensee conducted an analysis of the shipping environment and determined no detrimental impact occurred. This issue was entered into the licensees corrective action program as Nuclear Notifications NN 202160749, NN 201960027, and NN 20191118.

The finding is more than minor because it is associated with initiating events cornerstone attribute of equipment performance and affected the associated cornerstone objective to limit the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations.

Using Inspection Manual Chapter 0609.04, Phase 1 - Initial Screening and Characterization of Findings, and Inspection Manual Chapter 0609, Appendix A,

The Significance Determination Process (SDP) for Findings At-Power, the inspectors determined the finding to be of very low safety significance because it did not result in exceeding the reactor coolant system leak rate for a small loss of coolant accident or result in a total loss of systems used to mitigate a loss of coolant accident. This finding does not have a cross-cutting aspect because the associated performance deficiency was not representative of current performance.

(Section 4OA5.4)

Licensee-Identified Violations

A violation of very low safety significance identified by the licensee has been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensees corrective action program. This violation and associated corrective action tracking number are listed in Section 4OA7 of this report.

REPORT DETAILS

Summary of Plant Status

Prior to the event, San Onofre Unit 2 was shut down for refueling outage and Unit 3 was operating at 100 percent rated thermal power with no plant evolutions in progress. On January 31, 2012, Unit 3 control room operators received an alarm that indicated a primary-to-secondary reactor coolant leak from steam generator 3E0-88. The alarm received was from the main condenser air ejector radiation monitors, which continuously samples from a vent line for the purpose of rapidly identifying steam generator tube leaks. Although the leak rate was small, it increased enough in a short period of time for the licensee to perform a rapid shutdown. The estimated leak rate was 75 gallons per day. The facility license allows full power operation with a steady state leak rate of less than 150 gallons per day. On February 2, 2012, Unit 3 reached cold shutdown conditions. The licensee reviewed the amount of gaseous radioactivity released and estimated a dose of approximately 0.0000452 mrem to a member of the public. The annual regulatory limit to a member of the public is 100 mrem per year. At the time of the inspection, both Unit 2 and 3 were in a shutdown and cooldown status.

4OA3 Follow-up of Events and Notices of Enforcement Discretion

.1 (Closed) Licensee Event Report (LER) 05000362/2012-001-00: Unit 3 Manual Reactor

Trip due to Steam Generator Tube Leak On January 31, 2012, the Unit 3 reactor was manually tripped following a rapid power reduction after plant instrumentation detected a tube leak in one of the two steam generators. The inspectors responded to the control room and evaluated plant status, mitigating actions and the licensees procedure compliance in performance of the abnormal operating instructions for a steam generator tube leak. The event was reported to the NRC as Event Notification 47628 and documented in the licensee corrective action program as Nuclear Notification NN 201836127, which included a root cause evaluation. The inspectors observation of the crew response for the reactor trip is documented in the Augmented Inspection Team Report 05000361/2012007 and 05000362/2012007. See Section 4OA5.1 of this report for a discussion of an NRC-identified finding associated with this event.

4OA5 Other Activities

Inspection Procedure 93800, Augmented Inspection Team Unresolved Items For detailed information on the background of each unresolved item, refer to Inspection Report 05000361/2012007 and 05000362/2012007.

.1 (Closed) Unresolved Item 05000362/2012007-01, Adequacy of the Trip/Transient and

Event Response Procedure

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

On March 19, 2012, the team requested to review the results of operations post trip/transient evaluation of the January 31, Unit 3 tube leak event.

Operations Procedure SO123-0-A8, Trip/Transient and Event Review, Revision 8, required a detailed post trip review following unplanned reactor trips. However, a formal trip/transient and event review was not available because operations personnel determined the Unit 3 event was planned and therefore a formal review was not required.

The inspectors reviewed operator logs for a period from 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> prior to the January 31, 2012, event, to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> after that event, to determine how much time had elapsed between the first indication of a tube leak and the subsequent downpower and reactor trip. The inspectors interviewed Operations personnel who had been involved in implementing Operations Procedure SO123-0-A8, "Trip/Transient and Event Review," following that event. The inspectors also reviewed the revision history of Procedure SO123-0-A8, to determine when and under what circumstances the text that referred to unplanned reactor trips had been inserted into the procedure.

b. Observations and Findings

Introduction:

The inspectors identified a Green non-cited violation of 10 CFR 50, Appendix B, Criterion V, Instructions, Procedures and Drawings, because Operations Procedure SO123-0-A8, "Trip/Transient and Event Review, Revision 5, was not adequate.

Description:

Control room logs showed that on January 31, 2012, while the unit was operating at 100 percent power, the Operations crew entered Abnormal Operating Instruction SO23-13-14, "Reactor Coolant Leak," Revision 16, at 3:05:33 p.m. due to indications of an emergent steam generator tube leak of greater than 5 gallons/day.

At or about 4:30 p.m., in accordance with Abnormal Operating Instruction S023-13-14, "Reactor Coolant Leak," Revision 16, the licensee initiated a rapid power reduction, and at 5:31 p.m., the reactor operator manually tripped the reactor from 35 percent power.

Operations Procedure SO123-0-A8, "Trip/Transient and Event Review," Revision 5, required the licensee to complete a detailed post-trip review for all unplanned reactor trips in order to determine the cause of the reactor trip and that the plant and equipment responded as required. For the January 31, 2012, event, the licensee began compiling materials for that review, but suspended their activities when staff members questioned whether that event had constituted an unplanned reactor trip.

In late March 2012, the licensee continued compiling materials for post-trip review of the subject event. At the time of this inspection in August 2012 the licensee had completed all but the final steps of that review.

While reviewing the circumstances associated with the January 31, 2012, event, the inspectors found that Procedure SO123-0-A8, Revision 5, included instructions for responding to unplanned reactor trips, but did not define what unplanned meant.

The inspectors noted that Inspection Procedure 71152, Performance Indicator Verification, does not define unplanned. However, as referenced in Inspection Procedure 71152, Nuclear Energy Institute (NEI) document NEI 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 6, has been accepted for use by the NRC for reporting and tracking performance indicators.

Document NEI 99-02, "Regulatory Assessment Performance Indicator Guideline, includes a section that discusses the Unplanned Scrams Per 7,000 Critical Hours performance indicator, and that section states that scrams directed by abnormal, emergency, or annunciator response procedures are included in this performance indicator (scram is synonymous with trip). On January 31, 2012, in accordance with Abnormal Operating Instruction S023-13-14, "Reactor Coolant Leak,"

Revision 16, the reactor was tripped; therefore, the reactor trip was unplanned.

The inspectors determined that the text regarding unplanned reactor trips had been inserted into Procedure SO123-0-25, Trip/Transient Review, Revision 5 (the predecessor of Procedure SO123-0-A8), in January 2003, and had been included in Procedure SO123-0-A8, Revision 0, when that procedure had superseded Procedure SO123-0-25 in October 2003. The inspectors determined that the licensee had not revised the text that referred to unplanned reactor trips in any of the subsequent revisions of Procedure SO123-0-A8.

Analysis:

The licensees failure to define unplanned reactor trip in Procedure SO123-0-A8, "Trip/Transient and Event Review," Revision 5, was a performance deficiency with respect to the requirement in 10 CFR 50, Appendix B, Criterion V, for procedures to be appropriate to the circumstances. This finding is more than minor because if left uncorrected the performance deficiency could be viewed as a precursor to a significant event. Using Inspection Manual Chapter 0609.04, Phase 1 - Initial Screening and Characterization of Findings, and Inspection Manual Chapter 0609, Appendix A, The Significance Determination Process (SDP)for Findings At-Power, the inspectors determined the finding to be of very low safety significance because a reactor trip was initiated with no loss of mitigating equipment or other associated initiators. Because the text, unplanned reactor trip, had been inserted into the procedure in January 2003, and had not been revised since then, the inspectors determined that this finding does not have a cross-cutting aspect because the associated procedure change was not representative of current performance.

Enforcement:

Title 10 CFR 50, Appendix B, Criterion V, requires, in part, that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances. Contrary to the above, from January 2003 through January 31, 2012, the licensee failed to provide an appropriate (adequate) procedure for an activity affecting quality. Specifically, for Procedure SO123-0-A8, the licensee failed to define what constituted an unplanned reactor trip, and Step 6.1.6 requires the entire procedure to be completed for unplanned manual reactor trips and the operators did not complete the entire post trip review as required. Because this finding is of very low safety significance and was entered into the licensees corrective action program as Nuclear Notifications NN 201915602 and NN 202161665, this violation is being treated as a non-cited violation (NCV), consistent with Section 2.3.2 of the Enforcement Policy:

NCV 05000362/2012010-01, Inadequate Trip/Transient and Event Review Procedure.

.2 (Closed) Unresolved Item 05000362/2012007-02, Evaluation of Unit 3 Vibration and

Loose Parts Monitoring System Alarms

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

During the review of operational differences between Unit 2 and 3 steam generators the team identified a significant difference in number of valid vibration and loose parts monitoring system alarms. The vibration and loose parts monitoring system was designed to provide continuous monitoring and conditioning of loose parts accelerometer signals. Two separate accelerometers were installed on each of the steam generators. The location of these instruments are on the steam generators lower supporting structures and provide acoustic information about loose parts impacts specifically on the reactor coolant or primary side of the steam generators. The vibration and loose parts monitoring system real time functions consist mainly of impact alarm validation of suspected loose part events and recording acoustic data. Long term vibration monitoring and loose part event trending were done by engineering personnel using recorded data.

. . . The team noted that Unit 2 steam generators did not receive the same number and type of alarms during a similar period of steady state operations. Engineering personnel also compared hot leg temperature changes linked to Unit 3 operations from February 18, 2011, to January 31, 2012, and confirmed about 30 valid alarms during this period were not associated with thermal transients.

The inspectors reviewed the regulatory guidelines which established the primary purpose of the vibration and loose part monitoring system to be designed for early detection of loose metallic parts in the primary system. Early detection can provide the time required to avoid or mitigate damage to, or malfunction of, primary system components. The inspectors assessed whether the licensee appropriately responded to the alarms associated with the Unit 3 vibration and loose part monitoring system in accordance with alarm response procedures and vendor recommendations. The inspectors reviewed documents and interviewed engineering personnel regarding their efforts to validate and determine the cause of numerous loose part alarms.

The inspectors also reviewed the Unit 3 Root Cause Evaluation NN 201836127, operational history, nuclear notifications, work orders, operator logs, and station procedures used to respond and evaluate the loose part alarms. In addition, the inspectors reviewed the licensees application of operating experience that was related to the vibration and loose parts monitoring system, including consultation with the vendor.

b. Observations and Findings

The inspectors determined that the licensee properly responded to and evaluated the alarms and followed the applicable station alarm procedures and vendor recommendations. Subsequently, the licensee requested from the vendor an in-depth evaluation of the available acoustical data, which was documented in Nuclear Notification NN 201818719. This evaluation established the likely source of the alarms. The results were inconclusive because of limitations with the monitoring system. Specifically, because of sensor locations (lower portion of the steam generator below the tube sheet in the support structure) and sensitivity, it was not possible to determine the exact source of the Unit 3 alarms. Westinghouse engineering personnel performed an evaluation (Evaluation 201818719-SPT-2) of acoustical data and determined from the shape and intensity of the particular responses that the acoustic source was not likely from the upper bundle of the replacement steam generator or related to the tube-to-tube wear.

As a result of the licensee evaluation, additional actions have been taken to improve the planned replacement of the obsolete vibration and loose part monitoring system with a newer design to increase reliability and sensitivity. The licensee is considering additional sensor locations which are not required, but may help with monitoring the upper bundle region of the steam generator during power operation. The results of this additional monitoring and increased sensor sensitivity may provide the licensee with a potential means to monitor for tube-to-tube degradation.

The inspectors concluded that no performance deficiency existed since the licensee operated the vibration and loose part monitoring system in accordance with procedures and vendor recommendations.

No findings were identified.

.3 (Closed) Unresolved Item 05000362/2012007-03, "Evaluation of Retainer Bars Vibration

during the Original Design of the Replacement Steam Generators

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

In February 2012, the licensee identified wear indications in Unit 2 replacement steam generators at the tube locations in contact with the retainer bars (see figure below). Some of the indications showed excessive wear with a maximum degradation of 90 percent through wall. The team identified that the design of the replacement steam generators did not expect any potential vibration concerns in the area of the tube bundle where the retainer bars were located. . .

However, upon identification of retainer bar-to-tube wear in Unit 2 replacement steam generators, Mitsubishi performed an evaluation to identify the cause of excessive wear. The analysis considered three vibration mechanisms: fluid-elastic instability, vortex-induced vibration, and turbulence-induced vibration (random vibration). The analysis for turbulence-induced vibration determined that random vibration was the possible cause of the retainer vibration, based on the peculiar flow around the retainer bar, combined with the rather low natural frequency of the retainer bar. The analysis used the two phase flow conditions around the retainer bars and identified various modes of vibration at those flow conditions that could lead to retainer bar vibration and consequently to tube wear.

The inspectors reviewed corrective action program documents and supporting engineering evaluations associated with this unresolved item to determine if a performance deficiency existed or if the issue constituted a violation of NRC requirements. The inspectors reviewed Nuclear Notification NN 201843216 and the associated cause evaluation performed to address the mechanistic cause of the inadequate design of the retainer bars in Unit 2 and Unit 3 replacement steam generators. The inspectors also reviewed the status of Edisons and Mitsubishis cause evaluation to identify the organizational and programmatic factors leading to the inadequate design of the retainer bars.

The inspectors reviewed the certified design specification for the replacement steam generators to identify the applicable design standards for the retainer bars. The review of design information included design basis documents for the original steam generators to identify any design requirements for flow-induced vibration and to determine if those requirements were properly translated into the certified design specification. The inspectors reviewed the applicable design standards to identify information that would have prompted the licensee to identify the inadequate design of the retainer bars. The inspectors reviewed Mitsubishis design calculations against the applicable design standards to determine if a performance deficiency occurred in the implementation of design requirements.

Additionally, the inspectors interviewed licensee staff and reviewed documentation about the licensee interactions with Mitsubishi during the design of the replacement steam generators in relation to the retainer bars, in order to determine if Edison had a reasonable opportunity to identify the design issue. The inspectors also reviewed applicable quality assurance requirements and site procedures for the verification of supplier documents to assess whether the licensee had a reasonable opportunity to identify the retainer bar design issue based on the requirements and guidance contained in site procedures.

b. Observations and Findings

The inspectors reviewed a licensee-identified non-cited violation of 10 CFR Part 50, Appendix B, Criterion III, Design Control, for the licensees failure to verify the adequacy of the replacement steam generator design for Units 2 and 3 with respect to the susceptibility of the retainer bars in the anti-vibration bar assembly to flow-induced vibration. This licensee-identified violation is described in Section 4OA7 of this report.

As shown in the figure below, the retainer bars are internal parts of the steam generator anti-vibration bar assembly that are attached to a series of retaining bars to capture a small group of tubes with the purpose of restraining movement of the anti-vibration bar assembly during manufacturing, upending during installation; and a main steam line break design basis event. The anti-vibration bar assembly was designed to float in between the tubes and be held in place by friction. There are 24 total retainer bars of two different diameter sizes in each steam generator. There are two groups of twelve bars where each group has six large and six small diameter bars. The different sized bars were needed to accommodate the different size of tube gaps in different areas of the bundle. Only tubes adjacent to the smaller diameter bars experienced wear indications. The retainer bars capture two short rows of tubes, which physically stop anti-vibration bar movement into or out of the tube bundle. Only those tubes immediately adjacent to both sides of the retainer bars (23 inside and 24 outside) were affected or potentially affected by vibration wear between the retainer bar and tubes. The first captured row was not affected by the retainer bar vibration.

The first indication of tube wear due to retainer bar vibration was identified on February 5, 2012, as part of the planned in-service eddy current examinations of the Unit 2 steam generator tubes. The Unit 2 in-service eddy current examinations were in progress at the time that the Unit 3 primary-to-secondary leak was identified on January 31, 2012. The licensee performed eddy current examinations in the Unit 2 and Unit 3 replacement steam generators to identify the extent of the condition. The eddy current examinations identified six tubes with wear indications in Unit 2 as a result of retainer bar vibration. The total number of tube-to-retainer-bar wear indications in Unit 2 was seven since one tube had more than one indication. Out of the six tubes with indications, four tubes had indications greater than 35 percent through wall degradation. In Unit 3, the licensee identified four tubes with one tube-to-retainer-bar wear indication each. Three of the four tubes had indications greater than 35 percent through wall degradation. For Unit 2, the maximum through wall degradation was determined to be 90 percent in tube 2E-089 R119 C133. This tube was in-situ pressure tested and it maintained pressure with zero leakage during the testing and therefore met the technical specification requirements for tube integrity.

The Cause Evaluation Report NN 201843216 identified that the mechanistic root cause of the tube-to-retainer-bar wear was due to inadequate design of the smaller diameter retainer bars in that the bar size (diameter and length) was insufficient to prevent excessive flow-induced vibration. The retainer bars were designed and fabricated with longer and smaller diameter than previous designs, which resulted in a lower natural frequency when compared with previous configurations, increasing the susceptibility to flow-induced vibration. This resulted in tube wear from vibration contact between the retainer bars and tubes inside the steam generators. The vibration source was characterized as turbulent two-phase flow across the retainer bars.

Mitsubishis technical evaluation Document SO23-617-1-M1562, Revision 4, stated that turbulence-induced vibration, buffeting, or random vibration were not considered in the design phase because generally the fluid force of this mechanism was small enough in the normal two-phase flow in the steam generator secondary side, and the natural frequency of the structure was high enough that structures were not expected to vibrate excessively. However, further evaluation of this mechanism found that this was considered to be the most probable cause of the vibration in the smaller diameter retainer bars, based on flow forces near the retainer bars, combined with the rather low natural frequency of the retainer bar. The evaluation concluded that the cross-flow velocities in the replacement steam generators could result in large vibration amplitude of the smaller diameter retainer bars.

Edisons Cause Evaluation Report NN 201843216 identified that in 2006, Edison personnel questioned Mitsubishi about the potential for tube wear as a result of tube contact with the retainer bar during operation. Mitsubishi indicated that the tubing and retainer bars were not designed to be in direct contact with each other and operational experience showed no tube wear issues from retainer bars in previous steam generator designs. However, as a conservative measure in the design (in case the retainer bars and tubes actually contacted each other) Mitsubishi provided additional chromium plating of the retainer bars to reduce the wear coefficient and minimize any potential tube wear.

The cause evaluation report also noted that Edisons concerns for tube-to-retainer-bar wear were based on the fact that tubes could vibrate with sufficient amplitude to result in contact with the retainer bar. Edison staff did not recognize the potential of vibration in the retainer bar itself since Mitsubishi had stated that the retainer bars would not contact the tubes. As such, Edison accepted the additional chromium plating as a reasonable response based on the predefined spacing gap and the fact that the tubes themselves would not significantly move.

The certified design specification for SONGS replacement steam generators, Document SO23-617-01, Revision 4, Section 3.9.3.7, stated, The tube support design shall: preclude tube damage due to wear caused by flow-induced vibration.

The Replacement Steam Generator shall address flow-induced and turbulence-induced vibration of the tube supports to demonstrate that fatigue failures and excessive fretting and wear of the tubes will not occur. Specifically, the Supplier shall demonstrate that its design will minimize vibration-induced tube wear or fatigue in the tube bend area of the tube bundle.

Procedure SO123-XXIV-37.8.26, Processing of Supplier Documents, Revision 8, provided instructions for processing of engineering documents submitted by suppliers to fulfill specification or purchase order requirements. Section 6.4 required that the responsible engineer shall determine the document review and approval requirements in accordance with Procedure SO123-XXIV-1.1, Document Review and Approval Control. Procedure SO123-XXIV-1.1, Revision 13, stated that technical reviewers should use the portions of Attachment 4 as an aid in performing design reviews. Procedure SO123-XXIV-1.1, Attachment 4, contained specific guidance for design reviews that included verification of correct inputs, appropriate design methods, and reasonable outputs compared to design inputs.

As discussed in the root cause evaluation report, Mitsubishi did not typically perform an analysis of the retainer bar that addresses flow-induced vibration.

Document SO23-617-1-C749, Analytical Report of Anti-vibration Bar Assembly, Revision 4, performed by Mitsubishi for the original design of the replacement steam generators, focused on the structural analysis of the retainer bars and did not address the susceptibility of the anti-vibration bar assembly to flow-induced vibration.

Additionally, Document SO23-617-1-C157, Evaluation of Tube Vibration, Revision 3, did not specifically address the susceptibility of the retainer bars to flow-induced vibration. Both reports were reviewed and approved by SONGS in the design stage of the replacement steam generators.

The licensee did not meet Procedure SO123-XXIV-37.8.26 requirements to ensure the design of the retainer bar was adequate with respect to the certified design specification. Specifically, the licensee failed to ensure that there was sufficient analytical effort in the design methodology of the anti-vibration bar assembly to support the conclusion that tube wear would not occur as a result of contact with the retainer bars due to flow-induced vibration. The inspectors determined that the requirements for flow-induced vibration in the certified design specification, along with the expectations in Procedure SO123-XXIV-37.8.26, provided sufficient information to reasonably foresee the inadequate design of the retainer bars during the review and approval of design Calculations SO23-617-1-C749 and SO23-617-1-C157, including the associated design drawings provided by Mitsubishi. The associated violation for this performance deficiency is described in Section 4OA7 of this report.

.4 (Closed) Unresolved Item 05000362/2012007-05, Shipping Requirements not in

Accordance with Industry Standards

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

Specification SO23-617-01, Section 3.16.3, specifies the supplier shall be responsible for monitoring and maintaining nitrogen atmosphere inside the steam generators during their shipping from Mitsubishi to the California port discharge point. The team noted that Unit 3 steam generators did not require, monitoring or control of dew point, oxygen concentration, inside nitrogen pressure. The team could not identify if this was properly evaluated (Reference Section 5 of shipping and handling procedure SO23-617-1-M1350).

As part of the evaluation of this unresolved item, the inspectors reviewed the shipping requirements documented in the licensee certified design specification; various standards; replacement steam generator shipping procedures, including supporting engineering assessments documented in the corrective action program; the Unit 3 root cause evaluation; shipping timelines; and data reports for Unit 3 replacement steam generators.

b. Observations and Findings

Introduction:

The inspectors identified a Green non-cited violation of 10 CFR 50, Appendix B, Criterion XIII, "Handling, Storage, and Shipping," for the licensees failure to take appropriate measures to control preservation of safety-related equipment during shipping, specifically, the protective environment provided for the Unit 3 steam replacement generators was not appropriately specified or monitored.

Description:

The inspectors determined that shipping of the Unit 3 replacement steam generators was not done in accordance with the industry standards and the original shipping requirements. In particular, during shipping of the Unit 3 steam generators on an open ocean transport, the process of monitoring and maintaining a protective inert gas environment was not used. The licensee did not provide specific or appropriate monitoring of the protective environment for the Unit 3 replacement steam generators. The Unit 3 steam generators were shipped using a simplified nitrogen fill sequence without the required dew point, positive pressure, or monitoring during shipment. This shipping change was a result of schedule slippage associated with the Unit 3 divider plate repairs. The change in the shipping requirements minimized additional schedule delays.

The original shipping requirements included the following: "Each RSG shall have a nitrogen supply available to the vessel on the primary and secondary side. The supply would be used to replenish nitrogen in the vessels should its pressure drop below the recommended minimum. Calibrated redundant compound pressure gauges shall be provided for the primary and secondary side to indicate the nitrogen pressure; valve connections shall be provided for adding nitrogen as necessary. "

On October 23, 2009, a teleconference meeting between Mitsubishi Heavy Industries and Southern California Edison was held to discuss the Unit 3 divider plate repair plan options and accelerated repair plans. Southern California Edison proposed deleting the conformed design specification requirements for measuring dew point and monitoring nitrogen pressure during shipment but requiring a special protective environment involving removal of the air and filling the replacement steam generators with nitrogen. Different actions were assigned to Mitsubishi Heavy Industries which included submitting a supplier deviation request to remove dew point measurement, positive pressure and monitoring of special protective environment required in the original design specification. In addition, Mitsubishi Heavy Industries was assigned action to simplify the nitrogen fill sequence without control of dew point, positive pressure, or monitoring of the special protective environment.

On December 1, 2009, Mitsubishi Heavy Industries submitted Supplier Deviation Request SDR 10041870-09091, which recommended deviating from the Unit 3 replacement steam generators design specifications that stated the replacement steam generators shall be pressurized on both the primary and secondary side with dry nitrogen. The deviation request stated MHI [Mitsubishi Heavy Industries] wants to propose not to control the positive pressure, the dew point of nitrogen, and the oxygen content on the primary and secondary side of Unit 3 replacement steam generators to accelerate delivery schedule." This request was approved by Southern California Edison engineering personnel on December 22, 2009.

Procedure, SO23-617-1-M1350, Shipping and Handling, Revision 6, and Design Specification, SO23-617-01, Design Specification for Design and Fabrication of RSGs for Unit 2 and 3, Revision 4, required implementation of ANSI N45.2-1977, Quality Assurance Program Requirements for Nuclear, and Regulatory Guide 1.38, "Quality Assurance Requirements for Packaging, Shipping, Receiving, Storage, and Handling of items for Water-Cooled Nuclear Power Plants." On July 28, 2010, a revision was made to Procedure SO23-617-01, Section 3.16, which stated, "For Unit 3 RSGs [replacement steam generators], nitrogen purge shall be applied on the primary and secondary side. But it is acceptable not to control the dew point of nitrogen, pressure, and oxygen content in the primary and secondary side of the RSGs, and not to monitor these parameters prior to and during shipment."

The licensee reviewed shipping records and validated that the Unit 3 replacement steam generators were purged and filled with nitrogen gas prior to shipping; although, no measurements of oxygen or moisture content were conducted. In addition, upon receipt, the licensee did not perform any verifications as to the atmospheric content of the steam generator internals. The licensee conducted an analysis of the shipping environment and determined no detrimental impact occurred. This concern is documented in Nuclear Notifications NN 202160749, NN 201960027, and NN 20191118.

Analysis:

The failure of engineering personnel to implement appropriate measures to ensure a protective environment was monitored and maintained was a performance deficiency. The finding is more than minor because it is associated with initiating events cornerstone attribute of equipment performance and affected the associated cornerstone objective to limit the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Using Inspection Manual Chapter 0609.04, Phase 1 - Initial Screening and Characterization of Findings, and Inspection Manual Chapter 0609, Appendix A, The Significance Determination Process (SDP) for Findings At-Power, the inspectors determined the finding to be of very low safety significance because it did not result in exceeding the reactor coolant system leak rate for a small loss of coolant accident or result in a total loss of systems used to mitigate a loss of coolant accident. This finding does not have a cross-cutting aspect because the associated performance deficiency occurred in August 2010 and was not representative of current performance.

Enforcement:

Title 10 CFR 50 Appendix B, Criterion XIII, "Handling, Storage, and Shipping," requires, in part, that, measures shall be established to control the handling, storage, shipping, cleaning and preservation of material and equipment in accordance with work and inspection instructions to prevent damage or deterioration.

In addition, when necessary for particular products, special protective environments, such as inert gas atmosphere, specific moisture content levels, and temperature levels, shall be specified and provided. Contrary to the above, between August 2 and August 18, 2010, Southern California Edison failed to provide adequate measures to ensure that a special protective environment was specified and provided for the Unit 3 replacement steam generators. Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notifications NN 202160749, NN 201960027, and NN 20191118 this violation is being treated as a non-cited violation (NCV), consistent with Section 2.3.2 of the NRC Enforcement Policy: NCV 05000362/2012010-02, "Failure to Comply with Requirements for Handling, Storage, and Shipping."

.5 (Closed) Unresolved Item 05000362/2012007-06, Shipping Requirements not in

Accordance with Design and Fabrication Specifications

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

Based on the information gathered by the team on shipping and handling specifications associated with the Unit 2 and 3 replacement steam generators, the team could not determine that Mitsubishi or SCE adequately considered the potential impact of not providing methods of tube bundle supports as required in Specification SO23-617-01.

The inspectors assessed whether the licensee appropriately shipped the replacement steam generator in accordance to the design and fabrication specifications. The inspectors reviewed the Unit 3 root cause evaluation and Nuclear Notification NN 201921176 which assessed the methods of tube bundle support during the shipping and handling of Unit 3 replacement steam generators.

The inspectors also reviewed design drawings, sagging measurements, and procedures for the shipping of replacement steam generators. The licensee methodology was assessed by the inspectors to determine if the appropriate evaluations were completed and if a method of tube support was deemed not necessary to preclude tube bundle damage.

b. Observations and Findings

The design specifications required the supplier to specify all shipping and handling requirements to preclude component damage, including tube bundle support methods. However the shipping and handling procedure did not specify methods for tube bundle support. The licensee, prior to shipment of the replacement steam generators, deemed a temporary tube bundle support structure was unnecessary based on field test results which demonstrated that tube bundle sagging would not result in tube deformation. The licensee completed additional evaluations as a result of the Augmented Inspection on the shipping effects of the replacement steam generators which are documented in Unit 3 Root Cause Evaluation NN 201836127-0029 and Nuclear Notification NN 201921176. Mitsubishi engineering personnel did additional analysis and determined that the tubes would not plastically deform. Eddy current data was also reviewed and if deformation due to tube sagging occurred, it would be identified on the tubes at tube sheet 7. No pinched tube indications were identified.

As a result of the inspectors review of nuclear notifications, drawings, stress evaluations, and eddy current data, it was determined that the licensee assessment for lack of tube bundle support was appropriate. Consequently, the inspectors concluded that no performance deficiency existed as the licensee adequately demonstrated that shipping requirements for the tube bundle support had been properly evaluated.

No findings were identified.

.6 (Closed) Unresolved Item 05000362/2012007-07, Unit 3 Steam Generator 3E0-88

Stresses Related to Handling

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

Unit 3 replacement steam generator 3E0-88 accelerometers indicated up to a 1.23 g spike with a simultaneous recording on all three of the attached accelerometers.

Mitsubishi provided an evaluation of the forces which showed loads were within allowable stress limits but exceeded stress for an operating basis earthquake. The team was not able to determine if this was properly considered.

The inspectors assessed whether the licensee had properly evaluated the Unit 3 replacement steam generators stresses related to horizontal shipping or the handling activities which resulted in the observed accelerometer recordings. The inspectors reviewed the Unit 3 root cause evaluations, relevant accelerometer recordings, and the subsequent stress evaluations. In addition, the inspectors reviewed the licensee conclusions documented in Nuclear Notification NN 201921165 and 201952341.

b. Observations and Findings

The licensee evaluations of the effects of shipping on the replacement steam generators were documented in Unit 3 Root Cause Evaluation Nuclear Notification NN 201836127-0029 and Nuclear Notifications NN 201921165 and 201952341. As part of the evaluation the licensee assessed the potential of coincidental accelerometer recordings during shipment of the Unit 3 steam generator as well as shipping stresses related to shipping and handling compared to the shipping stresses expected by seismic forces.

The licensee reviewed the transportation logs and confirmed that at the time of the largest accelerometer recordings, the replacement steam generators were stationary. The licensee reviewed the largest accelerometer recordings during the shipping of the Unit 3 replacement steam generators and determined that the accelerometers readings had mostly occurred as individual events or recordings caused by the lashing of the replacement steam generators or by other rigging activities in the proximity of the accelerometers. The licensee evaluation of maximum accelerations and seismic loads indicated that tube loading stress remained below the allowable limits for plastic deformation of the steam generator tubes and internals.

As a result of the inspectors review of nuclear notifications, drawings, and stress evaluations, it was determined that the licensee assessment of the effects of shipping on the replacement steam generator was appropriate. The inspectors concluded that no performance deficiency existed as a result of the accelerometer recordings and that the licensee evaluated the data in accordance with the procedures and vendor recommendations.

No findings were identified.

.7 (Discussed) Unresolved Item 05000362/2012007-08, Non-Conservative Thermal-

Hydraulic Model Results

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

The team identified an unresolved item associated with the adequacy of Mitsubishis FIT-III thermal-hydraulic code. The FIT-III code predicted nonconservative low velocity and low void fraction results which were used as inputs to the vibration code FIVATS. These non-conservative thermal-hydraulic results lead Mitsubishi to conclude that margins to instability were significantly larger than they actually were.

The inspectors reviewed corrective action program documents and supporting engineering evaluations associated with this unresolved item to determine if a performance deficiency existed or if the issue constituted a violation of NRC requirements. The inspectors reviewed Nuclear Notification NN 201836127 and Southern California Edisons cause evaluation performed to address the mechanistic cause of the non-conservative results of the FIT-III thermal-hydraulic model and flow-induced vibration analysis developed by Mitsubishi for the design of Unit 2 and Unit 3 replacement steam generators. The inspectors also reviewed the status of Southern California Edisons and Mitsubishis cause evaluation to identify the organizational and programmatic factors leading to the non-conservative thermal-hydraulic model.

Additionally, the inspectors reviewed the certified design specification for the replacement steam generators to identify the applicable design standards for thermal-hydraulic modeling and flow-induced vibration. The review of design information included design basis documents for the original steam generators to identify any design requirements for thermal-hydraulic modeling and flow-induced vibration in order to determine if those requirements were properly translated into the certified design specification. The inspectors reviewed the applicable design standards to identify design information that would have prompted the licensee to identify deficiencies in the thermal-hydraulic model and flow-induced vibration analysis. Particularly, the inspectors reviewed the technical justification for critical assumptions and design inputs.

The inspectors interviewed licensee staff and reviewed applicable quality assurance requirements and site procedures for the verification of supplier documents to assess whether the licensee had a reasonable opportunity to identify any deficiencies with the thermal-hydraulic modeling and the flow induced-vibration analysis based on the requirements and guidance in site procedures.

b. Observations and Findings

The inspectors identified that the cause evaluation for the non-conservative results of the FIT-III thermal-hydraulic model was still in-progress at the time of the inspection and no final conclusions were reached for the cause of the non-conservative flow velocities, which were used as inputs in the tube vibration analysis and resulted in non-conservative stability ratios. Since the licensee had not completed the cause evaluation for this unresolved item, the inspectors were not able to make a final determination of whether a performance deficiency or violation of NRC requirements occurred.

The inspectors were informed that Mitsubishi was performing an evaluation of the potential factors that contributed to the low flow velocities in FIT-III relative to the velocities calculated by the ATHOS model developed after the tube leak event in Unit 3. This evaluation was included in Document SO23-617-1-M1530, Revision 1, which also intended to demonstrate the validity of FIT-III results for the original tube vibration analysis. This evaluation was still being finalized and not yet approved by Edison.

The licensee and Mitsubishi continued to evaluate this unresolved item and no final conclusions were reached at the time of the inspection. The NRC is continuing to perform independent reviews of existing information, and will conduct additional reviews as new information becomes available.

.8 (Closed) Unresolved Item 05000362/2012007-09, Evaluation of the Effects of Divider

Plate Weld Repairs in Unit 3 Replacement Steam Generators

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

The cracking of the divider plate weld in both Unit 3 replacement steam generators required extensive repairs affecting the channel head, divider plate, and tubesheet. Based on interviews with licensee personnel and the review of documentation for the repairs, the team determined that Mitsubishi did not perform a comprehensive evaluation to assess the impact of the divider plate repairs on the integrity of the tube bundle.

The inspectors reviewed the results of the licensees examination of the possible effects of additional rotations of the steam generators on the tube bundle. The inspectors also reviewed the licensees plan for repair of the divider plate to channel head weld joint and cladding in the Unit 3 Replacement Steam Generators, and their plan for post-weld heat treatment requirements for that repair. In addition, the inspectors reviewed the licensees heat transfer analysis to confirm that the heating plan for post-weld heat treatment of the channel head to tubesheet weld would achieve the required temperature range, the licensees post-weld heat treatment report, and the report that evaluated the post-weld heat treatment results.

Furthermore, the inspectors reviewed Nuclear Notification NN 202102763 Request 72, which described the post-repair tube sheet flatness checks performed by the licensee.

b. Observations and Findings

In Root Cause Report NN 201836127, dated May 7, 2012, the licensee documented an analysis conducted to determine if the divider plate weld failure and repair caused directly or contributed to the free span wear (tube-to-tube) on Unit 3 Replacement Steam Generators (RSGs). In that analysis, the licensee documented, in part, that:

1. Mitsubishi Heavy Industries determined that the calculated displacement attributed to the divider plate weld failure had not been sufficient to cause plastic tube deformation and thus was not related to the free span wear.

2. Mitsubishi Heavy Industries had concluded that the change in tube-to-AVB gap due to hydrostatic testing was of no consequence to tube wear, given the elastic nature of the displacement.

3. Because replacement steam generator 3A (3E088) had undergone twice the number of hydrostatic tests as replacement steam generator 3B (3E089), yet on replacement steam generator 3A less-severe cracking of the divider plate weld toe was visible, the licensee concluded that given the elastic nature of the displacement, the number of hydrostatic tests performed would have no bearing on the consequence of the weld failure.

4. Mitsubishi Heavy Industries had determined that the tubes calculated to lose contact with the AVBs due to rotation were primarily in the peripheral tubes, with little change calculated in the region that the significant free span wear was observed.

5. Temperature profiles for the tubesheet and tubes during post-weld heat treatment were determined analytically, monitored, and evaluated. Mitsubishi Heavy Industries concluded that the temperatures were not sufficient to produce plastic deformation.

6. Channel head to tubesheet welding and post-weld heat treatment processes were the same for all 4 replacement steam generators, while only the Unit 3 replacement steam generators exhibited the significant free span wear.

Additionally, these processes would affect only the peripheral tubes, which were not observed to have free span wear.

From this analysis, the licensee concluded that the likelihood of the divider plate weld failure and associated repairs being the cause of the free span wear is judged to be of a very low level.

The inspectors determined that the analysis described in Nuclear Notification NN 201836127 was a thorough evaluation of the likelihood that the divider plate weld failure and repair had caused or contributed to the free span wear on the Unit 3 replacement steam generators.

No findings were identified.

.9 (Closed) Unresolved Item 05000362/2012007-10, Evaluation of Departure of Method of

Evaluation for 10 CFR 50.59 Processes

a. Inspection Scope

NRC Inspection Report 05000361/2012007 and 05000362/2012007described this unresolved item, in part, as follows:

The NRR technical specialist reviewed SCEs 10 CFR 50.59 evaluation contained in Engineering Change Packages 800071702 and 800071703 for the Unit 2 and Unit 3 replacement steam generators, respectively, in which SCE determined that the impact of the replacement steam generators on the current licensing basis and any need for NRC approval as required by 10 CFR 50.59. . .

The NRR technical specialist identified one unresolved item associated with a change in the method of evaluation as described in the updated final safety analysis report.

Additional review and followup will be required to review the departure of the method of evaluation used during the stress analysis calculations associated with the replacement steam generators.

The structural analysis of the original steam generators used ANSYS software for the thermal and stress analyses while the replacement steam generators were analyzed using ABAQUS software. ANSYS was described in the updated final safety analysis report as a large-scale, general-purpose, finite element program for linear and nonlinear structural and thermal analysis of the reactor coolant loop components.

To address the use of ABAQUS instead of ANSYS for reactor coolant system structural integrity analyses, the inspectors reviewed: the changes made to the Updated Final Safety Analysis Report to describe this change; the 10 CFR 50.59 evaluations prepared by the licensee to evaluate this change for Unit 2 and Unit 3; and, a report that compared the results of ABAQUS and those of ANSYS for a number of test cases.

For loss of coolant accident analysis, the original steam generator used STRUDL computer program to calculate displacement histories and then ANSYS computer program to calculate tube stresses. The tube stresses for the replacement steam generators were determined using ANSYS computer program based on the blowdown forces. For the original steam generators, stresses were determined using a combination of events. For the replacement steam generators, the loss of coolant accident, design basis earthquake, and the main steam line break events were combined as one limiting event, which Southern California Edison considered to be a more conservative method of evaluation relative to the original steam generators.

To address the use of ANSYS instead of STRUDL and ANSYS for tube-wall-thinning analyses, the inspectors reviewed: the 10 CFR 50.59 evaluations prepared by the licensee to evaluate this change for Unit 2 and Unit 3; the report of the licensees analysis in accordance with Regulatory Guide 1.121 to determine an appropriate tube repair limit for the tubes in the replacement steam generators; and, License Amendments 220 and 213 associated with the tube repair limit for Unit 2 and Unit 3, respectively.

b. Observations and Findings

ABAQUS Instead of ANSYS The inspectors determined that the change of methods would not have required a license amendment based on the NRC approval for the use of ABAQUS at other nuclear power plants in similar applications. However, the inspectors identified a minor violation of 10 CFR 50.59(d)(1), which requires that the licensee to maintain records of changes in the facility for changes that do not require license amendments.

The inspectors reviewed this issue with respect to 10 CFR 50.59 (c)(2)(viii), which requires a licensee to obtain a license amendment prior to making a change to their updated final safety analysis report if the change would result in a departure from a method of evaluation as described in the update final safety analysis report. However, if the new method had been approved by the NRC for use in essentially the same manner, then that method can be used by the licensee without obtaining a license amendment.

During the Augmented Inspection, the inspectors had identified an unresolved item associated with the licensee changing their structural integrity analysis from ANSYS to ABAQUS. Southern California Edisons 50.59 review stated that the change to ABAQUS was a change in an element of method. The 50.59 evaluation further identified that Southern California Edison compared the results of both programs on test cases and found the results to be essentially the same (less than 1 percent difference) and therefore, did not require NRC review and approval. However, the inspectors determined that the licensees decision to use ABAQUS instead of ANSYS for reactor coolant system structural integrity analyses constituted changing a method of evaluation described in the San Onofre Nuclear Generating Station Updated Final Safety Analysis report.

In response to the inspectors questions, the licensee identified several documents which described uses of ABAQUS for applications (including research done for the NRC)similar to RCS structural integrity analyses, including the following:

• ORNL/NRC/LTR-04/15, Probabilistic Structural Mechanics Analysis of the Degraded Davis-Besse RPV Head, September 2004 (ML042600455)

• D. Rudland, D.J. Shim, H. Xu, and G. Wilkowski, Summary Report on Evaluation of Circumferential Indications in Pressurizer Nozzle Dissimilar Metal Welds at the Wolf Creek Power Plant to Nuclear Regulatory Commission Washington DC, April 2007 (ML071560398)

• NUREG/CR-6854, Fracture Analysis of Vessels - Oak Ridge FAVOR, v04.1, Computer Code: Theory and Implementation of Algorithms, Methods, and Correlations, September 2004 (ML061580369)

• NUREG/CR-6765, Development of Technical Basis for Leak-Before-Break Evaluation Procedures, May 2002 (ML021720594)

• NUREG/CR-6774, Validation of Failure and Leak-Rate Correlations for Stress Corrosion Cracks in Steam Generator Tubes, May 2002 (ML021510286)

(In the list above, the text in parentheses identifies the corresponding ADAMS access numbers.)

The inspectors reviewed the Comanche Peak Updated Safety Analysis Report, Section 3.6B.2.2.2 (High-Energy Piping Other Than RCS Main Loop) that described using ABAQUS for piping dynamic responses resulting from a postulated pipe rupture.

Based on reviewing the documents described above, the inspectors determined that the NRC had approved using ABAQUS for reactor coolant system structural integrity analyses. Therefore, the inspectors determined that the change from ANSYS to ABAQUS did not require the licensee to obtain a license amendment prior to implementing the change.

However, the inspectors noted that the 10 CFR 50.59 written evaluation for this change, did not state that ABAQUS had been approved by NRC for the intended application.

Therefore, the inspectors determined that the evaluation was not adequate, in that it did not provide a correct basis for the licensees determination that the change from ANSYS to ABAQUS did not require a license amendment prior to implementing the change.

Title 10 CFR 50.59(d)(1) requires that the licensee maintain records of changes in the facility that include a written evaluation which provides the bases for the determination that the change, test, or experiment does not require a license amendment . The licensees failure to provide an appropriate basis for the licensees determination that the change from ANSYS to ABAQUS did not require a license amendment prior to implementing the change therefore constituted a violation of 10 CFR 50.59(d)(1).

Because this violation impacted the regulatory process, the inspectors assessed it in accordance with the NRC Enforcement Policy, as directed by MC 0612, Appendix B, Issue Screening. NRC Enforcement Manual contains specific processes and guidance for implementing this Policy. NRC Enforcement Manual, Section 2.10.D.6 states, in part, that minor violations include the failure to meet 10 CFR 50.59 requirements that involve a change to the final safety analysis description where there was no reasonable likelihood that the change would ever require NRC approval per 10 CFR 50.59. As described above, the change from ANSYS to ABAQUS did not require a license amendment prior to implementing the change, so with respect to section 2.10.D.6 of the NRC Enforcement Manual, there is no reasonable likelihood that the change from ANSYS to ABAQUS would ever require NRC approval. Therefore, in accordance with the NRC Enforcement Manual, the inspectors determined that the licensees change from ANSYS to ABAQUS was a minor violation of 10 CFR 50.59(d)(1).

ANSYS Instead of STRUDL and ANSYS The inspectors review of this issue did not identify a violation of 10 CFR 50.59. The inspectors noted that the licensee had used ANSYS to calculate tube stresses for the both the original steam generators and replacement steam generators, so their use of ANSYS for this purpose did not constitute a change from the method described in the Updated Final Safety Analysis Report. The inspectors noted that for the original steam generators, the licensee had analyzed a combination of events using STRUDL to calculate displacement histories for those events. This provided additional margin for analysis done by ANSYS. However, for the replacement steam generators, the licensee had analyzed for the most limiting event, and had sufficient margin, so STRUDL was not needed. Based on this, the inspectors determined that the licensee had changed from using ANSYS and STRUDL to analyze several events for the original steam generators, to using only ANSYS to analyze a single limiting event for the replacement steam generators. Therefore, because the licensee did not change the method described in the Updated Final Safety Analysis Report, the inspectors concluded that the licensee did not need to obtain a license amendment prior to implementing that change.

No findings were identified.

4OA6 Meetings

Exit Meeting Summary

On September 28, 2012, the inspectors presented the inspection results to Mr. D. Bauder, Vice President and Station Manager, and other members of the licensee staff. The licensee acknowledged the issues presented. Proprietary information was provided to the team and all proprietary information was either returned to Southern California Edison or destroyed.

4OA7 Licensee Identified Violations

The following violation of very low safety significance (Green) was identified by the licensee which met the criteria of the NRC Enforcement Policy for being dispositioned as a Non-Cited Violation.

• Title 10 CFR Part 50, Appendix B, Criterion III, Design Control, requires, in part, that design control measures shall be established to provide for verifying or checking the adequacy of design, such as by the performance of design reviews, by the use of alternate or simplified calculational methods, or by the performance of a suitable testing program.

Contrary to the above, on December 11, 2007, and July 3, 2008, the licensee failed to establish design control measures during the review of Mitsubishis design Calculations SO23-617-1-C749 and SO23-617-1-C157, respectively, to verify or check the adequacy of the retainer bars design with respect to the susceptibility of the smaller diameter retainer bars to flow-induced vibration. Specifically, the licensee failed to ensure that there was sufficient analytical effort in the design methodology of the anti-vibration bar assembly to support the conclusion that tube wear would not occur as a result of contact with the retainer bars due to flow-induced vibration. Consequently, the smaller diameter retainer bars vibrated during normal operation causing wear on the adjacent tubes, that challenged the integrity of the reactor coolant system boundary. The inspectors determined that the licensees failure to verify the adequacy of the retainer bar design as required by Procedure SO123-XXIV-37.8.26 was of very low safety significance (Green)based on Inspection Manual Chapter 0609.04, Phase 1 - Initial Screening and Characterization of Findings, and Inspection Manual Chapter 0609, Appendix A, The Significance Determination Process (SDP) for Findings At-Power, because the finding did not involve a degraded steam generator tube condition where one tube could not sustain 3 times the differential pressure across a tube during normal full power, steady state operation and none of the replacement steam generators violated the accident leakage performance criterion in plant Technical Specifications as a result of the retainer bar vibration. The licensee also implemented actions to inspect all affected tubes in Unit 2 and 3 and remove from service all those tubes surrounding the smaller retainer bars that could wear due to vibration of the retainer bar. Because this violation has been determined to be of very low safety significance (Green) and has been entered in the licensees corrective action program as Nuclear Notification NN 201843216, it will be dispositioned as a non-cited violation in accordance with Section 2.3.2 of the NRCs Enforcement Policy.

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

D. Bauder, Vice President and Station Manager

J. Brabec, Project Manager, Steam Generator Recovery Program

R. Treadway, Nuclear Regulatory Affairs

J. Davis, Operations Manager

M. Stevens, Regulatory Affairs

M. Liu, SONGS Plant Engineering

M. Pawlaczyk, Regulatory Affairs

L. Pentecost, Technical Specialist/Scientist

B. Olech, Edison Design Engineering

NRC Personnel

D. Beaulieu, Project Manager, NRR/DPR/PGCB

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000362/2012010-01 NCV Inadequate Trip/Transient and Event Review Procedure (Section 4OA5.1)

05000362/2012010-02, NCV Failure to Comply with Requirements for Handling, Storage, and Shipping (Section 4OA5.4)

Closed

05000362/2012-001-00 LER Unit 3 Manual Reactor Trip due to Steam Generator Tube Leak

05000362/2012007-01 URI Adequacy of the Trip/Transient and Event Review Procedure

05000362/2012007-02 URI Evaluation of Unit 3 Vibration and Loose Parts Monitoring System Alarms

05000362/2012007-03 URI Evaluation of Retainer Bars Vibration during the Original Design of the Replacement Steam Generators

05000362/2012007-05 URI Shipping Requirements not in Accordance with Industry Standards

05000362/2012007-06 URI Shipping Requirements not in Accordance with Design and Fabrication Specifications

05000362/2012007-07 URI Unit 3 Steam Generator 3E0-88 Stresses Related to Handling

05000362/2012007-09 URI Evaluation of the Effects of Divider Plate Weld Repairs in Unit Replacement Steam Generators Attachment

05000362/2012007-10 URI Evaluation of Departure of Method of Evaluation for 10 CFR 50.59 Processes

Discussed

05000362/2012007-08 URI Non-Conservative Thermal-Hydraulic Model Results