ML20198A821

Insp Rept 99900104/98-01 on 980921-25 & 1116-17. Noncompliance Noted.Major Areas Inspected:Conformance of Fabrication & Exam Activities of Nrc,Asme Code & Customer Requirements

ML20198A821
Person / Time
Issue date:	12/14/1998
From:	NRC (Affiliation Not Assigned)
To:
Shared Package
ML20198A795	List: ML20198A790 ML20198A805 ML20198A821
References
REF-QA-99900104 NUDOCS 9812170141
Download: ML20198A821 (21)

Text

- __. _ _ _ . . ._ - ._ . . _ . . . _

i U.S NUCLEAR REGULATORY COMMISSION OFFICE OF NUCLEAR REACTOR REGULATION l

. 1

_ Report No: -99900104/98-01 l

1 Organization: Westinghouse Electric Corporation Pensacola Plant 8301 Scenic Highway -

Pensacola, Florida 32515-7810

Contact:

James C. Woeber, Director, Steam Generator Product Line Nuclear Industry Activity: Manufacture of steam generators Dates: September 21-25 and November 16-17,1998

~

inspectors: lan Barnes, Technical Assistant, Division of Reactor Safety Region IV William M. McNeill, Reactor inspector Division of Reactor Safety Region IV Robert L. Pettis, Senior Raactor Engineer Quality Assurance, Vendor Inspection and Maintenance Branch Division of Reactor Controls and Human Factors Approved by: Robert A. Gramm, Chief Quality Assurance and Safety Assessment Section Quality Assurance, Vendor Inspection

! and Maintenance Branch Division of Reactor Controls and Human Factors i

i Enclosure 2

l l

I I 1 Inspection Summary Westinghouse Electric Corporation Pensacola Plant (WPP) currently holds ASME i Certificate of Authorization N-1669 for manufacture of Class 1,2, and 3 vessels and piping systems, Class CS core support structures, and as a material organization supplying ferrous and nonferrous bars, threaded fasteners, forgings, plates, clad plate, flanges, fittings welded without filler metal, fittings made from NPT stamped tubular products, seamless and welded with (NPT stamped) and without filler metal tubular products, welding material, structural shapes, sheet, and rounds and hollows. This inspection was performed at the WPP manufacturing facility and was focused on manufacture of South Texas Project, Unit 1 replacement steam generators.

During this inspection, the inspectors assessed conformance of fabrication and examination activities to NRC, ASME Code, and customer requirements. Specific subject areas reviewed during the inspection were procurement control, control of special processes, control of tubing operations, implementation of 10 CFR Part 21, and l corrective actions taken in response to a November 1997 stop work order from STP Nuclear Operating Company (STPNOC).

l The inspection bases were as follows:

Appendix B " Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," to Part 50 of Title 10 of the Code of Federal Reaulations (10 CFR Part 50),

10 CFR Part 21," Reporting of Defects and Noncompliance," and WPP's Nuclear Quality Assurance Program Manual, approved March 18,1998.

Overall, the results of the inspection indicated that WPP had established appropriate program criteria for control of fabrication and examination activities, with implementation noted generally to be good. In addition, the inspectors found that WPP was in compliance with the provisions of 10 CFR Part 21 and was effectively implementing the Assessment Recovery Program that was developed in response to the November 6, 1997, stop work order from STPNOC. However, the inspection identified that WPP did not conform to certain NRC and procedural requirements pertaining to procurement of submerged arc welding flux. This nonconformance is discussed in Section 3.4.1.

2 Status of Previous inspection Findings The scope of this inspection was limited to the manufacture of rep!acement steam generators by the Westinghouse Pensacola Plant (WPP) for South Texas Project, Unit 1.

Although the NRC staff previously accepted WPP corrective actions for findings that l resulted from inspection Report 99900104/92-01, dated May 21,1992, the inspectors did l not verify implementation of the corrective actions during this inspection.

2

3 Inspection Findings and Other Comments 3.1 Insoection for Comoliance with 10 CFR Part 21

a. Insoection Scoce 1 The inspectors reviewed WPP's implementation of and compliance with the requirements of Title 10 of the Code of Federal Reaulations (10 CFR) Part 21, as i applied to the manufacture of replacement steam generators for South Texas Project, Unit 1. The steam generators were being supplied in accordance with STPNOC Technical Specification 4R129NS1014, " Specification for Replacement Steam j Generators," Revision 2, dated March 11,1998.

b. Observations and Findinos l b.1 Implementation of 10 CFR Part 21 1 i

The inspectors reviewed WPP Procedure PQ-02-007, " Identification & Reporting of Conditions Adverse to Safety," Revision 9, dated August 19,1998. The procedure was noted to require the WPP Safety Review Committee to evaluate deviations and failures to comply, and to report the evaluation results to the Energy Systems Business Unit Safety Review Committee located in Monroeville, PA. The Energy Systems Businc ,s J Unit Safety Review Committee was responsible for review of the WPP Safety Review Committee evaluation results and determination of whether the deviation or failure to .

comply was reportable pursuant to the requirements of 10 CFR Part 21.  !

The inspectors noted with respect to Section 21.21 of 10 CFR Part 21, which requires l the filing of an interim written report if an evaluation of an identified deviation or failure to l

comply potentially associated with a substantial safety hazard cannot be completed '

within 60 days from discovery, that paragraph 6.4.3 of Procedure PQ-02-007, j Revision 9, required notification of the Chairman of the Energy Systems Business Unit 1 Safety Review Committee if it could not be conclusively determined within a timely  !

fashion whether a condition adverse to safety existed. WPP explained that this reflected ,

the understanding that the discovery period, defined by Westinghouse as the period of l information gathering and completion of the documentation, associated with a given i issue is not governed by a time limit, but instead it is the evaluation by the Safety j Review Committee of such issue that is governed by the 60-day reporting requirement.  ;

Once the issue is reported to the Energy Systems Business Unit Safety Review Committee and a Potential Issue is opened, the 60-day requirement of Energy Systems Business Unit Quality Policy / Procedure 21. Revision 2, would be in effect. The inspectors found this explanation acceptable and consistent with the provisions of Energy Systems Business Unit Quality Policy / Procedure 21, Revision 2.

I The inspectors also verified WPP's compliance to the posting requirements of Section 21.6 of 10 CFR Part 21, which requires the posting of current copies of the regulation, including adopted procedures, and Section 206 of the Energy Reorganization ,

Act of 1974. The inspectors noted that WPP had posted copies of Revision 15 of the 3

l l

l

l Westinghouse Electric Company poster, dated March 4,1997, which satisfied the intent of the regulation. The poster was viewed by the inspectors in numerous locations throughout the facility, including the administration building lunch area, manufacturing floor locations, and lobbies to both manufacturing and administration buildings.

1 b.2 Review of Incorporation of 10 CFR Part 21 Requirements in Procurement Documents The inspectors selected several WPP purchase agreements for review that were applicable to basic components that were to be utilized in the South Texas Project, Unit 1, replacement steam generators. Included in the review were purchase agreements i placed with: (a) Japan Steel Works (:hannel head forgings); (b) Ansaldo Energia S.P.A.

(upper shell barrels); (c) Creusot-Marrel, Inc. (transition cone forgings); (d) Kobe Steel, Ltd. (tube sheet forgings); (e) Sandvik Steel (seamless tubes); and (f) Equipos Nucleares SA (channel head cladding, machining and assembly). In addition, one purchase agreement placed with Forgemasters Engineering Ltd. was reviewed, pertaining to tube sheet forgings for Shearon Harris replacement steam generators. .

Each purchase agreement was found to reference a Quality Note which corresponded I to an inspection code for a particular supplier activity. The Quality Notes, numbered .

HA0051 through HA0058, contained the applicable specific quality system requirements, l including compliance to Section 21.31, " Procurement Documents," of 10 CFR Part 21.

The Quality Notes also contained technical and documentation requirements related to nondestructive examination, certificates of compliance, and certified material test reports.

Item 1 of Quality Note HA0051, " Material for ASME Code and Safety Related Applications," invoked the reporting requirements of 10 CFR Part 21 for all U.S. firms; Item 2 required non-U S. firms to report all deviations to WPP for evaluation and disposition; and item 3 required compliance to NCA-3800 of the ASME Boiler and Pressure Vessel Code and the applicabie portions of 10 CFR Part 50, Appendix B, quality assurance program requirements, as qualified by WPP or the ASME by issuance of a Quality System Certificate.

The inspectors examined the subject material of several of the Material Disposition Reports that had been received by WPP from the above suppliers in the 1995-1998 time frame, and reviewed the 10 CFR Part 21 fHes to identify if any of the Material Disposition Reports had resulted in a deviation requiring the opening of a Potential Issue. This review identified two issues which had been evaluated as a potentially reportable condition. The inspectors noted no other Material Disposition Reports in the sample reviewed that appeared to warrant evaluation for reportability. The first issue, Material Disposition Report 372765, pertained to the detection by Equipos Nucleares SA of an imbedded foreign object during nondestructive examination of replacement steam generator shell barrels that had been furnished by Creusot-Marrel for the Joseph Farley Nuclear Station. At the request of WPP, the issue was reviewed as a potential Part 21 by the Chairman of the Energy Systems Business Unit Safety Review Committee. On September 18,1998, WPP concluded that the issue had been adequately addressed by l the Safety Review Committee and that no condition adverse to safety existed. The inspectors concurred with this conclusion.

4

The second issue related to an evaluation of bobbin and rotating pancake coil indications at tube support plate intersections that were detected in the pre-shipment inspection of the Shearon Harris replacement steam generators. The indications were indicative of the presence of dings (i.e., mechanical damage causing a localized reduction in tube diameter ) and were most frequently located at the top and bottom of the 9th tube support plate. The WPP evaluation concluded that the potential for stress corrosion cracking at the dings was judged to be negligible and that no corrective action was required. This conclusion was based primarily on: (a) absence of detected stress I

corrosion cracking, after up to 18 years of operational service, in steam generators I tubed with thermally treated inconel 600 tubing and which contained dings larger than those present h the Shearon Harris replacement steam generators; (b) a similar absence of detected stress corrosion cracking in steam generators containing thermally treated Inconel 690 tubing, the tubing material selected for the South Texas Project, Unit 1, replacement steam generators, after up to 8 years of operational service; and (c) the superior corrosion resistance exhibited by thermally treated Inconel 690 tubing in laboratory corrosion testing. On September 18,1998, the WPP Safety Review Committee concluded that based on the evaluation of the bobbin indications, no l condition adverse to safety existed. The inspectors concurred with this conclusion.

b.3 Review of Anti-Vibration Bar End Cap Laser Weld Failure I

A review was performed by the lead inspector during November 16-17,1998, of the WPP evaluation of a failure on October 29,1998, of an anti-vibration bar end cap assembly during staking and dimpling to an anti-vibration bar (Steam Generator 4, Shop Order 12272). The end cap assembly consists of a channel-shaped component to which an end piece is attached by laser welding to each of the channellegs. The machine operator and inspector heard a noise during the staking and dimpiing operation, that was subsequently established to have been caused by the separation of one side of the attached piece because of a lack of penetration of the laser weld into the channelleg. The welding of the anti-vibration end cap assembly was performed approximately 1 year before the staking and dimpling operation, with the exact date unable to be established due to the non-assignment of unique identification numbers for individual assemblies.

The inspector reviewed WPP Report " Root Cause Analysis for the STP RSG AVB End Cap Laser Weld," Revision 1. This analysis concluded that the most probable cause of the failure was improper power settings by the welding operator, resulting from a mixup in settings between that used for anti-vibration bar end caps and that used for welding non-nuclear diaphragms. The root cause was attributed to be the periodic production interruptions experienced during welding South Texas Project, Unit 1, anti-vibration bar end caps for Shop Order 12272 (Steam Generator 4), as a result of higher scheduler priority for diaphragms. Additional contributing factors were determined to be the l absence of independent verification of power settings and inadequacies in the visual inspection process used for verification of weld penetration. The inspector reviewed the l

defined visual inspection criteria and confirmed that they were inadequate for establishing conformance of the v eld to drawing requirements. The inspector also 5

l l considered, however, that quality assurance staff or inspection personnel should have l previously identified these inadequacies. Overall, the inspector considered the root  !

cause analysis conclusions to be reasonable. Re-inspection of end cap assemblies was initiated in the four South Texas Project, Unit 1, steam generators, whh gas tungsten arc weld repairs performed for identified lack of penetration of laser welds.

The anti-vibration bar end cap design used for South Texas Project, Unit 1, replacement steam generators was previously used for steam generators for Point Beach, Shearon Harris, and Kori. An analysis of design loads on the anti-vibration bar end cap assembly was performed, which conservatively estimated the maximum axial and transverse loads to be, respectively,100 lb and 300 lb. To establish the acceptability of installed anti-vibration bar end cap assemblies in completed steam generators, pull-out tests (i.e., I measuring the axial load required to pull the anti-vibration bar out of the end cap) and pull-off tests (i.e., measuring the transverse load required to separate end cap l components) were performed for different fabricaticn conditions. The test conditions evaluated for the pull-out tests included one and two unacceptable laser welds, no end piece, and one tack weld with no laser welds. The minimum axial load value obtained was approximately 1800 lb for the no laser welds with one tack weld condition, a factor of 18 relative to the 100 lb maximum design load. The test conditions evaluated for the pull-off tests included one and two unacceptable laser welds, no laser welds coupled with an autogenous tack weld each side, and no laser welds coupled with a filler welded tack each side. The minimum transverse load value obtained was 2695 lb for the no laser welds coupled with a filler welded tack each side condition, a factor of approximately 9 relative to the maximum design load, The WPP Safety Committee concluded that the anti-vibration bar end cap failure was not reportable under 10 CFR Part 21. The inspector concurred with this conclusion and considered that the WPP approach to evaluation of the failure was excellent.

c. Conclusions The inspectors concluded from the review and evaluation of a sample of material disposition reports and purchase orders and the evaluation performed on anti-vibration end cap assemblies, that WPP was effectively implementing the provisions of the Energy Systems Business Unit quality program and was in compliance with the requirements of 10 CFR Part 21.

3.2 Procurement Control

a. Insoection Scoce The inspectors reviewed the WPP technical and quality requirements that were imposed by purchase order and material specification on the vendors selected for manufacture of

pressure boundary components for South Texas, Unit 1, replacement steam generators.

Components included in the review were the tubing, feedwater and auxiliary feedwater nozzles, the upper and lower shell barrel forgings, the transition cone forging, the elliptical head forging, and the tube plate forging. The WPP material specifications examined during this review were B163C23, "Therrnally Treated Alloy UNS N06690 6

l l

l

. j (Alloy 690) Tubing for South Texas Unit No.1 Replacement Steam Generators (Section ill-NB, SB-163, Code Case N-20-3)," Revision D; A508C25, "SA-508 Class 3a l Nozzle Forgings (Section lil-NB)," Revision D; A508C23, "SA-508 Class 3a Shell(Ring) l Forgings (Section lil-NB)," Revisions C and D; A508C24,"SA-508 Class 3a Elliptical I Head Forgings (Section ill-NB)," Revision C; and A508C20, "SA-508 Class 3a Tube Plate Forgings (Section lil-NB)," Revision D.

In addition, the inspectors examined the conformance of the vendor certification, for the l Steam Generator 4 (Shop Order 12272) pressure boundary items, to the requirements of the procurement documents, ASME Sections ll and ill Code, and STPNOC Specification 4R129NS1014, Revision 2.

A limited review of welding material procurement and receipt inspection was also performed which has been documented in Section 3.4.1 (Control of Welding) of the '

inspection report.

b. Observations and Findinos The inspectors found that the WPP material specifications, utilized for the procurement of steam generator pressure boundary materials, were consistent with ASME Sections 11 and ll1 Code requirements and the technical and quality requirements contained in STPNOC Specification 4R129NS1014, Revision 2. The vendor certification for the Steam Generator 4 (Shop Order 12272) pressure boundary items was ascertained to be in compliance with ASME Section lil Code and WPP material specification sampling and testing requirements.

c. Conclusions The inspectors concluded from the review of WPP material specifications and vendor documentation that: (1) present conformance to ASME Code and STPNOC Specification 4R129NS1014, Revision 2, requirements was good; and (2) specification reviews performed as part of the corrective action response to the stop work order from STPNOC had been effective.

3.3 Control of Tubino Ooerations

a. Insoection Scoce The inspectors reviewed WPP Drawings 6488E99, " Steam Generator Upper Shell Detail Assy," Sub 3; 6488E93, " Steam Generator U-tube Insertion and Assy with Helium Leak Test," Sub 4; and 6488E68, " Steam Generator Tube Plate and Lower Barrel Weld Assembly and Machining," Sub 4. The inspectors also reviewed Detailed Manufacturing Procedures: DMP-5563, " Steam Generator Tubing Tube Tack Expansion," Revision 22; l DMP-6255, " Foreign Object Control Procedure for a Complete Steam Generator,"

l Revision 06; and DMP-6401, " Steam Generator Hydraulic Tube Expansion," Revision 8; and Quality Inspection Procedures: OlP-3150, " Tube Insertion Inspection Requirements," Revision 16; and OlP-3605, "Hydroswage Equipment Calibration,"

7

l 1

l Revision 01. The team observed the insertion of tubes, tack expansions, and crevice expansions of tubes.

b. Observations and Findinas The team found that procedural controls for tubing operations were satisfactory and consistent with the requirements of STPNOC Specification 4R129NS1014, Revision 2.

The team also observed insertion of tubes into Steam Generator 3 (Shop Order 12271, Traveler Operation 60), and noted that the tubing operation was, in general, proceeding smoothly and without problems. Tubing of the steam generator was performed in a Level"A" clean room, the highest level of foreign object control. Level"A" controls imposed limitations on personnel access and tools in the area and required daily '

sweeping and mopping of the area. Personnel performing tubing operations were also required to wear head coverings, rubber shoe covers, gloves, and coveralls.

The inspectors noted certain conditions which, although not considered to represent a significant foreign object or tubing damage concern, appeared to be inconsistent with the overall program cleanliness controls. An abrasive cloth used to remove minor markings from tubes was observed to not be discarded after it fell on the floor of the clean room. When a new shipping box of tubes was introduced to the area, the unpacking process resulted in loosening of some staples and some splintering of the wood on the box edges. Staples held down plastic fiber strapping that secured the iubes during shipping. Some staples had only one leg in the wood because the straps were torn rather than cut. Some staples could be removed with only finger pressure.

After the inspectors identified these undesirable conditions, WPP staff taped over the staples and splinters and noted the conditions in a log book so that subsequent shifts would become aware of the conditions. In addhion, WPP staff held a " stand down" meeting with the work crews to reinforce sensitivity to the Level"A" requirements.

The inspectors observed the performance of hydraulic tack expansions of tubing in Steam Generator 3 (Shop Order 12271) and full-depth hydraulic tube-to-tube sheet expansions in Steam Generator 2 (Shop Order 12270), Operations 070 and 060, respectively, on their travelers. The tack expansion secured the tube for welding, and the crevice expansion eliminated the crevice between the outside diameter of the tube and the inside diameter of the tube sheet holes. The team observed the expansion processes and observed the quality control checks made of the expansion processes.

These checks included verification of the pressures used for expansions, plug gauging the tube inside diameters after expansion, and measurement of the expansion depth into the tube. The crevice expansion diameters were sampled and measured automatically. An online computer recorded and plotted control charts of the measurements for acceptance. These measures appeared to assure expansion of the tubes to the proper depth.

c. Conclusions The inspectors found the overall WPP tubing criteria and practices to be consistent with the requirements of STPNOC Specification 4R129NS1014, Revision 2. Corrective 8

l actions were immediately taken in response to inspector obsen/ations regarding use of  !

abrasive paper, loose staples, and wood splinters. l l

3.4 Control of Soecial Processes 3.4.1 Control of Welding l

a. Insoection Scope 1 The inspectors reviewed Procedure DMP-5881, " Shop Control of Welding Materials ASME Boiler Pressure Vessel Code," Revision 20, to determine the measures used to control order, receipt, conditioning, and issue of welding materials. The inspectors j toured the welding material crib and observed the materialidentification practices and  ;

the temperatures of storage ovens ( for covered electrodes and submerged arc welding l flux). In addition, the inspectors reviewed the issue records for E7018 and E9018 covered electrodes, SFA 5.23 Class EM-2 electrode, and Oerlikon OP-121-TT j submerged arc welding flux and examined the applicable certified material test reports l and material specifications. The material specifications reviewed were: (1) C051C02, j

" Carbon Steel Electrode SFA 5.1 Class E7018 for Shielded Metal Arc Welding l

(SMAW)," Revision C; (2) C05502, " Low Alloy Steel Electrode SFA 5.5 Class E9018M for Shielded Metal arc Welding," Revisions B and C; (3) C523C04, "Oerlikon Type i OP-121TT Flux for Submerged Arc Welding (SAW) Low Alloy Steel," Revision C; and (4) C523C05, " Low Alloy Steel Electrode SFA 5.23 Class EM-2 and Oerlikon OP-121TT  ;

Flux for Submerged Arc Welding (SAW)," Revision A. The inspectors also visited the l warehouse where bulk welding materials were stored to examine the identification on submerged arc welding flux containers and review storage conditions.

Limited welding activities were in process on the South Texas Project, Unit 1, replacement steam generators during the onsite inspection. The inspectors were able to observe performance of: (1) tube-to-tube sheet welding in Steam Generator 4, Shop Order 12272, using Welding Procedure Specification 4164-1, Revision 13; and (2) performance of a shielded metal arc weld buildup (Detail H) on the inside diameter of the "H" upper shell barrel for Steam Generator 2, Shop Order 12270, using Welding Procedure Specification 4222, Revision 5.

b. Observations and Findinos Weldina Materials The inspectors determined that appropriate technical controls had been developed for shop control of welding materials, with no problems noted with respect to storage oven temperatures, identification of covered electrodes, and issue control. Review of certified material test reports for E7018 and E9018 covered electrodes showed the chemistry and mechanical properties were in conformance with the requirements of the respective WPP material specifications and ASME Sections ll and ill Code.

9

,e The inspectors observed, however, during examination of the submerged arc welding l flux storage ovens, that a single WPP heat code had been assigned to what appeared l

potentially to be multiple lot numbers of flux (i.e., the flux ovens were marked as i containing Oerlikon OP-121-TT flux, assigned Heat Code SZ47, from Manufacturing Lot l N o s. 1719-7-3, 1719 4, 1719-7-5, 1719-9 -6, 1719 1, 1719- 10-2, 1719- 14-6, 1719-15-1,1719-15-2, and 1719-15-3). Additional inspection regarding the procurement, receipt inspection, and qualification testing of this flux (which is used for l submerged arc welding of the steam generator low alloy steel pressure boundary weld I joints) identified the following: .

l

. The applicable procurement document for the submerged arc welding flux was ascertained by the inspectors to be WPP Purchase Agreement E64876, dated I June 28,1996. This purchase agreement required the vendor, Welding Engineering Supply Co., to supply Oerlikon Type OP-121TT automatic submergod j arc welding flux per WPP Material Specification C523C04, Revision C, and the requirements of the attached Quality Note HA0051. i

. WPP Material Specification C523C04, "Oerlikon Type OP-121TT Flux for Submerged Arc Welding (SAW) Low Alloy Steel," Revision C, stated that its purpose was to establish the requirements for Oerlikon Type OP-121TT flux for submerged arc welding to comply with ASME Section ll, Part C, and ASME Section Ill, Division 1, Subsection NB, paragraph NB-2400. The material specification included in its requirements that: (1) certified test reports should include a statement of compliance to the purchase order and the specification and revision letter; and (2) each unit package of flux should be identified with material classification and specification numbers, supplier's name and trade designation, and lot number.

. The inspectors noted that WPP Interna l Quality Release 055814, dated November 11,1996, identified the flux received in response to WPP Purchase Agreement E64876 as Oerlikon Type OP-121TT, Heat No. 1719-3-6. The Certificate of Compliance issued by Welding Engineering Supply Company for the WPP Purchase Agreement E64876 order identified, however, the type of flux as Hobart 121/SD3. A Hobart Certificate of Compliance for the WPP purchase agreement number was also provided to the inspectors. This document indicated the type of flux to be Hobart 121 and identified that the flux had been produced under Quality System Program FMQP 01, Revision 4, audited and approved by WPP as meeting ASME Section lil Code, Subsection NCA-3800, and the applicable portions of 10 CFR Part 50, Appendix B. Both Certificates of Compliance indicated the lot number of the flux was 1719.

. The inspectors questioned WPP personnel concerning the supply and acceptance of a Hobart manufactured submerged arc welding flux, when the applicable procurement documents required Oerlikon Type OP-121TT flux to be furnished.

The inspectors were informed that WPP became aware that Oerlikon (a Swiss manufacturer of welding products) had discontinued its ASME Quality System Certificate as a materials organization, thus necessitating, for use of the vendor 10

1 l

products in ASME Section lli Code welding applications, an audit by WPP to verify that a quality assurance program was in effect which conformed to the provisions of paragraph NCA-3800 in Section lll of the ASME Code. Upon ascertaining that Hobart, a domestic manufacturer of welding products, was licensed to use Oerlikon formulations in the manufacture of welding consumables, WPP opted to perform an i NCA-3800 audit at Hobart. This audit was performed on August 1-2,1996, and the  !

vendor conditionally approved pending receipt of corrective actions for two l ident;fied findings. l A copy of the Oerlikon licensing agreement was obtained by WPP from Hobart at the inspectors' request. The inspectors noted during review of this agreement that it required the licensee to use its own trade names for the products and did not permit the licensee to use Oerlikon brand or trade names. No information was i provided by WPP personnel with respect to why Purchase Agreement E64876 and l WPP Material Specification C523C04 were not appropriately revised to reflect the use of the Hobart manufactured product. The failure of receipt inspection to identify that the flux identity (contained in the submitted vendor documentation) did not conform to that specified in the procurement documents, as required by Criterion Vll, " Control of Purchased Material, Equipment, and Services," and Criterion XV," Nonconforming Materials, Parts, and Components," of Appendix B to 10 CFR Part 50, was identified as Nonconformance 99900104/98-01-01.

. Paragraph 3.1 of WPP Material Specification C523C04, Revision C, included in its requirements that flux containers be identified with the supplier's name, trade designation, and lot number. The inspectors noted, however, during examination of containers in warehouse storage, that the information on the container i identification labels did not include the flux manufacturer's name and showed the trade designation to be OP-121 TT. The failure of receipt inspection to identify that the flux containers did not contain the manufacturer's name was identified as an additional example of Nonconformance 99900104/98-01-01.

. The inspectors observed that the identification labels on flux containers in warehouse storage showed the same three number type format (e.g., 1719-5-1) for lot identity as was previously noted on flux storage oven documentation. The inspectors requested information on what this format represented, in that the scope of WPP testing of this flux (with respect to the requirement in paragraph NB-2420 of the ASME Section lil Code to test each combination of heat of bare electrode and lot of submerged arc flux) appeared to be based on the assumption that all of the flux containers were from a single lot. WPP contacted Hobart to obtain the information and were informed that the three numbers represented, respectively, lot number, batch number, and mix number. An initial complete chemicallot was made up which was then subdivided into batches, followed by further subdivision into smaller quantities for mixing with a binder. The inspectors were informed, in response to questions regarding uniformity of flux composition from batch to batch, that a weld test pad was made for each batch and chemical analysis performed.

Copies of the batch analyses for Lot 1719 were obtained from Hobart by WPP.

11

1' l

i

! The inspectors concluded from review of the analyses that the values were reasonably uniform from batch to batch.

. During review of mechanical properties of submerged arc weld deposits made using Lot 1719, the inspectors noted that the scope of testing performed by WPP was not consistent with the requirements of Materials Specification CS23C05, " Low Alloy Steel Electrode SFA 5.23 Class EM-2 and Oerlikon OP-121TT Flux for

. Submerged Arc Welding (SAW)," Revision A. The specification required performance of additional Charpy-V and drop weight tests to those required by  ;

ASME Section 111 Code. The scope of actual notch toughness testing performed I was limited to that required by ASME Section 111 Code. After identification of the l discrepancy by the inspectors, WPP revised Material Specification C523C05 to eliminate the requirement for the additional Charpy-V and drop weight tests.

Production Weldina ,

l The inspectors noted good compliance to welding procedure specification requirements during observation of gas tungsten are tube-to-tube sheet welding and performance of a shielded metal arc weld pad buildup.

c. Conclusions The inspectors found that welding material storage and issue were satisfactorily centrolled, with good compliance to welding procedure specification requirements noted during limited observations of production welding. A nonconformance was identified with respect to the procurement of submerged arc welding flux.

3.4.2 Post-Weld Heat Treatment

a. Insoection Scoce The inspectors reviewed the conformance of two WPP post-weld heat treatment procedures to the requirements of the ASME Section ill Code. The selected procedures were: Procedure DMP-6181, " Post Weld Heat Treatment Steam Generator Upper Assembly and Closure Seam," Revision 14; and DMP-6420, " Heat Treat Procedure including Preheat, interpass, Gouging, Hydrogen Baking and Post Weld Heat Treatment," Revision 03. The inspectors examined the post-weld heat treatment records for two South Texas Project, Unit 1, replacement Steam Generator 3 assemblies (i.e., Upper Shell Assembly, Serial No. 8962J; and Transition Cone to Lower Shell Barrel B Assembly, Serial No. 8910J) to verify conformance to procedural requirements. In addition, the inspectors reviewed Quality inspection Procedure OlP-3146, " Post Weld Heat Treat Data Sys (SG&PRZ)," Revision 09, to verify that appropriate provisions had been made to monitor accumulated post-weld heat treatment time with respect to the qualification times that had been used in the testing of base materials, welding materials, and welding procedure qualifications.

12

.-. . . . - - - . ~ . ~ . _- - - --- -- - - - - - -..

I

b. Observations and Findinos The inspectors verified that the procedural (DMP-6181, Revision 14, and DMP-6420, Revision 03) requirements for hold time and permissible heating rate, cooling rate, and temperature gradient were consistent with ASME Section lli Code requirements. The inspectors noted from review of the procedures that significant numbers of thermocouples were used to monitor assembly temperatures, with the selected attachment locations providing a high degree of assurance of required temperatures and uniformity being achieved. No probls ns were identified during review of the post-weld heat treatment records for the South Texas Project, Unit 1, replacement Steam Generator 3 Upper Shell and Transition Cone to Lower Shell Barrel B assemblies, with good procedural compliance noted. The inspectors determined from review of Procedure OIP-3146. Revision 09, that appropriate provisions had been made for monitoring accumulated post-weld heat treatment time with respect to the qualification j times used during testing of base materials, weld materials, and welding procedure l qualifications. The inspectors additionally noted that the utilization of a hydrogen baking practice on completed welds significantly reduced the number of required post-weld heat treatment cycles; thereby, maintaining accumulated post-weld heat treatment times well l below the qualification times used for materials and welding procedure qualification testing.

c. Conclusions The procedural requirements for conduct of post-weld heat treatment on South Texas l Project, Unit 1, replacement steam generator assemblies were consistent with the

! ASME Section 111 Code, with good procedural compliance noted during review of l

- post-weld heat treatment records. Appropriate actions had been taken to monitor component accumulated post-weld heat treatment time with respect to qualification  !

l times used in materials and welding procedure qualification testing.

3.4.3 Nondestructive Examination Insoection Scoce a.

l The inspectors reviewed the conformance of WPP nondestructive examination i procedures to the requirements of the ASME Section V Code and STPNOC Specification 4R129NS1014, Revision 2. Included in this review were Nondestructive Examination Procedures 8001, " Guidelines for RT/UT Layout of Weld," Revision 01; 8105, " Solvent Remover Liquid Penetrant Exam," Revision 16; 8106, " Water Washable Liquid Penetrant Exam," Revision 13; 8139, " Water Washable Liquid Penetrant Exam,"

Revision 02; 8567, " Ultrasonic Examination of Austenitic Welds," Revision 05; 8573,

" Ultrasonic Examination of Welds," Revision 17; 8141, " Solvent Remover Liquid Penetrant Exam," Revision 01; 8330, " Radiographic Procedure," Revision 01; and 8609,

. " Visual Examination of S/G Welds," Revision 01.

The inspectors witnessed the ultrasonic examination of Weld P (Elliptical Head to Upper i Shell Barrel J) in Steam Generator 3 (Shop Order 12271) for South Texas Project, 13 i

Unit 1. The inspectors e'so reviewed the radiographic film for the following South Texas Project, Unit 1, steam generator welds: (1) Steam Generator 2 (Shop Order 12270),

Weld C (Lower Shell Barrel A to Lower Shell Barrel B), G (Lower Shell Barrel B to Transition Cone), and Y (Lower Shell Barrel A to Tube Sheet); and (2) Steam l Generator 1 (Shop Order 12269), Weld R (Feedwater Nozzle to Upper Shell Barrel H). I As of the onsite inspection, radiographic examinations of Walds S and SO (Manways to  ;

Upper Shell Barrel J) had not been performed. As a result, the radiographic film was l l

reviewed for the corresponding welds in replacement Steam Generator 3 for Shearon Harris (Shop Order 12245).

b. Observations and Findinas The inspectors found that the nondestructive examination procedures were co,1sistent with the requirements of ASME Section V Code and STNOC Specification 4R129NS1014, Revision 2.

The inspectors verified during review of the Steam Generators 1 and 2 radiographic film that WPP had used the correct image quality indicator (penetrameter) and film type, and that film density values were acceptable. No disagreements were noted with the WPP film interpretations. In general, the radiographic results indicated that the welds were of high quality and free of apparent defects. Minor exceptions were a few views which showed acceptable levels of porosity and slag inclusions. The inspectors noted during review of radiographic film for Welds S and SO in Steam Generator 3 (Shop Order 12245) that, because of the section thickness variations in the manway welds, certain views required " double film viewing" (i.e., film overlapping) to achieve an acceptable film density. The use of " double film viewing" was not documented, however, on the radiographic inspection reports (reader sheets). The inspectors noted no problems at the locations where " double film viewing" was required to achieve an acceptable film density, but considered that its use should be appropriately documented on the reader sheets. WPP re-reviewed the radiographs for the Shearon Harris replacement steam generator manway welds subsequent to the onsite inspection, and documented when double film Vewing was used.

As part of the observation of the ultrasonic examination of Weld P in Steam Generator 3 (Shop Order 12271), the inspectors witnessed calibration of the instrument using Calibration Block SV80. The inspectors verified that the calibration block was manufactured from the same material type as the vessel being inspected, the correct reference level was used, and the instrument exhibited required linearity and alignment.

Axial and circumferential scans of Weld P with 60*,45 , and O' search units were observed. The examination personnel were noted to conform to the procedural requirements with respect to couplant, search units, instrumentation, settings and techniques. Scanning was performed at 14 dB above the reference level and with the I proper overlap, coverage, and speed. Intermittent rejectable indications were found by l WPP personnel during these examinations, which were located at the weld center line i about 0.5 inches from the weld root. The indications were appropriately documented in an inspection report.

14 l

c. Conclusions The inspectors found that the WPP nondestructive examination procedures were consistent with the requirements of ASME Section V Code and STPNOC Specification 4R129NS1014, Revision 2. Performance of radiographic and ultrasonic examinations was noted to conform to procedural requirements. An observation made regarding docurnentation of the use of " double film viewing" during interpretation of manway weld radiographs was appropriately responded to by WPP quality assurance staff.

3.5 Corrective Action Proaram - Stoo Work Order Backoround On November 6,1997, STPNOC issued a stop work order pertaining to all Westinghouse activities associated with the procurement and fabrication of South Texas Project, Unit 1, replacement steam generators. The order applied to all Westinghouse and Westinghouse contractor facilities worldwide, but excluded replacement steam generator licensing activities. The basis for the order resulted from identified engineering and manufacturing deficiencies and STPNOC concerns regarding the effectiveness of WPP oversight of its vendors.

The most significant engineering deficiency resulted in the nozzle openings for the feedwater and auxiliary feedwater nozzles being cut at incorrect locations in the "H" upper shell barrels for the four STP, Unit 1, replacement steam generators. The barrels were fabricated and machined by Ansaldo in accordance with Detail F of WPP Shell Fabrication Drawing 6488E61, Revision 4, which was revised by Engineering Change Notice 33447, dated November 1996, to show the final placement and dimensions of the FW and AFW nozzle openings. The view presented in Detail F of the upper shell barrel was a " Bottom View" looking up at the replacement steam generator. However, WPP General Arrangement Drawing 6488E99, Revision 4, utilized the typical " Top View" convention for the replacement steam generator, and indicated: (1) the feedwater nozzle to be located at 29* from the "Z" axis (South) towards the "W" axis (East), and (2) the auxiliary feedwater nozzle to be located at 29 from the "Z" axis towards the "Y" axis (West). The use of different conventions resulted in the nozzle locations shown in Detail F on the fabrication drawing sent to Ansaldo being 180' from the locations required by the general arrangement drawing. The error was identified by WPP after receipt of the shell barrels from Ansaldo and prior to further fabrication operations.

Material Deficien::y Reports 373042,373043,373044, and 373046, dated November 5, 1997, were issued, with corrective actions taken including cutting two of the upper shell barrels to make a new one and ordering of replacement upper shell barrels.

The most significant vendor error was documented in Material Deficiency Report 367490, dated April 30,1997. In this instance, Ansaldo cut a manway opening in an upper shell barrel 3 inches above the required location. The final disposition was to scrap the barrel.

15

l

! 1 1

1 l Contingent upon release of the stop work order, STPNOC requested that Westinghouse  !

develop a total assessment / recovery plan that included a historical confirmation that all l

work to date, and resultant work products, were in compliance with established l requirements and were technically and programmatically correct. By letter dsted l January 9,1998, WPP submitted its Assessment Recovery Program l (ST-W2-NOC-000358, Revision 2) to STPNOC for review and approval. The program consisted of two parts: (1) a historical review of manufacturing, quality assurance and purchasing; and (2) a root cause analysis of events in the areas of supplier oversight, cause and prevention, quality assurance and management oversight, design assurance, and program compliance. Implementation of the Assessment Recovery Program was accomplished by use of review teams. Reviews of manufacturing routings and records, purchase orders and quality requirements, material heat code packages, and rionconformance documents were performed by WPP technical services and product assurance personnel. One team, the " Historical Design Engineering Review Team,"

consisted of individuals who were not involved in th; / SP operation. The stop work order was lifted by STPNOC in January 1998 following review and approval of the Assessment Recovery Program,

a. Insoection Scoce The inspectors reviewed work products of the Assessment Recovery Program and interviewed participating personnel. In addition, a review was performed of : (1) the assessment results documented in Report NSD-E-MSI-98-039, " Historical Review Design Engineering, South Texas Project, Replacement Steam Generators," Revision 1, dated July 1998; and (2) the root cause analysis documented in Report OS-98-0083, dated January 20,1998, which was transmitted to STPNOC by WPP Letter ,

ST-W2-NOC-000363, dated January 27,1998.

b. Observations and Findinas Report NSD-E-MSI-98-039, Revision 1, was found to address four review subjects (i.e., l design and manufacturing drawings, material specifications, process specifications, and engineering design). The inspectors noted that the report made several recommendations to WPP management in regard to the design document verification

process. The inspectors ascertained from review of the root cause analysis report that WPP had concluded that managerial oversight was at the center of the cause. WPP management made a commitment to establish on a forward basis a commitment to total quality programs, which included absolute compliance to technical specification and procedural requirements and appropriate overview of WPP quality system performance to identify, eliminate or resolve problems either internally or at the supplier.

The inspectors reviewed several work products of the Assessment Recovery Program corrective actions which included: (1) a material specification review (PEN-98-326,

! dated July 29,1998); (2) a report discussing the cobalt and carbon content of welding consumables for South Texas Project (NPD/E/ PEN-98-311, dated July 29,1998), which was prepared to verify if cobalt and carbon contents of stainless steel and Inconel filler materials meet STPNOC Technical Specification requirements; and (3) a check list for l

16 i

^

l I

. l drawings (NPD/E/ PEN 98-331, dated August 3,1998) and a check list for Process Specifications (NPD/E/ PEN 98-337, dated August 6,1998), both documenting the resolution of comments made in the historical review, i

The inspectors also reviewed the implementation of all five design assurance corrective actions. Revisions made to procedures as a result of the corrective action plan included: (1) revisions to PQ-02-031, Revision 5, which added a conformance matrix; (2) revisions to PQ-02-006, Revision 11, which required all drawings be independently checked by drafting personnel; (3) revisions to PQ-02-033, Revision 2, v.hich  ;

implemented varicus calculation note requirements; (4) revisions to PQ-02-003, I Revision 8, which required an impact assessment of design changes on engineering orawings; and (5) revisions to numerous procedures to incorporate scratch criteria to ensure that WPP, and its customers, have a clear understanding of the expectations in this area and the levet of scratches considered acceptable. The procedures reviewed l by the inspectors, to verify effective implementation of the scratch cnteria, included the following: WPP Process Specification 87111WU, "Special Engineering Requirements During Tube Bundle Assembly," Revision G; WPP Material Specifications B163C20, Revision G, and B163C27, Revision E, used, respectively, for thermally treated Alloy 690 tubing for Arkansas Nuclear One, Unit 2 and Farley Units 1 and 2 replacement I steam generators; Sandvik Control Procedures (for Arkansas Nuclear One, Unit 2) 5404, " Visual Standards," Revision 2,5488, " Inspection of U-Bent Tubes," Revision 2, and 5482, " Inspection of Straight Tubes," Revision 1; WPP Procedure DMP-5563,

" Steam Generator Tubing, Tube Tack Expansion," Revision 22; and WPP Quality inspection Procedure 3150, " Tube Insertion Inspection Requirements," Revision 16.

c. Conclusions l

Based on the review and evaluation of the WPP work products that were generated in response to the STPNOC stop work order, the inspectors concluded that WPP was effectively implementing the provisions of the Assessment Recovery Program which was submitted to the utility on January 9,1998, and was in compliance with the applicable provisions of 10 CFR Part 50, Appendix B.

3.6 Entrance and Exit Meetinas At the entrance meeting on September 21,1998, the lead inspector discussed the scope of the inspection, outlined the areas to be inspected, and established interfaces with WPP management. In the exit meetings on September 25 and November 17, 1998, the lead inspector discussed the findings and observations of the inspection.

Documents reviewed during the inspection which were identified as containing proprietary information included WPP material specifications, Report NSD-E-MSI-98-039, Revision 1, and Report QS-98-0083, dated January 20,1998. No information was included from these documents in the inspection report that was considered proprietary.

17

l<

PARTIAL LIST OF PERSONS CONTACTED Westinahouse Pensacola Plant J. Alexander, Controller -

J. Allen, Senior Quality Assurance Engineer S. Anderson, Purchasing Manager D. Arnott, Works Engineering Manager G. Bieberbach, Project Manager, South Texas T. Coriale, Product Assurance Manager R. Eubanks, Area Manager, Manufacturing R. Frisbey, Human Resources Manager L. Garrett, Engineering Aide D. Jennings, Quality Control Supervisor M. Kachmar, Engineering Manager J. Land, Design Engineer L. Lavoie, Area Manager, Manufacturing O. Machado, Manager, Nuclear Projects and Marketing K. Merritt, Quality Assurance Manager K. Olmstead, Quality Assurance / Reliability Engineer U. Schneider, Manager, Technical Services C. Schaishuhn, Plant Manager M. Weatherly, Quality Assurance Engineer / Nondestructive Examination Levellli J. Woeber, Director, Steam Generator Product Line STP Nuclear Oneratina Comoany C. McIntyre, Director, Engineering Projects R. Rehkugler, Director, Quality

-C. Sist, Resident inspector Arkwriaht Mutual Insurance Comoany W. Jones, Authorized Nuclear inspector ITEMS OPENED Ooened 99900104/98-01-01 Para. 3.2 NON ' Procuremer,t of submerged arc welding flux 18

.a LIST OF DOCUMENTS EXAMINED South Texas Project Nuc_ lear Ooeratina Comoany Soecification 4R129NS1014, Replacement Steam Generators, Revision 2 Westinahouse Process Soecification  ;

l 87111WU, "Special Engineering Requirements During Tube Bundle Assembly," Revision G Waglicahouse Material Soecifications A508C20, SA-508 Class 3a Tube Plate Forgings (Section Ill-NB), Revision D A508C23, SA-508 Class 3a Shell(Ring) Forgings (Section lil-NB), Revisions C and D 1

A508C24, SA-508 Class 3a Elliptical Head Forgings (Section Ill-NB), Revision C A508C25, SA-508 Class 3a Nozzle Forgings (Section ill-NB), Revision D B163C20, Thermally Treated Alloy UNS N06690 (Alloy 690) Tubing for Arkansas Nuclear One Unit No. 2 Replacement Steam Generators (Section lil-NB, SB-163, Code Case N-20 3), Revision G B163C23, Thermally Treated Alloy UNS N06690 (Alloy 690) Tubing for South Texas Unit No.1 Replacement Steam Generators (Section Ill-NB, SB-163, Code Case N-20-3, Revision D B163C27, Thermally Treated Alloy UNS N06690 (Alloy 690) Tubing for Farley Units No.1 and 2 Replacement Steam Generators (Section ill NB, SB-163, Code Case N-20-3), Revision E C523C04, Oerlikon Type OP-121TT Flux for Submerged Arc Welding (SAW) Low Alloy Steel, Revision C l l

C523C05, Low Alloy Steel Electrode SFA 5.23 Class EM-2 and Oerlikon OP-121TT Flux for Submerged Arc Welding (SAW), Revision A CO51C02, Carbon Steel Electrode SFA 5.1 Class E7018 for Shielded Metal Arc Welding (SMAW),

Revision C C055C02, Low Alloy Steel Electrode SFA 5.5 Class E9018M for Shielded Metal Arc Welding, R'.:vnions B l and C Ouality insoection Proceduras 3150, Tube Insertion inspection Requirements, Revision 16 3605, Hydroswage Equipment Calibration, Revision 01 19

.c Qtawings 6488E99, Steam Generator Upper Shell Detail Assy, Sub 3 6488E93, Steam Generator U-tube Insertion and Assy with Helium Leak Test, Sub 4 6488E68, Steam Generator Tube Plate and Lower Barrel Weld Assembly and Machining, Sub 4 6488E71, Replacement Steam Generator Outline, Revision 4 Detailed Manufacturipa Procedures DMP-0200, General Welding Specification, Gas Tungsten Arc Welding - GTAW ASME Boiler and Pressure Vessel Code Section IX and Section 111, Revision 22 DMP-0700, General Welding Repair Procedure ASME Boiler and Pressure Vessel Code Section IX and Section Ill, Revision 19 DMP-5524, Heat Treatment Procedure including Preheat, Interpass, Gouging, Hydrogen Baking and Post Weld Heat Treatment, Revision 50 DMP-5563, Steam Generator Tubing Tube Tack Expansion, Revision 22 DMP-5881, Shop Control of Welding Materials ASME Boiler Pressure Vessel Code, Revision 20 DMP-6255, Foreign Object Control Procedure for a Complete Steam Generator, Revision 06 DMP-6401, Steam Generator Hydraulic Tube Expansion, Revision 8 DMP-6420, Heat Treatment Procedure including Preheat, Interpass, Gouging, Hydrogen Baking and Post Weld Heat Treatment, Revision 03 DMP-6424, Instructions for Assembly and inspection of Anti-Vibration Bars in Delta 94 Replacement Steam Generators, Revision 01 Nondestructive Examination Procedures 8001, Guidelines for RT/UT Layout of Weld, Revision 01 8105, Solvent Remover Liquid Penetrant Exam, Revision 16 8106, Water Washable Liquid Penetrant Exam, Revision 13 8139, Water Washable Liquid Penetrant Exam, Revision 02 8567, Ultrasonic Examination of Austenitic Welds, Revision 05 l

20

2-l'6:

o 8573, Ultrasonic Examination of Welds, Revision 17 l

l 8141, Solvent Remover Liquid Penetrant Exam, Revision 01 8330, Radiographic Procedure, Revision 01 8609, Visual Examination of S/G Welds, Revision 01 i

. Sandvik Control Procedures 5404, Visual Standards, Revision 2 5488, inspection of U-Bent Tubes, Revision 2 6482, inspection of Straight Tubes, Revision 1 l

l 1

1 I

i 21 e

-. --