IR 05000458/1993029

Insp Rept 50-458/93-29on 931129-1203.No Violations Noted. Major Areas Inspected:Procurement Activities W/Emphasis on Lessons Learned as Result of Problems Incurred During Procurement for Reactor Vessel Feedwater Safe Forgings

ML20059F759
Person / Time
Site:	River Bend
Issue date:	01/07/1994
From:	Constable G NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:
Shared Package
ML20059F716	List: IR 05000458/1993029 ML20059F714
References
50-458-93-29, NUDOCS 9401140141
Download: ML20059F759 (12)

Text

. _ . _ . _ _ ._ . _ _ . _ ._ _ .

t W

APPENDIX

,

U.S. NUCLEAR REGULATORY COMMISSION

REGION IV

Inspection Report: 50-458/93-29 License: NPF-47 Licensee: Gulf States Utilities P.O. Box 220  ;

St. Francisville, Louisiana

Facility Name: River Bend Station Inspection At: St. Francisville, Louisiana Inspection Conducted: November 29 through December 3, 1993 Inspectors: M. E. Murphy, Reactor Inspector, Plant Support Section Division of Reactor Safety, Region IV J. E. Whittemore, Reactor Inspector, Plant Support Section Division of Reactor Saihty, Region IV  :

i_i N Approved: / ; -

le,~ Chief, Plant Support Section

//V[4</

~Date'  !

Division of Reactor Safety Inspection Summary  ;

Areas Inspected: Routine, announced inspection of procurement activities with emphasis on lessons learned as a result of problems incurred during j procurement for reactor vessel feedwater safe end and elbow forgings 1 replacemen Results:

• Procurement problems associated with the reactor-feedwater safe end and elbow forgings replacement had been satisfactorily resolved. Changes to the licensee's procurement procedures, together with the recognition that time must be allowed in preplanning to resolve quality issues that 1 arise, should preclude further. problems in procuring ASME Code-related  ;

material (Section 1.1.3). l

• Licensee procedures used within the procurement function were not well integrated (Section 1.2.3).  ;

!

9401140141 940107 PDR ADOCK 05000458-G PDR  :

-

- -

. , _ . , . . _ . _ _ . , - - .

_ _ _ __ ._

. .

'b

,.

P-2- i

. The licensee's receipt inspection process was not well proceduralized and relied heavily on the receipt inspector's experience to determine ,

the correct inspection attributes, which were not reviewed prior to the material being committed to the field for installation (Section 1.3.3). ,

. The licensee's procurement program satisfactorily addressed the issue control, and post issue control of quality and safety-related material .

(Section 1.3.3).

. The licensee's handling of insufficient material staging space by the initiation of a material pledging program was considered a strength (Section 1.3.3).

. Overall, the licensee's program for procurement, receipt, storage, and'

handling of safety-related items was in conformance with regulatory requirements, licensee commitments., and industry guides and standards '

(Section 1.4).

Summary of Inspection Findings: )

. None

'

Attachment:

. Attachment - Persons Contacted and Exit Meeting

.

!

. _ .

^

,

-3-l DETAILS 1 PROCUREMENT AND MATERIALS HANDLING (38701,38702)

This inspection assessed the licensee's program for procurement, receipt, _

'

storage, and handling of safety-related items. This inspection also evaluated the effectiveness of the licensee's corrective actions in response t previously identified weaknesses in the procurement program, specifically, the violation associated with the replacement of the N4A reactor vessel feedwater-nozzle safe end and the planned replacement of the first upstream elbow, documented in NRC Inspection Report 50-458/92-2 .1 Feedwater Safe End and Elbow Forginas Replacement The inspectors reviewed the procurement issues associated with the replacement of the N4A reactor vessel feedwater nozzle safe end and the planned replacement of the first upstream elbow with regard _to the effect of programmatic changes resulting from lessons learned on the procurement of ASME Code-related materia . Background Prior to Refueling Outage 4, the licensee identified the presence c T . racks in the N4A vessel feedwater nozzle safe en In addition, slight wall thickness degradation in the first upstream elbow was identified subsequent to the start of procurement for the safe end repair. The decision was made to replace this elbo During procurement of the ASME. Code-related material to implement the repair, several issues arose that hindered, or in the case of the elbow, precluded the completion of the desired work. Two ring forgings consisting of-the safe end and the thermal transition sleeve were successfully replaced as planned, but the elbow was not replace The NRC issued Violation 458/9221-01 for the licensee's failure to assure adequate testing and inspection prior to accepting ASME Code-related material. This violation was closed in NRC Inspection Report 50-458/93-1 .1.2 Task Related Procurement Problems In order to support the safe end repair, the licensee attempted to purchase the following material in accordance with the requirements of ASME Code,Section III, Class I, Subsection NB:

. Two safe end ring forgings

• Two thermal sleeve transition ring forgings e One forged large bore elbow e Three 10 feet lengths of various size and schedule pipe The inspectors reviewed the procurement documents associated with the purchase of the ASME Code, safety-related material. Purchase Order (PO) 91-J-73927, Revision 0, was issued on September 10. 1991. The P0 specified one line item,

- . .. . ~. . . =_- -. . - .

_

.

I

.

-4-the safe end ring forging, with approximate dimensions. Additionally, the P0 specified technical and code attributes the material was to meet and required the submission of certified material test reports (CMTR). There were no specified drawings for the vendor to follo Revision 1 to the P0 was issued February 24, 1992. The line items specified by this purchase order included the safe end ring forgings, the thermal sleeve transition ring forgings, three 10-feet lengths of pipe, and a forged elbo The P0 included specified drawings for the two different ring forgings but did not specify drawings for the elbow. Two additional technical requirements were added for heat treating of both ring forgings and material test coupon ,

When the material was received on site, a receipt inspection performed by the 1-licensee and documented in Receipt Inspection Report'(RIR) 92-0522 failed to identify problems with heat creating of a test coupon for one of the two different ring forgings. Subsequent to the licensee's receipt and acceptance of this material, an NRC inspector determined that test coupon heat treating did not meet ASME Sections 11 and III requirement Revision 2 to the P0 was issued to correct the heat treatment requirements for one of the ring forgings and the associated test coupo The licensee subsequently identified that the test coupons that had been used for testing the material from which the safe end forgings had been manufactured did not meet ASME code requirements for test specimen location and size. The licensee contacted the vendor, who in turn was able to obtain proper test coupons from scrap material, and retest the material to meet the ASME Code requirement Inspection of the large bore elbow that was to replace the existing elbow due to a minor indication identified by the licensee subsequent to discovery of the safe end indications was reported in RIR 92-9550. This inspection

"

resulted in the item being placed on hold while documentation problems were addressed, but the item was eventually dispositioned to use-as-is.- Subsequent to this acceptance, a maintenance order was issued to obtain data for the elbow outer radius wall thickness by ultrasonic testing (UT) examinatio This data was to be used for the erosion / corrosion program baseline dat This UT examination revealed that areas of the elbow outer radius wall thickness were less than the design required minimu Revision 3 to the PO was issued to document the return of the elbow to the .

vendor for weld repair of the minimum wall thickness. RIR 92-0729 documented the receipt inspection of the elbow after it had been weld repaired. Due to indeterminate wall thickness quality by UT examination, the elbow was not ,

accepted. The decision _ was made to reinstall the original elbow as indication i of degraded wall thickness was minimal and would be monitored by the existing l licensee program. Revision 4 of the P0 documented the return of the l

'

unacceptable elbow to the vendo After the repair was complete, using the original elbow with a new safe end and thermal transition sleeve, the licensee performed a surveillance and a i thorough investigation to identify all problems associated with the ]

.-

- . . - .

_ _

~

'

!

-5- c

!

procurement action related to the repair. These efforts were successful in identifying the specific causes for the individual errors associated with the '

'

complex procurement activit Condition Report 93-0134 was issued to develop the corrective action for significant findings, which included the recognition that time must be allowed in preplanning to resolve quality issues that aris The inspectors determined that problems associated with this specific *

procurement action had been satisfactorily resolved and, as discussed in [

Section 1.2.2, revisions to the licensee's procurement procedures should '

preclude similar problems in future procurement of ASME Code-related materia .

The inspectors agreed that time must be allowed in preplanning to resolve quality issues that aris l 1.1.3 Conclusions Procurement problems associated with the reactor feedwater safe end and elbow forgings replacement had been satisfactorily resolved. Changes to the i licensee's procurement procedures, together with the recognition that time must be allowed in preplanning to resolve quality issues that arise, should -

preciude further problems in procuring ASME Code-related materia '

1.2 Procurement Control The inspectors reviewed the licensee's administrative procedures for the l control of procurement of safety-related items and the departmental procedures that implement those procurement activities. This review determined that the .

licensee's procedures established basic elements in the procurement documents, i such as: specific identification needs, technical requirements, access to the supplier's plant or records, certification documents, establishment of the requisite quality assurance (QA) program consistent with Appendix B to 10 CFR Part 50, and the applicability of 10 CFR Part 21. Discussions were held with individuals who would be involved in the revision of licensee procedures to reflect the new corporate requirements, policies, and philosophy of the organization that would result after the planned merger. The inspectors observed that the procedures controlling individual department activities did not clearly reflect an integrated program, which is discussed belo . Merger Related Procedure Revisions During the discussions with licensee personnel, it was re'vealed that upper tier Gulf States Utilities procedures had already been reviewed against Entergy Operations, Inc. procedures. The results of this review had identified needed changes to existing procedures in order to implement new operating requirements, policies, and philosophy. The licensee agreed with'

the inspectors' observation that in some cases, the procedure scope would change. The licensee was in the process of reviewing results and drafting revised upper tier procedures to conform with the different operational approac The inspectors asked licensee representatives how a change in scope of an

. upper tier procedure would capture and identify the changes necessary in lower

-- .. . _ - . -

i s l l-6- l l

tier procedures. The initial reply was that the upper tier procedure  ;

reference list would provide the input to those lower tier procedures that '

needed to be change Subsequently, the inspectors were advised that the ;

licensee's procedure for revising and control of procedures, ADM 0005, should l provide this administrative contro I The inspectors reviewed Procedure ADM 0005, " Revision, Control, and Use of Procedures," Revision 19, and held additional discussions with licensee representatives. Sections 6.8.3 and 6.8.11 of ADM 0005 implemented the !

requirement to perform a cross-discipline review. The attributes of this review provided criteria to assure accurate definition of the procedure scop and further assured that the procedure fully addressed the defined scope. The inspectors determined that this process would account for identifying any applicable supporting procedures and assure that the broad scope of the upper l tier procedures is addressed in the lower tier procedures. The inspectors I were informed that ADM 0005 was scheduled to be cancelled and rep' d by one or more procedures; however, the licensee intended to preserve tb requirements of Sections 6.8.3 and 6.8.11 in the replacement procedure (s).

I 1. Procurement-Related Procedures .

During the conduct of the inspection, the inspectors reviewed several of the licensee's procedures that addressed activities within the procurement function. The majority of these procedures'were related to procurement engineering, material handling and storage, and QA/ quality control (QC)

activities. The inspectors observed a lack of consistency in the methods by which procedures were controlled and revised, which could lead to difficulty in understanding the procedures. This~ inconsistency was most visible in the activities re'ated to temporary or interim procedure change * Material handling procedures (MHP) under temporary change were identified by placing the change notice in front of the procedure. The change was reflected in the text by pen and ink change to the affected pages. These procedures were generally easy to use and the purpose of the change was easily understoo l

• QC instructions (QCI) or procedures had change notices placed behind the procedure text and attachments. The affected pages of text were corrected and reprinted to make it appear like a normal procedure, with the changed areas identified. These procedures were easy to use but the-purpose of the change filled in on the front page of the change notice was usually inadequate to identify the purpose of the chang * Engineering department procedures (EDP) inserted the change notice in front of the procedure and the reason for the change was usually understandable. There were no changes or indications made to the changed areas in the text portion of the procedures. This method of implementing changes provided the greatest potential for error during use of a changed procedur . _ __ . . -- ...

_- -. _ ._.. _ __ _ _ _ _ __

.

-7-Also, during discussions with the licensee's staff, it became apparent that-the different disciplines that worked within the procurement function were not aware of each other's rol ,

The inspectors reviewed Procedure EDP-AA-01, " Preparation, Review, Approval, Revision, Cancellation, and Issue of Engineering Department Procedures,"

Revision 11. This procedure was used by engineering to develop engineering i procedures. The procedure was noted to contain provisions for extensive i review and a screening process to assure compliance with 10 CFR Part 50.5 !

During the procurement efforts associated with the reactor vessel safe end repair, discussed previously in this report, procurement engineering established technical and quality requirements in accordance with Procedure EDP-EQ-01, " Technical, Quality, and Documentation Requirements for Procurement Documents," Revision 4. Part of the corrective action to address the '

violation involved changing this procedure to reflect that special considerations were needed to address the procurement of some items. The special considerations adopted, to address the corrective action for the !

violation, was to assure proper testing was specified in the purchase order for ASME Code class forgings and that minimum wall thickness was to be ;

specified for code class forged elbows. However, these changes were never ;

made to EDP-EQ-01. The licensee decided that the procedure was becoming too l'

large and opted to create a new procedur Procedure EDP-EQ-22, " Items With Special Considerations for Procurement, Transfer or Upgrade," Revision 0 was developed in accordance with EDP-AA-0 'i The inspectors reviewed the development package and verified that the procedure had been developed in accordance with the licensee's requirement ;

Taken together, these procedures reflected an acceptable procurement proces Although the inspectors did not identify any weaknesses in Procedure EDP-EQ-22, it was noted that there were no comments from any of the reviewers and the procedure was apparently approved exactly as drafted. The package contained documentation indicating that the required 10 CFR Part 50.59 screening review, had been performed, reviewed, and approve . Conclusions ,

-

,

The licensee's process was adequate to assure that procedure changes required by the merger would maintain the overall scope within all levels of current procedure Licensee procedures used within the procurement function were acceptable, but not well integrated. A significant indication of this ,

condition was the inconsistency in development, revision, review, and approval ,

of procedures used by the different disciplines that worked within the t procurement function. Through review of procedures and interviews, the inspectors acquired the perception that the disciplines. involved in the procurement function operated with a high degree of independenc ,

. .- .

..

.

( -8-

1.3 Material Handling The inspectors toured the licensee's warehousing, receipt inspection, and materials laboratory facilities; interviewed personnel in each area; and reviewed the licensee's administrative procedures for the receipt, storage,

, and handling of safety related equipment and materials. The interviews and l review determined that the licensee had established controls for inspection of safety related material at receipt against the attributes included in the purchase order. Acceptable items were marked for storage or provisions were made for immediate issue if required. Nonconforming items were properly marked, segregated, documented and dispositioned. Organizations affected by nonconforming material were required to be notified and pertinent information was required to be provided to the 0A organization responsible for supplier qualification. Controls for the conditional release of nonconforming items were in effect and required the proper justification, documentation and authority for the release of an item. Storage of safety related items was .

'

controlled with written requirements for levels of storage with appropriate environmental controls. Maintenance and care of items in storage, including an effective shelf life program, were provided in the licensee's administrative control The inspectors also reviewed several receipt inspection reports (RIR) related to specific purchase orders. This review effort included all RIR related to the replacement of the N4A reactor vessel feedwater nozzle safe end and the planned replacement of the first upstream elbo . Receipt Inspection (RI) Program About two years prior to this inspection, the licensee's program required the QA function to invoke RI attributes. The current program required the QC receipt inspector to invoke the RI attributes as interpreted from the official warehouse copy of the purchase order (FO).

The RI activity was implemented by Procedure QCI-3.0, " Receiving Inspection," ;

Revision 12. This procedure was administrative in nature and described the inspection and reporting processes. Receipt inspection planning was addressed !

in Procedure QCI-3.7, " Quality Control. Inspection Planning," Revision I Previously, this procedure had not been used for RI plar.ning, but a temporary change was implemented when it was decided that the QA organization would no ,

longer invoke RI attributes. This temporary change required the receipt inspector to select from Procedure QCIP-0028, " Inspection Plans," Revision 0, the_ proper attributes to perform a receipt inspection. Procedure QCIP-0028 was a generic list of attributes such as cleanliness, workmanship, physical damage, and traceability. Each attribute was coded, had a requirement reference, and contained an inspection description. This document was ;

approved in December 1991. The inspectors determined that there was no !

supervisory oversight of the receipt inspectors' attribute selection activitie A completed RIR was subjected to a peer review, but this review i was administrative in nature and only verified that the RIR was complete. The only opportunity to identify a missed attribute on an inspection was during

, ,

. . - - . .- _ - .-

.

t

'

,

-9- 3

i periodic audits, when inspected material could have been already installed and ,

performing a safety-related functio The receipt inspector was required to review the current revision of the P0,  ;

identify the specifications and requirements, and then extract _ the correct inspection attributes from a generic inspection plan list. The RI process did not specify any baseline attributes for material that was classed QC Level 1, '

ASME Code-related, or other required codes and specifications. The receipt inspector was expected to interpret the technical and quality requirements  :

specified by the P0 and any related drawings, and determine the correct  ;

inspection attributes to assure that technical and quality requirements had  !

been me ,

i The R1 attribute determination process was further complicated by the multiple ,

methods available to procurement engineering personnel to specify quality and >

technical requirements. Procurement specifications were routinely specified - i by engineering using one of the following methods: l

• Mini-Specification  ;

• Product Data Page l

• Technical and Quality Checklist ,

• Detailed Specifications  !

• Boilerplate Notes The inspectors observed that the purchase order for the N4A safe end repair l material had been procured with technical and quality requirements and the i addition of seven special/ additional requirements. Conspicuously missing from !

the first two iterations of the P0 were requirements for heat treating test  ;

coupons and elbow wall thickness. Therefore, the receipt inspector could not

'

have identified the attributes necessary to assure these requirements were met

'

from the information provided in the P ,

1. ssue Control and Post Issue Control of Material  ;

To efficiently issue material in support of modifications and maintenance ,

requires the ability to identify the needed material -in the material control system. The licensee is presently operating with two separate data bases, one 1 is. the Material Management System and the other is the Plant Maintenance i

. System. The differences in the two systems for identifying material had . ;

caused the licensee significant problems in the timely planning and execution  :

of modifications and maintenance. To alleviate this condition, the licensee ]

had established a " Help Desk," staffed by selected personnel with expertise in :

cross-referencing the two information sources and providing the necessary 1 information for identifying and locating needed material or the lack of it within the existing syste The licensee had also established a process referred to as " pledging." This process identified and located material in the various storage locations and then provided for a unique tag that precluded that item from any other use but

. ._ -______ _ _ -

~ -

'

.

.

-10-i what is was " pledged" for. This process should improve the delivery of i material needed to support unscheduled and scheduled plant outage l

!

The inspectors verified that the licensee's processes addressed controlling

'

the issue of Quality Level I material and the post issue control of this '

.

material. The authorization for withdrawal of material from a warehouse'was restricted to individuals that had been approved by management. This process l was enforced by maintaining an authorized signature list at each material :

issue point. The instructions for maintaining the list were found in :

Procedure MHP-15-006, " Issuance or Release of Material From Warehouse,"

Revision 2, Sections 4.2 and The supervisor of warehouse operations was ?

responsible for maintaining the signature list of personnel authorized to t approve spares requisitions. Authorized site supervisors and foremen could i authorize by memoranda other individuals to act in their place, either ,

temporarily or on a permanent basi ?

Maintenance procedures addressed the post issue control of parts by requiring ;

personnel to return unneeded material to the warehouse. Procedure ADM-0023,

" Conduct of Maintenance," Revision 10C, Section 5.6.1 instructed personnel to return the material in accordance with Procedure MHP-15-010, " Return of Material to Warehouse," Revision 1. Procedure MHP-15-010 required the generation of documentation by maintenance and procurement personnel to :

maintain the traceability of material, or to reclassify i '

1. Conclusions The licensee's process for planning and conducting receipt inspections was not well proceduralized. The integration of engineering and QA procedures to enhance the receipt inspection process had not been considered. The process

'

relied heavily on a receipt inspector's experience to determine the correct ,

inspection attributes. This resulted in the need for receipt inspector '

interpretation which was not reviewed before material was committed to the field for installation. The establishment of the pledging process by the

'

licensee is considered a strength in the material control area, since it compensates for the lack of in-plant space for pre-staging outage support ,

material, but still provides for improved material deliver The licensee's '

program addressed the issue control and the post issue control of quality and ;

safety-related materia ,

1.4 Overall Conclusions  !

i The licensee's program for procurement, receipt, storage, and handling of safety-related items was in conformance with regulatory requirements, licensee commitments, and industry guides and standard .

I f

. - , _ . . - - - - - - . - _ - ,

- . - . . . __ .. _ . - -

.

.

,

ATTACHMENT

!

1 PERSONS CONTACTED Licensee Personnel

• R. Biggs, Supervisor, Quality Asiurance Systems-- .

• J. Booker, Assistant to the Vice President, River Bend Nuclear Group

• D. Castleberry, Senior Qu311ty Assurance Engineer

• T. Crouse, Director, Engineering Support Services J. Fisicaro, Manager, Safety Assessment & Quality Verification >

• R. Frayer, Director, Procurement Services & Materials

• K. Giadrosich, Director, Quality Assurance  !

• J. Hamilton, Manger, Engineering

• R. Hebert, Supervisor, Inventory Management

• T. Hoffman, Supervisor, Civil / Structural Design Engineering l K. Kennedy, Supervisor, Records Management

• T. LaRocque, Quality Engineer

• J. Levines, Supervisor, Nuclear Safety Assessment Group '

C. Loebs, Quality Assurance Engineer R. Ludwig, Senior Mechanical Engineer

• Malik, Supervisor, Corrective Action & Reviews

,

J. McGaha, Vice President, River Bend Nuclear Group l

• J. McQuirter, Licensing Engineer

• R. Redmond, Senior Quality Assurance Engineer

• K. Skaggs, Quality Control Inspector i

• M. Small, Senior Quality Engineer  !

B. Smith, Supervisor, Mechanical Maintenance ,

• A. Suni, Supervisor, Equipment Qualification & Specifications l

• M. Stein, Director, Plant Engineering

• K. Suhrke, Manager, Site Support I

• C. Walker, Supervisor Contracts & Procurement Services R. Whitley, Supervisor, Quality Control 1.2 Stone & Webster Personnel

• N. Zink, Engineer 1.3 Cajun Electric Power Personnel

• Curran, Site Representative 1.4 NRC Personnel

• Smith, Senior Resident Inspector In addition to the personnel listed above, the inspectors contacted other personnel during.this inspection perio * Denotes personnel that attended the exit meeting.

.

. , . , , ._, , , - - . . - - r- .-. .- - . .e,.. - , , e - .-

-.. . .- . .- .- . _ - _- .

- - .

.

..-

-2-

.

2 EXIT MEETING An exit meeting was conducted on December 3, 1993. During this meeting, the inspectors reviewed the scope and findings of the report. The licensee did not express a position on the inspection findings documented in this repor The' licensee did not identify as proprietary any information provided to, or- 1 reviewed by, the inspector l l

l l

.

l l

!

l

i

!

'

.

l

,

l l

!

t j

i

!

~

~ -. .. -