ML17161A012
| ML17161A012 | |
| Person / Time | |
|---|---|
| Issue date: | 12/17/1996 |
| From: | NRC (Affiliation Not Assigned) |
| To: | |
| Shared Package | |
| ML17161A009 | List: |
| References | |
| REF-QA-99901304 99901304-96-01, 99901304-96-1, NUDOCS 9612190394 | |
| Download: ML17161A012 (34) | |
Text
U S.
NUCLEAR REGULATORY CQNISSION OFFICE OF NUCLEAR REACTOR REGULATION Report No.:
99901304/96-01 Organization:
CAD Charte Power Systems.
Incorporated 1400 Union ~eetin" Road Blue Bell, Pennsylvania 19422-0858 Contact Address,:
Terry J.
Kinden, Director of equality Assurance Washington E Cherry Streets Conshohocken, PA 19428 (610) 825-2150 ext.
245 Nuclear Industry Activity:
Class lE n -..'ear power plant station batteries, battery chargers, and seismically qualified station battery racks.
Dates of Inspection:
September 16-18, 1996 9 Attica, Indiana, and October 7-9, 1996 8 Conshohocken, Pennsylvania Inspectors:
Kamalakar R. Naidu, Senior Reactor Engineer Yueh-Li C. Li, Mechanical Engineer Joseph J. Petrosiro, g.A. Specialist Saba N.
- Saba, Electrical Engineer Approved by:
Gregory C. Cwalina, Chief Vendor Inspection Section Special Inspection Branch Division of Inspection and Support Programs Office of Nuclear Reactor Regulation Enclosure 3
CKD is a registered trademark of CAD Charter Power Systems, Inc.
0
1 l}5PECTIN SlNOIARY During this inspection, the NRC inspectors reviewed the implementation of selected portions of CSD Charter Power Systems, Incorporated (CED) quality assurance (gA) program, reviewed activities associated with the manufacture of lead-acid stationary batter' used in Class IE applications, and reviewed pertinent design and engineering documents.
The inspection bases were:
Aopendix B, "guality Assurance C it-.ria for Nuclear Power Plants and Fuel Reprocessing Plants,"
to Part 50 of Title 10 of the Code of Federal
~R1 ti (10 CFR P
50)
~
10 CFR Part 21, "Reporting of Defects and Noncompliance" The American National Standard Institute/The American Society of Mechanical Engineers (ANSI/ASHE) Standard N45,2-1977, "guality Assurance Program Requirements for Nuclear Facilities" (ANSI N45.2-1977),
as endorsed by U.
S. Nuclear Regulatory'Commission Regulatory Guide (RegGuide) 1.28, "guality Assurance Program Requirements (Design and Construction)," Revision 2, February 1979 (RegGuide 1.28)
The team found that CED's quality program for its control of nuclear power plant Class IE station batteries, at the Attica, Indiana facility, contained weaknesses regarding the consistency of assembly and manufacturing process controls as a result of not providing its assembly and machine operators with the required procedures or instructions as required.
The team determined that even though procedures were established for C8D's manufacturing craftsmen, and CED's quality system program required that they be used; the team identified that in many manufacturing areas the craftsman were not aware nor being provided with the required procedures/in~tructions as discussed in Section 3.6.
A correlation to this process control inconsistency.was seen when the team reviewed a
CKD laboratory inspection report that discussed a phenomena regarding prematurely degraded LCY-35 battery cells in Class 1E safety-related systems at the Edwin I. Hatch nuclear plant (Hatch).
The results of the CLD investigation that was delineated in the laboratory inspection report prompted CLD to state that the excess sedimentation suggested that the cause was due to "improper positive plate processing,"
most likely because of "imoroper or incomplete steam curing with resultant weak accive material st. cloture."
The team was informed that CED dedicateP its battery cells and associated components at the Attica facility after all manufacturing and assembly activities were completed.
C5D staff stated that dedication occurred at the time of a successful battery capacity discharge test, performed in accordance with guidance in The Institute of Electrical and Electronics Engineers, Incorporated (IEEE) Standard-450, "Recom.:=nded Practice for Haintenance, As defined in Section 21.3 of 10 CFR Part 21.
I'esting, and Repl;acement of Large Lead Storage Batteries fur Generating Stations and Substations"-1987.
Although Part 21 requires that the dedication process be conducted in accordance with an Appendix B to 10 CFR Part 50 quality assurance
- program, the team found t>>t C&D had not documented or established a battery cell assembly dedication process or program as required by NRC regulations.,
Additionally, since C&D claimed to dedic
'.e their batteries after manufacture, it would be expected that the batteries would be manufactured as commercial grade items (CGIs),
and that all of the ritical characteristics would be verified upon dedication.
- However, even Lhoug,'>
CED characterized its battery cells as being handled as CGIs to the NRC inspection team and NRC licensee
- auditors, as discussed in Section 3.3, it appeared that C8D was taking credit for in-process inspections and verifications during design and manufacturing activities.
The team also noted that C'D's battery cell design for safety-related applications is controlled under C&D's quality system program that was established to meet ANSI N45.2-1977.
As a result of this finding, the team asse'ssed the adequacy of C8D's quality system program manual (QSPH) implementation since it appeared that the adequacy of the Class IE batteries manufactured at Attica received the benefit of being processed in accordance with C&D's documented QSPM controls.
During this inspection, two violations of IO CFR Part 2I were identified and are discussed in Sections 3.2 and 3.3 o'his report, one was a non-cited violation.
Four instances where C8D Charter Power Systems, Incorporated failed to conform to NRC requirements (10 CFR Part 50 Appendix B) imposed upon them by NRC licensees were also identified, The nonconformances are discussed in Sections 3.6, 3.7.and 3.9 of this report.
2 STATUS OF PREVIOUS INSPECTION FINDINGS This was the first NRC inspection of C&D's facilities at Attica, Indiana, and Conshohocken, Pennsylvania.
3 INSPECTION FINOIN6$ ANO OTHER COIINENTS
- 3. I Qgli~t Assurance Pro ram a.
gt~~e The inspection team (team) reviewed the establishment and implementation of selected portions of the C&D quality program, which was documented in the CSD Quality System Program Hanual.
The QSPH stated that its purpose was to establish the basic operating policies and procedures to be eaployed by C&D, and to meet applicabl'e requirements nf International Organization for Standardization (tSO) Standard 9001-1994, ANSI fN]45.2-1977 and.ther imposed specificati.-s.
b.
The team was apprised by C8D that it assures that its components conform to che speci fied requirements through implementation of C80's quality system program that is delineated i" the QSPH.
The QSPH's fo'reword explains that C8D's quality system program is implemented through the
- QSPH, departmental standard operating procedures, work instructions, drawings, bills oi -materials, material specifications, process specifications, and quality and test procedures,Section I, "Hanagement Responsibility," of the QSPM indicates that C8D management has the responsibility and authority to ensure that the policy and object,ives defined in the
- QSPH, and its supporting procedures and instructions are unaer stood, implemented, and m~intained at all levels in the company.
rv ti n n
findin s The team noted that the C8D facilities that are under C8D's quality system program are specified in the QSPH, specifically: (I) Corporate Headquarters-Blue Bell, PA-Control and maintenance of design and qualification aspects of all C8D safety-related components; (2) Attica, IN-Class 1E station batteries; (3) Dunlap, TN-Class JE station battery chargers; (4) Conshohocken, PA-seismically qualified, Class IE station battery racks; (5) Leola, PA-Class IE station batteries, round cell; (6) Conyers, GA-commercial products; (7) Huguenot, NY-commercial products; and (8) Ratelco, Incorporated,
- Seattle, MA-electronics, including battery charger assemblies used in safety-related applications of C8D battery chargers.
The team ascertained that six of the eight facilities manufacture and control components that are destined for safety-related applications at nuclear power plants.
Since six of the eight C8D facilities manufacture components that are used in safety-related applications, the team looked at the overall QSPH program establishment to determine whether a quality system that addresses 10 CFR Part 50 Appendix B requirements such as, ISO-9001 and ANSI N45.2, was in existence and controlled for the eight C8D facilities that supplied components for safety-related applications.
The team observed that C8D's Policy Implementation stated that the C8D Quality Program has been designed to assure compliance with ISO-9001 (1991),
ANSI [N]45.2 (1977),
and CSA-9001.
- However, although the team was informed by C8D staff that each facility complied to ANSI N45.2-1977, the team determined that only 2 of the 8 facilities had been certified to ISO-9001 and nono of the 8 facilities had been certified to CSA-9001.
The team was also informed that all of C8D's Attica facility manufacturing, in-process verification and assembly activities were considered coneercial
- grade, and that C8D dedicated each station battery cell for Class IE safety-related application at the time of successful final testing using guidance conta'wed in IEEE Standard-450-1987.
Based on that position, C80 con~idered Attica s manufacturing and assembly activities that occurred before the IEEE-450 test to be activities that did not fall under an Appendix B
QA program even though 4
ac its
()SPY was established to meet ANSI N45.2-1977 requirements and its certificates of compliance for Class IE batteries attested to compliance with licensee purchase orders (which imposed Appendix B) and C&D's gSPH, fdLSJlli The team observed that C&D's quality system program manual was generally developed to follow the requirements of ANSI-N45.2-1977, which can, if established and implemented in accordance with NRC RegGuide 1.28, "Quality Assurance Requiremen.s (Design and Construction),"
meet the requirements of Appendix B.
The team concluded that C&D's position regarding battery dedication would have beep acceptable had C&D complied with appropriate regulatory and industry guidance regarding dedication.
That guidance requires that the battery cell critical characteristics be identified and their acceptability verified by appropriate inspection,
- tests, or analysis.
In addition, the dedication process must be conducted and controlled in accordance with the applicab1e provisions of 10 CFR Part 50 Appendix B.
The team found that C&D had not established nor implemented a dedication program to comply with NRC regulations or industry guidance.
The team did not find, nor was it provided with any battery ceH dedication procedure documents.
Additiona11y, the team was not provided with any documented basis of C&D's position regarding battery dedication.
- Further, the team determined tnat the "dedication" test was indicative of the battery capacity at the time of the test but did not adequately demonstrate the battery's functionality for its guaranteed life.,
Therefore, the team determined that several manufacturing steps were crucial to demonstrate that the battery would meet purchase order requirements.
Those steps need to be verified by the dedication process or controlled in accordance with Appendix B.
Consequently, the team concluded that this quality assurance program area was not well established or implemented in accordance with 10 CFR Part 21 and 10 CFR Part 50 Appendix 8, and industry guidance.
The team also found that one area of the gSPM, regarding work instructions and procedures, was not well implemented at the Attica f~cility.. The, team's findings in this area are detailed in* Section 3.6.
FR P rt Pro ram
- a. ~m The inspectors evaluated the procedure adopted by C&D to implement the requirements of 10 CFR Part 21 by reviewing Standard Policy and Procedure Number A-I4-4, "Repo.ting of Defects and Nonconformances in accordance with Federal Regula.~on 10 CFR 21 (I ~ Nuclear Regulatory Commission)," dated Harch 18, i996.
4
b.
C.
rv i
n n
fin in s
After discussing the implementation of the Part 21 responsibility in detail with the C8D gA Director, the team informed him that Procedure A-14-4 was inconsistent with the regulation.
The failure to adequately establish the requirements specified in
$ 21.21 constitutes a violation of minor significance and is being treated as a non-cited violation, consistent with Section IV of the NRC Enforcement Hanual (NUREG-1600).
Baseo upon discussions with the i.aD staff the team was concerned regard ng C8D compliance with oiner Part 21 requirements.
Specifically, the team discussed requirements that are contained in 521.21(b) of 10 CFR Part 21 with C&D staff that require a supplier of basic components to inform purchasers or affected licensees within five working days if the supplier is unable to perform an evaluation of a deviation or failure to comply.
The team also informed the C8D gA Director that its procedure did not define or reference 10 CFR Part 21 specific terms such as, discovery and evaluation.
As discussed in Section 3.4 of this report, over 40 of the 120 safety-related station battery cells that C8D manufactured and supplied in late 1993-early 1994 to the Hatch plant exhibited premature degradation'.
The battery was degraded due to excessive sedimentation but continued to be operable according to the licensee.
Therefore, the team reviewed the Hatch event with regard to C&D's responsibility under 10 CFR part 21.
The team noted C&D's June 1996 laboratory inspection report of the cell degradation indicated "an improper or incomplete steam curing."
C&D staff stated that it reviewed its manufacturing
'and quality control (gC) records during that time period and did not identify other potentially affected customers.
C&D further stated that no other customers (comnercial or nuclear) with similar battery cells from that time period have reported similar problems.
The team had no concerns with C8D's efforts regarding the Match issue.
1 sions The team concluded that C&D's procedure'adopted to implement Part 21 was not adequately established and lacked clarity.
Specifically, the team found that the procedure did not address the preparation of an interim report i.
on evaluation of a deviation r r failure to corn-v cannot be completed within 60 days of discovery.
Additionally, the team concluded that C&D was not required to inform t.ne NRC or other customers of the manufacturing process control weakness, as discussed in Sections 3.6 and 3.7, since it determined that no other customers were affected during its review of the circumstances of the problem.
I
~
~
3.3 i
fCl s
St tion atteries a.
$ZdUE The te-m found that the control of the manufacture of C&D's battery cells and associated components could.be viewed as being implemented in one of the following two ways:
( I) performing dedication activ>ties of its Class lE battery cells, as conditionally permitted by 10 CFR Part 2l, in accordance with the applicable requirements of Appendix 8 to 10 C.-R Part 50, or (2) manufac tu :ng its Class 1E battery cells as basic compunents, under its gSPM.
b.
rv tions nd findin s C&D staff informed the team that its station batteries were processed in their manufacturing and assembly areas as commercial grade items (CGIs) and were not consid~.-ed basic components until after each battery cell passed a test (which C&D considers its dedication activity) that C&D performs to meet guidance contained in IEEE-450-1987.
- Although, due to a lack of adequate documentation and conflicting information, the team was not able to determine how C&D Coes, in fact, produce its safety-related batteries (whether by dedication or by manufacture in accordance with Appendix 8), the team discussed C&D's dedication position to determine its compliance with NRC requirements.
The team also reviewed the manufacturing process for compliance with Appendix 8, as discussed elseiIhere in this report.
Additionally, the team noted that the C&D safety-relat'ed battery cells are typically used at licensee facilities for back-up electrical requirements for.instrumentation and control and other power needs to various safety-related
- circuits, nverter power and emergency and normal Class 1E vital bus applications.
Typical critical characteristics of lead-acid battery cells that would be required to be verified include characteristics such as:
ampere-hour capacity; cell voltage; dimensions and configuration; case and component materials, such as paste mixture for positive and negative plates; and grid, plate, or electrode construction and assembly, such as internal plate assembly-to-post
.connections.
Section 21.3 of 10 CFR Part 21 states that dedication is an acceptable process undertaken to provide reasonable assurance that a commercial grade i iem
>ll perform its intended safety function and, in this
- respect, is deemed equivalent to an item designed and manufact'ured under a
10 CFR Part 50, Appendix 8, quality assurance program.
This assurance is achieved by identifying the critical characteristics of the item and verifying their acceptability by inspections,
- tests, or
- analyses, and that the dedication Process must be conducted in accordance with the applicable pr~ isions of Appendix B.
Section 21.3 also svates that a comner".ial grade item means a structure,
- system, or component, or part therefore that affects i'.s safety function, that was not designed and manufactured as a basic component.
g ~
t
<tems
[CGIs] do not include items where the design and manufacturing pro".ess require in-process inspections and verifications to ensure that defects or failures to comply are identified and corrected (i.e.,
one or more critical characteristics of the item cannot be verified).
- Further, Section 21.21 of 10 CFR Part 21 requires a dedicating entity to maintain auditable records
~or the dedication process.
The team determined that C&D: (1) di.'ot establish or have a dedication procedure or program that complied with the applicable provisions of Appendix B, or 10 CFR Part 21, (2) did not identify or verify critical characi.eristics that were necess: r~ to perform an adequate dedication
- process, (3) did not take exceptions to licensee purchase order requirements that imposed Appendix B to 10 CFR Part 50, and (4) indicated on its certificates of compliance that its Class 1E batteries are in compliance with the applicable licensee purchase orders and specifications which the team found to typically impose Appendix B to 10 CFR Part 50 and C&D'SPM.
- Further, the team noted that C&D appeared to take credit for the implementation of manufacturing activities at Attica under its gSPM program For compliance to Appendix B to 10 CFR Part 50, as attested to in its CoCs.
Conversely, C&D appeared to characterize the same manufacturing activities to the NRC inspection team and licensee
- auditors, that occurred before IEEE-450 testing, as being commercial processes not subject to Appendix B.
The team also noted that C&D's dedication position of its battery cell assembly dedication had been found to be acceptable by recent
'censee audit group. inspections.
The team did not find any objective evidence to indicate that C&D was manufacturing its battery cells destined for Class 1E applications as comnercial grade items followed by dedication.
The team concluded that even though Section 21.21 of 10 CFR Part 21 requires that a dedicating entity is responsible for maintaining auditable records for the dedication
- process, and Section 21.3 requires that the dedication process must be conductod in accordance with 10 CFR Part 50 Appendix B, C&D did not have specific records to delineate its battery cell assembly dedication or to substantiate that its activities were carried out in accordance with 10 CFR part 50 Appendix B.
The team identified Violation 99901304/96-01-01 in thi-area.
The team concluded that although C&D stated that it was performing battery cell dedication for customers who imposed Appendix B to 10 CFR Part 50, C&D did not establish a dedication procedure to identify and verify the adequacy of each of the critical characteristics important to ensure that the battery cells would perform satisfactorily in service.
There<'ore C&D did not ensure tha. its battery cell dedication activities affecting quality were prescribed and accomplished in accordance with documented instructions as required by Criterion V,
- i
'Instructions, Procedures, and Drawings," of Appendix B to 10 CFR Part 50.
The team identified Nonconformance 99901304/96-01-04 in this area.
Additionally, the liack of a formalized dedication program to address critical characteristics and the lack of an -effectively implemented Appendix B to 10 CFR Part 50 type of a quality system program could affect the overall life and functionality of C&D's battery cells that are currently in-use at operating nuclear power plant facilities.
3.4 attaiie for Hatch Nuclear P~~er,'1ant
- a. ~ie The inspectors reviewed the circumstances surrounding premature degradation of over 40 of the 120 C&D LCY-35 type battery cells that it manufactured and supplied to Hatch in 1994-1995.
The scope of the team's review included:
(1)
C&D documentation to support the LCY-35 design; (2) llicensee purchase order requirements; (3) C&D's documentation substantiatina CAD's certificate of compliance (CoC) for Match; and (4) the C&D analysis of the failure of the batteries supplied to Match.
b.
Observations and findin s
~B.i i
Ap ii 19 5, I'.,"RC i f d
b iii "9i i" Company (GPC) that excessive
. edimentation was observed in 21 of 127 LCY-35 type batteries supplied by C&D, and by Hay 1996, approximately 43 battery cells exhibited excessive sedimentation.
The batteries were supplied by C&D in 1994 to the Hatch nuclear power plant (Hatch).
C&D committed to the licensee to replace the prematurely degraded battery cells with new battery cells.
e i n Verification The inspectors reviewed the discharge characteristics curves of LCY (35 through 39 plates),
and LCUN-33 (33 plates) type batteries at C&D's facility in Conshohocken, PA.
These batteries are used in switch-gear and control applications at'commercial nuclear power plants and are tested utilizing test procedures meeting the IEEE 450 standards.
The inspectors reviewed the data for a battery with voltage terminal of 125 volts, 1.215 electrolyte specific gravity at :7'F (25'C) and found it a~ ~ptable.
The team reviewed test data of' typical duty cycle of LCY-35 and LCY-37 cells to verify that the cell voltage at the one-minute discharge rate included the effect of the Coup de Fouet'.
The team's review of the test data confirmed that all the one-minute rates did include the Coup de Fouet.
A phenomena of additional battery volta",e drop when the battery is subjected to a discharge after long duration float charges.
The inspectors reviewed the implementation of the engineering change notices (ECKs) and engineering change requests (ECRs).
A typical ECR gave detailed information on the reasons for the change and its impact on manufacturing,
- whereas, a typical ECN referred to the
- ECR, and was sent to different departments, including the manufacturing facilities.
The team did not id~i cify any concerns in thjs area.
The inspectors established that one of the principal differences between Class 1E and commercial grade cells is the separator.
In Class 1E
- cells, ChD uses a rubber separate.,
- whereas, in the commercial battery cells CED uses a microporous polye.hylene separator.
The separator app"ared to be the only component difference between the commercial and Class lE battery cells.
The installation of the rubber separator is an example of a manufacturing activity that requires inspection or verification dur'ing dedication or control by Appendix B quality program.
H t h Purchase Order The inspectors revi'ewed GPC's PO 60135700000, dated December 1,
- 1994, to C8D for the supply of LCR-25 and LCY-35 125/250 Vdc, lead calcium, 120 cell batteries and accessories for Hatch.
The PO inc'luded the following requirements:
~
Applicable portions of 10 CFR 50, Appendix B, apply I
Items shall be supplied ne~
and not used or refurbished in any way
~
10 CFR 21 applies to safety related nuclear items/services
~
CoC certifying the material/components meet the requirements of Specification NP-93024, Revision 2, Section 3.A.4.
Certificate's of Com liance CoC The inspectors reviewed CKD's CoC dated February 25, 1994, for Hatcl and determined that it certified that 127 LCY-35 wet cells with accessories were brand new and had been manufactured in accordance with the requirements of specification NP-
- 93024, Revision 2 and the quality standards specified therein, and that it met IEEE-450-1987 capacity acceptance test.
The discharge rate was 459 amperes for a discharge time of 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> and the acceptance volts per cell was 1.75 average.
The Capacity Discharge test sheets were attached to the CoC.
These test sheets indicated that the average volts per cell exceeded 1.75V at the end of the test..
The inspectors noted that CAD issued a
CoC certifying that the batteries met all the GPC requirements implying that CLD's gA program complied with 10 CFR Part 50, Appendix B, even though CSD was aware that its instructions and procedures affecting the quality of the components supplied were not comprehensive enough in some areas.
For example, procedures and instructions that did not adequately address all of the Attica facility work activities, as discussed in Section 3.6.
A review of a sample population nf other NRC licensee purchase order packages at the Attica facility determined that licensees are procuring Class 1E battery cells and accessories on POs which typically impose 10
'ppendix B to 10 CFR Part 50, 10 CFR Part 21 and the nuclear related quality program that is delineated in C&D's gSPM (ANSI-N45.2-1977),
All of the C&D CoCs. reviewed in the different PO packages stated compliance with the applicable licensee PO and 'icensee specification.
Additionally, the CoCs contained other compliance statements as
- required, such as, "all batteries/cells are manufactured in accordance
'with the [licensee]
PO requirements, C&D's quality system program policy
- manual, meeting the applicable requirements of 10 CFR 50, Appendix 8, ANSI N45.2-1977, and other applicable industry standards (e.g.,
IEEE 450-1987, and IEEE 535-1986)."
Iii the PO packages
- reviewed, the inspectors observed that C&D die', nut take any exceptions to any of the licensee contractual requirements or additional requirements imposed on
- them, nor did any of the CoCs or gSPM discuss or address a dedication basis as discussed in Section 3.3 of this report.
rvi ecti R~ort C&D examined the excessive sedimentation in the Hatch batteries.
According to C&D the excessive
- sediment, which looks like sponge deposits, appeared to be the result of active plate material being shed due to contamination or impurities introduced into the plate material at the factory.
The batteries were considered operable based on their tested performance to deliver over I16X of rated capacity.
On June 25, 1996, after inspecting the batteries, C&D transmitted its findings and included C&D's battery laboratory inspection report.
The inspectors reviewed the report which was based on C&D's analysis of one battery cell from Hatch, stated:
~
The individual cell voltage was more than 2.29 V per cell.
~
The positive and negative grids were in good condition.
~
The positive plate active surface material was depleted (shed) with approximately 50-100X of the plate surface being affected.
~
The negative active material was soft.
~
There was no mossing present either on sides of the frames or the top.
No sulfaration was evident.
~
The sediment chamber was filled with shed active material that had oxidized after making contact with the feet on the negative plate.
~
The glass mat was saturated at the bottom.
~
The plate lug strap burns were good.
~
Corrosion was normal on the positive post and strap assembly.
The report also stated Hatch's LCY. 35 battery suffered from sedimenr,ation, due to improper pos..ive plate processing, and the condition of the positive plate active material suggested improper or incomplete steam curing with resultant weak active material structure.
.a
During discussions with C&D personnel, the team was informed that C&D is currently in the process of considering additional design enhancements and process control changes to contr"'. the curing ovens more effectively for uniform curing of the positive plates.
-The team concluded inat C&D's laboratory report results and the team's observations of weaknesses in C&D's process control suggests that other Class IE batteries that C&D prov',ded in this same time frame may also prematurely degrade.
Therefore, this aspect was discussed with the C&D guality Assurance Director regard>ng whether C&D should be considering informing its applicable customers of a potential deviation that needs to be monitored and evaluated by certain licensees.
- However, as discussed in Section 3.2, C&D investigated this issue and concluded that it was not applicable to other licensee battery cells.
In general, the inspectors observed that the C&D adequately controlled the design of the batteries through ECNs and
- ECRs, including technical and engineering notices.
The inspectors determined that C&D's CoC implied that CED's gA program complied with.10 CFR Part 50, Appendix B, even though C&D was aware that its instructions and procedures affecting the quality of the components supplied did not meet the requirements of 10 CFR Part 50 Appendix B or ANSI N45.2-1977.
3Property "ANSI code" (as page type) with input value "ANSI N45.2-1977.</br></br>3" contains invalid characters or is incomplete and therefore can cause unexpected results during a query or annotation process..5 M
ement Audits The inspectors reviewed the implementation of the gA program relative to management audits'he plant manager performed audits to satisfy the requirements of the gA Manual.
Paragraph 1.5 of Section 1.0 of the C&D gSPM requires executive management to review the quality system program at defined intervals in accordance with Management Review procedures'his review is intended to ensure the continuing suitability and effectiveness of the established guality Management System and the Company's stated quality policies and objectives.
As an integral part of the process, confirmation as to preventive actions and results of audits are to be submitted.
Standard Operating Procedure (SOP)01-022.2
'outlines the requirements to implement'the above.
Paragraph 6.3 requires the plant manager to conduct a quality system program.review on a quarterly basis one month prior "o the end of the fiscal quarter or sooner as indicated by necessity or by di< ~
F corporate guality counsel.
The inspectors reviewed the plant manager's audits performed on December 31,
- 1995, and March 29,
- 1996, and determined that the audits covered the following: the need to develop addi'ional process instructions and work instructions, the need to revis~ training for hourly employees, enhancements in process improvei,'nt programs ana quality improvement
- programs, and status of corrective actions to a previous external audit.
Copies of the audit were distributed to the "Corporate guality Counsel"'2
0
~ 4
which consists of: the President/CEO, the Director of gA, the VP of
- Finance, Power Com., Motive, Technology, Power Com Operations, and the Director of Motive Products Services.
The team noted that the audit recognized the necessity to develop procedures to control the
'manufacturing process as required by C&D's quality system program.
3.6 M n f cturin Process Control a.
~Sco e
The team observed the process cuntrols used by C8D in the manufacture of their batteries and related components.
The team found that C&D's goal was to have all of its facilities certified to ISO-9001.
The main document which establishes the basic operating policies and procedures empioyed to meet the requirements of ISO-9001 is the gSPM.
The gSPM states that the C8D quality system program is implemented through the use of operating procedures, work instructions, drawings, quality and test procedures, etc.
At the time of the inspection, the primary documents used to control manufacturing activities were Product Process Specifications (PPSs!
-rd Factory Procedures (FPs).
b.
Observations and Findin s
Ade c
of rocedure and instructions The team found that in approximately 1993-1994, when the current C&D gA Director assumed his current position, he recognized that the existing PPSs and FPs that had been established for mang of
- he C&D facilities would not support ISO-9001 certification.
The team was told that the PPSs and FPs were written to encompass multiple facilities and, as a result, were
- not, specific to the battery manufacturing process control.
Therefore, C8D started a program to establish specific instructions and procedures, for each particular work activity at each of its facilities to replace the existing PPSs and FPs.
The replacement documents for the PPSs and FPs are process instructions (PIs) and operating/work instructions (OIs).
The team was informed that the PIs and OIs for the Leola facility had been completed but C&D had just begun to develop PIs and OIs for its Attica and Dunlap facilities.
Th-team reviewed some of C80's draft OIs and PIs for the Attica fa~ility and conducted discussions with C8D process engineering per sonnel who are responsible fur develop',.,~
e documents.
The team iict~d that the new OIs and PIs appear comprehensive and easy to use, and the team was informed that the new work instruction documents have the benefit of receiving input from the personnel that are performing the work activity.
The gA Director stated that C8D expects to complete the PIs and OIs at Attica and Dunlap in the Fall of 1998.
Use r
e n
i t ti.ng Discussions with manufacturing personnel revealed that the personnel were experienced and knowledgeable regarding their respective areas.
The team found that this knowledge was apparently obtained from comprehensive on-the-job training (OJT) l3
programs.
During the discussions, the team asked what procedures and instructions were used.
The team found that craft personnel typically were not aware of specific procedures or work instructions that controlled the activities that they performed, with the exception of those working in the paste mixing area.
Section 3, "Responsibilities and Authorities," of C8D's QA Procedure 01-003.6, "Document and Data Control (Corporate),"
sLates that department managers are responsible for generating, approving, distributing and ma'.)t ining work instructions fur 'heir department.
Additionally,
'd part...ent managers and superv>>ors are responsible to assure that the correct documents are available at the locations where needed and that they are being used".
The team determined that C&O's quality system program of procedures and instructions was not adequately established to ensure effective and consistent manufacturing processes control.
That determination is supported by C80's effort to replace all of its PPSs and FPs.
The team concluded that this weakness was not controlled in accordance with corrective action requirements of ANSI-N45.2-1977, 10 CFR Part 50 Appendix B or C&0's
- QSPH, which has been established to'address ANSI N45.2-1977.
Although this matter could have been cited as a
nonconformance being contrary to licensee requirements imposed on C80, the team did not identify this matter as a noncompliance to 10 CFR Part 50 Appendix B because the QA Director had identified the problem and had initiated corrective action.
3.7 The team found that C8D has been aware of the practice of not providing instructions and procedures and has been attempting to correct the problem by establishing specific Cis and PIs for each of its facilities since approximately 1992-1993.
- However, the team noted that until this activity is complete, C&D's Attica activities are contrary to Criterion V, "Instructions, Procedures, and Drawings," of Appendix B to 10 CFR Part 50, and Section 6, "Instructions, Procedures and Drawings," of ANSI-N45.2-1977, regarding implementation of QSPH.
Nonconformance 99901304/96-01-02 was identified in this area.
rv ti n of Q rk A tivities a 0 IRRHCt
'he team inspected several areas at the Attica battery cell and associated product manufacturing and assembly facility including: paste mixing, paste application, curing, hydrosetting and drying, cell
- assembly, and leak testing.
The team observed the manufacturing process control activities that were being performed and conducted discussions wi;h crafts personnel.
A. discus' in Section 3. 1, C80's quality 14
,E4
~
~
I'ystem program has been established to encompass all activities affecting product conformance to specified requirements through its system of oIIerating procedures, work instructions, drawings, bills of material, specifications, and quality and test procedures.
b.
Observations and Findin s
~yii th t
p t
h:iidthp f ttgp t f application on the grids.
The paste operate~
formulated the paste acccrding to two tables observed t~ be in a
~&D procedure; one tab1e was for ;hi positive p1ate and the o;n~,
tabie was for the negative plate.
These tab1es provided information on the components to be used, the materials specification, the quantities and tolerances, and the acceptance criteria for the resulting paste.
The team noted that the paste mixing area was the only area at the Attica manufacturing facility where the craftsmen were observed to have a procedure or work instruction availabl for the craftsmen.
I
~fti P
T pttdt th g
d hyg yft f
hopper to the grid's pasting operations where it is applied to the positive and negative grids.
Finished grids are visually inspected for non-uniform phsting.
Grids which are rejected during the initial inspection are run a second time through the process and if the non-uniformity persists, the grid is rejected and scrapped.
Following
- pasting, positive plates are sent to the curing area while negative platets go directly to hydrosettina.
The inspectors observed the pasting of positive plates.
During this
- period, a few plates which C&D personnel observed to be irregu1ar were put through the process a second time.
The p'roduction rate of this specific run appeared to operate smoothly with few slow-downs and stops during the period observed.
After the pasting was completed, the plates were moved to the curing process area.
gain In this area, the positive plates were subjected to a humidity curing process (also known as steam curing).
Curing is performed to obtain mechanical
- strength, grid paste
- adhesion, porosity and proper lead oxide crystal structure.
The plates were cured in an oven required to be maintained at certain humidity and temperature levels specified by a chart.
The insp~it-s observed that the only parameters of the curing that are recorded are the oven temperature and humidity.
An alarm has not been installed to announce any failure of the heating or the humidity sources.
The inspectors also observed that the access to the oven door~
were not mechanically controlled.
The inspectors observed this as a
weakness in CKD's curing process because plate curing is one of the critical and important stages in tl e manufacture of the lead acid batteries.
The aim of the curing p ocess is.to convert the wet paste to dry, crack-free material ~ith suffiiient strength and adhesion to the grid plate.
Improperly cured plates can affect the performance and longevity of the battery cells.
15
0 an
'epending on the type of the lead oxide used in the manufacture of positive plates, they are either dried immediately or subjected to a hydroset process befn~o. drying.
All negative plates are hydroset before drying.
Hydroset curing is performed at high humidity and relative low temperature to assure the required grid adhesion, mechanical strength and porosity.
During the hydrosettir. process, the free lead content'of the active material*is reduced to specific levels depending on the application of the battery.
All types of plates are subjected to a drying process at 215 i 5'F.
The final moisture content depends on the type of plates.
The team noted
. during several times of observing this area that the overhead doors are periodically left open without any obvious reconciliation or verification of the humidity and temperature levels.
fJIlli<< ii ii i i i;
i i -i >>
ii iiit'attery cell configurations (such as 4XTL13 and KCR9, that are used in safety-related applications).
The batteries were assembled in sequence including: (1) stacking, (2) plate-to-strap
- bonding, (3) element insertion into the container, (4) cover installation.
(5) terminal-connector post-to-cover insert bonding, and (6) leak testing.
The team noted that PPS-X4-3, "Standby Cell Assembly," Harch 28,
- 1991, described the assembly operation and emphasized proper alignment of
'plates and separators.
The team c '.ermined these aspects were critical to the appearance, performance, and life of the battery cell.
The element assembly consisted of two distinct procedures:
stacking and plate to strap bonding.
A stacked element consists of alternating negative and positive plates, interleaved with separators.
After the plates were stacked so that the lugs are aligned, the positive and negative burning lugs were on the opposite sides.
The plate was then bonded to the strap and visually inspected for voids, porosity, entrapped
- dross, lead run downs, and poor bonding.
(At this stage the assembly is cleaned if necessary,)
After acceptance, the operator raised the element into the vertical position and inserted support combs and moss shields.
The element fitted snugly into the battery container.
The cover was then glued on the container.
The team i')served that after the glue was inserted onto the mating surface of the cover at the cover-to-jar mating area, the cover was placed on the jar, and tapped into place with a mallet.
The craft-person then placed a wooden frame over the cover,, and placed a banding type of nylon strap with a ratchet tightening device around the entire jar/cover assembly.
The craft-person hand-tightened the ratchet until the strap appeared to be snug.
Th: cell was allowed to sit at that work station fn~ some undetermined amoui t of time
'.o allow the glue to dry.
16
( i P
The team observed that the above activity was contrary to the-requirements of PPS-X4-3..
The procedural requirement that supposedly contro',led this area (Section 5,3.4 of PPS-X4-3),
stated:
" Immediately following cover installation, the cell/unit shall be held in place for a minimum of 30 minutes with a pressure equal to 80-100 psig.
The cover shall be held to the jar in a way to eliminate movement and to distribute the pressure uniformly over the entire sealing area."
As a result of comparing the a tual work practice and the procedural requirement, the team noted that: (I) although the banding strap appeared to be firmly tighteneu, th~ craftsmen did not have any indication of the actual pressure that was applied each time that the operation was performed, and (2) it was not obvious whether C&D's wooden frame actually "distributed the pressure uniformly."
The inspectors discussed the process with the craft personnel and determined the craftsmen were not aware of any requirement for a specific pressure to be applied nor were they aware of a procedure that they were supposed to be using for that area of the assembly line.
The team asked C&D management
- bout the applied pressure requirement and was informed that the requirement had been modified in practice, but it was left in Procedure PPS-X4-3 because the procedure was scheduled 'to be replaced with the new Ols and PIs.
The team asked C&D engineering if it had approved the deletion of the requirement because it appeared to the team that it was a design requirement affecting the operability and service life of the battery.
C&D gA staff informed the team that C&D engineering had approved the r moval of the specified pressure requirement.
- However, the team was subsequently told at the Conshohocken facility that engineering had not approved the discontinuance of the PPS specified activity.
The C&D Chief Engineer in the battery area stated that the work activity step was a
C&D engineering requirement and that no engineering approval had permitted the requirement to be discontinued.
The team concluded that C&D management failed to ensure that craft personnel were supplied the procedure that provided the specific design requirement.
That failure resulted in craft personnel implementing a
change to the manufacturing process which, in turn, resulted in an unreviewed change to the design requirements.
The team informed C&D that this was contrary to Criterion III, "Design Control," of Appendix B
to i0 CFR Part 50, which requ>r~s that design changes be subject to ue~>gn control measures commensurate with '"" " 'i-inal design and approved by the organization that performed the original design.
The inspectors were unable to determine how long this practice was prevalent.
Nonconformance 99901304/96-01-05 was identified in this area.
Logk '
of 1
The tear.
~oted that C&D procedures require operators to perform leak tests
~n all cells usi,ig a specific pressure for several seconds to confirm that the container will not leak.
The team also noted that final formation of the cells, the electrolyte 17
g
fillinq of cells, charging currents and duration of the different charges are also discussed in CKD procedures.
The team found these procedures were in the plant supervisor's office.
Overall, during discussions with the craft personnel, several of the craft personnel
.admix<ed that they were not-aware of any specific work instructions or procedures for their specific job function, but relied instead on the OJT programs and co-worker experience and assistance. Generally, the team noted several weaknesses regarding C80's cor,tr~l of their manufacturing process.
Primary among those was the unreviewed design change to the baiiery assembly requirement.
1'he inspectors were also concerned with the lack of control for the curing oven temperature and humidity, including lack of process alarms and lack of control for the oven doors.
In addition, the lack of control of manufacturing area overhead doors during certain manufacturing steps could also affect required humidity and temperature levels.
c.
Conc1usions The team concluded that craft personnel were knowledgeable and appeared to be experienced in their job.
Based upon discussions with craft personnel, it was apparent that knowledge and experience was due primarily to OJT.
The failure to use work procedures or instructions is discussed in Section 3.6.
One ronconformance was identified in this area.
3.8 Ca cit Dischar e Tests The team observed capacity discharge tests in progress on a short duration and long duration battery to confirm that battery cell capacities met the design specifications.
A short duration battery is designed to supply relatively high currents for a short period of time.
These type of batteries, used in UPS applications at nuclear plants, are manufactured as off-the-shelf items.
A long duration battery is designed to supply relatively smal1 loads over a much longer period of time, typically eight hours, Long duration batteries are manufactured for a specific order."
The team was informed that C8D monitors individual cell voltages during the discharge test to identify potentially defective cells.
-mi ut~ ",sch r e rate of a short d ration batter The
+earn noted that the test was successfully performed for the full 15 minutes.
The team observed that the first low cell voltage alarm occurred at 15 minutes and 40 seconds.
The test was terminated at
- 17. 10'minutes froid the start indicating a capacity of over ll4X.
r i ch r e r t f a ion duration b tt r During observation of the test, the team noted that, when the test monitor indicated a low voltage on a cell, the test operat(
was able to identify the faulty cell using a hand-held voltmeter and confirmed that it was a bad wire 18
connection.
The connection was repaired and the voltage on the cell returned to normal.
The test continued thereafter without interruption.
The test was not completed until after the inspection.
The inspectors noted that C&D had adequate instrumentation to monitor the individual-battery cell voltages
-and the discharge currents-during the tests.
C&D routinely performed these tests to establish a benchmark rating of the battery before shipman>>t.
The inspectors had no concerns in this area.
3.9 S
SH'C UA IFICATION R PORT a.
~S~e To assess C&D's seismic qualification basis for its LCR-29 and LCY-35 batteries installed at Hatch, the team reviewed the seismic qualification portior.s of C&D's Report gR-66171-01, "Environmental and Seismic gualification Report of TYPE LCR-29 and LCY-35 Battery for Hatch Nuclear Plant," dated January 22, 1993.
The team also reviewed Wyle Laboratories, Seismic Simulation Test Reports 43450-1, "Seismic Simulation Test Program on a Battery Rack and Batteries,"
December 7,
- 1976, and 44467-1, "Seismic Simulation Test Program on a Battery Rack Containing Two LC-25 Battery Cells and a Battery Rack Containing A
3DCU-5 and 3DCU-7 Battery Cells," Harch 1979.
b.
bservations and findin s
The performance requirement specified in the report for the batteries is that the battery shall be capable of supplying the design loads for the required durations, without the voltage at the terminals falling below 210 volts.
The battery shall be capable of supplying design loads while experiencing any single or: combination of the following normal and design service conditions:
Temperature-77'F annual
- average, 65'F
- minimum, 110'F maximum; Pressure-atmospheric; Radiation-less than 10
- rads, 40 year total integrated duse; and Seismic-OBE
.& DBE Seismic Spectra for turbine building elevation 116'-0."
The seismic qualification for the 125/250 Vdc lead-acid storage battery assembly for the LCR-29 and LCY-35 batteries at Hatch plant was provided in accordance with the requirements of Georgia Power Company Purchase Order 6010129, dated December 16,
- 1992, as well as the requirements of IEEE Sta<<aard 344-1975.
The basis for qualification was by combination of test and analysis.
Several seismic qualification tests were previously performed for various "L" type battery cells at Hyle Laboratories.
The tests were performed in accordance with IEEE 344-1975 and the results were found to be well documented.
The seismic s:mulation test included five Operating Base Earthquake (OBE) leve', tests p ior to one Design Base Earthquake (DBE) test.
In addition to the seismic s>mul tion test, pre-seismic and post-seismic battery discharge capacity test" were also.performed.
'l
The seismic qualification of the Hatch LCR-29 and LCY-35 was demonstrated by similarity to those previously successfully tested battery assemblies'with the. same de~ign and configuration.
A detailed description of the similarity evaluation was found and reviewed.
The evaluation included comparison of the battery design with emphasis on
. material, dimension~,--weights, aging mechanism, battery service conditions, and service life.
The comparison that was reviewed indicated that the design of the LCR-29 and LCY-35 battery assembly was equa) or better than the originally qualified battery assembly.
The evaluation also indicated toad the se)smic acceleration experienced by the specimen assembly enveloped the seismic response spectrum.
Therefore, the loads experienced by the tested specimen enveloped the design loads for the plant Hatch batteries.
The evaluation reviewed by the team was based on the well-designed LCR-29 and LCY-35 battery assemblies and its data base did not include battery cells with excess amounts of sedimentation similar to the existing LCY-35 battery at Hatch nuclear plant.
Therefore, the staff's inspection findings and conclusions are not applicable to those existing batteries at Hatch.
The team observed that C&0 did not have a documented procedure for performing the seismic qualification by similarity.
The team informed C&0 management that Criterion V of Appendix 8 to 10 CFR Part 50, requires safety-related activities, such as the preparation of the Seismic
- Report, be performed in accord >nce with written procedures.
The team emphasized that in order to assure consistency and quality of seismic qualification reports, it is necessary to have a documented procedure which includes appropriate quantitative or qualitative acceptance criteria for determining that important technical aspects concerning the'imilarity have been satisfied.
The team also noted that C&0 was not required by Hatch to perform any specific seismic simulation test with C&0's Hatch battery to ensure that the existing Hatch battery will remain functional during and after a
seismic event.
cl sions The inspectors concluded that r&0 has documented an adequate technical basis for the seismic qualification for the LCR-29 and LCY-35 battery
- assembly, and no concerns in this area were ioeniified.
The team concluded that because CED did not have a formal dedication program to verify battery cell critical characteristics it was unclear as to how CED had ensured that it has maintained its seismically qualified configurations during it manufacturing process control.
20
",0 0
The team apprised CLD that failure to have a written procedure for performing teismic analysis was contrary to Criterion Y of Appendix B to 10 CFR Part'50.
Nonconformance 99901304/96-01-03 was identified in this area.
3.33
~PR NE CONTACTED C8D Charter Power S stems Inc.
Attica Indiana
+
C. Brown, Plant Hanager
+
- 8. OeSutter, General Supervisor, Casting
+
B. Donavan, General Supervisor, West Side
+
M. Foster, General Supervisor, Pasting
+
H. Guthrie, guality Control Manger R. Keller, Process Engineer
+
M. Lucas, Haterial Control I
CAD Charter Power S'tems Inc.
Conshohocken Penns lvania D, Meimer, Manager, Design a ".".cumentation 1'+
T. Kinden, Oit ector, equal;ty t'-~;urance E. Urbanski,
- Hanager, Process Engineering F.
- Wagner, Chief Engineer A.
Milliamson,
- Manager, Product Test Laboratories
- +
C.
Mood, Coordinator Supplier Ouality CSD Ch rter Power S stems Inc.
Bl~e Bell Penns lvania G. Walker, Manager, Applications Engineering
+ - Attendance at entrance meeting on 9/16/96 Attendance at exit meeting on 9/18/96
~
Attendance at entrance meeting on 10/7/96 Attendance at exit meeting on 10/9/96 21