ML20059C670
| ML20059C670 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon  |
| Issue date: | 08/19/1993 |
| From: | AFFILIATION NOT ASSIGNED |
| To: | |
| References | |
| OLA-2-I-MFP-083, OLA-2-I-MFP-83, NUDOCS 9401060023 | |
| Download: ML20059C670 (24) | |
Text
r y
jyri cprit"iI QW
~
[g) - 2 76 G 23 -
(%42:, % s.
EL T- /HFP' F NCR DCl-92-MM-N033 Rev. 00 February 24, 1993
'93 J
- 1 C s
MANAGEMENT
SUMMARY
On July 1,
1992 during a preventative maintenance (PM) on Centrifugal Charging pump (CCP) 2-1, several discrepancies were found on the motor hold down bolts.
Subsequent investigation determined that during the original plant equip 2ent procurement, the procurement specification and vendor supplied information did not adequately address the configuration of motor / pump hold-down bolts for skid mounted equipment.
Several corrective actions to prevent recurrence were put in place to include :(1) A hold down bolt hit team has been organized to ir pect the hold down bolts and to track the corrective actions for the components identified for inspection;(2) Existing pump / motor hold down bolts configuration data was compiled into a single engineering document; (3) A mechanism, will be provided to assure that future purchase specifications addresses specific hold down bolt configuration information on rotating equipment;(4) A POA was provided for other components that may have similar configurations like the charging pumps.
This draft, dated February 24, 1993, provides the meeting minutes from the February 24, 1993 TRG meeting.
This TRG reconvened to review the investigative and corrective actions to prevent recurrence results and if possible to sign-off this NCR.
The revised text below also includes new additional information on the investigative and corrcctive actions.
_UlON COMPLFTION OF THIS NCR WRITE-UP, COPIES WILL BE DISTRTBUTED TO THE TRG MEMBERS FOR REVIEW AND SIGN-OFF PRIOR SUBMISSION 10 THE PSRC.
wm o:awaw ca: un Q,jl[. L d <. ~ r9 v ((0>
n- '. e O c lF g uit 5 a g SLj$Z'Uf 5-v 1
- e w E.y I -i. s
..-t/
V i T'
o Ai! m iW1___ ___
h j,
92NCRWP/92MMNO33.PGD Page 1 of 21 9401060023 930819 PDR ADOCK 05000275 C
. \\
NCR DL1-92-MM-NO33 Rev. 00 February 24, 1993 NCR DC1-92-MM-NO33 PROBLEMS WITH HOLD DOWN MOTOR BOLTS ON CENTRIFUGAL CHARGING PUMPS.
j I.
Plant Conditions On July 1, 1992, Unit 1 was in Mode 1 (Power Operation) at 100% power and Unit 2 was in Mode 1 at 100% power.
II.
Rescription of Event A.
Summary:
i During a preventative maintenance (PM) on Centrifugal Charging pump (CCP) 2-1, several discrepancies were found on the motor hold down bolts.
B.
Event
Description:
On July 1, 1992, during a Preventive Maintenance (PM) work, on Centrifugal Chagrging Pump (CCP) 2-1 the following conditions were found on the hold-down bolts:
1.
The bolts were unmarled indicating a low carbon steel, such as ASTM A-307 grade A or B.
2.
The vendor drawing indicated that these bolts should be AAE grade 5 torqued at a higher value (275 vs 125 lb for a 3/4" bolt).
3.
The above conditions were identified by the QC department on AR A0269883.
These bolts were replaced with ASTM A-193 B-7.
Further investigation by QC revealed CCP 1-1 bolts had been previously torqued to 275 ft/lbs in accordance witn Work Order C0073104-01.
In addition to no markings on the head, the washers were deformed indicating a high torque value that 2
would be expected for a low carbon steel bolt.
These conditions are indicated on AR A0270096.
These conditions are identified on AR a0270096.
These bolts were replaced with ASTM A-307 Cgrade B bolts in accordance with Work Order C0101510.
i As a result of these findings, a TRG was convened.
On July 2, 1992, a further investigation revealed that per WO C0073104-01 (AR A0270096) a specific torque of 275 ft/lbs was used to tension the motor hold down bolts on CCP 1-1.
Each bolt had three washers which appeared to be deformed such that the 92NCRWP/92MMN033.PGD Page 2 of 21 3
b NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 center of the holes of the washers were pulled in towards the hole and the outside of the washer bowed upwards.
On July 3, 1992 the motor hold down hole sizes on CCP 2-2 were found to be out of tolerance.
They were
.90" instead of.940"
+.040
.000".
On July 4, 1992 during the replacement of CCP 1-1 motor hold down bolts inder Work Order C101510, the following conditions were noted:
1.
Slotted motor hold down bolt holes.
2.
Bolts at the NE and NW corners were machined down to root diameter for a distance of 1 1/2" from the head, had an effective thread length of 3/4' and a total length of 2 1/2".
3.
All bolts had circular carbon steel (CS) flat washers of 2" OD.
I 4.
As-found torque values ranged from 135 ft-lbs to-155 ft-lbs.
It was determined that less than adequate procurement specification and vendor supplied information regarding configuration of motor / pump hold down bolt connections for skid mounted equipment caused the conditions described in this NCR.
C.
Inoperable Structures, Components, or Systems that Contributed to the Event:
None.
D.
Dates and Approximate Times for Major Occurrences:
1.
July 01, 1992:
Event / Discovery date.
During PM work on CCP 2-1 unacceptable hold down bolting were uncovered and subsequently a Corrective Maintenance (CM) work was initiated.
j 2.
July 02, 1992:
Further investigations were conducted and AR A0270096 was written to identify WO C0073104 in which a specific torque value of 275 FT.lbs was used to tension the motor hold down bolts for CCP 1-1.
92NCRWP/92MMN033.PGD Page 3 of 21
~_
NCR DCl-92-MM-NO33 Rev, 00 February 2",
1993 E.
Other Systems or Secondary Functions Affected:
None.
F.
Method of Discovery:
During PM work on CCP 2-1 unacceptable hold down bolting were uncovered.
H.
Operator Actions:
N/A.
I.
Safety System Responses:
None.
III.
Cause of the Event A.
Immediate Cause:
Hold down bolts on CCP 2-1 were found machined to their root diameter.
Elongated bolt holes and stacked washers were noted.
Investigations revealed that these conditions were standard practices during the plant construction to meet the allowable clearance's criteria and perform the required alignment.
B.
Determination of Cause:
N/A C.
Root Cause:
Less than adequate procurement specification and vendor supplied information regarding configuration of motor / pump hold down bolt connections for skid mounted equipment.
D.
Contributory Causes:
1.
Prior to construction, procurement specifications did not call out these types of detailed such as bolting requirements (i.e. skid mounted equipments were provided to DCPP) a.
Seismic analysis was not probably a manufacturing consideration.
92NCRWP/92MMNO33.PGD Page 4 of 21
NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 2.
During construction / installation, it was a routine field practice to modify hold down bolt configurations for alignment purposes.
3.
Subsequent maintenance practices were not clear / controlled in this area (i.e., hold down bolt configuration / installation).
4.
The configuration of the connection was not clearly defined in engineering documents.
5.
Not implementing Westinghouse technical bulletin 86-07 resulted in a missed opportunity to identify this problem at i
an earlier date.
IV.
Analysis of the Event A.
Safety Analysis:
An initial review performed by NECS Engineering on a sample of components with configuration similar to the Centrifugal Charging Pumps (CCP's) determined that these components, with this condition, had a 20 percent or more stress margin at the worst pump-motor hold down bolt location.
NECS Engineering further assessed that the use of originally installed bolts on slotted holes does not affect the tension capacity, but may reduce the shear strength of the joint by approximately 15% (per AISC). The available margin in the seismic qualification appears to adequately compensate for the shear capacity reduction in the joint.
Therefore, from their initial review, NECS Engineering judged that the operability of the pump / fan motor assembly would not be jeopardized if the above stated deviated conditions were found.
This evaluation assumed that those bolts which have been overtorqued were not broken or fractur-4 The as-found condition of the motor mounting bolts for CCP 1-1 and 1-2 confirm this assumption.
The machining of the bolts for alignment purposes has a very small effect on the tension capacity of the bolts.
Based on the above evaluation, the health and safety of the public have not been affected by this condition.
B.
Reportability:
1.
Reviewed under QAP-15.B and determined to be non-conforming j
in accordance with Section 2.1.5.
2.
Reviewed under 10 CFR 50.72 and 10 CFR 50.73 per NUREG 1022 and determined to be not reportable.
j l
l 92NCRWP/92MHNO33.PGD Page 5 of 21 j
i
NCR DCl-92-MM-NO33 Rev. 00 February 24, 1993 l
3.
This problem will not require a 10 CFR 21 report, since it does not involve defects in vendor-supplied services / spare parts in stock.
4.
This problem will not be reported via an INPO Nuclear Network entry.
5.
Reviewed under 10 CFR 50.9 and determined to be not reportable since this event does not have a significant implication for public health and safety or common defense
)
and security.
6.
Reviewed under the criteria of AP C-29 requiring the issue and approval of an OE and determined that an OE is not required.
V.
Corrective Actions A.
Immediate Corrective Actions:
1.
All four CCP motors have been evaluated and corrective actions implemented to assure that they are properly mounted.
2.
CCP 1-1 and 1-2, ASTM A-193 grade B-7 hold down bolts with fabricated plate washers were installed and correctly torqued in accordance with DCN N-47569.
3.
CCP 2-1, bolts that were confirmed to be ASTM A-307 grade B (per inspection of bolt holes and inspection analyses ) were reinstalled and correctly torqued.
4.
Implementation of DCN N-46154 Rev.
1, ensures that CCP 2-2 has been properly installed and its hold down bolts terqued correctly.
B.
Investigative Actions:
1.
Evaluate the past operability of CCP 1-1 based upon the as-found information (slotted holes, bolt material / configuration, torque).
RESPONSIBILITY:
M.
Khan DEPARTMENT:
NECS Engineering Tracking AR:
A0270096, AE #04 STATUS:
COMPLETE.
92NCRWP/92MMN033.PGD Page 6 of 21
NCR DCl-92-MM-NO33 Rev. 00 February 24, 1993 l
i 2.
Evaluate the past operability of CCP 1-2 based upon the as-found data (bolt hole size, bolt material / configuration, torque).
RESPONSIBILITY: M.
Khan ECD:
07/06/92 DEPARTMENT:
NECS Engineering Tracking AR:
A0270246, AE #01 STATUS:
RETURN.
3.
Plant metallurgist to perform evaluation and document results, to determine material, type, and condition for all eight hold down bolts.
Refer:
AR A0270096, AE 05 and 06.
RESPONSIBILITY:
W.
Barkhuff DEPARTMENT:
Quality Control.
Tracking AR:
A0270629, AE #01 STATUS:
COMPLETE.
4.
Identify the other components that may have questionable hold down configurations.
RESPONSIBILITY:
M.
Khan DEPARTMENT:
NECS Engineering Tracking AR:
A0270629, AE #02 STATUS:
RETURN.
S.
Based on the list of components provided by NECS Engineering, Mechanical Maintenance te develop an inspection plan for those components identified by NECS Engineering.
NECS Engineering, System Engineering, Work Planning Center, and Quality Control to assist in the development of this plan.
RESPONSIBILITY:
C.
Seward DEPARTMENT:
Mechanical Maintenance.
Tracking AR:
A0270629, AE #03 STATUS:
RETURN.
6.
Provide a POA for other components that may have similar configurations like the charging pumps.
In addition, advise if it is warranted to perform inspections on other components during upcoming outages 1R5/2R5.
RESPONSIBILITY:
M.
Khan STATUS: RETURN DEPARTMENT:
NECS Engineering 92NCRWP/92MMNO33.PGD Page 7 of 21
NCR DCl-92-MM-N033 Rev. 00 February N, 1993 Tracking AR:
A0270629, AE # 04 STATUS:
RETURN.
7.
As discussed during the TRG meeting of 10/13/92, evaluate the need to inspect pump and fans base hold down bolts at this time, since the original concern was based on motor hold down bolt configuration as it relate to alignment.
RESPONSIBILITY: M.
Khan STATUS:
RETURN 4
DEPARTMENT:
NECS...gineering Tracking AR:
A0270629, AE # 07 STATUS:
RETURN.
NOTE:
M.
Khan to write in AE # 7 the rationale for other pumps to be scheduled for inspection during 1R6 and 2R6. M.
Khan to write another AR to cover this subject.
C.
Corrective Actions to Prevent Recurrence-1.
Provide an engineering approved document specifying pump / motor hold down bolt configuration for components listed in AR A0270629 AE # 2.
RESPONSIBILITY:
J.
Hodges ECD: 07/01/93 DEPARTMENT:
NECS OPEG Engineering Tracking AR:
A0270629, AE # 08 Outage Related?
No OE Related?
No NRC Commitment?
No CMD Commitment?
No 2.
Compile existing pump / motor hold down bolts configuration data into a single engineering document.
RESPONSIBILITY:
J.
Hodges DEPARTMENT:
NECS OPEG Engineering Tracking AR:
A0270629, AE # 09 Outage Related?
No OE Related?
No NRC Commitment?
No CMD Commitment?
No STATUS:
RETURN.
92NCRWP/92MMN033.PGD Page 8 of 21
NCR DCl-92-MM-N033 Rev, 00 February 24, 1993 3.
Provide a generic DCN as requested in AR A0285214.
This information will provide acceptance criteria for Unit 2 pump / motors hold down bolt inspection.
RESPONSIBILITY:
J Hodges DEPARTMENT:
NECS OPEG Engineering Tracking AR:
A0270629, AE # 10 Outage Related?
No OE Related?
No NRC Commitment?
No CMD Commitment?
No STATUS:
RETURN.
4.
Provide the rational for pump hold down bolt configuration inspection.
Rationale to include pumps sampling plan, acceptance criteria and recommendations based on results of the sample inspection.
Also includes, basis for sampling plan.
RESPONSIBILITY:
M.
Khan DEPARTMENT:
NECS OPEG Engineering Tracking AR:
A0270629, AE # 11 Outage Related?
No j
r OE Related?
No NRC Commitment?
No CMD Commitment?
No STATUS:
RETURN.
5.
Provide a mechanism, i.e. procedure revision, to assure that future purchase specifications addresses specific hold down bolt configuration information on rotating equipment.
RESPONSIBILITY:
J.
Hodges.
ECD- 07/01/93.
DEPARTMENT:
NECS OPEG Engineering (NCFN).
Tracking AR:
A0270629, AE # 12 Outage Related?
No OE Related?
No NRC Commitment?
No CMD Commitment?
No 6.
A hold down bolt hit team has been organized to track corrective actions for the components identified for inspection.
The hit team leader shall compile and maintain this list to assure completion of the inspections of all component identified as a result of this TRG.
92HCRWP/92MMN033.PGD Page 9 of 21
a w
e;
.4 5-G
-nn,-M.m 5
,,M,s Q
u n
i+.-+4 45 e
su i a
ag s
m---4%a 5
t NCR DCl-92-MM-NO33 Rev. 00 February 24, 1993 RESPONSIBILITY:
L.
Price.
ECD: 06/01/93 DEPARTMENT:
Mechanical Maintenance.
Tracking AR:
A0270629, AE / 13 Outage Related?
Yes OE Related?
No NRC Commitment?
No CMD Commitment?
No D.
Prudent Actions (not required for NCR closure)
None.
VI.
Additional Information A.
Failed Components:
None.
B.
Previous Similar Events:
1.
NCR DC1-83-MM-N013
- Mounting bolts on component cooling water pump 1-1 were turned down to a smaller diameter for future motor alignment purposes.
Cause: The cause of this event was due to personnel errors and inaccuracies in the motor casting that prohibited proper motor alignment.
Corrective Action: Training was given to maintenance foremen, mechanical engineers, and technical assistants covering how 4
work is documented for ASME Section XI components.
These corrective action on the above event did not preclude the events leading to the present NCR.
2.
NCR DC1-86-TI-N082 RHR Pump discharge pressure mounting bolts.
These bolts were purchased on PO 4R-72436 which specifies " bolting shall be ASTM A-307" which in turn
'~s s specifies in part, that " bolt heads shall be marked to identify the manufacturer". The bolts used in this installation do not meet this criteria.
Cause: The cause of this event was due to limited inspection instructions and failure to perform adequate inspection to verify material conformance to purchase order requirements.
Supplier failed to supply materials in conformance to purchase order.
Corrective action:
NECS GC will be in conformance with the NPG procurement and dedication procedures by 09/01/86.
As an interim measure to assure that fasteners received by NECS-GC 92NCRWP/92MMNO33.PGD Page 10 of 21
NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 comply with all applicable requirements, NECS GC specifier f
and receipt inspectors shall be directed to specify and l
inspect for manufacturer markings on ASTM fasteners.
The corrective action on the above event did not preclude the events leading to the present NCR because these bolts were j
purchased and received by NECS GC.
In 1986, NECS GC operated under different requirements than DCPP.
Steum Generator mounting bolt. Were i
3.
NCR DC1-86-TI-N083 installed which do not conform to the requirements of installation specification 049238.
These bolts were purchased on PO 4R-72094 which specifies
" bolting shall be ASTM A-307" which in turn specifies in part, that " bolt head shall be marked to identify the manufacturer". The bolts used in this installation do not meet this criteria.
Cause: The cause of this event was due to limited inspection instructions and failure to perform adequate inspection to l
verify material conformance to purchase order requirements.
Supplier failed to supply materials in conformanceoto purchase order.
Corrective action:
NECS GC will be in conformance with'the NPG procurement and. dedication procedures by 09/01/86.
As an interim measure to assure that fasteners received by NECS-GC comply with all applicable requirements, NECS GC specifier and receipt inspectors shall be directed to specify and inspect for manufacturer markings on ASTM fasteners.
Same as above, the corrective action on the above event did not preclude the events leading to the present NCR because these bolts were purchased and received by NECS GC.
In 1986, NECS GC operated under different requirements than DCPP.
Several instances were identified 4.
NCR DCO-86-PG-N090 where bolts did not conform to ASTM A-307 as required by installation specification 049238.
These bolts were purchased on PO 4R-72094 which specifies
" bolting shall be ASTM A-307" which in turn specifies in part, that " bolt heads shall be marked to identify the manufacturer". The bolts used in this installation do not meet this criteria.
Cause: The cause of this event was due to limited inspection instructions and failure to perform adequate inspection to verify material conformance to purchase order requirements.
Supplier failed to supply materials in conformance to purchase order.
92NCRWP/92MMNO33.PGD Page 11 of 21 l
.~.
4 NCR DCl-92-MM-NO33 Rev. 00 February 74, 1993 i
Corrective actions: NECS GC will be in conformance with the NPG procurement and dedication procedures by 09/01/86.
As an interim measure to assure that fasteners received by NECS-GC comply with all applicable requirements, NECS GC specifier and receipt inspectors shall be directed to specify and inspect for manufacturer markings on ASTM fasteners. NECS-GC to initiate a procedure to be approved _by the PSRC regarding the sampling and testing of bolts installed in the plant.
NECS Engineering to finalize the acceptance criteria and specify tests for tl._ inctalled bolts based on the results of testing.
All unmarked bolts requiring marking which are stored in the warehouses and shop storage will be withdrawn from use and dispositioned.
These corrective actions on the above event did not preclude 4
the events leading to the present NCR. In 1986, NECS GC operated under different requirements than DCPP.
Same as above, the corrective action on the above event did not preclude the events leading to the present NCR because these bolts were purchased and received by NECS GC.
In 1986, NECS GC operated under different requirements than DCPP.
Boric Acid Transfer Pump Motor.
5.
NCR DCO-90-EN-N007 During an engineering evaluation of the seismic qualification l
of Boric Acid Transfer Pump motor 1-2, Westinghouse discovered that the motor hold down bolts size used in the analysis was 1/2" as opposed to as-installed bolt size of 3/8".
The non-conformance identified herein is determined to be an isolated occurrence based on the following:
l o
PG&E's audit of Westinghouse seismic qualification work i
has shown that Westinghouse has an acceptable quality l
assurance program regarding sub-supplier's control.
o Since the motor holddown bolts in question typically have significant safety margin, spot checking of these items by the primary vendor (Westinghouse) is not always performed.
O PG&E investigation has concluded that Mcdonald Engineering was involved in the seismic qualification of these pump / motor assemblies only.
92NCRWP/92MMN033.PGD Page 12 of 21
t NCR DCl-92-MM-N033 Rev 00 February 24, 1993 C.
Operating Experience Review:
1.
NPRDS:
Not applicable.
2.
NRC Information Notices, Bulletins, Generic Letters:
IE INFORMATION NOTICE NO. 86-25.
Traceability and material control of material and equipment, particularly fasteners.
The NRC found several instances of lack of traceability and loss of material control on fasteners installed in various nuclear power plants. These fasteners included equipment mounting bolts, nuts, studs and washers installed on safety related equipment.
3.
None.
D.
Proposed Trend Code:
Responsible department EN, and cause code C1.
E.
Corrective Action Tracking:
1.
The tracking action request is A0270629.
2.
Are the corrective actions outage related?
No.
f l
F.
Footnotes and Special Comments:
4 i
None.
G.
References:
1.
Initiating Action Request A0270096.
2.
Tracking AR A0270629.
3.
4.
I 92NCRWP/92MMN033.PGD Page 13 of 21 3.m.
,m.m.-
y
NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 i
H.
TRG Meeting Minutes:
1.
On July 9, 1992, the TRG convened and considered the follow-ing:
a.
Review of the events leading to this NCR. Three AR docu-menting the problems were reviewed. They are as follows:
1.
A0269883 - Preventative Maintenance on CCP 2-1 re-sulted in Corrective daintenance.
QC wrote this AR to document the concern on bolt torquing.
2.
A0270096 -
3.
A0270249 - CCP 1-1 "as-found" conditions.
b.
Discussion on the slotted holes and machined bolt's body.
c.
Review of the evaluations conducted on the hold down bolting.
d.
ISSUES Some hold down bolts have no marking.
3/4" hold down bolts were torqued to 27S ft/lbs.
Hold down bolts holes were slotted.
Hold down bolts body were machined to their root diameter.
e.
CCP 1-1 slotted holes on initial installation had no evaluation and were declared inoperable.
These slotted bolt holes were later evaluated by Engineering on a DCN.
f.
Westinghouse letter (CHRON # 148429), references the bulletin letter which allowed A-307 bolting material and 125 ft/lbs torquing.
g.
Previous NCR on the counterfeit bolts.
h.
Questions to be answered and actions to be taken:
1.
Was anything done to implement the Westinghouse letter?
92NCRWP/92MMNO33.PGD Page 14 of 21 l
l
\\
NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 2.
Is there a need to investigate other safety related motor's hold down bolts?
3.
Need to define what bolt material was used.
4.
Need to review the seismic design.
5.
Need to determine the bolt strength, material, etc..
6.
Need to look at machined bolts.
Did machini49 lower the bolt's strength?
i.
Three problems were identified:
1.
The CCP 1-1 hold down bolts were overtorqued for the material specified by Westinghouse.
2.
The slotted holes are not per vendor analyses and drawing.
3.
These hold down bolts are not per drawing and the bolt bodies are machined.
j.
The TRG held a telephone conference with NECS Engineering to attempt to address the concerns on those three issues and to discuss the operability issues.
k.
Immediate corrective actions:
1.
All four CCP motors have 'been evaluated and correc-tive actions implemented to assure that they are properly mounted.
2.
On CCP 1-1 and 1-2,-
ASTM A-193 grade B-7 hold down bolts were installed with fabricated plate washers, in accordance with DCN N-47569.
3.
On CCP 2-1, the bolts that were confirmed to be ASTM A-307 grade B (per inspection of bolt holes and inspection analyses) were reinstalled.
4.
Implementation of DCN N-46154 Rev.
1, ensures that CCP 2-2 has been properly installed and its hold down bolts torqued correctly.
1.
Investigative actions:
92NCRWP/92MMN033.PGD Page 15 of 21 4
__..,._,,4
..,o.c
t NCR DCl-92-MM-N033 Rev. 00 Februar; 24, 1993 1.
NECS Engineering to evaluate the past operability of CCP 1-1 based upon the as-found information (slotted holes, bolt material / configuration, torque).
2.
NECS Engineering to evaluate the past operability of CCP 1-2 based upon the as-found data (bolt holes size, bolt material / configuration, torque).
3.
Plant metallurgist to perform evaluation and document results, to uetermine material, type, and condition for all eight hold down bolts.
4.
NECS Engineering to identify the other components that may have questionable hold down configurations.
5.
Mechanical Maintenance to develop an inspection plan for those components identified by NECS Engineering.
6.
NECS Engineering to provide a POA for other compo-nents that may have similar configurations like the charging pumps. They need to identify the configura-tion of each of the pumps in the NCR writeup.
THE NEXT TRG WILL RECONVENE ON THE DATE TO BE ESTABLISHED BY THE CHAIRMANz 2.
On August 19, 1992, the TRG reconvened and considered the following:
a.
Review of the AE's on tracking AR A0270629.
b.
The different configurationc of the hold-down bolts.
c.
NECS Engineering to furnish the general criteria for the inspection of the different components using these hold-down bolts.
d.
The inspection should determine (1) if some washers have been utilized in the installation of the hold-down bolts and (2) the condition of the bolts.
e.
The inspection should address if the hold-down bolts are OK.
The results of the inspection was sent to NECS Engineering for evaluation and for contingency if applicable.
NECS Engineering will make an evaluation of the condition of the components.
92NCRWP/92MMN033.PG0 Page 16 of 21
NCR DCl-92-MM-NO33 Rev. 00 February 24, 1993 f.
Subsequently NECS Engineering needs to get together with Mechanical Maintenance to review the results of the inspection.
A generic DCN will be issued.
g.
NECS Engineering needs to specify, in the instances where the bolting material is not specified in the applicable drawing, what actions need to be taken.
I h.
It was agreed that the as-found condition was a result of the alignment process.
i.
In 1985, a potential problem with bolting was found in certain locations.
j.
NECS Engineering will supply Mechanical Maintenance with the bolting specs for bolting material installed in certain locations.
This information should indicate if unmarked bolts are acceptable in certain specified locations.
k.
NECS Engineering will supply Mechanical Engineering with preliminary information on what is acceptable.
TRG TO RECONVENE ON BEPTEMBER 10, 1992 TO REVIEW THE INSPEC-TION PLAN, SIMILAR EVENTS AND ESTABLISH THE IMMEDIATE CAUSE OF THIS EVENT.
3.
On September 9, 1992, the TRG reconvened and considered the following:
Review of NECS preliminary acceptance criteria supplied a.
by NECS Engineering to Mechanical Maintenance. This criteria is required for thc inspection of the hold down
- bolting, b.
Determine the visual inspections could be done without disassembly and finish it off with by removal type in-spections that can be accomplished by existing cms or by recurring tasks.
Several components that could be better done outside of an outage, such as the spent fuel pit pumps.
NECS Engineering was asked to provide the general crite-c.
ria for inspection to Mechanical Maintenance Engineering.
92NCRWP/92MMN033.PGD Page 17 of 21
NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 4
NECS Engineering said they can provide the material criteria and the bolt sizes.
However, they were not able i
to address the minimum thread engagement since the bolts have to be fully engaged. NECS should provide a criteria for minimum thread engagement such as 1 1/2 time bolt diameter.
d.
NECS Engineering was to provide DCPP Mechanical Maintenance Engineering with the necessary acceptance criteria for bolt material, bolt sizes, minimum thread engagement and bolt hole configuration.
The chairman specifically requested that NECS Engineering e.
provide Engineering assistance during the outage for the inspections and documentation of certain as found condi-tions where acceptance criteria have not been provided by them.
NECS Engineers will determine the proper bolting i
material, the correct bolt size, the minimum thread engagement, and the correct hole size, then the bolts will be put back in.
If the bolts do not meet the acceptance criteria, then Engineering will evaluate these conditions.
f.
The torque values will also need to be known before reinstalling the bolts.
The work order will need to de-scribe the work to be done and specify the acceptance criteria.
g.
It was considered to have the work order describe that Engineering will be gathering data for further evaluation rather than providing acceptance criteria for installing plant equipment.
h.
Since this work will be done during the Unit 1 outage.
NECS Engineering would supply one to two persons during that time frame.
i.
It vas suggested that since the bolts were to be removed for inspection, why not replace them on a one to one basis with new ones and prefabricated washers.
Bag and tag the as found bolts and washers for further evaluation by NECS.
j.
A call was made to NECS Engineering SF to request that they supply of minimum of two engineers to perform an inspection of the bolting and determine past operability if required.
I l
92NCRWP/92MMNO33.PGD Page 18 of 21
)
t a-w m,,--_-..
a
---.,,,n
.- - ~-,-
e-
,----,-,,----,m
NCR DCl-92-MM-NO33 Rev. 00 February 24, 1993 1.
OPEG Engineering may assist with one or two persons.
They are 21 safety related pumps that will need some work m.
during the outage.
TRG TO RECONVENE AFTER NECS ENGINEERING HAS ESTABLISHED A FIRM COMMITMENT FOR THEJJ ASSISTANCE IN THE BOLTS INSPECTION. THE TRG WILL RECONVENE TO DETERMINE THE IMMEDIATE CAUSE OF THE EVENT AND REVIEW THE BOLT INSPECTION RESULTS.
4.
On October 13, 1992, the TRG reconvened and considered the following:
Telephone conversation between the TRG and NECS a.
Engineering.
b.
Discussion on the inspection criteria of the bolts.
Continue with the current work with the motor hold down c.
bolts and discontinue inspection of the pumps and fans until NECS complete their evaluation.
f.
There is no operability concern as long as there are no broken bolts.
g.
IE Notic: 86-25 CHRON # 085165 - memo dated 12/29/92 from Gisclon to Grebel was reviewed and discussed as a possible basis for not inspecting the pumps at this time.
h.
NECS Engineering will submit the inspection guidelines to Mechanical Maintenance by October 14, 1992.
1.
NECS Engineering will evaluate the need to inspect pump and fan base hold down bolts at this time, since the original concern was based on motor hold down bolt configuration as it relate to alignment.
See IE notice 86-25 for previous completed bolt inspections on skid mounted components. (Discussed during the 10/13/TRG.)
Reference AE # 07.
TRG TO RECONVENE WHEN NECS ENGINEERING WILL PROVIDE THE REOUIRED INFORMATION.
5.
On December 2, 1992, the TRG reconvened and considered the following:
92NCRWP/92HMNO33.PGD Page 19 of 21
NCR DCl-92-MM-N033 Rev. 00 February 21, 1993 Initiation and review of the contributory causes and the root cause of this event.
It was agreed that during construction the unacceptable practices noted were standard procedures The corrective actions out of this NCR should include a good place where a store the information on hold down bolt connections.
It was suggested that a component database or a component history is needed to collect this information.
This information should be periodically up-dated.
The contributory causes of this event were discussed and established as:
1.
Prior to construction, procurement specifications did not call out these types of detailed such as bolting requirements (i.e. skid mounted equipments were provided to DCPP)
I a.
Seismic analysis was not probably a manufacturing consideration.
2.
During construction / installation, it was a routine field practice to modify hold down bolt configurations for alignment purposes.
3.
Subsequent maintenance practices were not clear / controlled in this area (i.e., uold down bolt configuration / installation).
4.
The configuration of the connection was not clearly defined in engineering documents.
5.
There was a missed opportunity when the West nghouse i
technical bulletin 86-07 was not implemented which resulted in the failure to identify this problem at an earlier date.
Based on a review of the contributory causes it was found that the root cause was due to less than adequate procurement specification and vendor supplied information 92NCRWP/92MMN033.PGD Page 20 of 21 i
. ~ -.
J P
NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 6
regarding configuration of motor / pump hold down bolt connections for skid mounted equipment.
Four corrective actions to prevent recurrence were established.
For more information see paragraph V (C) of this NCR.
As part of the corrective actions NECS Engineering (M.
Khan) is to write in AE # 7 that a sample of three pumps is to be inspected during 2RS.
Also, NECS Engineering is to create another AE, to develop the rationale for inspecting other pumps during 1R6 and 2R6.
TRG TO RECONVENE ON FEBRUARY 15, 1992 TO REVIEW THE INVESTIGATIVE AND CORRECTIVE ACTIONS RESULTS AND IF POSSIBLE SIGN-OFF THE NCR.
NCR CLOSURE DATE: August 1, 1993.
l 5.
On February 24 1993, the TRG reconvened and considered the following:
i Review of the results of the investigative actions.
Established two corrective actions to prevent recurrence as follows:
1.
NECS OPEG to provide a mechanism, i.e. procedure revision, to assure that future purchase specifications address specific hold down bolt configuration information on rotating equipment.
1 2.
A hold down bolt hit team has bec.. organized to track corrective actions for the components identified for inspection.
The hit team leader shall compile and maintain this list to assure completion of the inspections of all component identified as a result of this TRG.
It was decided that the voting members of this TRG will reconvene on March 3, 1993 to review and revise this NCR.
UPON COMPLETION OF THIS NCR WRITE-UP, COPIES WILL BE DISTRIBUTED TO THE TRG MEMBERS FOR REVIEW AND SIGN-OFF PRIOR SUBMISSION TO THE PSRC.
I.
Remarks:
92NCRWP/92MMNO33.PG0 Page 21 of 21
F NCR DCl-92-MM-NO33 Rev. 00 February 24, 1993 None.
J.
Attachment (s):
None.
4 t
i i
'l 92NCRWP/92MMN033.PGD Page 22 of 21 i
4
-.-r
i-NCR DCl-92-MM-N033 Rev. 00 i
February 24, 1993 i
PROBLEMS WITH HOLD DOWN MOTOR BOLTS ON CENTRIFUGAL CHARGING PUMPS.
ROOT CAUSE ANALYSIS.
Page 1 of 2 EFFECT CAUSE EVIDENCE C/A. BARRIER t
Several problems were found on Prior to While An engineering i
the motor hold down bolting of construction, performing-approved document Centrifugal Charging Pump 1-1 procurement work order C-will be issued to i
to include unmarked bolts, specifications did 0101510, used specify the bolts machined down to their not call out to implement pump / motor hold root diameter, washers had details such as DCP N-47569, down bolt elongated holes and were bolting these configuration 1
stacked. Other bolts were requirements for abnormal
{
overtorqued to 275 ft/lbs.
skid equipment.
conditions CC.
were noted.
WHY? In essence, some bolts During A generic DCN will were found unmarked, others construction it be initiated to were machined down to their was routine provide acceptance root diameter.
In certain practice to modify criteria cases, washers had elongated bolt configuration information for holes and were stacked and in the field for Unit 2 pump / motors bolts which were overtorqued alignment hold down bolt to 275 ft/lbs.
purposes.
CC.
i I
4 4
i NCR DCl-92-MM-N033 Rev. 00 February 24, 1993 PROBLEMS WITH HOLD DOWN MOTOR BOLTS ON CENTRIFUGAL CHARGING PUMPS.
j ROOT CAUSE ANALYSIS.
l Page 2 of 2
[
EFFECT CAUSE EVIDENCE C/A. BARRIER l
WHY?
Subsequent maintenance Less than adequate Not implementing practices were not clearly procurement Westinghouse i
controlled in this area (i.e.,
specification and technical bulletin i
hold down bolt vendor supplied resulted in a l
configuration / installation).
information missed opportunity i
CC.
regarding to identify the i
configuration of problem at an of motor / pump hold earlier date. **
i down bolt The rationale for connections for pump held down bolt skid mounted configt ration for equipment.
pumps motors will Root cause.
be established.
A rationale for other 1
ECCS pumps to be j
inspected during l
1R6 and 2R6 will be established. C/A i
CC.
CONTRIBUTORY CAUSE
- BARRIER l
\\
r i
l l
I 1
l i
t b
4+
4