ML20244A755: Difference between revisions
StriderTol (talk | contribs) (StriderTol Bot insert) |
StriderTol (talk | contribs) (StriderTol Bot change) |
||
Line 1: | Line 1: | ||
{{Adams | |||
| number = ML20244A755 | |||
| issue date = 04/03/1989 | |||
| title = Insp Rept 50-382/89-09 on 890308-09.Violations Noted.Major Areas Inspected:Licensee Action on Previous Insp Findings Associated W/Testing,Low Recirculation Flow Condition & Corrective Maint of HPSI Pump B | |||
| author name = Chamberlain D, Howell A | |||
| author affiliation = NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) | |||
| addressee name = | |||
| addressee affiliation = | |||
| docket = 05000382 | |||
| license number = | |||
| contact person = | |||
| document report number = 50-382-89-09, 50-382-89-9, IEIN-89-008, IEIN-89-8, NUDOCS 8904180178 | |||
| package number = ML20244A753 | |||
| document type = INSPECTION REPORT, NRC-GENERATED, INSPECTION REPORT, UTILITY, TEXT-INSPECTION & AUDIT & I&E CIRCULARS | |||
| page count = 8 | |||
}} | |||
See also: [[see also::IR 05000382/1989009]] | |||
=Text= | |||
{{#Wiki_filter:n -- | |||
". . | |||
b | |||
g- 't | |||
. APPENDIX ~ | |||
' | |||
' | |||
U.'S. NUCLEAR REGULATORY. COMMISSION | |||
REGION'IV | |||
NRC' Inspection Report: 50-382/89-09 Operating License: NPF-38- | |||
3: | |||
Docket: 50-382 | |||
' | |||
Licensee: Louisiana Power & Light Company (LP&L) | |||
317 Baronne, Street | |||
P.O.' Box 60340- | |||
New Orleans, Louisiana 70160 | |||
< | |||
Facility Name:' Waterford Steam Electric Station, Unit 3 | |||
Inspection At: 'Taft, Louisiana- | |||
Inspection Conducted: Marc 8-9, 1 89 | |||
. Inspector: - | |||
N _ [ _ | |||
, III, Pro 5eH Engineer, Reactor | |||
h8 | |||
Date | |||
, A. T. Howe | |||
. Project Section A | |||
Approved: M'8-8Y | |||
D. DV Chamberlain, Chief', Reactor Project Date | |||
Section A | |||
Inspection Summary | |||
Inspection Conducted March 8-9, 1989 (Report 50-382/89-09) | |||
Areas Inspected: Special, announced inspection of licensee action on previous | |||
inspection findings associated with the testing, low recirculation flow | |||
condition, and the corrective maintenance of the "B" high pressure safety | |||
injection (HPSI) pump. | |||
Results: Potential violations involving two examples of failure to test the | |||
"B" HPSI pump in accordance with some of the provisions of Article 3000 of | |||
Section XI of ASME Boiler and Pressure Vessel Code were identified. These two | |||
examples are apparent violations of Technical Specification (TS) 4.0.5.a. Also | |||
a potential violation of failure to properly implement a procedure during | |||
corrective maintenance of the "B" HPSI pump motor was identified. | |||
These potential violations are indicative of three broad problem areas. First, | |||
the licensee relies almost exclusively on surveillance procedure acceptance | |||
criteria to demonstrate component / system operability rather than emphasizing | |||
890418017e s9040g_ | |||
PDR ADOCK 050003B2 | |||
G PNV | |||
- _ _ _ _ _ _ _ - - - _ _ _ - - - - - _ _ _ _ | |||
_ _ - _ - _ - - _ - _ _ - _ _ _ _ | |||
. . | |||
I . .. | |||
2 | |||
. | |||
\ | |||
sound engineering judgement when evaluating components and systems for operability. | |||
Second, the failure to find and correct the cause of low "B" HPSI pump recirculation | |||
flow is indicative of continuing problems in the area of corrective action. | |||
Third, the heavy reliance on job specific work instructions and vendor technical | |||
manuals rather than component specific maintenance procedures (where appropriate | |||
to the circumstances) increases the likelihood of personnel error during the | |||
maintenance process. This could adversely impact the operability and reliability | |||
.of safetyrrelated. components and systems. | |||
. | |||
I | |||
1 | |||
i | |||
I | |||
_ - - - _ _ _ - - - _ . - . - - _ ) | |||
.- _ _ . ._ | |||
__ | |||
. | |||
1 | |||
... E. .f | |||
1 | |||
' | |||
3 , | |||
i | |||
. | |||
) | |||
DETAILS | |||
1.. -Persons Contacted l | |||
l | |||
Principal Licensee Employees | |||
*D. F. Packer, Assistant Plant' Manager, Operations and Maintenance | |||
*P. V. Pransankumer, Assistant Plant Manager, Technical Support | |||
J. J. Zabritski, Manager, Nuclear Quality Assurance (Acting) | |||
'*G. M. Davis, Manager, Events Analysis Reporting and Responses | |||
*L. W. Laughlin,' Site Licensing Supervisor | |||
*D. E. Marpe, Lead Maintenance Engineer | |||
*W. E. Day, Trending, Compliance and Response Supervisor | |||
*P. N. Backes, Assistant to Plant Manager | |||
R. B. Hereford, Systems Engineer | |||
*G. F. Koehler, Quality Assurance Supervisor | |||
A. M. Ciluffa, Maintenance Engineer | |||
D.' A. Shultz, Assistant Operations Superintendent | |||
B. G. Morrison, Licensing Engineer | |||
*Present at exit interview | |||
2. Followup of Previously Identified Items (92701) | |||
! | |||
a. (Closed) Unresolved Item 382/8901-02 - Failure to Perform | |||
Surveillance Testing of the "B" HPSI Pump in Accordance With | |||
TS 4.0.5.a and 4.0.3 ; | |||
HRC. Inspection Report 50-382/89-01 documented that the licensee had | |||
requested relief from Article IWP-3000 of the ASME Boiler and | |||
Pressure Vessel Code, Section XI (IWP-3100), that requires' that the | |||
resistance of the system shall be varied until either the measured | |||
differential pressure or the measured flow rate equals the | |||
corresponding reference value. Specifically, the licensee stated | |||
that test flow was approximately the same for each test because a | |||
flow restricting orifice was installed in the "B" HPSI pump test | |||
line. The licensee proposed that an alternate measurement of ' | |||
. differential pressure would be compared to the allowable ranges in | |||
lieu of the flow rate comparison. | |||
This written relief request was denied by the Office of Nuclear | |||
Reactor Regulation (NRR) in a May 20, 1988, letter to LP&L. The NRC | |||
inspector reviewed this letter and noted that the letter implicitly | |||
acknowledged that the flow of the system could not be varied, per . | |||
IWP-3100, because of the flow restricting orifice; however, | |||
differential pressure as well as flow could be measured and compared | |||
to allowable ranges. Specifically, the letter required the recording | |||
of all pararoeters in Table IWP-3100-2 in order to determine pump | |||
operability. With respect to flow, Table IWP-3100-2 requires the | |||
measuring of Alert and Required Action Ranges. In order to establish | |||
_ __ _ | |||
. . | |||
. i | |||
4 ! | |||
* i | |||
. | |||
Alert and Required Action Ranges, a reference value for flow (Qr) | |||
must first be established in accordance with IWP-3110. | |||
Discussions with licensee personnel revealed that not all of them | |||
were aware of the requirements of the May 20, 1988, letter. They | |||
noted that they were, in fact, measuring pump flow per Table IWP-3100-1 ; | |||
and that IWP-3100 only required varying the resistance of the system ' | |||
until either the measured differential pressure or the measured | |||
flow-rate equals the corresponding reference values. They'noted that | |||
this was being performed for differential pressure. | |||
The NRC inspector informed the licensee that flow had to be recorded | |||
in accordance with Table IWP-3100-2 as well as Table IWP-3100-1. | |||
Additionally, while IWP-3100 requires varying the resistance of the | |||
system until either measured differential pressure or the measured | |||
flow rate equals the corresponding reference value, it further | |||
requires that the test quantities in Table IWP-3100-1 be measured or | |||
observed, and that any deviations determined shall be compared with | |||
the limits given in Table IWP-3100-2 and the specified corrective i | |||
action taken. The NRC inspector noted that even though flow was | |||
measured per Table IWP-3100-1, it was not being measured per | |||
Table IWP-3100-2 since the denial of the relief request on May 20, | |||
1988. | |||
; | |||
'Further discussions.with licensee personnel revealed that LP&L | |||
responded on July 8, 1988, to the May 20, 1988, letter from NRR. In | |||
this letter, LP&L reiterated its request not to vary the resistance | |||
of the system since'the system design prevents varying the flow rate. i | |||
However, LP&L noted in the letter that for each surveillance test | |||
"the flow rate, differential pressure and vibration, are measured and i | |||
' | |||
compared to reference values." The NRC inspector noted that flow was | |||
not being compared to reference values during surveillance testing of | |||
the "B" HPSI pump or the two other HPSI pumps and the two low | |||
pressure safety injection (LPSI) pumps. In the February 7,1989, | |||
transmittal of the Safety Evaluation of the Waterford 3 inservice | |||
testing program for pumps and valves through Revision 5, NRR again | |||
reiterated the requirement to " measure and record all of the | |||
parameters in Table IWP-3100-2 in order to determine pump | |||
operability." | |||
The NRC inspector noted two instances, since the issuance of the | |||
July 8,1988, LP&L letter to the NRC, of "B" HPSI pump surveillance | |||
testing in which flow data was not recorded in accordance with | |||
Table IWP-3100-2. These tests occurred on August 30, 1988, and | |||
November 22, 1988. Failure to perform surveillance testing in | |||
accordance with the Article IWP-3000 standard is an apparent | |||
violation of TS 4.0.5.a. TS 4.0.5.a requires, in part, that | |||
Surveillance Requirements for inservice testing of ASME Code Class 1, | |||
2, and 3 components shall be applicable as follows: inservice | |||
testing of ASME Code Class 1, 2, and 3 pumps shall be performed in | |||
- _ _ | |||
n . | |||
. | |||
>- | |||
. | |||
, , | |||
E , , | |||
,,'' ~4 . | |||
' | |||
. . | |||
. | |||
A | |||
'' ' | |||
e'' | |||
t, - | |||
5- | |||
- , | |||
e | |||
accor' dance with= Section XI of the ASME- Boiler and Pressure Vessel .' | |||
, | |||
: Code, except where specific written relief has been granted by the' | |||
- Commission. | |||
i; b. .(Closed)' Unresolved Item 382/8901-03 - Failure to Take Prom)t | |||
- Corrective Action Following the Identification of Low "B" H)SI | |||
~ | |||
Pump Recirculation Flow | |||
- A'n NRC maintenance' team inspection-(NRC Inspection Report 50-382/89-01) | |||
noted that a low recirculation flow condition.for the "B" HPSI pump - | |||
was discovered by the licensee during a routine surveillance test of- | |||
the "B" HPSI pump on November 22, 1988. :The recorded value of- | |||
recirculation' flow was 19.0 gpm. . Although no acceptance criteria | |||
- | |||
was listed on-the' surveillance test data sheet, the vendor technical | |||
manual recommended a minimum recirculation. flow'of 25 gpm. . During the | |||
second week of the ~NRC maintenance team inspection of Waterford 3 | |||
(January 31' through February 3,1989) the team noted that the cause of | |||
the recirculation flow condition had not been determined'or. corrected. | |||
The team further noted et that time the "B" HPSI pump had not been- | |||
- declared-inoperable as a ~ result of the low recirculation flow | |||
condition. | |||
As a result of NRC concerns about'the operability of the "B" HPSI- | |||
- pump,,the licensee performed a 1 hour and 15 minute. surveillance test | |||
of -the pump on- January 31,_1989. During this test, the I&C | |||
technician observed that the flow orifice associated with Flow | |||
Instrument SI-IFI-7121 in the return line to the refueling water | |||
storage pool (RWSP) had apparently been installed backwards during' | |||
~ | |||
maintenance in December 1987. 'The orifice was'placed in the proper | |||
. | |||
position and the pump was retested. Recirculation flow indication | |||
increased approximately 4.5 gpm to a value of 24.5 gpm. | |||
The N'lC inspector' concluded that 'the reversed. orifice could not be | |||
the sole cause of the low recirculation flow condition since the | |||
"B" HPSI pump recirculation flow rate 'was still. several gpm less than | |||
the average flow that was recorded prior to. December 1987. Additionally, | |||
flow ~had increased from 20 gpm in' December 1987 to an average rate of | |||
25.5 gpm during the next 3 surveillance tests but.had decreased to | |||
19.0 gpm on November 22, 1988. The reversed flow orifice does not | |||
account for these apparent flow anomalies. | |||
The ."B" HPSI pump.was declared inoperable on February 1,.1989, due to | |||
high ' axial vibration. On February 2,1989, the licensee initiated | |||
Work Authorization (WA) 01031752.to determine the cause of excessive | |||
wear and to repair the "B" HPSI pump inboard and outboard thrust | |||
bearings. | |||
Discussions with licensee personnel revealed that the accelerated | |||
thrust bearing failure was caused by improper balance drum clearance. | |||
One of the purposes of the pump balance drum is to absorb axial pump | |||
forces during low flow conditions (particularly when the pump is | |||
! | |||
< | |||
w____---.___.___--___ ___ __ -- - _ . _ . _ . - _ _ - - . . _ _ _ . . | |||
- - _ _ _ _ | |||
. . | |||
. | |||
, | |||
i | |||
6 | |||
. | |||
. | |||
operating in the recirculation mode). Improper balance drum- | |||
clearance resulted in the pump's thrust bearings absorbing a | |||
disproportionate amount of the pump's axial forces during low flow | |||
conditions. This resulted in an accelerated deterioration of the | |||
thrust bearings. The balance drum clearance was apparently | |||
improperly set following replacement of the "B" HPSI pump rotor in | |||
April 1986. The licensee attributed the improper clearance to | |||
inadequate guidance in the vendor technical manual. A review of _ ; | |||
Ingersoll-Rand HPSI Instruction Manual 457000272, revealed that the | |||
balance drum clearance settings specified were correct, but that the | |||
level of detail for properly setting the balance drum clearance was | |||
lacking. The NRC inspector concluded that this aspect of the repair | |||
activity could not be considered within the skill of craft for | |||
maintenance personnel. The NRC inspector further concluded that this | |||
improper repair due to inadequate technical manual guidance could | |||
have been avoided if the licensee had developed a detailed corrective | |||
maintenance procedure for the HPSI pumps. This is discussed in | |||
further detail in paragraph 2.c of this report. | |||
The licensee returned the "B" HPSI pump to service following repair | |||
of the pump and motor bearings. The NRC inspector noted, however, | |||
that recirculation flow was still significantly less than the | |||
pre-December 1987 values and less than the 25 gpm minimum specified | |||
in the vendor technical manual. The "B" HPSI pump recirculation pump i | |||
flow was recorded as 22 gpm during a February 24, 1989, surveillance | |||
test of the pump. This flow was evaluated as acceptable by the | |||
licensee. The NRC inspector made the following observations: | |||
(1) In a' minimum flow evaluation report, "High Pressure Safety | |||
Injection Pump: Ingersoll-Rand 4x9c-9, Motor Driven," | |||
(January 27,1989) performed by the Ingersoll-Rand Company at | |||
the request of LP&L (which was collecting data for NRC | |||
Bulletin 88-04, " Potential Safety-Related Pump Loss"), the | |||
minimum flow for pump starting / stopping was 25 gpm. | |||
Ingersoll-Rand defined pump starting / stopping as pump operation | |||
of 15 minutes or less. For pump operation greater than , | |||
15 minutes, Ingersoll-Rand specified significantly higher | |||
minimum flow requirements. | |||
(2) In the same report, Ingersoll-Rand also stated that "in order to | |||
realize the pump life projected in this evaluation, periodic | |||
maintenance must be provided. As a minimum, the pump must be j | |||
' | |||
operated in accordance with the guidelines and recommended ' | |||
maintenance schedules as presented in the Ingersoll-Rand Pump | |||
Instruction Manual and good industry practice." The NRC | |||
inspector noted that one of the guidelines in the Pump | |||
Instruction Manual was to keep the recirculation line open when | |||
starting and stopping and at capacities less than 25 gpm. | |||
(3) The licensee primarily based its "B" HPSI pump operability | |||
determination on verbal information provided by the | |||
- - - - _ _ - - _ _ | |||
l | |||
- | |||
. . | |||
:. . | |||
7 | |||
. | |||
' | |||
. | |||
Ingersoll-Rand Company. The basis for this determination was | |||
that the pump parameters (noise, vibration, and temperature | |||
readings) all remained within acceptable limits during a 1 hour | |||
and 15 minute run of the pump in recirculation mode. One of the | |||
primary considerations was the stabilization of pump | |||
temperatures (including bearing temperatures) taken at various | |||
. | |||
l | |||
! locations on or near the pump. The NRC inspector noted, | |||
however, that in NRC Not. ice 89-08, " Pump Damage Caused by | |||
Low-Flow Operation," early component degradation from low | |||
recirculation flow may not be detected by routine inservice | |||
i tests. Furthermore, a review of the licensee's Inservice Test | |||
l Program, Revision 6, revealed that LP&L was requesting relief | |||
from the requirement to measure bearing temperature annually. | |||
If this exemption were granted, the licensee would not have the | |||
benefit of valuable operating temperature information in | |||
assisting in the determination of pump operability. | |||
(4) The cause of the low recirculation flow for the "B" HPSI pump | |||
has not been determined and corrected. The licensee plans to | |||
investigate the cause of the low flow condition once it obtains | |||
a highly accurate ultra sonic flow measuring device that is | |||
expected to be available in late March or early April. The NRC | |||
inspector noted, however, that had a reference value for the | |||
"B" HPSI pump been established on the basis of flow data | |||
obtained prior to July 8,1988, it appears that the '19.0 gpm | |||
value recorded on November 22, 1988, would have been lower than | |||
the " Low Value Required Action Range" limit. The NRC inspector | |||
further noted that IWP-3230(b) requires that if deviations fall | |||
within the " Required Action Range" of Table IWP-3100-2, the pump | |||
shall be declared inoperablo and not returned to service until | |||
the cause of.the deviation has been determined and the condition | |||
corrected. Discussions with licensee personnel revealed that | |||
they believed that their analysis of the low recirculation flow | |||
condition met the requirement of IWP-3230(c) which states, in | |||
part, that " correction shall be either replacement or | |||
repair . . . or shall be an analysis to demonstrate that the | |||
condition does not impair pump operability and that the pump | |||
will still fulfill its function." However, the NRC inspector | |||
noted that the licensee had not yet determined the cause of the | |||
deviation. | |||
Failure to determine the cause of the low flow condition prior to | |||
returning the "B" HPSI pump to service is a second exeinple of failure | |||
to meet the Surveillance Requirements for the "B" HPSI pump in | |||
accordance with TS 4.0.5.a. | |||
c. NRC Inspection Report 50-382/89-06 - Repair of the "B" HPSI Pump and j | |||
Pump Motor | |||
As previously noted in paragraph 2.b, the HPSI Pump Instruction | |||
Manual provided insufficient guidance for setting HPSI pump balance | |||
i | |||
_ _ _ _ _ _ _ _ _ _ _ _ | |||
_ _- . | |||
, | |||
' ' | |||
' | |||
. . . , | |||
y.. > | |||
a | |||
8 | |||
. | |||
. | |||
drum clearance. ! Because of the licensee's heavy reliance on the use | |||
.of work instructions and vendor manuals in lieu of Plant _ Safety | |||
Committee (PSC) reviewed and approved component specific corrective | |||
maintenance procedures,' the NRC inspector was concerned. that future | |||
instances of inadequate maintenance on safety-related. equipment was | |||
likely to occur., This concern was also documented in NRC Inspection | |||
Report 50-382/89-01. | |||
NRC Inspection Report 50-382/89-06 documented an instance in which | |||
the "B" HPSI pump motor oil slinger ring was improperly installed and | |||
that the bearing antirotation pins were not transferred from the old | |||
bearings to the newly-installed bearings. Two uncoupled motor runs | |||
were attempted.on February 20, 1989, but the motor had to.be secured | |||
because.cf excessive vibration. Upon removal of the bearing covers, | |||
bearing damage was observed. The damage was attributed to the. | |||
improper installation of the oil slinger ring. | |||
A review of Event Summary Report-(ESR) 89-003, revealed that a | |||
contributing cause of the "B" HPSI pump motor bearing damage was the | |||
omission of the checks for proper oil slinger ring operation which- | |||
are in the vendor instruction manual, but was not included in the | |||
work instructions of WA 01032220. Furthermore ESR 89-03 documented | |||
that the motor technical manual was.not the latest revision and did | |||
not provide guidance for the movement of the antirotation pins from | |||
the.old bearing to the new bearing. As noted in NRC Inspection | |||
Report 50-382/89-06, failure to install the antirotation pins would | |||
cause motor failure, but may not be detected during normal | |||
surveillance testing. | |||
Step 5.2.7 of Administrative Procedure MD-1-014, Revision 2. " Conduct | |||
of Maintenance," . requires, in part, that "if, during the performance | |||
of any activity requiring the use of vendor technical manuals, | |||
deviations fromithe recommendations or procedures in the_ manuals are | |||
necessary, such deviations shall receive a documented technical | |||
review by Maintenance Engineering and be a | |||
MAS [ maintenance assistant superintendent]pproved by the | |||
." The NRC inspector noted cogniza | |||
that the omission of the checks for proper oil slinger _ ring operation | |||
from WA 01032220 was not reviewed by Maintenance Engineering and | |||
approved by.the Electrical MAS. This is an apparent violation of | |||
MD-1-014. | |||
3. Exit Interview | |||
l | |||
The inspection scope and findings were summarized on March 9, 1989, with | |||
those persons indicated in paragraph 1 above. The licensee acknowledged | |||
the NRC inspector's findings. The licensee did not identify as | |||
i | |||
proprietary any of the material provided to, or reviewed by, the NRC | |||
; | |||
I inspector during this' inspection. | |||
! | |||
, | |||
- | |||
w. . | |||
, | |||
}} |
Latest revision as of 04:49, 17 February 2021
ML20244A755 | |
Person / Time | |
---|---|
Site: | Waterford ![]() |
Issue date: | 04/03/1989 |
From: | Chamberlain D, Howell A NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
To: | |
Shared Package | |
ML20244A753 | List: |
References | |
50-382-89-09, 50-382-89-9, IEIN-89-008, IEIN-89-8, NUDOCS 8904180178 | |
Download: ML20244A755 (8) | |
See also: IR 05000382/1989009
Text
n --
". .
b
g- 't
. APPENDIX ~
'
'
U.'S. NUCLEAR REGULATORY. COMMISSION
REGION'IV
NRC' Inspection Report: 50-382/89-09 Operating License: NPF-38-
3:
Docket: 50-382
'
Licensee: Louisiana Power & Light Company (LP&L)
317 Baronne, Street
P.O.' Box 60340-
New Orleans, Louisiana 70160
<
Facility Name:' Waterford Steam Electric Station, Unit 3
Inspection At: 'Taft, Louisiana-
Inspection Conducted: Marc 8-9, 1 89
. Inspector: -
N _ [ _
, III, Pro 5eH Engineer, Reactor
h8
Date
, A. T. Howe
. Project Section A
Approved: M'8-8Y
D. DV Chamberlain, Chief', Reactor Project Date
Section A
Inspection Summary
Inspection Conducted March 8-9, 1989 (Report 50-382/89-09)
Areas Inspected: Special, announced inspection of licensee action on previous
inspection findings associated with the testing, low recirculation flow
condition, and the corrective maintenance of the "B" high pressure safety
injection (HPSI) pump.
Results: Potential violations involving two examples of failure to test the
"B" HPSI pump in accordance with some of the provisions of Article 3000 of
Section XI of ASME Boiler and Pressure Vessel Code were identified. These two
examples are apparent violations of Technical Specification (TS) 4.0.5.a. Also
a potential violation of failure to properly implement a procedure during
corrective maintenance of the "B" HPSI pump motor was identified.
These potential violations are indicative of three broad problem areas. First,
the licensee relies almost exclusively on surveillance procedure acceptance
criteria to demonstrate component / system operability rather than emphasizing
890418017e s9040g_
PDR ADOCK 050003B2
G PNV
- _ _ _ _ _ _ _ - - - _ _ _ - - - - - _ _ _ _
_ _ - _ - _ - - _ - _ _ - _ _ _ _
. .
I . ..
2
.
\
sound engineering judgement when evaluating components and systems for operability.
Second, the failure to find and correct the cause of low "B" HPSI pump recirculation
flow is indicative of continuing problems in the area of corrective action.
Third, the heavy reliance on job specific work instructions and vendor technical
manuals rather than component specific maintenance procedures (where appropriate
to the circumstances) increases the likelihood of personnel error during the
maintenance process. This could adversely impact the operability and reliability
.of safetyrrelated. components and systems.
.
I
1
i
I
_ - - - _ _ _ - - - _ . - . - - _ )
.- _ _ . ._
__
.
1
... E. .f
1
'
3 ,
i
.
)
DETAILS
1.. -Persons Contacted l
l
Principal Licensee Employees
- D. F. Packer, Assistant Plant' Manager, Operations and Maintenance
- P. V. Pransankumer, Assistant Plant Manager, Technical Support
J. J. Zabritski, Manager, Nuclear Quality Assurance (Acting)
'*G. M. Davis, Manager, Events Analysis Reporting and Responses
- L. W. Laughlin,' Site Licensing Supervisor
- D. E. Marpe, Lead Maintenance Engineer
- W. E. Day, Trending, Compliance and Response Supervisor
- P. N. Backes, Assistant to Plant Manager
R. B. Hereford, Systems Engineer
- G. F. Koehler, Quality Assurance Supervisor
A. M. Ciluffa, Maintenance Engineer
D.' A. Shultz, Assistant Operations Superintendent
B. G. Morrison, Licensing Engineer
- Present at exit interview
2. Followup of Previously Identified Items (92701)
!
a. (Closed) Unresolved Item 382/8901-02 - Failure to Perform
Surveillance Testing of the "B" HPSI Pump in Accordance With
TS 4.0.5.a and 4.0.3 ;
HRC. Inspection Report 50-382/89-01 documented that the licensee had
requested relief from Article IWP-3000 of the ASME Boiler and
Pressure Vessel Code,Section XI (IWP-3100), that requires' that the
resistance of the system shall be varied until either the measured
differential pressure or the measured flow rate equals the
corresponding reference value. Specifically, the licensee stated
that test flow was approximately the same for each test because a
flow restricting orifice was installed in the "B" HPSI pump test
line. The licensee proposed that an alternate measurement of '
. differential pressure would be compared to the allowable ranges in
lieu of the flow rate comparison.
This written relief request was denied by the Office of Nuclear
Reactor Regulation (NRR) in a May 20, 1988, letter to LP&L. The NRC
inspector reviewed this letter and noted that the letter implicitly
acknowledged that the flow of the system could not be varied, per .
IWP-3100, because of the flow restricting orifice; however,
differential pressure as well as flow could be measured and compared
to allowable ranges. Specifically, the letter required the recording
of all pararoeters in Table IWP-3100-2 in order to determine pump
operability. With respect to flow, Table IWP-3100-2 requires the
measuring of Alert and Required Action Ranges. In order to establish
_ __ _
. .
. i
4 !
- i
.
Alert and Required Action Ranges, a reference value for flow (Qr)
must first be established in accordance with IWP-3110.
Discussions with licensee personnel revealed that not all of them
were aware of the requirements of the May 20, 1988, letter. They
noted that they were, in fact, measuring pump flow per Table IWP-3100-1 ;
and that IWP-3100 only required varying the resistance of the system '
until either the measured differential pressure or the measured
flow-rate equals the corresponding reference values. They'noted that
this was being performed for differential pressure.
The NRC inspector informed the licensee that flow had to be recorded
in accordance with Table IWP-3100-2 as well as Table IWP-3100-1.
Additionally, while IWP-3100 requires varying the resistance of the
system until either measured differential pressure or the measured
flow rate equals the corresponding reference value, it further
requires that the test quantities in Table IWP-3100-1 be measured or
observed, and that any deviations determined shall be compared with
the limits given in Table IWP-3100-2 and the specified corrective i
action taken. The NRC inspector noted that even though flow was
measured per Table IWP-3100-1, it was not being measured per
Table IWP-3100-2 since the denial of the relief request on May 20,
1988.
'Further discussions.with licensee personnel revealed that LP&L
responded on July 8, 1988, to the May 20, 1988, letter from NRR. In
this letter, LP&L reiterated its request not to vary the resistance
of the system since'the system design prevents varying the flow rate. i
However, LP&L noted in the letter that for each surveillance test
"the flow rate, differential pressure and vibration, are measured and i
'
compared to reference values." The NRC inspector noted that flow was
not being compared to reference values during surveillance testing of
the "B" HPSI pump or the two other HPSI pumps and the two low
pressure safety injection (LPSI) pumps. In the February 7,1989,
transmittal of the Safety Evaluation of the Waterford 3 inservice
testing program for pumps and valves through Revision 5, NRR again
reiterated the requirement to " measure and record all of the
parameters in Table IWP-3100-2 in order to determine pump
operability."
The NRC inspector noted two instances, since the issuance of the
July 8,1988, LP&L letter to the NRC, of "B" HPSI pump surveillance
testing in which flow data was not recorded in accordance with
Table IWP-3100-2. These tests occurred on August 30, 1988, and
November 22, 1988. Failure to perform surveillance testing in
accordance with the Article IWP-3000 standard is an apparent
violation of TS 4.0.5.a. TS 4.0.5.a requires, in part, that
Surveillance Requirements for inservice testing of ASME Code Class 1,
2, and 3 components shall be applicable as follows: inservice
testing of ASME Code Class 1, 2, and 3 pumps shall be performed in
- _ _
n .
.
>-
.
, ,
E , ,
,, ~4 .
'
. .
.
A
'
e
t, -
5-
- ,
e
accor' dance with= Section XI of the ASME- Boiler and Pressure Vessel .'
,
- Code, except where specific written relief has been granted by the'
- Commission.
i; b. .(Closed)' Unresolved Item 382/8901-03 - Failure to Take Prom)t
- Corrective Action Following the Identification of Low "B" H)SI
~
Pump Recirculation Flow
- A'n NRC maintenance' team inspection-(NRC Inspection Report 50-382/89-01)
noted that a low recirculation flow condition.for the "B" HPSI pump -
was discovered by the licensee during a routine surveillance test of-
the "B" HPSI pump on November 22, 1988. :The recorded value of-
recirculation' flow was 19.0 gpm. . Although no acceptance criteria
-
was listed on-the' surveillance test data sheet, the vendor technical
manual recommended a minimum recirculation. flow'of 25 gpm. . During the
second week of the ~NRC maintenance team inspection of Waterford 3
(January 31' through February 3,1989) the team noted that the cause of
the recirculation flow condition had not been determined'or. corrected.
The team further noted et that time the "B" HPSI pump had not been-
- declared-inoperable as a ~ result of the low recirculation flow
condition.
As a result of NRC concerns about'the operability of the "B" HPSI-
- pump,,the licensee performed a 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> and 15 minute. surveillance test
of -the pump on- January 31,_1989. During this test, the I&C
technician observed that the flow orifice associated with Flow
Instrument SI-IFI-7121 in the return line to the refueling water
storage pool (RWSP) had apparently been installed backwards during'
~
maintenance in December 1987. 'The orifice was'placed in the proper
.
position and the pump was retested. Recirculation flow indication
increased approximately 4.5 gpm to a value of 24.5 gpm.
The N'lC inspector' concluded that 'the reversed. orifice could not be
the sole cause of the low recirculation flow condition since the
"B" HPSI pump recirculation flow rate 'was still. several gpm less than
the average flow that was recorded prior to. December 1987. Additionally,
flow ~had increased from 20 gpm in' December 1987 to an average rate of
25.5 gpm during the next 3 surveillance tests but.had decreased to
19.0 gpm on November 22, 1988. The reversed flow orifice does not
account for these apparent flow anomalies.
The ."B" HPSI pump.was declared inoperable on February 1,.1989, due to
high ' axial vibration. On February 2,1989, the licensee initiated
Work Authorization (WA) 01031752.to determine the cause of excessive
wear and to repair the "B" HPSI pump inboard and outboard thrust
bearings.
Discussions with licensee personnel revealed that the accelerated
thrust bearing failure was caused by improper balance drum clearance.
One of the purposes of the pump balance drum is to absorb axial pump
forces during low flow conditions (particularly when the pump is
!
<
w____---.___.___--___ ___ __ -- - _ . _ . _ . - _ _ - - . . _ _ _ . .
- - _ _ _ _
. .
.
,
i
6
.
.
operating in the recirculation mode). Improper balance drum-
clearance resulted in the pump's thrust bearings absorbing a
disproportionate amount of the pump's axial forces during low flow
conditions. This resulted in an accelerated deterioration of the
thrust bearings. The balance drum clearance was apparently
improperly set following replacement of the "B" HPSI pump rotor in
April 1986. The licensee attributed the improper clearance to
inadequate guidance in the vendor technical manual. A review of _ ;
Ingersoll-Rand HPSI Instruction Manual 457000272, revealed that the
balance drum clearance settings specified were correct, but that the
level of detail for properly setting the balance drum clearance was
lacking. The NRC inspector concluded that this aspect of the repair
activity could not be considered within the skill of craft for
maintenance personnel. The NRC inspector further concluded that this
improper repair due to inadequate technical manual guidance could
have been avoided if the licensee had developed a detailed corrective
maintenance procedure for the HPSI pumps. This is discussed in
further detail in paragraph 2.c of this report.
The licensee returned the "B" HPSI pump to service following repair
of the pump and motor bearings. The NRC inspector noted, however,
that recirculation flow was still significantly less than the
pre-December 1987 values and less than the 25 gpm minimum specified
in the vendor technical manual. The "B" HPSI pump recirculation pump i
flow was recorded as 22 gpm during a February 24, 1989, surveillance
test of the pump. This flow was evaluated as acceptable by the
licensee. The NRC inspector made the following observations:
(1) In a' minimum flow evaluation report, "High Pressure Safety
Injection Pump: Ingersoll-Rand 4x9c-9, Motor Driven,"
(January 27,1989) performed by the Ingersoll-Rand Company at
the request of LP&L (which was collecting data for NRC
Bulletin 88-04, " Potential Safety-Related Pump Loss"), the
minimum flow for pump starting / stopping was 25 gpm.
Ingersoll-Rand defined pump starting / stopping as pump operation
of 15 minutes or less. For pump operation greater than ,
15 minutes, Ingersoll-Rand specified significantly higher
minimum flow requirements.
(2) In the same report, Ingersoll-Rand also stated that "in order to
realize the pump life projected in this evaluation, periodic
maintenance must be provided. As a minimum, the pump must be j
'
operated in accordance with the guidelines and recommended '
maintenance schedules as presented in the Ingersoll-Rand Pump
Instruction Manual and good industry practice." The NRC
inspector noted that one of the guidelines in the Pump
Instruction Manual was to keep the recirculation line open when
starting and stopping and at capacities less than 25 gpm.
(3) The licensee primarily based its "B" HPSI pump operability
determination on verbal information provided by the
- - - - _ _ - - _ _
l
-
. .
- . .
7
.
'
.
Ingersoll-Rand Company. The basis for this determination was
that the pump parameters (noise, vibration, and temperature
readings) all remained within acceptable limits during a 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />
and 15 minute run of the pump in recirculation mode. One of the
primary considerations was the stabilization of pump
temperatures (including bearing temperatures) taken at various
.
l
! locations on or near the pump. The NRC inspector noted,
however, that in NRC Not. ice 89-08, " Pump Damage Caused by
Low-Flow Operation," early component degradation from low
recirculation flow may not be detected by routine inservice
i tests. Furthermore, a review of the licensee's Inservice Test
l Program, Revision 6, revealed that LP&L was requesting relief
from the requirement to measure bearing temperature annually.
If this exemption were granted, the licensee would not have the
benefit of valuable operating temperature information in
assisting in the determination of pump operability.
(4) The cause of the low recirculation flow for the "B" HPSI pump
has not been determined and corrected. The licensee plans to
investigate the cause of the low flow condition once it obtains
a highly accurate ultra sonic flow measuring device that is
expected to be available in late March or early April. The NRC
inspector noted, however, that had a reference value for the
"B" HPSI pump been established on the basis of flow data
obtained prior to July 8,1988, it appears that the '19.0 gpm
value recorded on November 22, 1988, would have been lower than
the " Low Value Required Action Range" limit. The NRC inspector
further noted that IWP-3230(b) requires that if deviations fall
within the " Required Action Range" of Table IWP-3100-2, the pump
shall be declared inoperablo and not returned to service until
the cause of.the deviation has been determined and the condition
corrected. Discussions with licensee personnel revealed that
they believed that their analysis of the low recirculation flow
condition met the requirement of IWP-3230(c) which states, in
part, that " correction shall be either replacement or
repair . . . or shall be an analysis to demonstrate that the
condition does not impair pump operability and that the pump
will still fulfill its function." However, the NRC inspector
noted that the licensee had not yet determined the cause of the
deviation.
Failure to determine the cause of the low flow condition prior to
returning the "B" HPSI pump to service is a second exeinple of failure
to meet the Surveillance Requirements for the "B" HPSI pump in
accordance with TS 4.0.5.a.
c. NRC Inspection Report 50-382/89-06 - Repair of the "B" HPSI Pump and j
Pump Motor
As previously noted in paragraph 2.b, the HPSI Pump Instruction
Manual provided insufficient guidance for setting HPSI pump balance
i
_ _ _ _ _ _ _ _ _ _ _ _
_ _- .
,
' '
'
. . . ,
y.. >
a
8
.
.
drum clearance. ! Because of the licensee's heavy reliance on the use
.of work instructions and vendor manuals in lieu of Plant _ Safety
Committee (PSC) reviewed and approved component specific corrective
maintenance procedures,' the NRC inspector was concerned. that future
instances of inadequate maintenance on safety-related. equipment was
likely to occur., This concern was also documented in NRC Inspection
Report 50-382/89-01.
NRC Inspection Report 50-382/89-06 documented an instance in which
the "B" HPSI pump motor oil slinger ring was improperly installed and
that the bearing antirotation pins were not transferred from the old
bearings to the newly-installed bearings. Two uncoupled motor runs
were attempted.on February 20, 1989, but the motor had to.be secured
because.cf excessive vibration. Upon removal of the bearing covers,
bearing damage was observed. The damage was attributed to the.
improper installation of the oil slinger ring.
A review of Event Summary Report-(ESR)89-003, revealed that a
contributing cause of the "B" HPSI pump motor bearing damage was the
omission of the checks for proper oil slinger ring operation which-
are in the vendor instruction manual, but was not included in the
work instructions of WA 01032220. Furthermore ESR 89-03 documented
that the motor technical manual was.not the latest revision and did
not provide guidance for the movement of the antirotation pins from
the.old bearing to the new bearing. As noted in NRC Inspection
Report 50-382/89-06, failure to install the antirotation pins would
cause motor failure, but may not be detected during normal
surveillance testing.
Step 5.2.7 of Administrative Procedure MD-1-014, Revision 2. " Conduct
of Maintenance," . requires, in part, that "if, during the performance
of any activity requiring the use of vendor technical manuals,
deviations fromithe recommendations or procedures in the_ manuals are
necessary, such deviations shall receive a documented technical
review by Maintenance Engineering and be a
MAS [ maintenance assistant superintendent]pproved by the
." The NRC inspector noted cogniza
that the omission of the checks for proper oil slinger _ ring operation
from WA 01032220 was not reviewed by Maintenance Engineering and
approved by.the Electrical MAS. This is an apparent violation of
MD-1-014.
3. Exit Interview
l
The inspection scope and findings were summarized on March 9, 1989, with
those persons indicated in paragraph 1 above. The licensee acknowledged
the NRC inspector's findings. The licensee did not identify as
i
proprietary any of the material provided to, or reviewed by, the NRC
I inspector during this' inspection.
!
,
-
w. .
,