ML20116H262
| ML20116H262 | |
| Person / Time | |
|---|---|
| Site: | Seabrook  |
| Issue date: | 08/08/1996 |
| From: | Feigenbaum T NORTH ATLANTIC ENERGY SERVICE CORP. (NAESCO) |
| To: | NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM) |
| References | |
| NYN-96055, NUDOCS 9608120009 | |
| Download: ML20116H262 (10) | |
Text
r w%,
~
li North North Atlantic Energy Service Corporation
' ty Atlantic H
P.O. Box 300 Seahroot,siiO3874 (603) 474-9521 The Northeast Utilities System August 8,1996 Docket No. 50-443 NYN-96055 United States Nuclear Regulatory Commission Attention: Document Control Desk Washington, D.C. 20555 Seabrook Station Reply to a Notice of Violation 3
In a letter dated July 10,1996 the NRC described a violation regarding incorrect installation of mechanical seals in the turbine driven emergency feedwater pump that rendered the pump inoperable. The repon also identified weaknesses in the evaluation of this event in that it was not comprehensive nor commensurate with the safety significance of the event and did not appear to meet the station guidance for conducting such evaluations. Accordingly, the enclosure provides North Atlantic Energy Service Corporation's (North Atlantic) response to this violation and the identified weaknesses.
North Atlantic is making certain commitments in response to this violation and the identified weaknesses. The commitments are fully described in the enclosure to this letter.
Should you have any questions concerning this response, please contact Mr. Anthony M.
Callendrello, Licensing Manager, at (603) 474-9521, extension 2751. .
1 Very truly yours, l NORTH ATLANTIC ENERGY SERVICE CORP. ;
, I 7b&
T. C. Feigeptfa'um Executive Vice President and Chief Nuclear Officer
)
cc: H. J. Miller, Regional Administrator A. W. De Agazio, Sr. Project Manager i
_ J. B. Macdonald, NRC Senior Resident Inspector 9608120009 960808 -
/{}j
/<
{DR ADOCK 05000443 PDR \
./
2 NRC Inspection Report 96-04, dated July 10,1996, J. F. Rogge to T. C. Feigenbaum. )
6 4
ENCLOSURE 1 TO NYN-96055 1
1 I
c i REPLY TO A NOTICE OF VIOLATION l
NRC Inspection Report 96-04 described a violation regarding incorrect installation of mechanical seals in the turbine driven emergency feedwater (EFW) pump that rendered the pump inoperable. l The inspection report also identified weaknesses in the evaluation of this event in that it was not j comprehensive nor commensurate with the safety significance of the event and did not appear to l meet the Station guidance for conducting such evaluations. North Atlantic's response to this !
violation and the identified weaknesses are described below.
L Description of Violation The following is a restatement of the violation VIO 96-04-01:
10 CFR 50, Appendix B, Criterion V and UFSAR Section 17.1.1.5 requires that activities affecting quality shall be prescribed by documented instructions, procedures and drawings of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures or drawings. Instructions, procedures or drawings shall include l appropriate quantitative or qualitative acceptance criteria for determining that important activities i have been satisfactorily accomplished. I i
Contrary to the above, during refueling outage ORO 4 (November - December 1995), safety-related maintenance was performed on the turbine driven emergency feedwater pump, using Section 8.29 of maintenance procedure MS0523.21, " Emergency Feedwater Pump Maintenance." ;
The turbine driven EFW pump mechanical seals were improperly installed and aligned due to lack l of sufficient acceptance criteria to confirm critical tolerances were maintained. This caused the i turbine driven EFW pump to become inoperable on May 21,1996, during surveillance testing.
Subsequent analysis found the safety-related equipment would not have been able to perform its intended safety function if called upon to do so.
4 This is a Severity Level IV Violation (Supplement I).
IL Reniv to Violation Reason for Violation North Atlantic agrees with this violation. Upon receipt of the Notice of Violation, North Atlantic performed a formal root cause analysis to identify the primary and contributing causes of this ;
event. Prior to this, and as described in LER 96-003-00, " Emergency Feedwater Pump Mechanical Seal Failure," North Atlantic had performed a less rigorous Cause and Failure Analysis to identify the apparent causes. This analysis was identified as being too narrowly focused in NRC Inspection Report 96-04. Weaknesses associated with the original analysis are discussed in Section III of this response, below. The primary and contributing causes from the formal root cause evaluation are as follows:
1
Primaw Cause Two primary causes were determined for this event:
- 1. Design deficiency The design clearances and tolerances of the turbine driven EFW pump's mechanical seals were insufficient to prevent damage during operation unless the installation technique used non-customary methods (i.e., use of dialindicators and feeler gauges). The design permitted a stack up of tolerances greater than the clearance. The design did not preclude, as it could have, the interference between the seal and the shaft sleeve. This design deficiency also applies to the motor driven EFW pump mechanical seals.
- 2. Corrective actions for previously identified event were not adequate to prevent recurrence i l
Two failures occurred to the secondary seal of the motor driven EFW pump inboard seal assembly in January 1987 prior to the issuance of the full-power operating license. These failures l 1
did not affect the primary seal and there was no loss of feedwater inventory. The initial failure was at the start of a quanerly suweillance run. There is no written record of this event, but the recollection of the system engineer was that the seal rub became evident as soon as the pump was started. The seal assembly was replaced and reinstalled, with the pump vendor present. When the pump was re-started for its maintenance run, there was again catastrophic failure of the secondary seal. When the pump seal was disar.sembled, the vendor and the system engineer discovered the close clearance between the throttle bushing and shaft sleeve and the loose tolerances that allowed seal mb if not installed carefully. When the seal was reassembled, a dial indicator was used to assure that the gland was properly installed to the top ofits fit so that the throttle bushing, which i is pressed into the gland, would be centered.
As a result of the 1987 seal failures, maintenance procedure MS0523.21, " Emergency Feedwater Pump Maintenance," was revised to include the following caution statement:
"While tightening the mechanical seal gland ring bolting, the gland ring must be held in the top ofits fit to prevent seal failure during start-up."
However, this procedure revision did not indicate the need to use a dial indicator to verify that the gland ring was at the top ofits fit after tightening the bolting. It is believed that this step was omitted since, at that time, use of a dial indicator for seal alignment was believed to be within the skill of the worker, i.e., the worker will use the tools needed to properly accomplish the job.
North Atlantic does not currently expect workers to utilize dial indicators, or other precision measuring equipment, unless it is specified in a procedure.
Associated with the inadequate corrective actions from the 1987 seal failures, and consistent with the cause of the recent seal failure as described in LER 96-003-00, collectively, there was a failure to incorporate operating experience from the 1987 failures. Specifically, there was no documentation of the 1987 seal failures. Knowledge of these prior events was limited to the system engineer's and a mechanic's recollection. These individuals did not share the lessons 2
1*
- t learned from the earlier events with other personnel who worked on the EFW pump seals. As a result, the maintenance supervisor who performed work on the seals during ORO 4 was unaware !
of the prior events or the need to take precision measurements to verify proper installation of the seal assembly.
The corrective action process has changed substantially since 1987 and problems with the process that was in existence at that time are not germane to the current program. North Atlantic believes that the current process of documenting information in work requests and Adverse Condition Reports (ACRs) will ensure that operating experience from future events is adequately captured for future use. Notwithstanding, this event will be discussed at training sessions provided to appropriate personnel to highlight the need to ensure adequate documentation of operating experience. North Atlantic does not believe that additional actions are necessary to address the reason for why inadequate corrective actions were implemented for the seal failures in 1987, nor for the failure to incorporate operating experience from these prior events. l Contributing Causes :
- 1. Predictive maintenance techniques This event demonstrated that the predictive maintenance techniques for the EFW pumps were unable to identify degradation of the secondary seal assembly. For this failure, it is possible that cenain condition based maintenance techniques (e g., thermography) could have detected the gradual temperature rise on the shaft sleeve caused by the seal rubbing.
Corrective Actions l 1. Upon identification of this condition North Atlantic declared the turbine driven EFW pump inoperable and replaced the outboard mechanical seal. The inboard seal was checked and properly adjusted by centering the gland. The clearances of both mechanical seals on the motor driven EFW pump were also checked and adjusted. The system engineer evaluated the mechanical seals on the start-up feedwater pump and determined that an inspection was not required. North Atlantic determined that no other safety related pumps at Seabrook Station utilize a similar seal design.
- 2. Maintenance procedure MS0523.21, " Emergency Feedwater Pump Maintenance," will be reviewed and revised, as appropriate, to incorporate greater detail in the mechanical seal maintenance / installation sections to ensure that the seal clearances are adequate to preclude failure. This procedure review will be performed before the next time this procedure is used for seal repair / replacement. It is anticipated that this review will be completed by August 31,1996.
l
- 3. Engineering will evaluate an EFW pump mechanical seal design change to preclude catastrophic seal failure caused by close clearances. It is anticipated that this engineering evaluation will be completed by June 1,1997.
l 3
1
\
l
- 4. Nonh Atlantic will investigate possible monitoring techniques to detect incipient seal failures. It is anticipated that this investigation will be completed by December 31,1996.
l 5. North Atle.ntic will discuss this event at training sessions provided to appropriate personnel l to highlight the need to ensure adequate documentation of operating experience. It is anticipated that this training will be completed by May 31,1997.
- 6. The root cause analysis identified a number ofissues associated with the disassembly and assembly of the EFW pumps during the last two refueling outages. As a result, North Atlantic will perform a review of previous problems related to major maintenance of complex mechanical equipment and consider performing a root cause analysis if there ,
appears to be a repetative problem. It is anticipated that this review will be completed by l February 1,1997. l l
- 7. North Atlantic will issue a supplement to LER 96-003-00 to reflect the additional )
information obtained from the aforementioned formal root cause analysis. It is anticipated I that this supplement will be submitted by September 15,1996.
Discussion of Weaknesses identified in Inspection Report 96-04 NRC Inspection Report 96-04 requested that, in addition to the response to the aforementioned violation, North Atlantic address the weaknesses described in the inspection report regarding the initial evaluation of the turbine driven EFW pump seal failure. The following summarizes these weaknesses: I e The Significance Level A, Management Review Team (MRT) and SORC approved, completed ACR evaluation was not commensurate with the safety significance of the event and did not fully meet the guidance contained in the Operating Experience Manual (SSOE).
Specifically, the inspector concluded that SSOE guidance strongly suggested that a formal root cause evaluation in conjunction with an OE 4.1 Event Evaluation would have been appropriate in this case given the Significance Level A ACR designation.
e The limited Cause and Failure Analysis and associated corrective actions were too narrowly focused and symptom oriented and thus did not address the root cause(s). Additionally, the Cause and Failure Analysis and associated corrective actions did not consider generic implications or address the several defense-in-depth barriers that broke down for this event to occur. Furthermore, the corrective actions did not address the failure of corrective actions for a previously identified problem or event to prevent recurrence (failure to adequately incorporate operating experience) and thus were not focused on preventing recurrence.
. Other similar events were not fully considered when performing the evaluation. Specifically, in refueling outage ORO 4, ACR 95-457 documented the motor driven EFW pump thrust l bearing had been installed backwards. This ACR was not included as part of the ACR 96-413 evaluation, yet it documented incorrect installation of components associated with the 4
motor driven EFW pump and the same contract personnel worked on the failed turbine l driven EFW pump.
l Reasons for Identified Weaknesses Nonh Atlantic agrees with the weaknesses stated in the inspection report. Accordingly, North Atlantic performed a formal root cause using Barrier Analysis to determine the primary cause and contributing factors for the aforementioned weaknesses. These are described below.
Primary Cause The primary cause for the aforementioned programmatic weaknesses is inadequate policy guidance regarding when to perform a formal root cause evaluation and/or an event evaluation. In this regard, the MRT was not given the proper tools to make such determinations. Specifically, the guidance provided in the Seabrook Station Operating Experience Manual (SSOE) for when a root cause evaluation and/or an Event Evaluation is unclear. SSOE Chapter 2, Section 2.3 states that both an Event Evaluation and a root cause analysis are required for a reactor trip, ESF actuation, or a nuclear safety concern. However, the term " nuclear safety concern" is act defined.
Additionally, SSOE procedure OE 4.1, " Event Evaluation," states that an cvent evaluation I should be performed for a reactor trip, ESF actuation, or other significant events. However, the term "significant events" is not defined. Furthermore, SSOE procedure OE 4.3, " Root Cause Analysis," describes the process for performing a root cause analysis but does not indicate when it l should be used. Moreover, SSOE Chapter 2, Section 2.0 states that management will select the ;
appropriate evaluations to be completed depending on the significance of the adverse condition. j Again, " significance" is not defined.
~
1 The root cause analysis also determined that members of the MRT and SORC were clearly aware j that the EFW seal event was safety significant. However, these members were unaware that this event was one of the most safety significant/ risk significant events in recent history. This may 4
have influenced the decision to perform a Cause and Failure Analysis as opposed to a formal root cause analysis. Overall, as stated above, the MRT does not have clear guidance on how to judge safety significance.
The SSOE does not contain any guidance on how comprehensive a root cause analysis should be.
A root cause analysis is adequate, at the most basic level, ifit determines the cause of the adverse condition using logical information management techniques. Beyond that, a root cause analysis can be comprehensive in depth and breadth. A root cause in depth investigates all the aspects of i the event and looks at all possible failure causes. A root cause in breadth looks for common links, such as other components, processes, and conditions, that may have similarity with the adverse condition. Guidance is necessary to quantify the required depth and breadth of a root cause analysis.
Contributing Caus_ej; 5
- 1. Large volume of ACRs to process The large number of ACRs that are processed and reviewed by the MRT provides a distraction that makes identification of highly significant issues more difficult. Since the beginning of 1995, the relatively low threshold for initiating ACRs has resulted in more than 1100 ACRs being generated. The ACR threshold has continued to drop over time, resulting in approximately 700 ACRs being initiated in 1996 to date. The large number of ACRs also has a direct impact on available resources. This may have influenced the MRT's assignment of more rigorous, and more 3
resource intensive, root cause analyses.
1
- 2. Appreciation of the value of root cause analysis i
i The MRT does not fully appreciate the value of the root cause process as a result of their lack of l exposure to the process. The MRT would benefit from exposure to the various methods of root cause analysis. It should be noted that the MRT was only recently constituted, although the j adverse condition evaluation process has been in place for several years. As a result, some of the
! processes of reviewing ACRs are still evolving (e.g., appropriate level of evaluation; significance).
I 1
i 3. Over emphasis on immediate hardware failures without recognizing broader hardware
! and process issues and implications j The MRT has exhibited a tendency to focus on the resolution of immediate hardware issues l without recognizing broader hardware and process issues. In addition, the MRT has exhibited i excessive confidence that the cause to complex issues was known prior to rigorous evaluation.
This is likely to have convinced the MRT that further and more rigorous analysis was unnecessary for the EFW pump mechanical seal event. Notwithstanding this, it should be noted that the j original MRT recommended Cause and Failure Analysis conducted for this event did in fact
- identify the root cause for the improper installation of the EFW pump mechanical seal. It did not, j however, constitute a comprehensive evaluation as stated in NRC Inspection Report 96-04.
Corrective Actions to Address Weaknesses l
l 1. The SSOE will be revised to clearly define threshold criteria for when a root cause analysis
- and event evaluation are to be conducted. Guidance will also be provided to determine how l comprehensive the cause determination should be. It is anticipated that the SSOE will be i revised by November 11,1996.
l
- 2. North Atlantic will provide MRT and SORC members with root cause training, appropriate for their use, to familiarize them with the various methods and benefits of conducting root
{ cause analyses. It is anticipated that this training will be conducted by November 11,1996.
i
- 3. MRT members have received a briefing on the root cause analyses for this event. They have j also been directly involved with the review of the root cause analyses and the preparation of j the reply to the Notice of Violation. This has provided the MRT with valuable lessons 3
~
- 6 i
7*.- ,- ; -c - - ---- . .
4 learned regarding the pitfalls associated with over emphasis on immediate hardware failures without recognizing broader hardware and process issues and implications.
1 IV. Date When Full Compliance Will be Achieved North Atlantic is currently in compliance with regulatory requirements.
l j
l l
l f
l l
7