ML17265A319
| ML17265A319 | |
| Person / Time | |
|---|---|
| Site: | Ginna  |
| Issue date: | 06/08/1998 |
| From: | Mecredy R ROCHESTER GAS & ELECTRIC CORP. |
| To: | Vissing G NRC (Affiliation Not Assigned), NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM) |
| References | |
| 50-244-98-03, 50-244-98-3, NUDOCS 9806120243 | |
| Download: ML17265A319 (17) | |
Text
AND ROCHESTER GAS ANDELECTRIC CORPORATION i
i s
e 89 EASTAYENLJE, ROCHESTER, MY IrIbf90001 AREA CODE71d 5'.27K ROBERT C. MECREDY Vice Presiaens Yiucleor Operosions June 8,
1998 U.S. Nuclear Regulatory Commission Document Control Desk Attn:
Guy S. Vissing Project Directorate I-1 Washington, D.C.
20555
Subject:
Reply to a Notice of Violation NRC Integrated Inspection Report 50-244/98-03 Notice of Violation, dated May 5, 1998 R.E. Ginna Nuclear Power Plant Docket No. 50-244 and
Dear Mr. Vissing:
Rochester Gas and.Electric (RGEE) provides this reply within 30 days of receipt of the letter which transmitted the Notice of Violation.
During an NRC Inspection conducted on February 22 - April 5,
- 1998, a
violation of NRC requirements was identified.
In accordance with the "General'tatement of Policy and Procedure for NRC Enforcement Actions," NUREG-1600, the violation is listed below:
Criterion
- XVI, "Corrective Action,"
requires in part that measures be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, and defective material and equipment are promptly identified and corrected.
In the case of significant conditions adverse to quality, the measures shall assure that the cause of the condition is determined and corrective action taken to preclude repetition.
Contrary to the
- above, inadequate problem identification, root cause
- analyses, or corrective actions caused repetitive failures in two safety-related circuit breakers from June 1993 t'o March
- 1998, as described below:
0 I
A.
The'utput circuit breaker from the B-emergency diesel generator (B-EDG) to safeguards electrical bus 16 experienced five failures to close during periodic surveillance tests during the period from January 1995 to March 1998.
The March 1998 failure was attributed to a defective condition that resulted from a missing internal component which had not been installed since 1985.
Page 2
B.
The power supply circuit breaker for the B - service water pump (B-SWP) experienced three failures to reset following simulated over-current conditions during post-maintenance testing conducted in June
- 1993, May 1995, and.March 1998.
These failures were attributed to the same faulty component."
'I the reason for the violation, or, if contested, the basis for disputing the violation or severity level:
Rochester Gas E Electric Corporation (RGEE) accepts the violation.
Process deficiencies may have contributed to inadequate identification of root causes for these circuit breaker failures.
The processes that required strengthening are:
o Depth of root cause investigations 0
0 Vendor manual documentation and support Operational Experience
- review, both industry.-wide and internal In the past, Ginna Station Maintenance and Engineering personnel typically performed troubleshooting and apparent cause determinations for circuit breaker failures.
With this approach, it was assumed that these actions provided an acceptable determination of the root
- cause, sufficient to resolve each circuit breaker failure.
The approach had been to.identify a problem or operate the breaker to attempt to duplicate the
- failure, then correct the problem and test the breaker to verify proper operation.
In each of the examples noted by the
- NRC, a
valid cause was identified, and was believed to be the "root cause".
A formal root cause analysis process was not always utilized.
In the case of the B emergency diesel generator output breaker to safeguards bus 16 (EGlB1), the five failures had been attributed to:
o January 23, 1995: oxidized control power contacts 0
0 October 28, 1996:
secondary contacts misaligned I
December 26, 1996: intermittent dead spot on W-2 switch I
November 14, 1997: binding of shunt trip coil plunger o
March 23, 1998: tripper'ar spacer missing causing binding of the tripper bar (ACTION Report 98-0454)
Page 3
In the case of the B service water (SW) pump supply breaker (SWP1B),
the three failures had been attributed to:
o June 30, 1993:
misaligned alarm switch and lack of lubrication o.
May 10, 1995: misalignment of mechanical trip mechanism o
March 26, 1998: misaligned alarm switch Prior to the failure of EG1B1 on March 23, 1998 (detailed in
'CTION report 98-0454),
the root cause analysis process utilized for circuit breaker failures, and the corrective actions identified for these failures did not prevent future breaker failures.
In each
- case, a breaker failure mechanism was identified and the specific mechanism was corrected.
However, all potential causes for the breaker problems were not identified.
. When a single cause was identified, the troubleshooting, should have been continued
. until all potential causes were eliminated.
This permitted repetitive breaker failures.
(2) the corrective steps that have been taken and the results achieved:
0 0
The need for a
more detailed investigation into circuit breaker failures was recognized..
With the assistance of a failure investigation consulting
- firm, a potential cause matrix was developed.
This matrix-identifies all known failure mechanisms of Westinghouse DB breakers.
The matrix was developed using vendor,
- industry, and in-house operating experience information, as well as the collective experience of RG&E personnel and consultants.
Existing preventive maintenance procedures were assessed, to verify that vendor-specified tolerances were included.
Emphasis has been placed on quarantining 'failed breakers in order to more effectively determine a root cause.
Operations and Maintenance personnel have been informed of this emphasis on quarantining, designed to minimize condition changes in the event of a future breaker failure.
I The potential cause matrix was used to investigate the failure of the "B" Component Cooling Water (CCW) pump supply
- breaker, which occurred on April 20, 1998.
A thorough investigation was conducted and the root cause of the failure was identified.
0 Page 4
The potential cause matrix was used to investigate a
subsequent failure of the "B"
SW pump supply breaker (SWP1B),
which occurred on -May 12, 1998.
During this investigation, the failure could not be repeated.
However, the potential cause matrix did identify a probable cause for the failure.
The SWP1B breaker will be returned to Westinghouse for their assistance in further investigation into the root cause for this breaker 'failure.
(3) the corrective steps that will be taken'o avoid further violations:
0 0
A DB breaker program upgrade.has been initiated to strengthen the preventive maintenan'ce (PM) procedures.
As part of this
- upgrade, vendor, industry and in-house operating experience information will be
- reviewed, to ensure the PM procedures address potential failure mechanisms.
A study will be initiated to perform a."critical,component analysis".
This study will identify critical mechanical, electrical and control interfaces in the circuit breaker and associated electrical switchgear.
Revisions to the PM procedures will be performed, based on the results of this analysis.
Using available vendor documentation, the proper DB breaker configuration will be incorporated in the DB.breaker PM procedures.
Ro'ot Cause Analysis procedures will be evaluated and enhanced as a result of lessons learned from this event.
the date when full compliance will be achieved=
Full compliance was achieved on May 15,
- 1998, when the improved Root Cause Analysis process, including use of the potential cause matrix and other techniques developed with the assistance of a.
consultant, was successfully used to determine the problems with the
~"B" SW pump breaker, after having been successfully used to investigate the problems with the "B" emergency diesel breaker (March 27, 1998) and the "B"
CCW pump breaker (April 22, 1998).
The "B" SW pump supply breaker was returned to service on May 15, 1998.
Ve ly yours, Robert C. Mecr dy
Mr. Guy S. Vissing (Mail Stop 14B2)
Project Directorate I-1
. Division of Reactor Projects -'/II Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, D.C.
20555 Regional Administrator, Region I U.S. Nuclear Regulatory Commission 475 Allendale Road King of Prussia, PA 19406 U.S.
NRC Ginna Senior Resident Inspector Page 5
CATEGORY 1 REGULATORY INFORMATION DISTRIBUTION SYSTEM (RIDS) iti ACCESSION NBR:9806120243 DOC.DATE: 98/06/08 NOTARIZED: NO DOCKET FACIL:50-244 Robert Emmet Ginna Nuclear Plant, Unit 1, Rochester G
05000244
~
~
AUTH.NAME
'UTHOR AFFILIATION
'ECREDY,R.C.
Rochester Gas
&, Electric Corp.
RECIP.NAME RECIPIENT AFFILIATION VISSING,G.S.
SUBJECT:
Responds to NRC 980508 ltr re violations noted in insp rept 50-244/98-03.Corrective actions:recognized need for more detailed investigation into circuit b'reaker failures assessed existing preventive maint procedures.
D1'STR1BUTION CODE:
IEOID COPIES RECEIVED:LTR I
ENCL Q SIZE:
TITLE: General (50 Dkt)-Insp Rept/Notice of Violation Response NOTES:License Exp date in accordance with 10CFR2,2.109(9/19/72).
05000244 I
RECIPIENT ID CODE/NAME PD1-1 PD INTERNAL: AEOD/SPD/RAB DEDRO NRR/DRCH/HHFB NRR/DRPM/PERB OE DIR RGN1 FILE 01 COPIES LTTR ENCL RECIPIENT ID CODE/NAME VISSING,G.
P RPM/PECB NUDOCS -ABSTRACT OGC/HDS3 COPIES LTTR ENCL RNAL: LITCO BRYCE,J H NRC PDR NOAC NUDOCS FULLTEXT NOTE TO ALL "RZDS" RECIPIENTS:
PLEASE HELP US TO REDUCE WASTE. TO HAVE YOUR NAME OR ORGANIZATION REMOVED FROM DZSTRZBUTION LIST OR REDUCE THE NUMBER OF COPIES RECEIVED BY YOU OR YOUR ORGANIZATION, CONTACT THE DOCUMENT CONTRO DESK (DCD)
ON EXTENSION 415-2083 TOTAL NUMBER OF COPIES REQUIRED:
LTTR 17 ENCL Q'
AND ROCHESTER GAS ANDELECTRIC CORPORATION ROBERT C. MECREDY Vice President livclear Operations
~ B9 EASTAVENU"",ROD.ESTER N.Y. id6d9 OHI
+
AREA CODE7/6 546.2700 June 8,
1998 U.S. Nuclear Regulatory Commission Document Control Desk Attn:
Guy S. Vissing Project Directorate I-1 Washington, D.C.
20555
Subject:
Reply to a Notice of Violation NRC Integrated Inspection Report 50-244/98-03 Notice of Violation, dated May 5, 1998 R.E. Ginna Nuclear Power Plant Docket No. 50-244 and
Dear Mr. Vissing:
Rochester Gas and Electric (RGEE) provides this reply within 30 days of receipt of the letter which transmitted the Notice of Violation.'uring an NRC Inspection conducted on February 22 April 5,
- 1998, a
violation of NRC requirements was identified.
In accordance with the "General Statement of Policy and Procedure for NRC Enforcement
'ctions,"
NUREG-1600, the violation is listed below:
Criterion
- XVI, "Corrective Action,"
requires in part that measures be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, and defective material and equipment are promptly identified and corrected.
In the case of significant conditions adverse to quality, the measures
'shall assure that the cause of the condition is determined and corrective action taken to preclude repetition.
Contrary to the
- above, inadequate problem identification, root cause
- analyses, or corrective actions caused repetitive failures in two safety-related circuit breakers from June 1993 to March
- 1998, as described below:
A.
The output circuit breaker from the B-emergency diesel generator (B-EDG) to safeguards electrical bus 16 experienced five failures to close during periodic surveillance tests during the period from January 1995 to March 1998.
The March 1998 failure was attributed to a defective condition that resulted from a missing internal component which had not been installed since 1985.
'tt806120243 980608 PDR ADQCK 05000244 PDR 1
yr
water p
p xp owing simulated over-current conditions during post-maintenance testing conducted in June
- 1993, May 1995, and March 1998.
These failures were attributed to the same faulty component."
Page 2
B.
The power supply circuit. breaker for the B
service um (B-SWP) e erienced three failures to reset foll the reason for the violation, or, if contested, the basis for disputing the violation or severity level:
Rochester Gas
& Electric Corporation (RG&E) accepts the violation.
Process deficiencies may have contributed to inadequate identification of root causes for these circuit breaker failures.
The proce'sses that required strengthening are:
o Depth of root cause investigations o
Vendor manual documentation and support o
Operational Experience
- review, both industry-wide and internal In the past, Ginna Station Maintenance and Engineering personnel
,typically performed troubleshooting and apparent cause determinations for circuit breaker failures.
With this approach, it was assumed that these actions provided an acceptable determination of the root
- cause, sufficient to resolve each circuit breaker failure.
The approach had been to identify a problem or operate the breaker to attempt to duplicate the
- failure, then correct the problem and test the breaker to verify proper operation.
In each of the examples noted by the
- NRC, a
valid cause was identified, and was believed to be the "root cause".
A formal root cause analysis process was not always utilized.
In the case of the B emergency diesel generator output breaker to safeguards bus 16 (EGlBl), the five failures had been attributed to-0 0
January 23, 1995: oxidized control power contacts October 28, 1996:
secondary contacts misaligned 0
December 26, 1996: intermittent dead spot on W-2 switch November 14, 1997: binding of shunt trip coil plunger March 23, 1998:
tripper bar spacer missing causing binding of the tripper bar (ACTION Report 98-0454)
Page 3
In the case of the B service water (SW) pump supply breaker (SWP1B),
the three failures had been attributed to:
o June 30, 1993:
misaligned alarm switch and lack of lubrication o
May 10, 1995: misalignment of mechanical trip mechanism o
March 26, 1998: misaligned alarm switch Prior to the failure of EGlB1 on March 23, 1998 (detailed in ACTION report 98-0454),
the root cause analysis process utilized for circuit breaker failures, and the corrective actions identified for these failures did not prevent future breaker failures.
In each
- case, a breaker failure mechanism was identified and the specific mechanism was corrected.
However, all potential causes for the breaker problems were not identified.
When a single cause was identified, the troubleshooting should have been continued until all potential causes were eliminated.
This permitted repetitive breaker failures.
(2) the corrective steps that have been taken and the results achieved:
0 0
The need for a
more detailed investigation into circuit breaker failures was recognized.
With the assistance of a failure investigation consulting
- firm, a potential cause matrix was developed.
This matrix identifies all known failure mechanisms of.Westinghouse DB breakers.
The matrix was developed using vendor,
- industry, and in-house operating experience information, as well as the collective experience of RGEE personnel and consultants.
Existing preventive maintenance procedures were assessed, to verify that vendor-specified tolerances were included.
Emphasis has been placed on quarantining failed breakers in order to more effectively determine a root cause.
Operations and Maintenance personnel have been informed of this emphasis on quarantining, designed to minimize condition changes in the event of a future breaker failure.
0 The potential cause matrix was used to investigate the failure of the "B" Component Cooling Water (CCW) pump supply
- breaker, which occurred on April 20, 1998.
A thorough investigation was conducted and the root cause of the failure was identified.
h
0 Page 4
The potential cause matrix was used to investigate a
subsequent failure of the "B"
SW pump supply breaker (SWP1B),
which occurred on May 12, 1998.
During this investigation, the failure could not be repeated.
However, the potential cause matrix did identify a probable cause for the failure.
The SWP1B breaker will be returned to Westinghouse for their assistance in further investigation into the root cause for this breaker failure.
the corrective steps that will be taken to avoid further violations:
A DB breaker program upgrade has been initiated to strengthen the preventive maintenance (PM) procedures.
As part of this
- upgrade, vendor, industry and in-house operating experience information will be
- reviewed, to ensure the PM procedures address potential failure mechanisms.
A study will be initiated to perform a "critical component analysis".
This study will identify critical mechanical, electrical and control interfaces in the circuit breaker and associated electrical switchgear.
Revisions to the PM procedures will be performed, based on the results of this analysis.
Using available vendor documentation, the proper DB breaker configuration will be incorporated in the DB breaker PM procedures.
'oot Cause Analysis procedures will be evaluated and enhanced as a result of lessons learned from this event.
(4) the date when full compliance will be achieved:
Full compliance was achieved on May 15,
- 1998, when the improved Root Cause Analysis process, including use of the potential cause matrix and other techniques developed with the assistance of a
consultant, was successfully used to determine the problems with the "B"
SW pump breaker, after having been successfully used to investigate the problems with the "B" emergency diesel breaker
~
(March 27, 1998) and the "B"
CCW pump breaker (April 22, 1998).
The "B" SW pump supply breaker was returned to service on May 15, 1998.
Ver uly yours, Robert C. Mecr dy
0
Mr. Guy S. Vissing (Mail.Stop 14B2)
Project Directorate I-1 Division of Reactor Projects
- I/II Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, D.C.
20555 Regional Administrator, Region I U.S. Nuclear Regulatory Commission 475 Allendale Road King of Prussia, PA 19406 U.S.
NRC Ginna Senior Resident Inspector Page 5
I r~
0 4