ML033040408
| ML033040408 | |
| Person / Time | |
|---|---|
| Site: | Perry  |
| Issue date: | 10/24/2003 |
| From: | Kanda W FirstEnergy Nuclear Operating Co |
| To: | Document Control Desk, NRC/RGN-III |
| References | |
| EA-03-007, IR-02-008, PY-CEI/NRR-2726L | |
| Download: ML033040408 (6) | |
Text
FE'NOC Perry Nuclear Power Plant 10 Center Road FirstEnergy Nuclear Operating Company Perry, Ohio 44081 Wlliarn R. Kanda 440-2805579 Vce President - Nuclear Fax: 440-280-8029 October 24, 2003 PY-CEI/NRR-2726L United States Nuclear Regulatory Commission Document Control Desk Washington, D.C. 20555 Perry Nuclear Power Plant Docket Number 50-440 License Number NPF-48
Subject:
Revised Reply to a Notice of Violation; EA-03-007 Ladies and Gentlemen:
Enclosed is a revised response to Notice of Violation EA-03-007 issued to the Perry Nuclear Power Plant (PNPP) by the Nuclear Regulatory Commission on March 4, 2003. The details of the violation were documented in NRC Inspection Report Number 50-440/02-08. The PNPP responded to the Notice of Violation on April 3, 2003 (reference PNPP letter PY-CEI/NRR-2701L). The violation documented a failure to implement a procedure for electrical breaker installation, which resulted in the failure of the High Pressure Core Spray pump to start during routine surveillance testing on October 23, 2002. The violation is associated with a White significance determination process finding.
The root cause analysis report for this event has been revised. As a result of revising the root cause analysis report, the reply to the notice of violation is submitted to clarify the reason for violation, and provide an update of corrective steps that have been taken and corrective steps that will be taken to avoid further violation. Revision bars have been provided in the margin of the enclosure to identify revisions to the response. This revised response supercedes the submittal dated April 3, 2003.
There are no regulatory commitments contained-in-this submittal.-Any actions-discussed ir this-documrent-that represent intended or planned actions, are described for the NRC's information, and are not regulatory commitments.
If you have any questions or require additional informnation, please contact Mr. Vernon Higaki, Manager-Regulatory Affairs, at (440) 280-5294.
Very truly yours, Enclosure cc: NRC Region III Administrator NRC Senior Resident Inspector - PNPP NRR Project Manager - PNPP
PY-CEI/NRR-2726L Docket Number 50-440 License Number NPF-48 Enclosure Page 1 of 5 REPLY TO A NOTICE OF VIOLATION; EA-03-007 RESTATEMENT OF THE VIOLATION Technical Specification 5.4 requires, in part, that procedures be established, implemented, and maintained as recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978.
Regulatory Guide 1.33, Appendix A, Section 9, "Procedures for Performing Maintenance,"
recommends that maintenance activities that affect the performance of safety-related equipment should be performed in accordance with written procedures appropriate to the circumstances.
Procedure GEI-0135, Revision 1, March.30, 1999, "ABB Power Circuit Breakers 5 KV Types
-5HK250 and SHK350-Maintenance," Step 15[sic]-14.3-3-requires a-visual-check of the cell switch- - -
normally open contacts to verify they are in the flat horizontal position prior to breaker installation.
The procedure allows in a note to the step, that it may be acceptable for contact bars to not be in a flat horizontal alignment provided a clear make/break of the contacts is observed.
Contrary to the above, the licensee failed to implement procedure GEI-0135 during the installation and inspection of the high pressure core spray pump breaker from 1994 through October 23, 2002.
Specifically, the licensee did not verify that the contacts were in the flat horizontal position prior to breaker installation or that there was a clear make/break of the contacts. This failure to verify the alignment of the contacts resulted in degradation of the connection over time and failure of the pump to start during surveillance testing on October 23, 2002.
REASON FOR THE VIOLATION The root cause for the failure of the high pressure core spray (HPCS) pump to start is procedure inadequacy. Maintenance Instruction GEI-0135, revision 1 did not provide adequate criteria for the inspection of the cubicle cell and auxiliary switches when performing breaker maintenance. GEI-0135, revision 1 provided guidance on what to do in the event that a switch requires adjustment; however, the procedure lacked the direction to reach the conclusion that adjustment was required.
Personnel performing the inspections were instructed to ensure that the contacts were in the flat, horizontal position. A note contained in the procedure (GEI-0135, revision 1) as part of step 5.14.3 stated, "It may be acceptable for contact bars to not be perfectly straight vertical or flat horizontal aligned provided a clear make/break is observed." This minimizes the significance of the flat horizontal position and allows the use of subjective judgement to exit the step. This inspection was performed in 1994, 1998, and 2002, and did not identify a discrepancy with the position of the cell switch contacts.
As determined during the course of investigation, the only way to be certain of positive contact following rotation is to ensure that the normally open contacts are in the horizontal position, completely within the viewing window, which equates to approximately +/- 100 from horizontal, and observe the contacts during racking of the breaker. Following racking of the breaker, visual observation ensures that the contacts rotate approximately 90° so that the normally closed contacts
PY-CEI/NRR-2726L Docket Number 50-440 License Number NPF-48 Enclosure Page 2 of 5 are in a horizontal position and are completely within the viewing window. "Completely within the viewing window" means no part of the contact extends beyond the edge of the viewing window.
The instruction did not provide this objective criterion, and therefore left room for subjective observation resulting in inconsistent application of a weak standard.
A contributing cause to this event is the variation in the dimensions of the replacement breaker that was installed in 1994 during the performance of a breaker refurbishment program. This conclusion is based on the degree of misalignment observed during the investigation. It is physically impossible for this amount of misalignment to occur during normal operation. The location of the operating pin on the breaker truck and variations in the size of the truck can change the alignment of the-operatin-glinkage. 1This cobdition-was'notfrecognizedd-as-a-potentialrproblemi during-_
performance of the breaker refurbishment program. Satisfactory completion of the maintenance instruction followed by a successful post-maintenance test was determined to be sufficient to ensure reliable performance of the equipment, when in reality, it was not. The true verification is the alignment of the cell switch.
CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVED Failure of the HPCS pump to start was entered into the Perry Nuclear Power Plant (PNPP) corrective action program as Condition Report 02-03972 on October 23, 2002.
The cell switch linkage for the HPCS pump breaker was adjusted on October 23, 2002, to obtain proper alignment of the switch contacts. Adjustment of the cell switch to obtain proper alignment included shortening the cell switch linkage actuator arm. Proper cell switch operation was verified, both visually and electrically, during breaker racking evolutions. Proper operation of the HPCS system was verified on October 23, 2002 at 1848 hours0.0214 days <br />0.513 hours <br />0.00306 weeks <br />7.03164e-4 months <br /> when the HPCS pump was started in accordance with the system operating instruction. The system was declared operable at 1529 hours0.0177 days <br />0.425 hours <br />0.00253 weeks <br />5.817845e-4 months <br /> on October 24, 2002.
Investigation and diagnosis of cell switches found in 5KV breakers was conducted using a spare-cell switch and a breaker cubicle used for training. A cell switch typically has normally open and normally closed contacts positioned perpendicular from each other on the switch shaft.
Measurements taken on the spare switch showed that aligning the top or bottom edge of the open switch contact tab with the viewing window opening equates to approximately 10 degrees of rotation of the switch. Positive contact between the stationary contacts and the moveable contacts is achieved when the contact tabs are completely visible in the viewing window. The optimal switch contact position is when the contact tabs are centered in the viewing window. Utilizing the contact tab position centered in the viewing window as criteria for acceptability, walkdowns of the 5 KV breaker cubicles were performed.
A walkdown of 5KV safety-related and nonsafety-related switchgear involving 73 breaker cubicles was completed to identify any breaker cell switch contacts not centered in the viewing window.
Cell switches that did not satisfy this criterion were documented to require additional adjustment to
PY-CEI/NRR-2726L Docket Number 50-440 License Number NPF-48 Enclosure Page 3 of 5 optimize the cell switch contact alignment. The walkdown of 40 safety-related switchgear cubicles was completed on November 18, 2002. Work requests were submitted for 10 safety-related breakers that required cell switch adjustment. Voltage checks for the 10 safety-related breakers that were identified to require cell switch adjustments were completed on December 17, 2002, to verify they were providing circuit continuity. Voltage readings on each side of the switch contact were taken to confirm the breakers were ready to operate. The voltage checks verified cell switch contacts that permit breaker closure were making electrical contact even though their contacts were not optimally centered in the viewing window.
The walkdown of nonsafety-related switchgear involving 33 breaker cubicles was completed on
-Dbcembe 4,-2002.-A fewfonsafdty-relited breakerIcell sWitches Were idehtified for adjustmient and work requests were submitted. In addition, work requests were submitted for breaker auxiliary switches that were identified to require additional adjustment.
Maintenance instruction GEI-0135 was revised (revision 2) to provide objective criteria for the inspection of the cubicle cell switches when performing breaker maintenance. This procedure revision was made effective on February 25, 2003. The procedure revision clarified instructions, by revising step 5.14, to observe that the breaker auxiliary and/or cell switch contacts are centered in the viewing window prior to installation of the breaker into the cubicle. Guidance in Attachment 9, "Auxiliary Switch and Cell Switch Linkage Adjustment Guide" was also enhanced to include the criteria to ensure the contacts are properly aligned. Guidance for shortening the actuator arm was also included in Attachment 9. The note referenced in the violation was removed from the procedure. Revisions 3, 4, 5, and 6 to GEI-0135 were completed to further clarify and correct cell switch adjustment methods. Revision 6 to GEI-0135, made effective on September 12, 2003, established the acceptance criteria for the cell switch contact position as "completely within the viewing window."
Revision 8 to SOI-R22, "Metal Clad Switchgear 5 - 15 KV," was completed and made effective March 27, 2003, and includes guidance to plant operators for visual checks of cell switch contact position when racking in breakers. The revised procedure also states when racking in class-1 E-5KV breakers, the breaker "may not be declared operable until the breaker is cycled per an approved instruction." Training was conducted for non-licensed operators as documented in CR 02-03972.
Discussion included the draft revision of the procedure SOI-R22 and an actual cell switch for hands on application. The non-licensed operator training was completed on March 25, 2003. Revision 12 to SOI-R22 clarified the indication to verify proper change-of-state of the cell switch contacts for 5KV breakers as "completely within the viewing window."
Cubicle cell switches for five Division 2 breakers and one Division 3 breaker were adjusted during the ninth refueling outage that was completed May 31, 2003. Two of these breakers provide an automatic closure to support safety system functions. Two additional safety-related breaker cell switches were adjusted in July 2003 and one was adjusted in October 2003.
PY-CEIINRR-2726L Docket Number 50-440 License Number NPF-48 Enclosure Page 4 of 5 Condition Report 03-01546 was initiated on March 27, 2003, to address questions that required clarification or additional information that was not provided in the original investigation for CR 02-03972. As a result, root cause revision 1 was prepared and approved on July 22, 2003. The generic implications portion of the root cause was found to be inadequate during an NRC inspection conducted the week of July 28, 2003. Revision 2 of the root cause was prepared to re-examine the generic implications associated with the root and contributing causes of the HPCS pump failure to start. Revision 2 of the root cause was approved on September 13, 2003. Revision 3 of the root cause investigation report was prepared to ensure the completeness of the document and to clarify some details contained in the report. Revision 3 was approved on September 26, 2003.
-Electrical-maintenanc-p-ersoninel *vere provided triininigonEadjusting 5KV cellahnd auxiliaiy switches. Training was completed in September 2003.
Revision 2 of the root cause, to re-examine generic implications, identified that additional procedures needed revision to ensure the criteria for inspection of adjustable cell and auxiliary switches is consistent. In addition to procedures GEI-0135 and SOI-R22, procedures GEI-009, "ABB Low Voltage Power Circuit Breaker Types K-600 & K-600S through K-3000 & K-3000S Maintenance," GEI-012, "Inspection and Cleaning of Electrical equipment," and GEI-0136, "ABB Power Circuit Breakers 15KV Type 15HKI000 Maintenance," were revised to incorporate the wording "completely within the viewing window" as the objective criteria to evaluate acceptability of cell switch adjustment. The revised procedures GEI-012 (revision 5), GEI-0135 (revision 6) and GEI-0136 (revision 2) were made effective on September 12, 2003. Procedure GEI-009 (revision
- 8) was made effective on September 16, 2003. Procedure SOI-R22 (revision 12) was made effective on September 29, 2003.
CORRECTIVE STEPS THAT WILL BE TAKEN TO AVOID FURTHER VIOLATIONS The remaining cubicle cell switch is scheduled for adjustment during the normal maintenance schedule and will be completed by the end of the tenth operating cycle. The remaining cubicle cell switch is for an electrical bus source breaker.
A modification to the HPCS pump circuit wiring has been approved to provide positive Control Room indication through the pump control switch green indicating light of breaker closing circuit integrity. Implementation of this modification is being planned for November 2003.
The PNPP 345 KV switchyard utilizes the same type of rotary switch in auxiliary switch applications. During extent of condition reviews, the switchyard auxiliary switches were walked down. As a result, PNPP will coordinate with the FirstEnergy Corporation Northern Region management to communicate the findings of this investigation with regard to the required maintenance of ABB Type L2 rotary switches and review switches identified in the switchyard that potentially require adjustment. These actions are being addressed in the PNPP corrective action program.
PY-CEI/NRR-2726L Docket Number 50-440 License Number NPF-48 Enclosure Page 5 of 5 DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The HPCS pump was returned to an operable status on October 24, 2002. The procedural inadequacy was corrected by revising procedure GEI-0135, "ABB Power Circuit Breakers 5 KV Types 5HK250 and 5HK350 Maintenance," that became effective on February 25, 2003. During adjustment of cell switches during RFO9 and July 2003, enhancements to GEI-0135 were identified. The procedure modification in February 2003 was viewed to provide adequate assurance that the PNPP was in full compliance. Full compliance was achieved when training on the adjustment of cell and auxiliary switches was completed in September 2003.