IR 05000440/2003007

IR 05000440-03-007, on 07/28/03 - 08/01/03, Firstenergy Nuclear Operating Co., Perry Nuclear Power Plant. Supplemental Inspection IP 95001. Mitigating Systems

ML032330255
Person / Time
Site:	Perry
Issue date:	08/21/2003
From:	Grant G Division Reactor Projects III
To:	Kanda W FirstEnergy Nuclear Operating Co
References
EA-03-007 IR-03-007
Download: ML032330255 (17)

Text

ust 21, 2003

SUBJECT:

PERRY NUCLEAR POWER PLANT NRC SUPPLEMENTAL INSPECTION REPORT 50-440/2003007

Dear Mr. Kanda:

On August 1, 2003, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Perry Nuclear Power Plant. The enclosed report documents the inspection findings which were discussed on August 1, 2003, with you and other members of your staff.

The NRC performed this supplemental inspection to assess your evaluation of the October 23, 2002, failure of the high pressure core spray (HPCS) pump to start during routine surveillance testing. This failure occurred as a result of inadequate procedure implementation during installation and inspection of the HPCS pump breaker from 1994 through October 23, 2002. This performance issue was previously characterized as having low to moderate risk significance (White) in the NRCs final significance determination letter dated March 4, 2003. During this supplemental inspection, performed in accordance with Inspection Procedure 95001, significant deficiencies were identified with regard to your extent of condition evaluation. As a result of these concerns, the white performance issue associated with the HPCS pump failure to start will not be closed at this time. Instead, we will re-perform the supplemental inspection in accordance with Inspection Procedure 95001 following the completion of your actions in response to the concerns identified in this report.

Further, based on the results of this inspection, the inspectors identified an issue of very low safety significance (Green). The issue was determined to involve a violation of NRC requirements. However, because of its very low safety significance and because it has been entered into your corrective action program, the NRC is treating this issue as a non-cited violation in accordance with Section VI.A.1 of the NRCs Enforcement Policy.

If you contest the subject or severity of the non-cited violation, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S.

Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with a copy to the Regional Administrator, U.S. Nuclear Regulatory Commission -

Region III, 801 Warrenville Road, Lisle, IL 60532-4351; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001; and the Resident Inspector Office at the Perry Nuclear Power Plant. In accordance with 10 CFR 2.790 of the NRC's "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA by Steven A. Reynolds Acting for/

Geoffrey E. Grant, Director Division of Reactor Projects Docket No. 50-440 License No. NPF-58

Enclosure:

Inspection Report 50-440/2003007 w/Attachment: Supplemental Information

REGION III==

Docket Nos: 50-440 License Nos: NPF-58 Report No: 05000440/2003007 Licensee: FirstEnergy Nuclear Operating Company (FENOC)

Facility: Perry Nuclear Power Plan, Unit 1 Location: P.O. Box 97 A200 Perry, OH 44081 Dates: July 28 through August 1, 2003 Inspector : R. Powell, Senior Resident Inspector Approved by: Mark A. Ring, Chief Branch 1 Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000440/2003-007; 07/28/03 - 08/01/03; FirstEnergy Nuclear Operating Company; Perry

Nuclear Power Plant. Supplemental Inspection IP 95001. Mitigating Systems

Cornerstone: Mitigating Systems

The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess the licensees evaluation associated with the October 23, 2002, failure of the high pressure core spray (HPCS) pump to start during routine surveillance testing. This failure occurred due to the licensees failure to adequately implement procedures during installation and inspection of the HPCS pump breaker from 1994 through October 23, 2002. This performance issue was previously characterized as having low to moderate risk significance (White) in the NRCs final significance determination letter dated March 4, 2003 (VIO 2002008-02). During this supplemental inspection, performed in accordance with Inspection Procedure 95001, significant deficiencies were identified with regard to the licensees extent of condition evaluation.

As a result of these concerns, the white performance issue associated with the HPCS pump failure to start will not be closed at this time.

A. Inspector-Identified and Self-Revealed Findings

Cornerstone: Mitigating Systems

Green.

The inspector identified a licensee performance deficiency involving a Non-Cited Violation for failure to promptly identify and correct a condition adverse to quality. The inspector determined that while the licensees evaluation of the HPCS pump failure to start event properly identified the root cause of this issue to be inadequate procedural guidance for cell switch alignment and inspection, the licensee failed to identify that the same procedural inadequacy existed in other licensee procedures. Specifically, the licensee inaccurately concluded that 5kv and 15kv breaker auxiliary switches were not affected by the issue.

The inspector determined that the finding was more than minor because the failure to adequately identify extent of condition and take corrective actions to address degraded conditions could reasonably be viewed as a precursor to a significant event. This issue was of very low safety significance (Green) because no actual loss of safety function nor initiating event resulted from the performance deficiency. (Section 02.02d)

Licensee-Identified Violations

None.

Report Details 01

INSPECTION SCOPE

The NRC performed this supplemental inspection to assess the licensee's evaluation and corrective actions associated with the failure of the HPCS pump to start due to the licensees failure to adequately implement procedures during installation and inspection of the HPCS pump breaker from 1994 through October 23, 2002. This performance issue was previously characterized as White in the NRCs final significance determination letter dated March 4, 2003.

EVALUATION OF INSPECTION REQUIREMENTS 02.01 Problem Identification a.

Determination of who (i.e., licensee, self-revealing, or NRC) identified the issue and under what conditions As discussed in Inspection Report 50-440/02-08, the issue was self-revealed when the HPCS pump failed to start on October 23, 2002, during routine surveillance testing. The reactor was at full power at the time of the failure. The licensee made the required notifications, took actions to assess and address the component failure, documented the circumstances in their corrective action program, and submitted the appropriate licensee event report.

b.

Determination of how long the issue existed, and prior opportunities for identification Problems with cell switch alignment date back to the licensees initiation of a breaker refurbishment program in 1994. On June 5, 1994, the licensee replaced the EH 1304 breaker (HPCS pump breaker) per the refurbishment plan. As part of this activity, the licensee inspected the cell and auxiliary switches per procedure GEI-0135, ABB Power Circuit Breakers 5KV Types 5HK250 and 5HK350, Rev. 0. The 1994 inspection failed to identify that the cell switch contacts were not in a flat, horizontal position.

On January 22, 1998, the HPCS pump failed to start during performance of licensee Procedure SVI-E22-T2001, HPCS Pump and Valve Operability Test, Rev. 7. The licensee was unable to determine the cause of the failure because the as-found condition was compromised when the control room staff directed the breaker be racked out to the disconnect position. After the breaker was racked in during troubleshooting, the pump was successfully started and Condition Report 98-0125 was closed with no additional corrective action taken.

On June 13, 2001, the A reactor feedwater booster pump failed to start following maintenance activities. In this instance, the licensees troubleshooting successfully identified that the breaker cubicle cell switch was not making sufficient contact on all contact points. Once the problem was identified, the licensee initiated action to adjust the cell switch. The pump was subsequently successfully tested and returned to service. The licensees cause analysis identified that the breaker had been serviced in February 2001 and that the cell switch misalignment should have been identified at that time. The licensee failed to recognize the potential generic issue. As a result of this failure, the only additional corrective action assigned was awareness training for electrical maintenance supervisors and craft which was conducted as part of the electrical shops morning briefings. The inspector concluded that awareness training was ineffective in that a subsequent inspection of the HPCS pump breaker conducted on April 18, 2002, also failed to identify the HPCS pump breaker cell switch misalignment.

c.

Determination of the plant-specific risk consequences (as applicable) and compliance concerns associated with the issue The NRC and the licensee were in agreement that the risk assessments of the HPCS pump failure to start was of low to moderate increased importance to safety - a White inspection finding. The NRC also concluded that the problem was a violation of Technical Specification 5.4 for the failure to implement procedure GEI-0135 during installation and inspection of the HPCS pump breaker from 1994 through October 23, 2002. A Notice of Violation to this effect was transmitted to the licensee in a letter dated March 4, 2003.

This letter also contained the final significance determination for the issue.

02.02 Root Cause and Extent of Condition Evaluation a.

Evaluation of method(s) used to identify root cause(s) and contributing cause(s)

The licensees root cause evaluation of the HPCS pump failure to start used the Event and Causal Factor Charting method to describe the event, identify areas for further investigation, and to identify failure modes. The licensee also used System Improvement, Inc., TapRoot methodology to identify causes due to programmatic and human performance weaknesses. Overall, the two methods used were appropriate to identify the root cause and contributing causes.

The inspector reviewed Revision 1 of the licensees root cause evaluation. The licensee had previously identified errors in the original root cause in March 2003. The errors primarily concerned conflicting root cause statements within the document. Since the errors did not impact the necessary corrective actions, and the document inadequacies were entered into the licensees corrective action program, the inspector determined the issue to be of minor significance.

The licensees root cause analysis report, Failure of the HPCS Pump to Start on Demand, Rev. 1, was approved by licensee management on July 22, 2003. The licensee determined the root cause of the event to be procedure inadequacy in that licensee procedure GEI-135, Rev. 1 did not provide adequate criteria for the inspection of cubicle and auxiliary switches when performing breaker maintenance. The licensee identified the contributing cause to be variation in the dimensions of the replacement breaker that was installed in 1994.

b.

Level of detail of the root cause evaluation The inspector determined that the root cause evaluation was not conducted to a sufficient level of detail in that the extent of condition determination, as discussed in Section 02.02d of this report, was of insufficient scope. Although the licensee correctly determined the root cause of the HPCS pump failure to start as being procedural inadequacy associated with cell switch inspections, the licensees evaluation incorrectly determined the problem not to effect 5kv and 15kv breaker auxiliary switches as discussed in Section 02.02d of this report.

c. Consideration of prior occurrences of the problem and knowledge of prior operating experience The licensees evaluation included a review to see if similar problems had previously been reported with the cell or auxiliary switches. The inspector noted one industry operating experience report which was similar to the Perry HPCS issue, but it involved a breaker auxiliary switch. As stated in the licensees root cause evaluation, the issue was determined not to be an issue at Perry due to auxiliary switches having adequate alignment instruction contained in GEI-0012, Inspection and Cleaning of Electrical Equipment, Rev. 4. As discussed in the subsequent report section, the licensee inaccurately evaluated the effect of GEI-0012 and, as such, the determination of non-applicability was flawed.

d. Consideration of potential common cause(s) and extent of condition of the problem

Introduction.

A Green non-cited violation (NCV) was identified for failure to promptly identify and correct a condition adverse to quality in that while the licensees evaluation properly identified the root cause of this issue to be inadequate procedural guidance for cell switch alignment and inspection, the licensee failed to identify that the same procedural inadequacy existed in other licensee procedures. Specifically, the licensee inaccurately concluded that 5kv and 15kv breaker auxiliary switches were not affected by the issue.

Description.

The inspector identified that the licensees evaluation incorrectly determined the extent of problem at Perry is limited to 5kv cell switches. The 5kv and 15kv auxiliary switches have adequate procedural guidance. The inspector concurred that the licensee had adequate auxiliary switch guidance in licensee procedure GEI-0012, but determined that the periodicity of that procedure was not properly evaluated and did not support the licensees conclusion. GEI-0012 provided instruction for electrical bus maintenance and was performed with a periodicity on the order of 10 years (variable depending on specific component). The breaker maintenance procedures, GEI-0135 and GEI-0136, ABB Power Circuit Breakers 15KV Type 15HK1000 Maintenance, Rev. 1 for 5kv and 15kv switchgear, respectively, were performed more frequently and, most significantly, were the procedures used post-breaker installation. Guidance in GEI-0135 and GEI-0136 was equally inadequate for both cell and auxiliary switches. Thus, auxiliary switches initially aligned or inspected per either GEI-0135 or GEI-0136 were susceptible to the exact same problem as the HPCS pump breaker cell switch until GEI-0012 was properly performed on each individual switch. Absent such action, the condition of the switches could not be summarily dismissed as not a problem.

Analysis.

The inspector determined that the specified corrective actions for 5kv and 15kv auxiliary switches were inadequate due to the erroneous extent of condition determination.

The inspector noted that the situation with respect to 5kv auxiliary switches was significantly mitigated by the inspection efforts of the subject matter expert. Concurrent with the 5kv cell switch inspections, this individual also conducted auxiliary switch inspections. Following these inspections, 16 work orders were generated to adjust 16 auxiliary switches - approximately 20 percent of the switch population. The inspector noted that the licensee missed a significant opportunity to reevaluate their extent of condition determination given the relatively high percentage of auxiliary switches in need of adjustment. The inspector also noted that procedural changes to GEI-0135 to enhance cell switch inspection criteria were also applicable to the auxiliary switches. However, the inspector determined that no mitigating circumstances existed with respect to 15kv breakers. The inspection procedure had not been revised at the time of the inspection nor had the position of these switches been reviewed.

The inspector determined that the finding was more than minor because the failure to adequately identify extent of condition and take corrective actions to address degraded conditions could reasonably be viewed as a precursor to a significant event. Although the 15kv breakers were non-safety, the inspector noted the risk significant loads which could have been adversely impacted included the 13.8kV bus L10 which supplies the Class 1E 4.16kV buses, the motor driven feedpump, and the recirculation pump motor fast speed supply breaker. Due to the loads potentially affected, the inspector determined the performance deficiency to be a potential transient initiator contributor in accordance with the Significance Determination Process (SDP) Phase 1 Screening Worksheet. The inspector, however, determined the event to be of very low safety significance (Green)because all questions in the initiating event column of the SDP Phase 1 Screening Worksheet were answered no.

Enforcement.

Appendix B, Criterion XVI of 10 CFR Part 50 requires, in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and nonconformances are promptly identified and corrected. Contrary to this, the licensee failed to perform an adequate identification of extent of condition review as part of the HPCS pump failure to start root cause investigation and therefore failed to promptly identify and correct potentially degraded conditions. Because this violation was of very low safety significance and was entered into the licensees corrective action program, this violation is being treated as a NCV, consistent with Section VI.A of the NRC Enforcement Policy. (NCV 50-440/2003007-01)02.03 Corrective Actions a.

Appropriateness of corrective action(s)

The licensee took immediate corrective actions to make the HPCS system operable.

After troubleshooting identified the cause of the failure, the switch was promptly adjusted and the pump successfully tested.

Licensee corrective actions with respect to 5kv cell switches were generally prompt and thorough. Although the inspector noted that training on the breaker inspection procedures was not being expeditiously pursued, the inspector did note the personal involvement of the subject matter expert in switch inspections and adjustments, and determined that to be an effective interim corrective action. The inspector observed two cell switch adjustments and inspected three cell switches which the licensee determined to not require adjustment to verify proper alignment. No issues of significance were identified.

The inspector determined that specified corrective actions for 5kv and 15kv auxiliary switches were inadequate due to the erroneous extent of condition determination.

b.

Prioritization of corrective actions With respect to the 5kv cell switches, the actions were properly prioritized. The 5kv auxiliary switches were also, fortuitously, addressed in conjunction with the cell switches.

At the time of the inspection no actions had been identified for the 15kv auxiliary switches and as such, the prioritization review was not applicable.

c.

Establishment of schedule for implementing and completing the corrective actions With respect to the 5kv cell switches, the actions were implemented or are scheduled for implementation. The 5kv auxiliary switches were also, fortuitously, addressed in conjunction with the cell switches.

At the time of the inspection no actions had been identified for the 15kv switches and as such, the implementation review was not applicable.

d.

Establishment of quantitative or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence:

The licensee has an open action in their corrective action program to perform an effectiveness review. This review was scheduled beyond the completion date of this inspection but before the end of this year.

03.

MANAGEMENT MEETINGS

Exit Meeting Summary

The inspector presented the inspection results to Mr. W. Kanda and other members of licensee management at the conclusion of the inspection on August 1, 2003. The licensee acknowledged the findings presented. No proprietary information was identified.

ATTACHMENTS:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

W. Kanda, Vice President-Nuclear

P. Arthur, Manager, Work Control Section

P. Chatterjee, Electrical Engineer

K. Cimorelli, Acting Director, Nuclear Maintenance

M. Humphrey, Root Cause Coordinator, Work Control Section

D. Miller, Engineer, Compliance

V. Higaki, Manager, Regulatory Affairs

J. Lausberg, Supervisor, Compliance

T. Rausch, General Manager, Nuclear Power Plant Department

D. Watkins, Maintenance Engineer

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

50-440/2003007-01 NCV Inadequate Identification of Extent of Condition Associated With High Pressure Core Spray Pump Failure to Start

Opened and Closed

None

Closed

None

Discussed

50-440/2003008-02 VIO High Pressure Core Spray Pump Failure to Start