IR 05000261/2010014
Download: ML103410289
Text
December 7, 2010
EA-10-205 Mr. RobertII Vice President Carolina Power and Light Company H. B. Robinson Steam Electric Plant, Unit 2 3851 West Entrance Road Hartsville, SC 29550
SUBJECT: H. B. ROBINSON STEAM ELECTRIC PLANT - FINAL SIGNIFICANCE DETERMINATION OF ONE WHITE FINDING, NOTICE OF VIOLATION AND ASSESSMENT LETTER (NRC INSPECTION REPORT 05000261/2010014)
Dear Mr. Duncan:
This letter provides you with the final significance determination of two apparent violations (AVs) identified in Nuclear Regulatory Commission (NRC) Inspection Report (IR) No. 05000261/2010006, dated October 8, 2010.
The first AV was assessed using the Significance Determination Process and was preliminarily characterized as White, a finding of low to moderate safety significance. The finding involved: (1) the failure to promptly correct a condition adverse to quality involving the failure of "B" Emergency Diesel Generator (EDG) output breaker 52/27B to close in October 2008, as required by 10 CFR 50, Appendix B, Criterion XVI; and (2) the failure to ensure the "B" Emergency Diesel Generator remained operable as required by Technical Specification 3.8.1. A second AV involving the requirements of 10 CFR 50.9 was under consideration for escalated enforcement under the NRC's traditional enforcement process, and was related to the licensee's submission of materially inaccurate information to the NRC in Licensee Event Report (LER) 05000261/2009-001-000.
At your request, a Regulatory and Pre-decisional Enforcement Conference was held on November 8, 2010 to discuss your views on these issues. A meeting summary was issued on November 16, 2010, which included copies of the slide presentation made by Carolina Power and Light Company (doing business as Progress Energy Carolinas Inc (PEC)) (ADAMS Accession # ML103200618). During the conference, PEC representatives discussed the circumstances associated with the "B" EDG output breaker failures, the AV related to materially inaccurate information, and the cause analyses and corrective actions taken for both AVs.
CP&L 2 At the conference, PEC did not disagree with the NRC's characterization of the two issues as violations of regulatory requirements. However, for the issue assessed under the Significance Determination Process, PEC concluded that the 10 CFR 50, Appendix B, Criterion XVI violation was not reasonably within their ability to promptly identify and correct. PEC staff agreed that the first failure of the "B" EDG output breaker should have been entered into the Corrective Action Program (CAP) as a Nuclear Condition Report (NCR) for a more methodical problem solving process. However, PEC concluded that based on the randomness of the failure mode, their CAP would not have identified the cause of the breaker failures.
Additionally, PEC did not disagree with the NRC's preliminary risk determination discussed in IR 05000261/2010006, which resulted in a finding of low to moderate safety significance (White). However, PEC disagreed that the failure to enter the first breaker failure into the CAP should be linked to any period of EDG inoperability. After further review of the licensee's position, the NRC has concluded that the failure to correct the first "B" EDG output breaker failure in October 2008 constituted a performance deficiency which resulted in the inoperability of the "B" EDG in April 2009. As discussed in NRC Inspection Manual Chapter 0612, Power Reactor Inspection Reports, a performance deficiency is defined as an issue that is the result of a licensee not meeting a requirement or standard where the cause was reasonably within the licensee's ability to foresee and correct and therefore should have been prevented. The NRC determined that the first breaker failure, including the specific nature of the symptoms observed by maintenance personnel, provided a reasonable opportunity to identify and pursue correction of the breaker failure mode and therefore prevent the inoperability of the "B" EDG. Following the two occasions when the breaker failed to close in October 2008, PEC did not take corrective actions commensurate with the safety significance of the equipment or the narrow nature of the symptoms, as evidenced by the return of the breaker to service without performing thorough troubleshooting and without attempting to determine the exact cause of its failure. The NRC concluded that had H. B. Robinson Plant personnel dedicated the same level of effort and attention to the first breaker failure in accordance with the plant's CAP, as they did for the second failure approximately six months later, the inoperability of the "B" EDG could have been prevented.
After considering the information gathered during the inspection and the information provided by PEC during the conference, the NRC has concluded that the finding related to the failure to promptly correct the failure of "B" EDG output breaker to close in October 2008, as required by 10 CFR 50, Appendix B, Criterion XVI, should be characterized as a White finding of low to moderate safety significance. The bases for the NRC's significance determination of this finding are documented in NRC IR 05000261/2010006. Additionally, the differences between the licensee's and NRC's characterization of the finding are discussed in Enclosure 2. The key difference in the characterization was the NRC's determination that the cause of the finding was reasonably within the licensee's ability to foresee and correct. Finally, the NRC determined that this finding is a violation of 10 CFR 50, Appendix B, Criterion XVI, "Corrective Action," as cited in the enclosed Notice of Violation (Enclosure 1). The circumstances associated with the violation are described in detail in IR 05000261/2010006. In accordance with the NRC Enforcement Policy, the Notice is considered escalated enforcement action because it is related to a White finding.
CP&L 3 Regarding the AV involving materially inaccurate information that was assessed under the NRC's traditional enforcement process, based on the information gathered during the inspection and the information provided at the conference, it has been determined that a violation of NRC requirements occurred. The violation is cited in the enclosed Notice of Violation and the circumstances associated with the violation are described in IR 05000261/2010006. The cited violation involved the requirements of 10 CFR 50.9, which states, in part, that information provided to the Commission by a licensee shall be complete and accurate in all material respects. On June 18, 2009, PEC provided information to the NRC in LER 2009-001 that was not complete and accurate in all material respects. In this case, the violation impacted the regulatory process, in that the NRC relied on the information to exercise enforcement discretion for a violation that would likely have resulted in additional inspection effort. The information was material, for had it been complete and accurate at the time provided, it likely would have resulted in a reconsideration of a regulatory position or substantial further NRC inquiry. Based on the above, the NRC has concluded that the violation of 10 CFR 50.9 is appropriately characterized at Severity Level III, in accordance with the NRC Enforcement Policy.
In accordance with the Enforcement Policy, a base civil penalty in the amount of $70,000 is considered for each Severity Level III violation (i.e., the 10 CFR 50.9 violation). Because your facility has not been the subject of escalated enforcement action within the past two years, the NRC considered whether credit was warranted for Corrective Action in accordance with the civil penalty assessment process in Section VI.C.2 of the Enforcement Policy. As PEC presented at the conference, corrective actions regarding the 10 CFR 50.9 violation included, in part: (1) initiation of a CAP evaluation to strengthen processes that ensure valid information is provided to regulatory authorities; (2) correction of the work order, NCR, and LER associated with the violation; (3) development of interim information validation guidance for regulatory correspondence across the fleet; (4) revision of fleet procedural guidance pertaining to validation of information used in regulatory correspondence; (5) training and counseling for fleet licensing and maintenance personnel; and (6) fleet wide operating experience report issued to reinforce the importance of providing complete and accurate information to the NRC. Based on the above, credit is warranted for the factor of Corrective Action regarding the 10 CFR 50.9 violation. The NRC notes that the 10 CFR 50, Appendix B, Criterion XVI violation was dispositioned in accordance with the NRC's Reactor Oversight Process and is not subject to the civil penalty assessment process of the Enforcement Policy. Therefore, to encourage prompt and comprehensive correction of violations, I have been authorized, after consultation with the Director, Office of Enforcement, to propose that a civil penalty not be assessed in this case. However, significant violations in the future could result in a civil penalty. You have 30 calendar days from the date of this letter to appeal the staff's significance determination for the White finding or the Notice of Violation. An appeal of the White finding will be considered to have merit only if it meets the criteria given in NRC Inspection Manual Chapter 0609, Attachment 2.
As discussed in NRC Integrated Inspection Report 05000261/2010004, the NRC assessed the H.B. Robinson Steam Electric Plant's performance to be in the Regulatory Response Column of the NRC's Action Matrix due to the Unplanned Scrams per 7000 Critical Hours performance indicator crossing the green-to-white threshold. As a result of the incorporation of the White inspection finding discussed in this report into the plant performance review, we have assessed CP&L 4 the plant performance to remain in the Regulatory Response Column of the NRC's Action Matrix. We will conduct a supplemental inspection (Inspection Procedure 95001) when you notify us of your readiness for the NRC to review the actions taken to address the White Performance Indicator and the White inspection finding. In addition, issuance of the Severity Level III violation constitutes escalated enforcement action that may subject you to increased inspection effort. The NRC has concluded that information regarding the reason for the violations, the corrective actions taken and planned to correct the violations and prevent recurrence, and the date when full compliance was achieved is already adequately addressed on the docket in this letter and in the information presented by PEC at the conference. Therefore, you are not required to respond to this letter unless the description herein does not accurately reflect your corrective actions or your position. In that case, or if you choose to provide additional information, you should follow the instructions specified in the enclosed Notice. For administrative purposes, this letter is issued as NRC Inspection Report No.
05000261/2010014. Accordingly, AVs05000261/2010006-02 and 05000261/2010006-03 are updated consistent with the regulatory positions described in this letter. Therefore: (1) AV 05000261/2010006-02, Failure to Correct a Condition Adverse to Quality in the "B" Emergency Diesel Generator Output Breaker 52/27B, is updated as VIO 05000261/2010006-02 with a safety significance of White and a cross-cutting aspect in the area of Problem Identification and Resolution, P.1(a); and (2) AV 05000261/2010006-03, Materially Inaccurate and Incomplete Information Provided to the NRC in LER 2009-001 which Impacted the Regulatory Process, is updated as Severity Level III violation VIO 05000261/2010006-03. In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosures, and your response, if you choose to submit one, will be made available electronically for public inspection in the NRC Public Document Room or from the NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html. To the extent possible, your response should not include any personal privacy, proprietary, or safeguards information so that it can be made available to the Public without redaction. If personal privacy or proprietary information is necessary to provide an acceptable response, please provide a bracketed copy of your response that identifies the information that should be protected and a redacted copy of your response that deletes such information. If you request withholding of such information, you must specifically identify the portions of your response that you seek to have withheld and provide in detail the bases for your claim of withholding (e.g., explain why the disclosure of information will create an unwarranted invasion of personal privacy or provide the information required by 10 CFR 2.390(b) to support a request for withholding confidential commercial or financial information). The NRC also includes significant enforcement actions on its Web site at http://www.nrc.gov/reading-rm/doc-collections/enforcement/actions.
CP&L 5 Should you have any questions concerning this letter, please contact Mr. George Hopper at 404-997-4645.
Sincerely,/RA/ Luis A Reyes Regional Administrator Docket No.: 50-261 License No.: DPR-23
Enclosures:
Enclosure 1 - Notice of Violation Enclosure 2 - NRC Bases for Final Characterization and Significance Determination cc w/encls: (See page 6)
CP&L 5 Should you have any questions concerning this letter, please contact Mr. George Hopper at 404-997-4645.
Sincerely,/RA/ Luis A Reyes Regional Administrator Docket No.: 50-261 License No.: DPR-23
Enclosures:
Enclosure 1 - Notice of Violation Enclosure 2 - NRC Bases for Final Characterization and Significance Determination cc w/encl: (See page 6) X PUBLICLY AVAILABLE NON-PUBLICLY AVAILABLE SENSITIVE X NON-SENSITIVE ADAMS: X Yes ACCESSION NUMBER:__ ML103410289_____ X SUNSI REVIEW COMPLETE OFFICE RII:EICS RII: DRP RII:DRPRII:DRSRII:DRPRII:ORANRR OESIGNATURE /RA/ /RA/ /RA/ /RA/ /RA/ /RA/ NAME CEvans LWert RMusser GMacDonald GHopper LReyes DATE 12/07/2010 12/07/2010 12/07/2010 11/29/2010 12/07/2010 12/07/2010 E-MAIL COPY? YES NO YES NO YES NO YES NO YES NO YES NO YES NO YES NO OFFICE RII:DRS SIGNATURE /RA/ NAME JRivera-Ortiz DATE 12/07/2010 E-MAIL COPY? YES NO OFFICIAL RECORD COPY DOCUMENT NAME: S:\DRP\RPB7\PI&R\PI&R\INSPECTION REPORTS\ROBINSON\ROBINSON FINAL SDP LETTER IR 2010014 (EA-10-205).DOCX CP&L 6 cc w/encls: Division of Radiological Health TN Dept. of Environment & Conservation Electronic Mail Distribution Eric McCartney Vice President H. B. Robinson Steam Electric Plant Unit 2 Carolina Power & Light Company Electronic Mail Distribution Sandra Threatt, Manager Nuclear Response and Emergency Environmental Surveillance Bureau of Land and Waste Management Department of Health and Environmental Control Electronic Mail Distribution R. J. Duncan, II Vice President Nuclear Operations Carolina Power & Light Company Electronic Mail Distribution Brian C. McCabe Manager, Nuclear Regulatory Affairs Progress Energy Carolinas, Inc. Electronic Mail Distribution Christos Kamilaris Director Fleet Support Services Carolina Power & Light Company Electronic Mail Distribution Curt A. Castell Supervisor Licensing/Regulatory Programs Carolina Power & Light Company Electronic Mail Distribution B. C. White Manager Support Services - Nuclear Carolina Power & Light Company Electronic Mail Distribution S. D. West Superintendent Security H. B. Robinson Steam Electric Plant Progress Energy Electronic Mail Distribution Joseph W. Donahue Vice President Nuclear Oversight Carolina Power and Light Company Electronic Mail Distribution David T. Conley Associate General Counsel Legal Dept. Progress Energy Service Company, LLC Electronic Mail Distribution John H. O'Neill, Jr. Shaw, Pittman, Potts & Trowbridge 2300 N. Street, NW Washington, DC 20037-1128 Susan E. Jenkins Director, Division of Waste Management Bureau of Land and Waste Management S.C. Department of Health and Environmental Control Electronic Mail Distribution Scott Saunders Plant General Manager Carolina Power & Light Company Electronic Mail Distribution Robert P. Gruber Executive Director Public Staff - NCUC 4326 Mail Service Center Raleigh, NC 27699-4326 W. Lee Cox, III Section Chief Radiation Protection Section N.C. Department of Environmental Commerce & Natural Resources Electronic Mail Distribution (cc w/encls continued next page)
CP&L 7 (cc w/encls continued) Mark Yeager Division of Radioactive Waste Mgmt. S.C. Department of Health and Environmental Control Electronic Mail Distribution Public Service Commission State of South Carolina P.O. Box 11649 Columbia, SC 29211 Chairman North Carolina Utilities Commission Electronic Mail Distribution CP&L 6 Letter to R.from L. Reyes dated December 7, 2010
SUBJECT: H. B. ROBINSON STEAM ELECTRIC PLANT - FINAL SIGNIFICANCE DETERMINATION OF ONE WHITE FINDING, NOTICE OF VIOLATION AND ASSESSMENT LETTER (NRC INSPECTION REPORT 05000261/2010014) Distribution w/encls: R. Borchardt, OEDO R. Zimmerman, OE E. Julian, SECY B. Keeling, OCA Enforcement Coordinators, RI, RIII, RIV E. Hayden, OPA C. McCrary, OI H. Bell, OIG E. Leeds, NRR M. Ashley, NRR T. Orf, NRR C. Scott, OGC D. Decker, OCA G. Gulla, OE J. Circle, NRR L. Reyes, RII V. McCree, RII C. Christensen, RII L. Wert, RII J. Munday, RII G. Hopper, RII R. Musser, RII J. Hickey, RII J. Worosilo, RII J. Rivera, RII S. Sparks, RII C. Evans, RII L. Douglas, RII OEMAIL RIDSNRRDIRS PUBLIC Enclosure 1 NOTICE OF VIOLATION Carolina Power & Light Company Docket No.: 50-261 H.B. Robinson Steam Electric Plant License No.: DPR-23 Unit 2 EA-10-205 During an inspection and in-office review completed on August 26, 2010, violations of NRC requirements were identified. In accordance with the NRC Enforcement Policy, the violations are set forth below:
A. 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, requires in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, and non-conformances are promptly identified and corrected. Technical Specifications (TS) 3.8.1, Condition B, requires that an inoperable Emergency Diesel Generator (EDG) shall be restored to operable status within 7 days. Contrary to the above, on October 15, 2008, Carolina Power and Light Company (Licensee) failed to assure that a condition adverse to quality, involving an EDG output breaker 52/27B failure-to-close malfunction, was promptly corrected. Specifically, indications of control relay malfunction during post-modification testing existed but were not identified and corrected. As a result, a similar malfunction during a surveillance test caused the "B" EDG to become inoperable, from March 28 to April 23, 2009, which exceeded the TS allowed outage time. This violation is associated with a White Significance Determination Process finding. B. 10 CFR 50.9(a) requires, in part, that information provided to the Commission by a licensee shall be complete and accurate in all material respects. Contrary to the above, on June 18, 2009, the Licensee submitted information that was not complete and accurate in all material respects. Specifically, the Licensee submitted Licensee Event Report (LER) 05000261/2009-001-00 which described the corrective actions taken on October 15, 2008, for a similar "B" EDG output breaker failure. The LER stated that the breaker was tested in accordance with Preventive Maintenance (PM) Procedure, PM-163, "Inspection and Testing of Circuit Breakers for 480 Volt Bus E2" and that the procedure was successfully completed. The NRC determined that the Licensee did not conduct full testing as stated, and had only completed the instructions for returning the breaker to service. The information provided in the LER was material because the NRC relied on this information to exercise enforcement discretion for the 2009 failure which would likely have resulted in an additional inspection effort. This is a Severity Level III violation (Enforcement Policy, Supplement VII.C.1).
The NRC has concluded that information regarding the reasons for the violations, the corrective actions taken or planned to correct the violations and prevent recurrence, and the date when full compliance was achieved is already adequately addressed on the docket, and in the information presented by PEC at the November 8, 2010, Regulatory and Predecisional Enforcement Conference. However, you are required to submit a written statement or explanation pursuant NOV 2 EA-10-205 Enclosure 1 to 10 CFR 2.201 if the description therein does not accurately reflect your corrective actions or your position. In that case, or if you choose to respond, clearly mark your response "Reply to a Notice of Violation EA-10-205," and send it to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C. 20555 with a copy to the Regional Administrator, Region II, within 30 days of the date of the letter transmitting this Notice of Violation (Notice). If you contest this enforcement action, you should also provide a copy of your response, with the basis for your denial, to the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001.
Should you choose to respond, your response will be made available electronically for public inspection in the NRC Public Document Room or from the NRC's document system (ADAMS). To the extent possible, it should not include any personal privacy, proprietary, or safeguards information so that it can be made available to the public without redaction. ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html. If personal privacy or proprietary information is necessary to provide an acceptable response, then please provide a bracketed copy of your response that identifies the information that should be protected and a redacted copy of your response that deletes such information. If you request that such material be withheld from public disclosure, you must specifically identify the portions of your response that you seek to have withheld and provide in detail the bases for your claim (e.g., explain why the disclosure of information will create an unwarranted invasion of personal privacy or provide the information required by 10 CFR 2.390(b) to support a request for withholding confidential commercial or financial information). If safeguards information is necessary to provide an acceptable response, please provide the level of protection described in 10 CFR 73.21.
In accordance with 10 CFR 19.11, you may be required to post this Notice within two working days. Dated this 7th day of December 2010
Enclosure 2 NRC Bases for Final Characterization and Significance Determination On November 8, 2010, the Nuclear Regulatory Commission (NRC) held a regulatory conference with representatives of Progress Energy Carolinas Inc (PEC), H. B. Robinson Steam Electric Plant, to discuss a preliminary White inspection finding documented in NRC Inspection Report (IR) Nos. 05000261/2010006, dated October 8, 2010. This finding involved: (1) the failure to promptly correct a condition adverse to quality involving the failure of "B" Emergency Diesel Generator (EDG) output breaker 52/27B to close in October 2008, as required by 10 CFR 50, Appendix B, Criterion XVI; and (2) the failure to ensure the "B" Emergency Diesel Generator remained operable as required by Technical Specification 3.8.1. At the regulatory conference, PEC did not disagree with the NRC's preliminary risk determination discussed in inspection report 05000261/2010006, which resulted in a finding of low to moderate safety significance (White). However, PEC disagreed that the identification of the breaker failure mode was reasonably within their ability to promptly identify and correct. Therefore, PEC disagreed that the failure to enter the first breaker failure in the Corrective Action Program (CAP) and take the necessary actions to correct that condition adverse to quality should be linked to any period of EDG inoperability. The paragraphs below provide a summary of the technical differences and the NRC's bases for determining that the inoperability of the "B" EDG was reasonably within the licensee's ability to foresee and correct and should have been prevented.
1. PEC Input - At the conference, PEC recognized that a Nuclear Condition Report (NCR) should have been initiated to address the first breaker failure in 2008. However, the licensee stated that while this may have resulted in more methodical problem solving process, it would not have resulted in the discovery of the breaker failure mode.
NRC's Position - The NRC determined that the failure to initiate an NCR to methodically evaluate the breaker failure would likely have resulted in the discovery of the failure mode, preventing the inoperability of the "B" EDG five months later.
According to the Progress Energy CAP procedure (CAP-NGGC-0200) at the time of the failure, when an NCR is initiated to address an adverse condition it is processed through several stages until the issue is resolved. First, the Originating Supervisor reviews the NCR and prioritizes the condition in accordance with the procedure guidance. Then, the Superintendent of Operations and Licensing staff review the NCRs that have been identified as potential operability/reportability concerns by the Originating Supervisor. Then, the Unit/Section evaluators review the NCRs to verify they are characterized in accordance with the procedure, which includes correct priority, NCR ownership, and assignment of cause investigations commensurate with the NCR priority. Depending on the type of adverse condition and priority, the issue is evaluated using cause investigation tools documented in licensee procedures. The adverse condition is finally resolved through a corrective action plan that directly addresses the cause(s) identified by the cause analysis. Based on the procedural guidance in CAP-NGGC-0200, the failure of the "B" EDG output breaker would be have been at least a Priority 2 NCR, which would have required, as a minimum, an apparent cause evaluation.
2 Enclosure 2 As an example of the effectiveness and capability of this process, the second breaker failure in April 2009 was entered in the CAP through an NCR which resulted in a systematic problem solving approach. Specifically, an initial Five Step Problem Solving Plan (troubleshooting) was developed and implemented, which found no root cause for the breaker failure. Then, engineering personnel developed and executed a Complex Troubleshooting Plan, which also found no root cause for the breaker failure. Consequently, engineering developed and executed a Failure Mode Tree and Failure Mode Cause Table (Support/Refute Methodology), which also found no root cause for the breaker failure. Finally, the licensee made a decision to send the suspect breaker to the vendor facility to have further investigation performed on the breaker, which eventually identified the cause of the breaker failure. Therefore, the NRC determined that in October 2008 the licensee had an equal opportunity to implement all the actions taken six months later to investigate the cause of the second breaker failure. The NRC did not identify any circumstances associated with the first breaker failure that would have prevented the implementation of a thorough investigation to determine the cause of the breaker failure and prevent future operability concerns. Additionally, the first breaker failure in 2008 showed even more specific indications that the breaker control relay was experiencing mechanical binding. This was evidenced by the auditory and visual indications witnessed by maintenance personnel, which led the maintenance staff to mechanically manipulate the control relay and generate a work order that specifically described potential problems with the control relay. Based on the integrated assessment of the information concerning the facts related to the breaker failures in October 2008 and April 2009, obtained from interviews with licensee personnel, licensee and vendor cause determination documents, and the regulatory conference, the NRC concluded that there was reasonable assurance that both failures were likely caused by the same failure mechanism. 2. PEC Input - At the conference, PEC stated that in response to the 2008 EDG output breaker failure, the breaker was electrically cycled successfully approximately seven times. Since the failure could not be repeated and no abnormalities were identified, the control relay was not replaced and the breaker was returned to service. NRC's Position - The NRC concluded that the evaluation of the condition adverse to quality (to include troubleshooting, failure mode analysis, and extent of cause and condition) and the extent of corrective actions taken to address the first breaker failure was the licensee's main deficiency in the resolution of this issue. Upon recognition of the failure of a safety-significant and safety-related component, the licensee limited their investigative effort to cycling the breaker seven times and then returning the component back to service although more specific degradation symptoms were shown by the first failure when compared to the April 2009 failure. The NRC determined that returning the breaker to service without further effort to identify the cause of the problem was not commensurate with the safety significance of the equipment or the nature of the failure mode. This led the NRC to conclude that the licensee identified a condition adverse to quality and did not take the necessary actions to correct it.
3 Enclosure 2 3. PEC Input - At the conference, PEC stated that the identification of the breaker failure mode occurred during extraordinary investigation and testing by the vendor and RNP staff.
The licensee also stated that the randomness of the failure mode made its identification uncertain, even with extraordinary inspection and testing. Furthermore, the licensee sated that discovery of the failure mode was by chance during repeated testing at vendor facility.
NRC's Position - After reviewing the information gathered during the inspection and the information presented at the regulatory conference, the NRC recognized the difficulties and challenges associated with the discovery of the breaker failure mode. However, the NRC disagrees with the characterization that the identification of the breaker failure mode occurred during "extraordinary" investigation and testing. Based on the review of the vendor proprietary causal investigation report and the information provided by PEC at the conference, the NRC noted that the troubleshooting activities at the vendor facility consisted of cycling the breaker multiple times in a specific test cell for that breaker model at different voltage conditions until any malfunction was observed. The voltages applied to the control relay included normal plant condition voltage, nominal voltage, minimum design voltage, and maximum design voltage. Once the breaker failure was repeated, the failed condition of the breaker was preserved and a detailed visual inspection was conducted, which resulted in the successful identification of the cotter pin failure mechanism. The NRC did not consider any special testing process that would make this troubleshooting effort an extraordinary one, since the testing involved breaker cycling at expected operating conditions. Additionally, the NRC concluded that the discovery of the failure mode was the result of a deliberate and focused troubleshooting effort as evidenced by the methodical testing process implemented and the attention to important details such as preserving the failed state of the breaker. Therefore, the NRC concluded that the testing conducted at the vendor facility involved a systematic and thoughtful approach to troubleshooting, was not of extraordinary nature, and the success in identifying the failure mode was accomplished by cycling the breaker sufficient times to evoke the random probability of failure. 4. PEC Input - At the conference, PEC stated that it was not reasonable to conclude that the performance of preventive maintenance procedure PM-163 would have resulted in identification and prompt correction of the cotter pin issue, assuming the cotter pin was the failure mode in 2008.
NRC's Position - The NRC recognizes that the performance of procedure PM-163 would not have assured the discovery of the failure mode. However, the full completion of this procedure would have resulted in additional cycling of the breaker, which could have helped to repeat the breaker failure. In addition, as was evidenced in the investigation of the April 2009 failure, the failure of PM-163 to identify the failure mode was an essential and needed step to move the investigation to more comprehensive troubleshooting and failure modes analysis, ultimately leading to a vendor review. Regardless the capability of that procedure to identify the failure mode, and the NRC's understanding that the licensee fully implemented this comprehensive maintenance procedure to address the first breaker failure was considered a reasonable effort to investigate the cause of the breaker failure. However, once the NRC Problem Identification and Resolution inspection discovered that formal procedural testing using PM-163 was not performed and PM-163 was only used to return the breaker to service, the NRC concluded that the necessary corrective actions to address the breaker failure were not taken.
4 Enclosure 2 5. PEC Input - At the conference, PEC stated that had the control relay been replaced in 2008 per the work order instructions, the cotter pin would not have been part of the component replaced. Additionally, PEC stated that had the entire breaker been replaced in 2008, the cotter pin failure mode would still have remained undiscovered.
NRC's Position - The NRC concluded that had the breaker been replaced in 2008, the new breaker would likely have prevented the inoperability of the EDG in 2009 because the licensee's extent of condition review, following the failure in April 2009, concluded that the failed "B" EDG output breaker was the only breaker with cotter pin legs long enough to have caused mechanical binding of the control relay lift linkage. The susceptibility of the breakers to fail relied on the configuration of the cotter pin (i.e. how the cotter pin legs were bent and, in particular, the length of the legs) during installation by the vendor, which varied between breakers. Even though other similar breakers were considered potentially susceptible to the same failure mode, the breaker 52/27B that failed was the only one confirmed to be fully susceptible to fail due to the cotter pin properties. Therefore, even though removing the breaker from service as soon as it showed control relay malfunction indications may not have resulted in the immediate discovery of the failure mode at that time, it would have resulted in the removal of a defective component from a safety related application, which could have prevented the inoperability of the EDG. Furthermore, replacing the defective breaker would have provided a reasonable opportunity to conduct additional investigation of the failure, which may have resulted in an earlier identification of the failure mechanism. Finally, regardless of the discovery of the failure mode, replacing the control relay or the entire breaker would have been considered by the NRC as a reasonable attempt to correct the condition adverse to quality based on the observed symptoms. Conclusion:
Based on the information gathered during the inspection, the information presented by the licensee at the regulatory conference, and in consideration of the above, the NRC determined that the cause of the "B" EDG inoperability in April 2009 was reasonably within the licensee's ability to foresee and correct, and should have been prevented since the same component failed twice in October 2008 providing warning of mechanical binding degradation in the breaker control relay. The NRC concluded that licensee's effort to identify the cause of the breaker failure in 2008 was limited and not commensurate to the safety significance of the equipment or the nature of the failure mode. The NRC did not identify any circumstances associated with the October 2008 breaker failure that would have prevented the implementation of a thorough investigation to determine the cause of the breaker failure (as occurred in April 2009). In addition, the NRC determined that the October 2008 failure provided even more specific indications that the breaker control relay was experiencing mechanical binding, which should have driven the licensee to conduct a thorough investigation to identify the cause of the failure in order to prevent future operability concerns. As discussed in Enclosure 2 of IR 05000261/2010006, the NRC determined that the delta core damage frequency (CDF) increase for the finding associated with the failure to promptly correct a condition adverse to quality involving the failure of "B" EDG output breaker 52/27B to close after degradation was identified, is approximately 7E-6 (White). This significance determination was based on a 26 days exposure time using the best-estimate assumptions and with common cause adjustment and no recovery.