ML103200618
| ML103200618 | |
| Person / Time | |
|---|---|
| Site: | Robinson  |
| Issue date: | 11/16/2010 |
| From: | Hopper G Reactor Projects Branch 7 |
| To: | Duncan R Carolina Power & Light Co |
| References | |
| Download: ML103200618 (57) | |
Text
UNITED STATES NUCLEAR REGULATORY COMMISSION REGION II 245 PEACHTREE CENTER AVENUE NE SUITE 1200 ATLANTA, GEORGIA 30303-1257 November 16, 2010 Mr. Robert J. Duncan, II Vice President Carolina Power and Light Company H. B. Robinson Steam Electric Plant Unit 2 3581 West Entrance Road Hartsville, SC 29550
SUBJECT:
PUBLIC MEETING
SUMMARY
- CATEGORY ONE - REGULATORY AND ENFORCEMENT CONFERENCE MEETING - DOCKET NO. 50-261
Dear Mr. Duncan:
This refers to the Category 1 public meeting which was held on November 08, 2010, in Atlanta, GA. The purpose of this Regulatory/Enforcement Conference, scheduled at your request, was to discuss the safety significance of one preliminary White finding with one associated Apparent Violation (AV) and the severity level of one AV considered for potential Escalated Enforcement that were documented in NRC Inspection Report 05000261/2010006 (ML102810633). These findings dealt with degradation of the B Emergency Diesel Generator (EDG) output breaker. A listing of meeting attendees and information presented during the meeting are enclosed.
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter will be available electronically for public inspection in the NRC Public Document Room (PDR) or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS).
ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (Public Electronic Reading Room).
Should you have any questions concerning this meeting, please contact me at (404) 997-4645.
Sincerely,
/RA/
George T. Hopper, Chief Reactor Projects Branch 7 Division of Reactor Projects Docket Nos.: 50-261 License Nos.: DPR-23
Enclosures:
- 1. List of Attendees
- 2. Agenda and Summary of Violations
- 3. Progress Energy Presentation cc w/encls: (See page 2)
November 16, 2010 Mr. Robert J. Duncan, II Vice President Carolina Power and Light Company H. B. Robinson Steam Electric Plant Unit 2 3581 West Entrance Road Hartsville, SC 29550
SUBJECT:
PUBLIC MEETING
SUMMARY
- CATEGORY ONE - REGULATORY AND ENFORCEMENT CONFERENCE MEETING DOCKET NO. 50-261
Dear Mr. Duncan:
This refers to the Category 1 public meeting which was held on November 08, 2010, in Atlanta, GA. The purpose of this Regulatory/Enforcement Conference, scheduled at your request, was to discuss the safety significance of one preliminary White finding with one associated Apparent Violation (AV) and the severity level of one AV considered for potential Escalated Enforcement that were documented in NRC Inspection Report 05000261/2010006 (ML102810633). These findings dealt with degradation of the B Emergency Diesel Generator (EDG) output breaker. A listing of meeting attendees and information presented during the meeting are enclosed.
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter will be available electronically for public inspection in the NRC Public Document Room (PDR) or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (Public Electronic Reading Room).
Should you have any questions concerning this meeting, please contact me at (404) 997-4645.
Sincerely,
/RA/
George T. Hopper, Chief Reactor Projects Branch 7 Division of Reactor Projects Docket Nos.:
50-261 License Nos.:
Enclosures:
- 1. List of Attendees
- 2. Agenda and Summary of Violations
- 3. Progress Energy Presentation cc w/encls: (See page 2)
X PUBLICLY AVAILABLE G NON-PUBLICLY AVAILABLE G SENSITIVE X NON-SENSITIVE ADAMS: X Yes ACCESSION NUMBER:_________________________
X SUNSI REVIEW COMPLETE OFFICE RII:DRP RII:DRP RII:DRP SIGNATURE GTH1 RAM JGW1 NAME GHopper GMusser JWorosilo DATE 11/16/2010 11/16/2010 11/16/2010 E-MAIL COPY?
YES NO YES NO YES NO YES NO YES NO YES NO YES NO OFFICIAL RECORD COPY DOCUMENT NAME:
S:\\DRP\\RPB7\\PI&R\\PI&R\\INSPECTION REPORTS\\ROBINSON\\ENF\\ROBINSON ENF CONF MEETING
SUMMARY
.DOCX
CP&L 2
cc w/encls:
Division of Radiological Health TN Dept. of Environment & Conservation Electronic Mail Distribution Sandra Threatt, Manager Nuclear Response and Emergency Environmental Surveillance Bureau of Land and Waste Management Department of Health and Environmental Control Electronic Mail Distribution R. J. Duncan II, Vice President H. B. Robinson Steam Electric Plant Unit 2 Carolina Power & Light Company Electronic Mail Distribution Brian C. McCabe Manager, Nuclear Regulatory Affairs Progress Energy Carolinas, Inc.
Electronic Mail Distribution Christos Kamilaris. Director Fleet Support Services Carolina Power & Light Company Electronic Mail Distribution Curt A. Castell, Supervisor Licensing/Regulatory Programs Carolina Power & Light Company Electronic Mail Distribution B. C. White, Manager Support Services - Nuclear Carolina Power & Light Company Electronic Mail Distribution S. D. West Superintendent Security H. B. Robinson Steam Electric Plant Progress Energy Electronic Mail Distribution Joseph W. Donahue, Vice President Nuclear Oversight Carolina Power and Light Company Electronic Mail Distribution David T. Conley Associate General Counsel - Legal Dept.
Progress Energy Service Company, LLC Electronic Mail Distribution John H. O'Neill, Jr.
Shaw, Pittman, Potts & Trowbridge 2300 N. Street, NW Washington, DC 20037-1128 Susan E. Jenkins Director, Division of Waste Management Bureau of Land and Waste Management S.C. Department of Health and Environmental Control Electronic Mail Distribution Robert P. Gruber Executive Director Public Staff - NCUC 4326 Mail Service Center Raleigh, NC 27699-4326 W. Lee Cox, III, Section Chief Radiation Protection Section N.C. Department of Environmental Commerce & Natural Resources Electronic Mail Distribution Mark Yeager Division of Radioactive Waste Mgmt.
S.C. Department of Health and Environmental Control Electronic Mail Distribution Public Service Commission State of South Carolina P.O. Box 11649 Columbia, SC 29211 Chairman North Carolina Utilities Commission Electronic Mail Distribution Senior Resident Inspector U. S. Nuclear Regulatory Commission H. B. Robinson Steam Electric Plant 2112 Old Camden Rd.
Hartsville, SC 29550
CP&L 3
Letter to Robert Duncan from George T. Hopper dated November 16, 2010
SUBJECT:
PUBLIC MEETING
SUMMARY
- CATEGORY ONE - REGULATORY AND ENFORCEMENT CONFERENCE MEETING DOCKET NO. 50-261 Distribution w/encls:
C. Evans, RII EICS L. Douglas, RII EICS OE Mail RIDSNRRDIRS PUBLIC RidsNrrPMRobinson Resource
UNITED STATES NUCLEAR REGULATORY COMMISSION REGION II 245 PEACHTREE CENTER AVENUE NE SUITE 1200 ATLANTA, GEORGIA 30303-1257 Progress Energy Public Meeting H.B. Robinson Steam Electric Plant RegulatorylEnforcement Conference November 8, 2010 NAME TITLE AND ORGANIZATION PROGRESS ENERGY Ai
-T (2 -
p ALf
,DIL hkA4dLn
/fi/f1Cc2 I(//:IfC7// eY CLt I-cs ¶crL-
/2,/1/:
oberE LL V -
ccj p SVfi CO E
YVdb L t rj fC,.&-
1_s ssloc c
NRC 6iQ 2oA/z-
,ir 34 2-J N (LC./a) S (.
c 4 Lis C
(2 1)> we4- (Aci N
r5 y-2.fc41 EJ Lt 31
-,-( ikii Dw.5.f i-P-t (hri (Ii.tPSL j)gc c:YD /2b-c itU-Muss c
R P,co,cr L R 77
,4,s4S pi,A L,0L 1
7
,O/1
_*7- +/---
(?//7 / (ç 7 ir
.ie.\\ vt r# Z VA r AaJr
/i LiriI-LNti frfs NRC.
v+/SbT.v, 4
- nne#k Worb lO bZPl 2
rtiJ-t Entyti/
/-.h6
°j S-Y
P.-hc T-C,1e /ll-)
7
/S AUDIO TELE-CONFERENCING M/4 1./1}
77&4af //( r/E/
Z4 7
&/ Jçi S444 c7 1v7
Open Regulatory and Predecisional Enforcement Conference November 8, 2010 NRC Region II Office, Atlanta, GA Robinson Steam Electric Plant Unit 2
Agenda Opening Remarks and Introductions NRC Regulatory and Enforcement Policy Statement of Issues and Apparent Violations Carolina Power and Light Company Break/NRC Caucus NRC Follow Up Questions Closing Remarks
Apparent Violation #1 10 CFR 50.9(a) required, in part, that information provided to the Commission by a licensee shall be complete and accurate in all material respects.
Contrary to the above, on June 18, 2009, the licensee provided information to the NRC in LER 2009-001-00 that was not complete and accurate in all material respects. The information provided described corrective actions for a previous and similar B EDG output breaker failure in October 2008. The LER stated that in response to the breaker failure The breaker was tested in accordance with Preventive Maintenance Procedure (PM), PM-163, Inspection and Testing of Circuit Breakers for 480 Volt Bus E2." The same paragraph also stated: and the successful completion of PM-163 The NRC determined that the licensee had not performed formal procedural testing and had only completed the instructions in PM-163 for returning the breaker to service.
Apparent Violation #2 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, requires, in part, that measures shall be established to assure that conditions adverse to quality, such as failures, deficiencies, and defective material are promptly identified and corrected. As implemented by NGGM-PM-0007, Quality Assurance Program Manual, Revision 19, Procedure CAP-NGGC-0200, Revision 26, required an NCR be generated for adverse conditions involving critical component degradation and if a WR/WO document required corrective maintenance as defined by the conduct of work management of a critical component. Technical Specifications 3.8.1, Condition B required that an inoperable EDG shall be restored to operable status within 7 days.
Contrary to the above, on October 15, 2008, the licensee failed to assure that the failure of the Emergency Diesel Generator Output Breaker 52/27B to close, a condition adverse to quality in a critical component, was promptly corrected in that the condition was not evaluated through an NCR when the breaker revealed indications of control relay malfunction during post-modification testing. As a result of the licensees failure to correct the adverse condition to quality, the B EDG became inoperable from March 28, 2009, to April 23, 2009, which exceeded the TS allowed outage time.
Robinson P
d i i l E f t C f
Pre-decisional Enforcement Conference and Regulatory Conference Regulatory Conference November 8, 2010 Apparent violation of 10CFR50 9(a) 10CFR50.9(a) and Preliminary white finding Preliminary white finding EDG output breaker
Progress Energy Participants
Jim Scarola Senior Vice President and Chief Nuclear Officer
Bob Duncan Vice President - Robinson Nuclear Plant
Warren Farmer Director of Engineering - Robinson Nuclear Plant
Charlie Sandifer Senior Engineer - Robinson Nuclear Plant
Chuck Morris Maintenance Manager - Robinson Nuclear Plant g
Curt Castell Licensing Supervisor - Robinson Nuclear Plant B i M C b M
R l t Aff i N
l G
ti
Brian McCabe Manager Regulatory Affairs - Nuclear Generation Group
Bob Rishel Manager Probabilistic Safety Assessment -
Nuclear Generation Group Nuclear Generation Group 2
Agenda Opening remarks B. Duncan 50.9(a) apparent violation C. Castell C. Morris C. Morris Maintenance CAP usage/work practices C. Morris 2008/2009 EDG breaker issues W. Farmer C. Sandifer C. Sandifer Closing remarks J. Scarola 3
Opening Remarks Ensure understanding of facts Discuss lessons learned and comprehensive corrective actions associated with apparent violation of 10 CFR 50.9(a)
( )
Provide basis for differing view on potential white finding associated with EDG output breaker p
Discuss extensive investigation to discover cotter pin design flaw g
Discuss actions being taken to address Maintenance work practices and use of the Corrective Action Program p
g 4
Apparent Violation Apparent Violation 10 CFR 50.9(a)
Curt Castell Curt Castell Supervisor - Licensing/Regulatory Programs 5
Apparent Violation 10 CFR 50.9(a)
Inaccurate statement in LER 2009-001-00:
A work order was written due to the breaker failing to operate
A work order was written due to the breaker failing to operate.
The work order required that 52/27B be inspected and tested.
The breaker was tested in accordance with Preventive Maintenance Procedure (PM), PM-163, "Inspection and Testing of
(
),
p g
Circuit Breakers for 480 Volt Bus E2." During this testing, the breaker cycled successfully seven times. The same paragraph also stated: and the successful completion of PM-163 6
Apparent Violation 10 CFR 50.9(a)
Comprehensive investigation was conducted
==
Conclusion:==
Progress Energy agrees that it failed to provide accurate and complete information in LER 2009-001-00 R
t C Li i
d l
i t did t
id ifi
Root Cause: Licensing procedural requirements did not provide specific guidance for hierarchy of validation information and direct that source records should be used to validate statements of fact.
Investigation addressed broader contributors
Investigation identified work practice problems that contributed to inaccurate LER inaccurate LER
Maintenance Work Order documentation
Corrective Action Program documentation and validation
Supervisory oversight 7
Apparent Violation 10 CFR 50.9(a)
Source of the inaccurate information was the work order documentation documentation Breaker tested and checked as per PM-163. No problem found Investigation findings Applicable sections of PM-163 were not performed Supervisor review of the work order failed to correct the information Subsequent investigation (NCR 333530) relied on the work order q
g
(
)
information LER 2009-001-00 validation was based on the investigation Extent of condition/cause review 8
Corrective Actions Corrective Action Program Validation NCR initiated to address validation of CAP information Evaluation by fleet Self Evaluation Manager is in progress Objective is to strengthen processes that ensure valid information Correction of Documentation Correct the work order record Correct the CAP investigation record Correct LER 2009-001-00 9
Corrective Actions Fleet Licensing/Regulatory Affairs Fleet Licensing/Regulatory Affairs leadership developed corrective actions Circumstances surrounding validation error promptly reviewed with fleet Licensing units Interim validation guidance issued across the fleet Revised fleet procedure guidance pertaining to validation e sed ee p ocedu e gu da ce pe a g o a da o REG-NGGC-0016, Regulatory Correspondence & LER Development Training for fleet Licensing personnel on procedure revision Training for fleet Licensing personnel on procedure revision 10
Corrective Actions Fleet Licensing/Regulatory Affairs (Continued)
Root cause investigation report designated as required reading for fleet Licensing personnel Fleet Licensing/Regulatory Affairs leadership discussion of lessons learned with staff Counseling of RNP Licensing personnel by Site Vice President on expectations for completeness and accuracy in correspondence 11
Corrective Actions Fleet Rapid Operating Experience Report Issued Summary of event and violation Reinforced that completeness and accuracy of documentation is essential to nuclear safety Reinforced the importance of providing complete and accurate information to the NRC Reinforced that organizational learning is applicable to the entire fleet Shared with industry via INPO OE network (OE 32184) 12
Corrective Actions Maintenance Work Practices and Oversight Counseling of individuals involved in work order documentation inaccuracy Reinforcement of supervisor and superintendent accountability and expectations associated with work order completion documentation Additional training for Maintenance supervisors and technicians on work order documentation Maintenance self-assessment of work order completion documentation 13
Corrective Actions Maintenance Work Practices and Oversight (Continued)
Focused observations of work order completion documentation Work order completion identified as focus area for Nuclear Oversight 14
Conclusions Incorrect information in LER was unintentional Investigation was thorough and addressed organizational contributing causes Corrective actions are comprehensive Progress Energy has a history of excellent performance associated with 10 CFR 50.9 Progress Energy has demonstrated, and will continue to demonstrate, an unyielding commitment to open, complete, and accurate communication with the NRC 15 p
Apparent Violation 10 CFR Part 50, Appendix B, Criterion XVI Chuck Morris Maintenance Manager Charlie Sandifer Senior Engineer Warren Farmer Director of Engineering 16
Agenda Maintenance work practices and use of CAP Cotter pin overview 2009 EDG b k
f il i
2009 EDG breaker failure overview 2008 EDG breaker failure overview Investigation results Differing views pertaining to apparent violation 17
Maintenance Insights M i t W
k P ti Maintenance Work Practices U
f C ti A ti P
Use of Corrective Action Program 18
Maintenance Insights PI&R inspection provided valuable insights into problems associated with Maintenance work order documentation associated with Maintenance work order documentation, work practices, and use of Corrective Action Program NRC insights are consistent with our observations Work Order Documentation Problems D
t ti lit d
i i ht f
k d
Documentation quality and supervisor oversight of work order completion documentation Issues and corrective actions addressed earlier in presentation Issues and corrective actions addressed earlier in presentation 19
Maintenance Insights Maintenance Work Practices Manual manipulation of EDG output breaker following 2008 failure Similar to February 2009 tightening of EDG fuel oil fitting Similar to February 2009 tightening of EDG fuel oil fitting without work instructions C
h i
ti ti t k Comprehensive corrective actions were taken Standards are understood 20
Maintenance Insights Maintenance Use of Corrective Action Program Failure to initiate NCR following 2008 breaker failure Failure to initiate NCR following 2008 breaker failure Progress Energy recognized that an NCR should have been written (in addition to work request) and initiated NCR in 2009 Problem identification remains a focus area Reinforcing procedure requirements and importance of initiating NCRs Reinforcing procedure requirements and importance of initiating NCRs New CAP single point entry system and management review/oversight (fleet)
Site leadership changes Fleet initiative addressing performance improvement programs Fleet initiative addressing performance improvement programs 21
Summary The PI&R inspection report provided valuable insights RNP is addressing Maintenance work practice challenges, including work order documentation Progress Energy recognizes that an NCR should have been initiated following the 2008 breaker failure Progress Energy is not satisfied with CAP performance at RNP RNP leadership changes and ongoing fleet initiative are focused on CAP performance improvement p
p 22
2008/2009 EDG O B
k F il 2008/2009 EDG Output Breaker Failures 23
Key Regulatory Questions Assuming that the October 2008 and the April 2009 breaker failures had the same failure mode (cotter pin),
was that failure mode reasonably within RNPs ability to was that failure mode reasonably within RNP s ability to promptly identify & correct in 2008?
Th f
i it lid t li k th f
d fi i Therefore, is it valid to link the performance deficiency associated with the failure of the B EDG output breaker in October 2008 to a period of inoperability of the B EDG i
A il 2009?
in April 2009?
24
Cotter Pin Overview
Cotter pin holds operating linkage in place during assembly of control relay onto the breaker
Pin does not serve any function once breaker has been assembled.
Cotter pin could be removed except for its accessibility
When cotter pin was introduced into breaker design vendor
When cotter pin was introduced into breaker design, vendor concluded breaker critical characteristics had not changed and operating functions were not affected. Unrecognized failure mechanism was introduced (likely mid 1990s)
Vendor concluded that notification of end-users was not necessary, and thus no owners review of the change was conducted Th i
ti i
t f thi i
i th d
There were no inspection requirements for this pin in the vendor maintenance manual
2009 RNP breaker failure was the first such cotter pin problem p
p reported in the industry 25
Cotter Pin Overview 26 Lifting linkage in default position (correct position)
Cotter Pin Overview 27 Cotter Pin in failed position - Holding up lifting linkage
Cotter Pin Overview 28
2009 EDG Output Breaker Failure On April 20, 2009, EDG B output breaker (52/27B) failed to close twice during performance of surveillance test g p Five Step Problem Solving Plan (troubleshooting) found no root cause for the failure Engineering developed and executed a Complex Troubleshooting Plan and found no root cause for the failure Engineering developed and executed a Failure Mode Tree and Failure Mode Cause Table (Support/Refute Methodology) and found no root cause for the failure found no root cause for the failure Decision was made to send suspect breaker to Westinghouse (OEM) to have further investigation performed on breaker
(
)
g p
29
2009 EDG Output Breaker Failure Westinghouse conducted testing in DB-100 test cell Testing included cycling breaker successfully 5 times at 130 VDC (plant conditions), 5 times at 125 VDC (nominal voltage) and 5 times at 87 VDC (minimum voltage) voltage), and 5 times at 87 VDC (minimum voltage)
Test voltage was increased to 144 VDC (maximum lt
)
t hi h ti b
k f il d t l
voltage) at which time breaker failed to close No correlation between voltage levels and failure mechanism Testing suspended to preserve the failed condition g
p p
30
2009 EDG Output Breaker Failure Visual inspections determined that lifting linkage failed to reset mechanically after previous opening of breaker reset mechanically after previous opening of breaker Condition prevented control relay contacts from closing to energize closing solenoid coil energize closing solenoid coil Detailed inspection revealed that cotter pin had twisted so th t lifti li k ld t f ll t th t
iti that lifting link could not fall to the reset position Breaker unable to close 31
2009 EDG Output Breaker Failure 32
2009 EDG Output Breaker Failure 33
2009 EDG Output Breaker Failure
Westinghouse corrective action to address cotter pin design flaw 34
2009 EDG Output Breaker Failure Conclusions Failure mode associated with cotter pin was random, new to the industry, and unforeseen by the manufacturer Failure mode associated with cotter pin was not identified during Westinghouse Certificate of Qualification (CoQ) testing for the breaker (2008) o t e b ea e ( 008)
Cotter pin failure mode was not identified via substantial testing and troubleshooting at RNP, including performance of PM-163 Identification occurred during extraordinary investigation/testing by OEM and RNP Randomness of failure mode makes identification even with extraordinary inspection/testing uncertain. Discovery was by chance during repeated testing at OEM facility chance during repeated testing at OEM facility 35
2009 EDG Output Breaker Failure Conclusions Cotter pin design flaw reported by Westinghouse via 10 CFR Part 21 notification and Nuclear Safety Advisory Letter to i d t
industry Westinghouse identified 48 safety related breakers in extent of diti i
condition review Identification and reporting of cotter pin design flaw likely t d f t f il f thi t i
th i d t
prevented future failures of this type in the industry 36
2008 EDG Output Breaker Failure On October 15, 2008 (RO-25), EDG B output breaker (52/27B) failed to close twice during post modification testing (52/27B) failed to close twice during post modification testing Maintenance personnel reached behind control relay to check for mechanical binding for mechanical binding Breaker was successfully closed on third attempt Work Request written to investigate breaker/control relay issue Progress Energy agrees that NCR should have been initiated Progress Energy agrees that NCR should have been initiated.
Failure to initiate NCR was recognized and documented in CAP in 2009 37
2008 EDG Output Breaker Failure Subsequent troubleshooting identified no problems with breaker breaker Breaker electrically cycled successfully approximately 7 times Troubleshooting could not repeat the failure to close Since no abnormalities were identified, control relay was not replaced P i t
t t
f RO 25 th f ll i
ill Prior to startup from RO-25, the following surveillance tests were conducted which successfully cycled breaker OST-409-2 (Fast Speed Start)
OST-411 (24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> full load run)
OST-163 (Safety injection and loss of power) 38
Key Regulatory Questions Assuming that the October 2008 and the April 2009 breaker failures had the same failure mode (cotter pin),
was that failure mode reasonably within RNPs ability to was that failure mode reasonably within RNP s ability to promptly identify & correct in 2008?
Th f
i it lid t li k th f
d fi i Therefore, is it valid to link the performance deficiency associated with the failure of the B EDG output breaker in October 2008 to a period of inoperability of the B EDG i
A il 2009?
in April 2009?
39
Key Considerations The cause of the 2008 breaker failure was not determined because failure could not be repeated determined because failure could not be repeated OST-401-2 (B EDG Slow Speed Start) was
(
p
)
performed successfully 7 times between 2008 and 2009 failures. Breaker operated without incident during OST testing incident during OST testing Breaker was cycled successfully 14 times between 2008 and 2009 failures 40
Key Considerations
NRC PI&R Inspection Report Concluded that the information in the LER pertaining to the conduct of PM-Concluded that the information in the LER pertaining to the conduct of PM 163 was material because the NRC relied on the information in exercising enforcement discretion C
l d d th t th i f ti i
th LER t i i t
th d
t f PM Concluded that the information in the LER pertaining to the conduct of PM-163 was used to determine whether the licensees corrective actions for the breaker failure in October 2008 were adequate to prevent recurrence
Not reasonable to conclude performance of PM-163 would have resulted in identification and prompt correction of cotter pin issue (Assuming cotter pin was 2008 failure mode)
Applicable PM-163 sections performed in June 2008 without identification of cotter pin issue (prior to October failure)
Applicable PM-163 sections performed following April 2009 failure without identification of cotter pin issue 41
Key Considerations The 2008 event was reenacted with technician in October 2010 at RNP Maintenance Test Facility at RNP Maintenance Test Facility Insights Si l t d b k
diti i
il t
tt i
ti f ll t l
Simulated breaker conditions similar to cotter pin preventing full travel of lift linkage resulted in no buzzing or humming Technician indicated that he reached behind control relay to check for Technician indicated that he reached behind control relay to check for mechanical binding If control relay had been replaced in 2008 per WO instructions, the cotter pin would not have been part of component replaced cotter pin would not have been part of component replaced If entire breaker had been replaced in 2008, the cotter pin failure mode would still have remained undiscovered 42
Key Considerations The cotter pin is very difficult to access and observe.
43 Control Relay
Key Considerations The cotter pin is very difficult to access and observe. Since the pin was not an integral part of breaker operation, there were no inspection requirements in the vendor maintenance manual or RNP maintenance requirements in the vendor maintenance manual or RNP maintenance procedures (which are based on vendor maintenance manual), nor were there established criteria for cotter pin arrangement 44
Key Considerations A twisted cotter pin has little surface area contact holding it in place.
Thus, very little force/vibration is necessary to rotate it out of that position.
This contributes to the random, intermittent nature of this failure mode.
Repeatability and discovery of failure mode is problematic and uncertain.
45
Key Considerations Absent performance issues, the cotter pin failure mode was still not reasonably within RNPs ability to promptly was still not reasonably within RNP s ability to promptly identify & correct in 2008 Initiation of NCR - While this may have resulted in more methodical y
problem solving process, the cotter pin failure mode was random and intermittent. The randomness of the failure mode makes identification even with extraordinary investigation and testing uncertain. Discovery was by chance in 2009 during repeated testing at the OEM facility.
PM-163 Completion - PM-163 performance on at least two occasions did not detect cotter pin problem.
Breaker Manipulation - April 2009 failure mode was not identified through extensive RNP investigation (including PM-163), even when breaker was not manipulated.
p 46
Summary Progress Energy concludes that despite the performance issues discussed, the cotter pin failure mode was not reasonably within RNPs ability to promptly identify and correct in October 2008 RNP s ability to promptly identify and correct in October 2008.
This is based on:
The failure mode was random and intermittent The failure mode was random and intermittent The failure mode would likely disappear when breaker was moved The failure mode was new and unforeseen, even by the OEM The failure mode was not identified during OEM CoQ testing The failure mode was not identified during OEM CoQ testing The failure mode went undetected in the industry for many years The failure mode was identified by chance during extraordinary investigation and testing by RNP and OEM Progress Energy agrees that an NCR should have been initiated following the 2008 breaker failure, but disagrees that the deficiency should be linked to any period of EDG inoperability deficiency should be linked to any period of EDG inoperability.
47
Cl i
R k
Closing Remarks 48