IR 05000298/2007007: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
Line 1: Line 1:
{{Adams|number = ML071430289}}
{{Adams
| number = ML072970900
| issue date = 10/23/2007
| title = Acceptance of White Finding and Notice of Violation - NRC Special IR 05000298-07-007 - Cooper Nuclear Station
| author name = Hay M C
| author affiliation = NRC/RGN-IV/DRP/RPB-C
| addressee name = Minahan S B
| addressee affiliation = Nebraska Public Power District (NPPD)
| docket = 05000298
| license number = DPR-046
| contact person =
| case reference number = EA-07-090
| document report number = IR-07-007
| document type = Letter, Notice of Violation
| page count = 3
}}


{{IR-Nav| site = 05000298 | year = 2007 | report number = 007 }}
{{IR-Nav| site = 05000298 | year = 2007 | report number = 007 }}
Line 5: Line 20:
=Text=
=Text=
{{#Wiki_filter:
{{#Wiki_filter:
[[Issue date::May 22, 2007]]
[[Issue date::October 23, 2007]]


EA 07-090Stewart B. Minahan, Vice President-Nuclear and CNO Nebraska Public Power District P.O. Box 98 Brownville, NE 68321
EA 07-090Stewart B. Minahan, Vice President-Nuclear and CNO Nebraska Public Power District 72676 648A Avenue Brownville, NE 68321


SUBJECT: COOPER NUCLEAR STATION - NRC SPECIAL INSPECTIONREPORT 05000298/2007007
SUBJECT: ACCEPTANCE OF WHITE FINDING AND NOTICE OF VIOLATION - NRCSPECIAL INSPECTION REPORT 05000298/2007007 - COOPER NUCLEAR STATION


==Dear Mr. Minahan:==
==Dear Mr. Minahan:==
On April 24, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed a special inspection at your Cooper Nuclear Station. This inspection examined activities associated with the Emergency Diesel Generator 2 failure that occurred on January 18, 2007. During this event the emergency diesel generator automatically isolated from the electrical bus following an over-current condition as a result of a voltage regulator failure. The NRC's initial evaluation satisfied the criteria in NRC Management Directive 8.3, "NRC Incident Investigation Program,"
Thank you for your letter dated October 15, 2007, in response to our August 17, 2007, letterand Notice of Violation concerning failing to promptly identify and correct, to preclude recurrence, of a significant condition adverse to quality regarding a defective voltage regulator circuit board used in Emergency Diesel Generator 2. The significance of this finding was determined to be White, a finding with low to moderate safety significance. We have reviewed your reply and find it responsive to the concerns raised in our Notice of Violation. Based on the guidance in Inspection Manual Chapter (IMC) 0305, the NRC staff will perform a95001, "Supplemental Inspection for One or Two White Inputs in a Strategic Performance Area." This inspection is scheduled for the week of December 3, 2007. We will review the implementation of your corrective actions during this inspection to determine that full compliance has been achieved and will be maintained.
for conducting a special inspection. The basis for initiating the special inspection is furtherdiscussed in the Charter, which is included as Attachment 2 to the enclosed report. The determination that the inspection would be conducted was made by the NRC on January 26, 2007, and the inspection started on January 29, 2007.The enclosed special inspection report documents the inspection findings which were discussedon April 24, 2007, with Mr. Mike Colomb, General Manager of Plant Operations, and othermembers of your staff. The inspection examined activities conducted under your license as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your license. The team reviewed selected procedures and records, observed activities, and interviewed personnel.The enclosed report discusses one finding that appears to have low to moderate safetysignificance (White). As described in Section 6.0 of this report, the NRC concluded that the failure to establish appropriate procedural controls for evaluating the use of parts of indeterminate quality prior to their installation in safety-related systems resulted in Emergency Diesel Generator 2 failure on January 18, 2007. The safety significance of this finding was assessed on the basis of the best available information, including influential assumptions, using the applicable Significance Determination Process and was preliminarily determined to be a White (i.e., low to moderate safety significance) finding. Attachment 3 of this report provides a detailed description of the preliminary risk assessment. In accordance with NRC Inspection Nebraska Public Power District-2-Manual Chapter (IMC) 0609, "Significance Determination Process," we intend to complete ourevaluation using the best available information and issue our final determination of safety significance within 90 days of this letter.This finding does not represent an immediate safety concern because of the corrective actionsyou have taken. These actions included the installation of properly dedicated voltage regulator components for Emergency Diesel Generator 2 following the failure that occurred on January 18, 2007.Also, this finding constitutes an apparent violation of NRC requirements and is beingconsidered for escalated enforcement action in accordance with the NRC Enforcement Policy.


The current Enforcement Policy is included on the NRC's Web site at www.nrc.gov; selectAbout NRC, How We Regulate, Enforcement, then Enforcement Policy. Before we make a final decision on this matter, we are providing you an opportunity (1) topresent to the NRC your perspectives on the facts and assumptions, used by the NRC to arriveat the finding and its significance, at a Regulatory Conference or (2) submit your position on thefinding to the NRC in writing. If you request a Regulatory Conference, it should be held within30 days of the receipt of this letter and we encourage you to submit documentation at least oneweek prior to the conference in an effort to make the conference more efficient and effective. Ifa Regulatory Conference is held, it will be open for public observation. If you decide to submitonly a written response, such submittal should be sent to the NRC within 30 days of the receiptof this letter.Please contact Michael Hay at (817) 860-8144 within 10 business days of the date of this letterto notify the NRC of your intentions.If you choose to provide a written response, it should be clearly marked as a "Response to AnApparent Violation in Inspection Report No. 05000298/2007007: EA-07-090" and shouldinclude: (1) the reason for the apparent violation, or, if contested, the basis for disputing theapparent violation; (2) the corrective steps that have been taken and the results achieved; (3)the corrective steps that will be taken to avoid further violations; and (4) the date when fullcompliance will be achieved. Your response may reference or include previously docketedcorrespondence, if the correspondence adequately addresses the required response. If anadequate response is not received within the time specified or an extension of time has notbeen granted by the NRC, the NRC will proceed with its enforcement decision or schedule aRegulatory Conference.Since the NRC has not made a final determination in this matter, no Notice of Violation is beingissued for the inspection finding at this time. In addition, please be advised that the numberand characterization of the apparent violation described in the enclosed inspection report maychange as a result of further NRC review.The report also documents one NRC-identified finding, which was evaluated under the risksignificance determination process as having very low safety significance (Green). This findingwas determined to involve a violation of NRC requirements. However, because of the very lowsafety significance and because it is entered into your corrective action program, the NRC is Nebraska Public Power District-3-treating this finding as a noncited violation consistent with Section VI.A.1 of the NRCEnforcement Policy. If you contest the noncited violation in this report, you should provide aresponse within 30 days of the date of this inspection report, with the basis for your denial, tothe U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington,DC 20555-0001, with copies to the Regional Administrator, U.S. Nuclear RegulatoryCommission, Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; theDirector, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington,DC 20555-0001; and the NRC Resident Inspectors at the Cooper Nuclear Station.In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and itsenclosures will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html.
Sincerely,/RA/Michael C. Hay, Chief, Reactor Projects Branch CDocket: 50-298License: DPR-46 Nebraska Public Power District- 2 -cc w/


Sincerely,/RA/Arthur T, Howell III, DirectorDivision of Reactor ProjectsDocket: 50-298License: DPR-46
===Enclosures:===
Gene Mace Nuclear Asset Manager Nebraska Public Power District P.O. Box 98 Brownville, NE 68321John C. McClure, Vice President and General Counsel Nebraska Public Power District P.O. Box 499 Columbus, NE 68602-0499David Van Der Kamp Licensing Manager Nebraska Public Power District P.O. Box 98 Brownville, NE 68321Michael J. Linder, DirectorNebraska Department of Environmental Quality P.O. Box 98922 Lincoln, NE 68509-8922ChairmanNemaha County Board of Commissioners Nemaha County Courthouse 1824 N Street Auburn, NE 68305Julia Schmitt, ManagerRadiation Control Program Nebraska Health & Human Services Dept. of Regulation & Licensing Division of Public Health Assurance 301 Centennial Mall, South P.O. Box 95007 Lincoln, NE 68509-5007H. Floyd GilzowDeputy Director for Policy Missouri Department of Natural Resources P. O. Box 176 Jefferson City, MO 65102-0176Director, Missouri State Emergency Management Agency P.O. Box 116 Jefferson City, MO 65102-0116Chief, Radiation and Asbestos Control Section Kansas Department of Health and Environment Bureau of Air and Radiation 1000 SW Jackson, Suite 310 Topeka, KS 66612-1366Melanie Rasmussen, State LiaisonOfficer/Radiation Control Program Director Bureau of Radiological Health Iowa Department of Public Health Lucas State Office Building, 5th Floor 321 East 12th Street Des Moines, IA 50319John F. McCann, Director, LicensingEntergy Nuclear Northeast Entergy Nuclear Operations, Inc.


===Enclosure:===
440 Hamilton Avenue White Plains, NY 10601-1813Keith G. Henke, PlannerDivision of Community and Public Health Office of Emergency Coordination 930 Wildwood, P.O. Box 570 Jefferson City, MO 65102Paul V. Fleming, Director of Nuclear Safety Assurance Nebraska Public Power District P.O. Box 98 Brownville, NE 68321 Nebraska Public Power District- 3 -Electronic distribution by RIV:Regional Administrator (EEC)Deputy Regional Administrator (TPG)DRP Director (ATH)DRP Deputy Director (AXV)DRS Director (DDC)DRS Deputy Director (RJC1)Senior Resident Inspector (NHT)Branch Chief, DRP/C (MCH2)Senior Project Engineer, DRP/C (WCW)Branch Chief, EB1, DRS (WBJ)Team Leader, DRP/TSS (CJP)RITS Coordinator (MSH3)State Liaison Officer (WAM)Public Affairs Officer (VLD)K.S. Fuller, RC/ACES (KSF)Enforcement Specialist (GMV)C.A. Carpenter, D:OE (CAC)OE EA File (RidsOeMailCenter)SUNSI Review Completed: MCH ADAMS: Yes G No Initials: MCH Publicly Available G Non-Publicly Available G Sensitive Non-SensitiveR:\_REACTORS\CNS\2007\CNS 2007-07_ThankYouLtr.wpd  SPE:DRP/CC:DRP/CWCWalkerMCHay /RA/ /RA/10/23/0710/23/07OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax
NRC Inspection Report 05000298/2007007 Attachment 1: Supplemental InformationAttachment 2: Special Inspection Charter Attachment 3: Significance Determination Evaluation cc w/
 
===Enclosure:===
Gene Mace Nuclear Asset Manager Nebraska Public Power District P.O. Box 98 Brownville, NE 68321John C. McClure, Vice President and General Counsel Nebraska Public Power District P.O. Box 499 Columbus, NE 68602-0499P. V. Fleming, Licensing ManagerNebraska Public Power District P.O. Box 98 Brownville, NE 68321Michael J. Linder, DirectorNebraska Department of Environmental Quality P.O. Box 98922 Lincoln, NE 68509-8922 Nebraska Public Power District-4-ChairmanNemaha County Board of Commissioners Nemaha County Courthouse 1824 N Street Auburn, NE 68305Julia Schmitt, ManagerRadiation Control Program Nebraska Health & Human Services Dept. of Regulation & Licensing Division of Public Health Assurance 301 Centennial Mall, South P.O. Box 95007 Lincoln, NE 68509-5007H. Floyd GilzowDeputy Director for Policy Missouri Department of Natural Resources P. O. Box 176 Jefferson City, MO 65102-0176Director, Missouri State Emergency Management Agency P.O. Box 116 Jefferson City, MO 65102-0116Chief, Radiation and Asbestos Control Section Kansas Department of Health and Environment Bureau of Air and Radiation 1000 SW Jackson, Suite 310 Topeka, KS 66612-1366Daniel K. McGhee, State Liaison OfficerBureau of Radiological Health Iowa Department of Public Health Lucas State Office Building, 5th Floor 321 East 12th Street Des Moines, IA 50319Melanie Rasmussen, Radiation Control Program Director Bureau of Radiological Health Iowa Department of Public Health Lucas State Office Building, 5th Floor 321 East 12th Street Des Moines, IA 50319Ronald D. Asche, President and Chief Executive Officer Nebraska Public Power District 1414 15th Street Columbus, NE 68601Kevin V. Chambliss, Director of Nuclear Safety Assurance Nebraska Public Power District P.O. Box 98 Brownville, NE 68321John F. McCann, Director, LicensingEntergy Nuclear Northeast Entergy Nuclear Operations, Inc.
 
440 Hamilton Avenue White Plains, NY 10601-1813Keith G. Henke, PlannerDivision of Community and Public Health Office of Emergency Coordination 930 Wildwood, P.O. Box 570 Jefferson City, MO 65102Chief, Radiological Emergency Preparedness Section Kansas City Field Office Chemical and Nuclear Preparedness and Protection Division Dept. of Homeland Security 9221 Ward Parkway Suite 300 Kansas City, MO 64114-3372 Nebraska Public Power District-5-Electronic distribution by RIV:Regional Administrator (BSM1)DRP Director (ATH)DRS Director (DDC)DRS Deputy Director (RJC1)Senior Resident Inspector (NHT)Branch Chief, DRP/C (MCH2)Senior Project Engineer, DRP/C (WCW)Team Leader, DRP/TSS (CJP)RITS Coordinator (MSH3)Only inspection reports to the following:DRS STA (DAP)L. Trocine, OEDO RIV Coordinator (LXT)ROPreports CNS Site Secretary (SEF1)D. Starkey, OE (DRS)OE MailM. Ashley, NRR (MAB)M. Vasquez (GMV)V. Dricks (VLD)W. Maier (WAM)SUNSI Review Completed: _WCW ADAMS: YesG No Initials: __WCW Publicly Available G Non-Publicly Available G Sensitive Non-SensitiveR:\_REACTORS\CNS\2007\CNS2007-07 RP Special.wpd_RIV:DRSRI:DRP/CSRA:DRSC:DRP/CACESSPRutenkrogerNHTaylorMFRunyanMCHayMHaireE-WCWalkerT-Walker/RA//RA//RA/5/14/075/17/075/9/075/17/075/10/07D:DRPATHowell III/RA/5/22/07OFFICIAL RECORD COPYT=Telephone E=E-mail F=Fax Enclosure-1-U.S. NUCLEAR REGULATORY COMMISSION REGION IV Docket.:50-298 License:DPR-46 Report:05000298/2007007 Licensee:Nebraska Public Power District Facility:Cooper Nuclear Station Location:P.O. Box 98 Brownville, Nebraska Dates:January 28 through April 24, 2007Inspectors:N. Taylor, Resident InspectorS. Rutenkroger, PHD, Reactor InspectorApproved By:Arthur T. Howell III, Director Division of Reactor Projects Enclosure-2-
 
=SUMMARY OF FINDINGS=
IR 05000298/2007007; 01/28/07 - 04/24/07; Cooper Nuclear Station. Special Inspection inresponse to Emergency Diesel Generator 2 failure on January 18, 2007. This report documents the special inspection activities conducted by one resident inspector andone reactor inspector. One apparent violation and one non-cited violation were identified. Thesignificance of the issues is indicated by their color (Green, White, Yellow, or Red) and was determined by the significance determination process in Inspection Manual Chapter 0609.
 
Findings for which the significance determination process does not apply are indicated by the severity level of the applicable violation. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.A.NRC-Identified and Self-Revealing Findings
 
===Cornerstone: Mitigating Systems===
*TBD. The team identified an apparent violation of 10 CFR Part 50, Appendix B,Criterion V, "Instructions Procedures, and Drawings," for the failure to establish procedural controls for evaluating the use of parts of indeterminate quality prior to their installation in safety-related applications. This procedural deficiency resulted in the installation of a voltage regulator circuit board of indeterminate quality that adversely affected the function of Emergency Diesel Generator 2.
 
Specifically, following installation of the part on November 11, 2006, failure of the part occurred following 35 hours of operation resulting in an over-voltage trip of Emergency Diesel Generator 2 on January 18, 2007.The finding is greater than minor because it is associated with the equipmentperformance cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using NRC Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, a Phase 2 evaluation was required because the finding resulted in the loss of the safety function of Emergency Diesel Generator 2 for greater than the Technical Specification completion time.
 
The Phase 2 evaluation concluded that the finding was of low to moderate safety significance. A Phase 3 preliminary significance determination analysis also determined the finding was of low to moderate safety significance.
: '''Green.'''
The team identified three examples of a noncited violation of TechnicalSpecification 5.4.1.a involving the failure to establish adequate maintenanceprocedures for work performed on Emergency Diesel Generator 2. Theseinadequate procedures failed to identify a degraded condition in the voltage regulator off-manual-auto switch and contributed to an over-voltage trip of Emergency Diesel Generator 2 that occurred on November 13, 2006.
 
Enclosure-3-The finding is more than minor because it is associated with the MitigatingSystems cornerstone attribute of procedure quality and affects the associated cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events. Specifically, the performance deficiency resulted in:  (1) the failure to discover a degraded condition in the Emergency Diesel Generator 2 voltage regulator, and (2) an over-voltage trip during the tuning of Emergency Diesel Generator 2 on November 13, 2006.
 
Using the Manual Chapter 0609 Appendix G, "Shutdown Operations Significance Determination Process," Phase 1 Checklist, the finding is determined to have very low safety significance because one operable diesel generator was still capable of supplying power to the class 1E electrical power distribution subsystems. This finding has a cross-cutting aspect in the area of humanperformance in that the licensee's procedures were not complete and providedinadequate instructions for conducting maintenance associated with EmergencyDiesel Generator 2.
 
Enclosure-4-
 
=REPORT DETAILS=
1.0Special Inspection ReportThe NRC conducted this special inspection to better understand the circumstancessurrounding the failure of Emergency Diesel Generator (EDG) 2 that occurred on January 18, 2007. The failure of EDG 2 occurred during a routine monthly surveillance test following approximately four hours of operation. The cause of the event was the result of a failure of a voltage regulator card that had been installed in November 2006.
 
In accordance with NRC Management Directive 8.3, "NRC Incident Investigation Program," it was determined that this event met the deterministic criteria and had sufficient risk significance to warrant a special inspection. The team used NRC Inspection Procedure 93812, "Special Inspection Procedure," toconduct the inspection. The special inspection team reviewed procedures, corrective action documents, as well as design and maintenance records for the equipment of concern. The team interviewed key station personnel regarding the event, reviewed the root cause analysis, and assessed the adequacy of corrective actions. A list of specific documents reviewed is provided in Attachment 1. The charter for the special inspection is provided as Attachment 2. 2.0Review of EDG Voltage Regulator ProblemsThe team conducted a review of all work orders, surveillance test records, and conditionreports associated with the electrical components of EDG 2 written from January 2004 through January 2007. The following discusses several discrete periods of degraded performance of the EDG 2 voltage regulator.EDG 2 Voltage Regulator Performance in 2005On three occasions in 2005, operators initiated condition reports (CRs) documentingdifficulties maintaining stable reactive load on EDG 2. In all reported occurrences, kilovolt-ampere-reactive (KVAR) step changes, ranging from 100-300 KVAR, and spikes up to 2000 KVAR were received during manipulation of the lower-raise voltage regulator switch in the control room. This remote switch sends demand signals to the motor operated potentiometer (MOP) providing a reference voltage signal to the voltage regulator. On the basis of these observations, the licensee replaced the EDG 2 MOP in December 2005. In addition, the licensee initiated a preventive maintenance (PM) plan in June 2006 implementing MOP cleaning and inspection activities every operating cycleand MOP replacement every third operating cycle.The team noted there were five similar condition reports pertaining to EDG 1 in the2000-2007 time period. The KVAR changes documented in these condition reports were consistent with MOP issues affecting EDG 2. The team noted that corrective actions consisting of the previously discussed PM activities appeared adequate to address the problem.
 
-5-The licensee evaluated the impact of these KVAR step changes on the operability of theEDGs after each occurrence. The KVAR rating of the EDGs is 5000 KVAR, and the maximum reactive load seen during these transients was approximately 2000 KVAR.
 
During these KVAR step changes, the licensee did not observe any frequency or real load changes, and all acceptance criteria of the surveillance test procedures were satisfied. As such, the licensee determined the operability of the EDGs was not affected by the KVAR swings, but that the KVAR changes were indicative of a degraded condition in the voltage regulator system. The team reviewed the licensee's evaluation for this degraded condition and determined it adequately provided reasonable assurance of operability. EDG 2 Voltage Regulator Performance in 2006Despite the installation of a new MOP in December 2005, unpredictable KVAR changesbegan occurring again early in 2006. In addition to KVAR swings during manipulation of the lower-raise voltage regulator switch, the licensee noted KVAR swings during steadystate operation of EDG 2. This new trend suggested that a failure mechanism existed that was unrelated to the MOP. In response to this trend, the licensee initiated CR-CNS-2006-2729 and developed actions to review the overall health of the voltage regulating system and utilized vendor support to develop actions to resolve the problem.
 
One of the actions coming from the CR was to perform extenistive troubleshooting during Refueling Outage 23 (RE23), specifically directed at cleaning the MOP and checking circuit continuity throughout the voltage regulator system. Work Order (WO)4514076 was created to perform this task on EDG 2.EDG 2 Over-Voltage Event on November 11, 2006One of the troubleshooting steps included in WO 4514076 was to "wipe" the R13feedback adjust potentiometer on the voltage regulator printed circuit board. Thisinvolved turning a small screw on the potentiometer and exercising the slide wireresistor through its full range of travel. During performance of this step on November 8,2006, the resistor failed due to an open circuit condition. The licensee had anticipatedproblems with the printed circuit board and initiated a contingency in the WO to replace the board with a spare.Replacement of the printed circuit board required a tuning process to establish thecorrect voltage regulator response. WO 4514076 contained the procedural guidance to complete this task. This tuning process involved, in part, running EDG 2 unloaded, initiating small voltage changes, measuring the time response of the ensuing voltage transient, and making incremental adjustments in the position of the R13 potentiometer.
 
During this process the licensee had several pieces of test equipment installed to monitor the EDG output voltage response.During the unloaded run of EDG 2 for this activity, operations personnel shut themachine down because of unexpected fuel rack movement (the team reviewed this condition and determined that this was normal fuel rack behavior that did not affect EDG 2 operability). While the EDG was shutdown, electrical technicians investigated a potential problem with an installed oscilloscope being used as test equipment. The
-6-technicians connected a variac to the oscilloscope and introduced a 60 volt test signal. The technicians realized immediately that they had failed to disconnect the oscilloscope from the voltage regulator system and disconnected it. Unknown at the time to the technicians, this erroneous act resulted in blowing a fuse on one of three phases of the voltage regulator potential transformer, adversely affecting the voltage regulator summing network that senses generator output voltage. As a result, during the subsequent restart of EDG 2, the voltage regulator sensed an artificially low output voltage and drove voltage up until the EDG 2 over-voltage trip occurred at 6125 volts. The performance aspects of this event resulted in a non-cited violation of NRC requirements, which is described in NRC Integrated Inspection Report 05000298/2006005 (ML070360639).EDG 2 Over-voltage Event on November 13, 2006On November 13, 2006, the licensee replaced the blown fuse and continued with thevoltage regulator tuning process. The process required operators to make small changes in voltage around rated voltage, after which engineers reviewed the ensuing voltage transient as recorded on a strip chart. This was an iterative process that involved several voltage changes. Approximately ten seconds after one voltage changeEDG 2 experienced a pair of voltage spikes; the first to approximately 5500 volts andthe second to greater than 5900 volts, resulting in an over-voltage EDG trip.In response to this event, the licensee initiated CR-CNS-2006-9096. The licenseeconducted significant troubleshooting efforts to ascertain the cause of the over-voltage condition. These troubleshooting efforts included continuity checks, testing of voltageregulator connectors, and five monitored test runs of EDG 2. Unable to find a definitivecause for the trip, the licensee contacted a nuclear field service contractor experiencedin voltage regulator maintenance. In the apparent cause report attached to CR-CNS-2006-9096, the licensee reported that on the basis of the troubleshooting conducted, theacceptable operation of EDG 2 in the five test runs, and the vendor's experience, theapparent cause of the trip was the erratic behavior of one or both of the potentiometersof the voltage regulator card. The team reviewed the design of the voltage regulator,vendor instructions, industry maintenance guidelines, and industry operating experience,and was unable to find any information that corroborated this apparent cause. Theapparent cause report suggested corrective actions to improve the training of thoseinvolved in voltage regulator maintenance and to develop a voltage regulator testingprocedure to allow onsite personnel to tune the voltage regulator.Following the completion of troubleshooting, EDG 2 was run without incident in a number of surveillance tests, including a sequential load test, at the end of the RE23 outage. In addition, a routine four-hour surveillance test was completed successfully on December 12, 2006.EDG 2 Over-voltage Event on January 18, 2007On January 18, 2007, the licensee conducted Surveillance Procedure 6.2DG.101,"Diesel Generator 31 Day Operability Test."  After approximately 3 hours and 13 minutes at full load, the EDG 2 output breaker tripped open on an over-current
-7-condition, followed immediately by an over-voltage trip of EDG 2. During the transient,the KVAR loading of the machine reached a peak of 10,667 KVAR and the outputcurrent reached a peak of approximately 1700 amps when the EDG output circuitbreaker and the bus tie breaker to the safety related bus tripped open on over-current. With EDG 2 disconnected from the bus, output voltage rose rapidly until the machinetripped on over-voltage at 6009 volts.Following this third over-voltage trip of EDG 2, the licensee formed a root cause analysisteam and completed a significant troubleshooting effort with vendor support. Unable to locate a failed component, the licensee replaced the suspect voltage regulator card and performed a six hour loaded run of EDG 2. Based on acceptable performance during this test and the belief that the degraded condition that caused the January 18th trip nolonger existed, the licensee declared EDG 2 operable on January 22, 2007.3.0Root Cause and Immediate Corrective ActionsRoot Cause for Voltage Regulator FailuresAs documented in CR-CNS-2007-00480, the licensee determined that the root cause ofthe January 18, 2007 trip of EDG 2 was that the original procurement process did not provide technical requirements to reduce the probability of infant mortality failure in thevoltage regulator board in the EDG voltage regulating subsystem. The team reviewedthe licensee's root cause methodology and agreed with this conclusion.The procurement error associated with the voltage regulator card occurred in the Fall of1973 and involved the inadequate procurement of a commercial grade component foruse in safety related applications. The team noted that in the late 1980's, the NRCconducted a series of inspections to understand licensees' procurement andcommercial-grade dedication programs. These inspections resulted in the NRCproviding additional guidance related to acceptable licensee commercial gradeprocurement and dedication programs. Specifically, the NRC issued Generic Letter 91-05, "Licensee Commercial-Grade Procurement and Dedication Programs" that providedguidance for complying with the requirements of 10 CFR Part 50, Appendix B, "QualityAssurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," specificallyrelated to assuring the quality of items purchased and installed in safety relatedapplications.The team reviewed Engineering Procedure 3-CNS-DC-138.2, "Dedication," Revision 1,and determined that it had been adequately revised to define expectations for thededication of commercial-grade components based on the new guidance. The teamnoted that during the years in which substandard procurement practices were in placethat components of indeterminate quality had been accepted into the warehouse asspare parts acceptable for use in safety- related applications. The team determined thatthe licensee had  failed to establish adequate procedural guidance to evaluate the useof these parts of indeterminate quality prior to their installation in safety-relatedapplications.The enforcement aspects of this issue are fully described in Section 6 of this report.
 
-8-Immediate Corrective ActionsIn response to the failure of EDG 2 on January 18, 2007, the licensee formed anintegrated response team and staffed the Outage Control Center to track the status oftroubleshooting efforts.A prompt assessment was performed to determine whether the cause of the failure ofEDG 2 was applicable to EDG 1. The licensee demonstrated that during RE23, EDG 2had undergone a significant amount of invasive voltage regulator maintenance. Inaddition, EDG 2 twice experienced over-voltage trips during the outage. EDG 1 had notdemonstrated this behavior and did not undergo any invasive voltage regulatormaintenance. On the basis of these facts, the licensee determined that EDG 1operability was not affected by the failure of EDG 2.The licensee began a rigorous failure modes and effects analysis (FMEA), focusing onthe condition of any electrical component that could affect the control circuit in thevoltage regulator assembly. The FMEA included continuity checks, verification of relaysettings, and an in-situ test of the suspect voltage regulator printed circuit board. Theboard passed the in-situ test with no anomalies, and was then replaced with a spare partobtained from another nuclear utility. After tuning the new circuit board, the licenseeconducted a six hour surveillance run on EDG 2 followed by a full load reject test. Noanomalies were noted during the surveillance testing.On the basis of the FMEA results and the successful completion of the surveillance test,the licensee declared EDG 2 operable on January 22, 2007.4.0Review of Licensee Troubleshooting EffortsThe team examined vendor manuals, industry maintenance guidelines, and circuitdrawings for the Basler series boost static regulator (SBSR) voltage regulator andcompiled a list of all components, which could fail in a manner as to potentially replicatethe January 18 event. The team reviewed relevant operating experience to identifyadditional failure mechanisms experienced at other facilities. The team then comparedthe independently generated list to the licensee's FMEA to determine whether anyfailure modes had not been considered or eliminated inappropriately.In addition, the team observed a regularly scheduled surveillance test of EDG 1 onJanuary 29, 2007, conducted in accordance with Procedure 6.1DG.101, "DieselGenerator 31 Day Operability Test."  The team observed control room and localindications during EDG startup and steady state operations, reviewed the data collectedand the completed surveillance package. The team noted no anomalies in theperformance of EDG 1.The team reviewed the licensee's analysis concerning the potential effects that the highvoltage and current conditions could have on affected electrical equipment during thethree EDG 2 over-voltage transients described above. The team reviewed vendordocumentation, procurement records, drawings, relay setpoint calculations and
-9-conducted interviews with design and system engineering staff. The team determinedthat the protective devices installed, such as the over-voltage and over current relays, provided adequate protection for EDG 2 and connected equipment.During troubleshooting following the failure, a maintenance technician discovered twoloose terminal screws on the back of the EDG 2 OFF-MANUAL-AUTO switch (OMAS). The licensee had concluded that this was a potential cause of the January 18 failure. The team reviewed the design of the switch, its function in the circuit, and conductedseveral interviews with the technician who discovered the condition. On the basis of theintermittent nature of the OMAS discontinuity and the prolonged nature of the transienton January 18 (at least 30 seconds in duration), the team determined that the OMASdiscontinuity was a possible but implausible mode of failure. The team shared this viewwith licensee management during a debrief on February 1, 2007.The team identified several examples of failure modes that were not considered oreliminated without a technical basis, each of which was brought to the attention oflicensee management during a debrief on February 1, 2007. Following thesediscussions the licensee performed additional EDG 2 inspections on February 8, 2007, and performed eight hour loaded runs on February 8 and February 12, 2007. Theseadditional inspections and loaded runs did not identify any deficient conditions.5.0Offsite Testing of Suspect Voltage Regulator CardFollowing the over-voltage event on January 18, 2007, the licensee conducted an in-situtest of the voltage regulator printed circuit board to identify potential failure mechanisms. This test was similar to a factory acceptance test in that it powered up the board toidentify hard component failures. No such failures were detected during the in-situtesting.The licensee sent the board to a commercial test laboratory where, under vendorsupervision and with input from CNS engineering staff, a series of visual and electricaltests were conducted to identify failure modes. The laboratory identified that a zenerdiode on the printed circuit was in a failed state. CNS demonstrated that this failedzener diode could have caused the over-voltage event that occurred on January 18,2007.The failed circuit board was one of two circuit boards commercially purchased in 1973that were evaluated by the CNS staff as acceptable for use in safety related applications without obtaining reasonable assurance that the parts were of sufficient quality to support safety-related diesel generator functions. After the failure mode was identified, the team challenged the licensee regarding the treatment of the spare circuit board still in the warehouse. The licensee subsequently placed the spare circuit board in a blocked status pending further testing or evaluation of its level of quality.
 
-10-6.0Procurement of Spare Voltage Regulator Card
 
====a. Inspection Scope====
The team reviewed the quality assurance controls associated with the voltage regulatorprinted circuit board that contained the failed zener diode. Specifically, the team reviewed pertinent procurement documents and quality assurance program guidance to determine whether the requirements of 10 CFR 50 Appendix B, "Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," were satisfied.
 
====b. Findings====
 
=====Introduction.=====
The team identified an apparent violation of 10 CFR Part 50, Appendix B,Criterion V,  "Instructions, Procedures, and Drawings," involving the failure to establishappropriate procedural controls for evaluating the use of parts of indeterminate quality prior to their installation in safety-related applications. This deficiency resulted in the installation of a defective voltage regulator circuit board in EDG 2 that failed after 35 hours of operation.
 
=====Description.=====
On January 18, 2007, approximately three hours and thirteen minutes intoa loaded surveillance run, EDG 2 experienced an automatic trip because of a voltage regulator printed circuit board failure. The licensee's root cause report, documented in CR-CNS-2007-00480, determined that the cause of the failure was that the original procurement process did not provide technical requirements to reduce the probability of infant mortality failure in the voltage regulator board. The board had been installed in EDG 2 on November 11, 2006, as corrective maintenance to repair a failure of a potentiometer on the previously installed circuit board.The licensee determined that the failed circuit board had been purchased from theBasler Electric Company in 1973, but that the procurement of the part had not specified any technical requirements from the vendor. In effect, the part was purchased as a commercial grade item from a non-Appendix B source and placed into storage as an essential component, ready for use in safety-related applications, without any documentation of its suitability for that purpose. The licensee determined that thespecification of proper technical requirements, such as inspections and/or testing, wouldhave provided an opportunity to discover the latent defect prior to installing the card inan essential application.The team noted that in February 1992, the NRC had conducted an inspection at CNS toreview the implementation of Nebraska Public Power District's (NPPD) programs for theprocurement and dedication of commercial grade items used in safety-relatedapplications. The results, documented in NRC Inspection Report 05000298/1992-201,consisted of several identified deficiencies of the CNS quality assurance program. These deficiencies described that NPPD had been purchasing commercial grade itemsand dedicating them based solely on a part number verification, and warehousing theseparts as ready for essential use. The report stated this practice was insufficient to
-11-provide reasonable assurance that the parts were suitable for essential applications, andas a result, parts of indeterminate quality had been inappropriately labeled as essential.In response to these deficiencies, CNS made improvements to the existing qualityassurance program and performed a review of all commercial grade procurements made during the period of the inspection (January 1990 through February 1992). The team noted these process improvements failed to address that substandard qualification of essential components potentially occurred since new construction, and that any essential parts commercially procured prior to 1990, were potentially of indeterminate quality. This resulted in the licensee's failure to establish procedural controls to evaluatethe adequacy of these previously procured commercial grade parts that were deemed acceptable for use in safety related applications.The team reviewed the current requirements of Administrative Procedure 0.40.4,"Planning," Revision 2 and Site Services Procedure 1-CNS-MP-115, "Material Issuesand Staging," Revision 6. Procedure 0.40.4 establishes the steps taken by work planners to select the correct quality class part for a given work order, and provides instructions for planners if the required part safety classification does not match the available asset. Procedure 1-CNS-MP-115 defines the activities taken by the warehouse personnel to verify that parts obtained from the warehouse satisfy the requirements identified by the work planners. These program requirements included checks for shelf life, post work testing, and other appropriate barriers. However, as previously discussed, neither procedure evaluated the use of parts that were inadequately procured for use in safety related applications prior to 1990. On the basis of the deficiencies in the CNS procurement program previously identified by the NRC, and a review of site quality procedures, the team determined that the licensee had failed to establish appropriate procedural controls for evaluating the use of parts ofindeterminate quality prior to their installation in safety-related applications.The team noted that if the licensee had implemented administrative controls to reviewthe suitability of pre-1990 procurement items prior to their use in safety-related applications, then the licensee could have had the opportunity to discover the inadequate measures taken to qualify the voltage regulator circuit boards for essential applications. Given this information, the licensee could have either procured a new board from an essential source, or dedicated the old part using Engineering Procedure3-CNS-DC-138.2, "Dedication."  Procedure 3-CNS-138.2 directs the user to EngineeringProcedure 3-CNS-DC-138, "Technical Evaluation Process," for the identification ofcritical characteristics important to provide reasonable assurance that a part is ready foressential use. Attachment 3 to Procedure 3-CNS-DC-138 provides examples ofappropriate critical characteristics, one of which is a "burn-in" endurance test. The teamreviewed industry standards for such tests, including MIL-STD-750D, "Department ofDefense Test Method Standard for Semiconductor Devices," which recommends a96 hour burn in test. Given that the voltage regulator card failed after approximately 35hours of service, the team concluded that a burn-in or other equivalent test would havegiven CNS the opportunity to discover the latent defect. To better understand currentCNS standards for acceptance of safety-related electrical components, the inspectorsreviewed Change Evaluation Document (CED) 6017822, that is being implemented toinstall new circuit cards in the Average Power Range Monitor system. Purchase
-12-Order 4500055181 procured the new circuit cards and imposed a burn-in test prior toshipment. The team also noted that the non-safety related electrical components forCEDs 6010820 and 6016542, being implemented to install new reactor water level andfeedwater control systems, have been energized on site for testing for over one year inan attempt to complete logic verification and identify latent defects.The team noted that the failure to evaluate these parts of indeterminate quality resultedin the failure to identify the latent defect in the voltage regulator circuit board in EDG 2.
 
Specifically, this deficiency resulted in installing a voltage regulator circuit board of indeterminate quality in EDG 2 on November 11, 2006, that experienced an infant mortality failure after 35 hours of operation on January 18, 2007.Analysis:  This finding is considered to be a performance deficiency because thelicensee failed to establish appropriate procedural guidance to ensure that commerciallyprocured components are of sufficient quality prior to their installation in safety related applications. This finding is more than minor because it is associated with theequipment performance attribute of the Mitigating Systems cornerstone and adverselyaffects the cornerstone objective of ensuring the availability, reliability, and capability ofsystems that respond to initiating events.The team evaluated the issue using the Significance Determination Process (SDP)Phase 1 Screening Worksheet provided in Manual Chapter 0609, Appendix A,"Significance Determination of Reactor Inspection Findings for At-Power Situations." The screening indicated that a Phase 2 analysis was required because the findingrepresents a loss of safety function for EDG 2 for greater than its TechnicalSpecification allowed completion time. The Phase 2 and 3 evaluations preliminarilyconcluded that the finding was of low to moderate safety significance (See Attachment 3for details).
 
=====Enforcement:=====
10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, andDrawings," requires, in part, that activities affecting quality shall be prescribed by procedures of a type appropriate to the circumstances and shall include appropriateacceptance criteria for determining that important activities have been satisfactorilycompleted. Contrary to this, the licensee failed to establish appropriate procedural controls for evaluating the use of parts of indeterminate quality prior to their installation in safety-related applications. This deficiency resulted in the installation of a voltage regulator circuit board of indeterminate quality in EDG 2 that prematurely failed after 35 hours of operation. This issue was entered into the licensee's corrective action program as CR-CNS-2007-00480. Pending determination of the finding's final safety significance, this finding is identified as Apparent Violation (AV)  05000298/2007007-001, "Inadequate Procedures Result in Failure of Emergency Diesel Generator VoltageRegulator."
 
-13-7.0Review of EDG 2 Maintenance Procedures
 
====a. Inspection Scope====
The team assessed the maintenance procedures and work orders used to perform preventive and corrective maintenance activities on EDG 2 between 2005 and the failure on January 18, 2007. In addition to reviewing the written procedures, the team interviewed procedure writers and maintenance technicians to determine if the work instructions provided were adequate to achieve their intended objectives.
 
====b. Findings====
 
=====Introduction:=====
The team identified three examples of a Green noncited violation (NCV) ofTS 5.4.1.a regarding inadequate maintenance procedures associated with EDG 2.Description:  Maintenance Procedure (MP) 7.3.8.2, "Diesel Generator ElectricalExamination and Maintenance," is performed by the licensee approximately every eighteen months and is intended to perform inspections and preventive maintenance onengine and generator electrical components. This procedure was conducted on EDG 2on March 14, 2006, and November 8, 2006. MP 7.3.8.2, Revision 19, contained  aspecific section on voltage regulator maintenance that included instructions to checklugs and screw terminals for tightness and integrity. The team noted that the scope of the voltage regulator checks was limited to only those components within the voltage regulator cabinet. This deficiency prevented maintenance technicians from identifying that loose terminal screws existed on the voltage regulator OFF-MANUAL-AUTO switch (OMAS) because it is physically located on the EDG 2 metering panel. These loose terminal connections were discovered during troubleshooting efforts following the January 18, 2007, failure of EDG 2. The licensee demonstrated that these loose terminations represented a degraded condition that could cause an over-voltage event similar in nature to that experienced on January 18, 2007. MP 7.3.8.2 was inadequate in that it did not contain sufficient procedural guidance to identify the degraded conditions in voltage regulator system components outside the voltage regulator cabinet. The licensee documented this performance deficiency in CR-CNS-2007-00794.In the second example, WO 4514076 was planned for the RE23 refueling outage as anassigned corrective action from CR-CNS-2006-2729. The intent of the work order was to perform a thorough inspection of EDG 2 voltage regulator system components to identify the cause of continued voltage and reactive load perturbations experienced during surveillance testing. WO 4514076 contained specific instructions for checking the integrity of electrical connections in the EDG 2 voltage regulator cabinet and metering panel. On the basis of interviews conducted with the system engineer who wrote the work instructions, the team learned the engineer had intended for the maintenance personnel to specifically check the terminal screws on the OMAS switch for tightness. The guidance to perform this check was contained in a note at the front of the procedure and not in the individual work order step for the switch. As a result, the OMAS switch connections were not checked for tightness, resulting in another missed opportunity to discover the loose terminal connections on the switch. The licensee documented this performance deficiency in CR-CNS-2007-01021.
 
-14-In the third example, WO 4514076 contained inadequate instructions for tuning the EDG2 voltage regulator during the RE23 refueling outage. On November 11, 2006, the printed circuit board in the EDG 2 voltage regulator was replaced to correct a degraded potentiometer. WO 4514076 contained instructions for tuning the new voltage regulator card following installation, but was inadequate in that it contained acceptance criteria that were inappropriate for the voltage regulators installed at CNS. The instructions directed maintenance personnel to adjust the R13 potentiometer to obtain "quarter wave dampening" in the EDG output voltage response.
 
The technicians noted that adjusting R13 did not change the amplitude of the sinusoidal response, and determined that the acceptance criteria in WO 4514076 could not be satisfied. Technicians then set the R13 potentiometer resistance on the new card to the same value as found on the old card. During subsequent measurements of the voltage regulator response, the technicians noted that the time required for the output voltage to oscillate through one complete cycle had increased from 3.1 to 3.8 seconds. The technicians accepted this new response characteristic without any engineering evaluation, procedural guidance, or vendor technical reference demonstrating its acceptability. A subsequent evaluationperformed by engineering demonstrated that this change did not interfere with the safetyfunction of the EDG. Additionally, EDG 2 experienced an over-voltage trip on November13, 2006, during the tuning process. In condition report CR-CNS-2006-9096, thelicensee documented the apparent cause of the EDG trip as "the erratic behavior of oneor both of the potentiometers on the voltage regulator card" and went on to explain thatindustry operating experience and vendors both recognize that this over-voltage tripcould have been caused by the tuning process. The corrective actions proposed in the apparent cause report included improvements to the EDG voltage regulator tuning process and additional training for maintenance personnel performing the activity. The licensee documented this procedural inadequacy in CR-CNS-2007-1307.Analysis:  The performance deficiency associated with this finding involved thelicensee's failure to provide adequate instructions for performing maintenance on EDG 2. The finding is more than minor because it is associated with the Mitigating Systems cornerstone attribute of procedure quality and affects the associated cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events. Specifically, the performance deficiency resulted in:  (1) the failure to discover a degraded condition in the EDG 2 voltage regulator, and (2) an over-voltage trip during the tuning of EDG 2 on November 13, 2006. Using the Manual Chapter 0609 Appendix G, "Shutdown Operations Significance Determination Process,"
Phase 1 Checklist, the finding is determined to have very low safety significance because one operable diesel generator was still capable of supplying power to the class 1E electrical power distribution subsystems.This finding has a cross-cutting aspect in the area of human performance in that thelicensee's procedures were not complete and provided inadequate instructions forpersons conducting maintenance on safety related equipment.Enforcement. Technical Specification 5.4.1.a requires that written procedures beestablished, implemented, and maintained, covering the activities specified in Regulatory Guide 1.33, Revision 2, Appendix A, dated February 1978. Regulatory Guide 1.33, Appendix A, Section 9 (a), requires that maintenance affecting the
-15-performance of safety-related equipment should be performed in accordance withwritten procedures. Contrary to this, Maintenance Procedure 7.3.8.2 and Work Order4514076 did not contain adequate instructions to identify the degraded condition of the EDG 2 OFF-MANUAL-AUTO switch. In addition, Work Order 4514076 did not contain adequate instructions for the tuning of EDG 2 following the replacement of the voltage regulator printed circuit board, resulting in an over-voltage trip of EDG 2 on November 13, 2006. Because the finding is of very low safety significance and has been entered into the licensee's corrective action program in Condition Reports CR-CNS-2007-00794,CR-CNS-2007-01021, and CR-CNS-2007-01307, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy:  NCV 05000298/2007007-002, "Inadequate Procedures for Conducting Maintenance on Emergency Diesel Generator 2."8.0Long-Term Corrective ActionsThe team interviewed system and design engineering personnel to ascertain the longterm plans the licensee intends to implement to improve EDG electrical performance.
 
The licensee has already implemented measures to improve their ability to monitor EDG performance during surveillance testing through the installation of special test equipment. Future plans include installation of permanent external test connections to minimize the unavailability time required to hook up the test equipment for each scheduled surveillance.The licensee has initiated actions to establish technical requirements for burn-in or otherequivalent testing for safety-related DG system circuit boards. Additionally, the licensee plans to define other safety related systems with circuit boards that need similar treatment.The licensee also plans to implement several modifications to improve EDG reliability inupcoming refueling outages. The changes include installation of a digital MOP during the next refueling outage and replacement of the entire voltage regulator system with a digital system in the subsequent outage.9.0Potential Generic IssuesThe team noted that CNS submitted an operating experience report to alert the industryof the potential failure of zener diodes in Basler SBSR voltage regulators. The team did not identify any potentially generic issues during the inspection.
 
-16-4OA6 Meetings, Including ExitOn February 1, 2007, the preliminary results of this inspection were presented toMr. M. Colomb and other members of his staff who acknowledged the findings. Following additional in-office reviews, the final results of the inspection were presented to Mr. Colomb and his staff on April 24, 2007. The team confirmed that the supporting details in this report contained no proprietary information.ATTACHMENT 1: 
 
=SUPPLEMENTAL INFORMATION=
 
==KEY POINTS OF CONTACT==
 
===Licensee Personnel===
: [[contact::M. Bergmeier]], Operations Support Group Supervisor
: [[contact::D. Buman]], System Engineering Manager
: [[contact::K. Cohn]], Engineering Support
: [[contact::M. Dickerson]], Design Engineering
: [[contact::J. Flaherty]], Licensing Engineer
: [[contact::P. Fleming]], Licensing Manager
: [[contact::C. Gaedeke]], Maintenance
: [[contact::T. Hottovy]], Equipment Reliability Manager
: [[contact::J. Larson]],  Quality Assurance Supplier Supervisor
: [[contact::M. McCormack]], Electrical Systems/I&C Engineering Supervisor
: [[contact::E. McCutchen]], Regulatory Affairs Senior Licensing Engineer
: [[contact::M. Metzger]], System Engineer
: [[contact::B. Morris]], Maintenance Support Superintendent
: [[contact::R. Noon]], Root Cause Team Leader, Corrective Action & Assessments
: [[contact::S. Norris]], Assistant Operations Manager
: [[contact::R. Rexroad]], System Engineering
: [[contact::K. Sutton]], Risk Management Supervisor
: [[contact::D. Willis]], Operations Manager
===NRC Personnel===
: [[contact::S. Graves]], Reactor Inspector
: [[contact::M. Haire]], Enforcement Specialist
: [[contact::R. McIntyre]], Quality & Vendor Branch A
: [[contact::R. Pettis]], Quality & Vendor Branch A
: [[contact::P. Prescott]], Quality & Vendor Branch A
: [[contact::S. Rutenkroger]], PHD, Reactor Inspector
: [[contact::S. Schwind]], Senior Resident Inspector
: [[contact::N. Taylor]], Resident Inspector
: [[contact::D. Thatcher]], Chief, Quality & Vendor Branch A
1A1-2
==LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED==
Opened05000298/2007007-01AVInadequate Procedures Result in Failure of EmergencyDiesel Generator Voltage RegulatorOpened and
===Closed===
: [[Closes finding::05000298/FIN-2007007-02]]NCVInadequate Procedures for Conducting Maintenance onEmergency Diesel Generator 2LIST OF ACRONYMSAVApparent ViolationCFRCode of Federal RegulationsCNSCooper Nuclear StationCRcondition reportEDGemergency diesel generatorFMEAfailure modes and effects analysisKVARkilovolt-ampere-reactiveMOPmotor operated potentiometerMPmaintenance procedureNCVnoncited violationNPPDNebraska Public Power DistrictNRCU.S. Nuclear Regulatory CommissionOMASoff-manual-auto switchPMpreventive maintenanceRE23refueling outage 23SBSRseries boost static regulatorSDPsignificance determination processWOwork order
==LIST OF DOCUMENTS REVIEWED==
===Procedures===
: Reviewed:Administrative Procedure 0.40.4, "Planning," Revision 2Site Service Procedure 1-CNS-MP-15, "Material Issues and Staging," Revision 6Engineering Procedure 3-CNS-DC-138, "Technical Evaluation Process," Revision 1Engineering Procedure 3-CNS-DC-138.2, "Dedication," Revision 1
: 1A1-3Quality Assurance Instruction
: QAI-9, "Guidelines for Establishing Quality Classifications ofComponents and Materials," Draft 10/3/73Condition Reports:CR-CNS-2005-00938,
: CR-CNS-2005-07806,
: CR-CNS-2005-08336,
: CR-CNS-2006-02091,
: CR-CNS-2006-02140,
: CR-CNS-2006-00279,
: CR-CNS-2006-02963,
: CR-CNS-2006-05149,
: CR-CNS-2006-08798,
: CR-CNS-2006-08999,CR-CNS-2006-09096,
: CR-CNS-2006-09301,
: CR-CNS-2007-00480Work Orders:4338439,
: 4424754,
: 4458094,
: 4472125,
: 4499755,
: 4514076,
: 4536182,
: 4535878,
: 4536371,4548656,
: 4548698,
: 4548841,
: 4548860, 4551090Controlled Drawings:Basler Electric Company drawing
: 9032100910, Revision N01
: Basler Electric Company drawing
: 9072400910, Revision N03
: 14EK-0144, Rev. N17
: BR 3012, Sheet #3, Rev. N17
: BR 3012, Sheet #6, Rev. N15
: 23R0558, Sheet #33, Rev. N2214DK0921, Rev. N01BR 3257, Sheet 48H, Rev. N01BR 3251, Sheet 11, Rev. N17G5-262-743, Sheet 10A, Rev. N03Miscellaneous Documents:Purchase Order 73440, September 21, 1973Nonconformance Report 002, October 27, 1973Purchase Order 75149, November 15, 1973Certificate of Compliance for Purchase Order 75149, December 3, 1973CNS Vendor Manual
: VM-0246 [Basler Type SBSR HV Series Boost Exciter-Regulator]
: 1A1-4Instruction Manual
: SM-100, "Synchronous Motors, Generators, D.C. Exciters & BrushlessEquipment," Ideal Electric (no date)Memo from Ideal Electric and Manufacturing Company to Cooper Bessemer Company, dated8-28-70 (provided specific ratings for CNS generators)CNS Design Criteria Document,
: DCD-1, "Diesel Generators"EPRI Technical Report
: 1011110, "Basler SBSR Voltage Regulators for Emergency DieselGenerators," Final Report, November 2004NUREG/CR-6819, Vol. 1, "Common-Cause Failure Event Insights, Emergency DieselGenerators"NRC Information Notice 96-23, "Fires in Emergency Diesel Generator Exciters DuringOperation Following Undetected Fuse Blowing," April 22, 1996IEEE Standard 336-1971, "Installation, Inspection and Testing Requirements forInstrumentation and Electric Equipment During The Construction of Nuclear Power Generating Stations"
: 2A2-1SPECIAL INSPECTION CHARTERJanuary 25, 2007MEMORANDUM TO:Nicholas H. Taylor, Resident Inspector, Cooper Nuclear StationProject Branch C, Division of Reactor ProjectsDr. Scott P. Rutenkroger, Reactor InspectorEngineering Branch 1, Division of Reactor SafetyFROM:Arthur T. Howell III, Director, Division of Reactor Projects /RA/ AVegel for
: SUBJECT: SPECIAL INSPECTION CHARTER TO EVALUATE THE COOPERNUCLEAR STATION EMERGENCY DIESEL GENERATOR FAILUREA Special Inspection Team is being chartered in response to the Cooper Nuclear Station emergency diesel generator (EDG) failure.
: The EDG failed during surveillance testing on January 18, 2007.
: You are hereby designated as the Special Inspection Team members.
: Mr. Taylor is designated as the team leader.
: The assigned senior reactor analyst (SRA) to support the team is Mike Runyan.A.BasisOn January 18, 2007, during performance of a monthly surveillance test, EmergencyDiesel Generator 2 automatically isolated from the electrical bus following an over-
current condition.
: The licensee determined this condition resulted from a high voltage condition.
: The licensee has preliminarily identified the cause of the failure to be either:
(1) a loose electrical connection affecting the voltage regulator circuit, or (2) a latent failure of the voltage regulator printed circuit board.
: The licensee has experienced previous voltage regulator problems, resulting in replacement of EDG 2 voltage regulator components during the last refueling outage (RE23).
: The most recent failure of EDG 2, and previous licensee efforts to identify and correct EDG 2 voltage regulator problems, draws into question the effectiveness of the licensee's corrective actions.
: Additionally, prior to the failure on January 18, 2007, EDG voltage regulator troubleshooting and postmaintenance activities have resulted in additional automatic trips due to high voltage conditions.This Special Inspection Team is chartered to review the circumstances related tohistorical and present EDG 2 voltage regulator problems and assess the effectiveness of the licensee's actions for resolving these problems.
: The team will also assess the effectiveness of the immediate actions taken by the licensee in response to the EDG 2
failure that occurred on January, 18, 2007.
: 2Attachment 2A2-2B.ScopeThe team is expected to address the following:1.Develop an understanding of the EDG degraded conditions and failures relatedto voltage regulator problems.2.Assess licensee effectiveness in identifying previous EDG voltage regulatorproblems, evaluating the cause of these problems, and implementation of corrective actions to resolve identified problems.3.Identify and assess additional actions planned by the licensee in response to thedeclining performance of the EDG 2 voltage regulator, including the timeline for completion of these actions.4.Assess the licensee's root cause evaluation, the extent of condition, and thelicensee's common mode evaluation.5.Evaluate pertinent industry operating experience and potential precursors to theJanuary 18 event, including the effectiveness of licensee actions taken in response to the operating experience.6.Determine if there are any potential generic issues related to the failure of theEDG 2 voltage regulator.
: Promptly communicate any potential generic issues to Region IV management.7.Determine if the Technical Specifications were met when the EDG failed.
: 8.Collect data as necessary to support a risk analysis.C.GuidanceInspection Procedure 93812, "Special Inspection," provides additional guidance to beused by the Special Inspection Team.
: Your duties will be as described in Inspection
===Procedure===
: 93812.
: The inspection should emphasize fact-finding in its review of the circumstances surrounding the event.
: It is not the responsibility of the team to examine the regulatory process.
: Safety concerns identified that are not directly related to the event should be reported to the Region IV office for appropriate action.The Team will report to the site, conduct an entrance, and begin inspection no later than January 28, 2007.
: While on site, you will provide daily status briefings to Region IV
management, who will coordinate with the Office of Nuclear Reactor Regulation, to ensure that all other parties are kept informed.
: A report documenting the results of the inspection should be issued within 30 days of the completion of the inspection.This Charter may be modified should the team develop significant new information thatwarrants review.
: Should you have any questions concerning this Charter, contact me at
(817) 860-8144.
: 3A3-1SIGNIFICANCE DETERMINATION EVALUATIONSignificance determination process Phase 1:In accordance with NRC Inspection Manual Chapter 0609, Appendix A, "SignificanceDetermination of Reactor Inspection Findings for At-Power Situations," the inspectors conducted a significance determination Phase 1 screening and determined that the finding resulted in loss of the safety function of Emergency Diesel Generator 2 for greater than the Technical Specification allowed completion time.
: Therefore, a Significance Determination Process Phase 2 evaluation was required.Significance determination process Phase 2:In accordance with Manual Chapter 0609, Appendix A, Attachment 1, "User Guidancefor Determining the Significance of Reactor Inspection Findings for At-Power Situations," the inspectors evaluated the EDG 2 failure using the Risk-Informed Inspection Notebook for Cooper Nuclear Station, Revision 2.Assumptions1.The Phase 2 analysis assumed that the EDG 2 was unable to perform itsfunction beginning on November 16, 2006.
: This date assumes that the failure ofthe voltage regulator was a run-time degradation (consistent with the licensee'sroot cause) and recognizes that in the licensee's risk model EDG 2 must becapable of providing its safety function for 24 hours following an accident.
: Thisresulted in an applied exposure time of 64 days.2.Recovery credit was not appropriate due to the lack of a useable procedure oroperator training in operating the EDG 2 voltage regulator in the manual modeduring a loss of offsite power.3.The loss of EDG 2 affects the full mitigation credit for other safety functions onthe Loss of Offsite Power worksheet (reference special useage rule 1.6).4.Phase 2 Analysis Results:
: The Phase 2 analysis indicated that the significanceof the finding was potentially Greater than Green.
: The dominant accident sequence involved a loss of offsite power with recovery within the 8 hour life of the CNS batteries, and a loss of offsite power without high pressure injection and recovery within one hour.
: Completion of all applicable sequences in the Loss of Offsite Power worksheet resulted in one sequence with a score of 8, three sequences with a score of 7 and one sequence with a score of 6.
: Based on this result the issue screened as "White" in the SDP Phase II analysis.
: 3A3-2Significance determination process Phase 3:The analyst estimated the risk increase resulting from the EDG 2 voltage regulatorfailure.
: The diesel was run at the following times with durations reported as the period of time that the voltage regulator was energized (all of these operational runs were conducted after the performance deficiency occurred):Date of EDG RunDuration of EDG RunNovember 13, 20061 hr 30 minNovember 14, 20066 hrs 46 min November 15, 20061  hr 35 min November 16, 20069 hrs 23 min November 17, 20065 hrs 3 min November 18, 20062 hrs 28 minDecember 12, 20065 hrs 41 minJanuary 18, 20074 hrs 16 min (point of failure)Assumptions:1. It is assumed that the voltage regulator degraded only during times that it wasenergized, which is closely correlated to engine runtime.
: This implies that no degradation occurred while the EDG was secured and in a standby status.
: It is further assumed that the failure was a deterministic outcome set to occur after a specific number of operating hours.
: Therefore, it is assumed that EDG 2 would have failed to run at four hours following a loss of offsite power (LOOP) demand at any time during the 37 day period from its last successful surveillance test on December 12, 2006, until the test failure that occurred on January 18, 2007.
: Prior to this date, EDG 2 would have run and failed at 10 hours for the 24 day period from November 18, 2006, to December 12, 2006.
: The EDG was run for approximately 17 hours in the two days preceding November 18, 2006.
: Because very little exposure existed prior to November 18, 2006, this date is chosen as the cutoff for this analysis.2.The voltage regulator could not have been repaired in place or replaced with anew unit in time to affect the outcome of any of the core damage sequences.
: Also, procedures did not exist and training was not conducted to operate the
: EDG in a manual voltage regulation mode.
: Therefore, it is assumed that the
: EDG 2 voltage regulator failure would not have been recoverable for any accident sequence.
: 3A3-33. For the purpose of this analysis, it is assumed that EDG 2 would not beunavailable or fail to operate for the first four hours or 10 hours of its hypothetical start demand during the 37 day and 24 day exposure periods, respectively.
: This introduces a slight inconsistency to the risk estimate, but because it would similarly affect both the base and current case, it does not significantly influence the result of this analysis.4.
: Common cause vulnerabilities for EDG 1 did not exist, that is, the failure mode isassumed to be independent in nature.
: This is because the root cause investigation determined that the failure was the result of an infant mortality of a voltage regulator conponent, which had been installed for only two months.
: The same component in EDG1 had been installed for several years and had operated reliably beyond the "burn-in" period without experiencing failure from manufacturing defects.The CNS SPAR model, Revision 3.31, dated October 10, 2006, was used in theanalysis.
: A cutset truncation of 1.0E-12 was used.
: Average test and maintenance was assumed.To represent the assumed failure of EDG 2, the basic event
: EPS-DGN-FR-DG1B(diesel generator 1B fails to run) was set to one.
: A flag set house event for "DG1B out of service" was added to the EDG1B fault tree and set to FALSE in the base case, and
: TRUE in the current case in order to remove non-minimal cutsets.
: Also, the common cause probability for fail-to-run events was restored to its nominal value.Internal Events Analysis:A.Risk Estimate for the 37-day period between December 12, 2006 andJanuary 18, 2007:During this exposure period, EDG 2 is assumed to have been capable of running forfour hours.
: The LOOP frequency used in the analysis was adjusted to reflect the situation that only LOOPs with durations greater than four hours would result in a risk increase attributable to the voltage regulator failure.
: The base LOOP frequency is
: 3.59E-2/yr.
: The 4-hour non-recovery of offsite power is 0.1566.
: The 4-hour non-
recovery of diesel generators is 0.4835.
: To account for having only one EDG to recover during the first four hours (since recovery of EDG 2 is assumed to be running during the first four hours of the event), the EDG non-recovery factor was adjusted to the square root of the base non-recovery factor.
: This adjusts the recovery from a one out of two
: EDG recovery to a one out of one recovery.
: This factor is (0.4835)1/2 = 0.695.
: Thereforethe adjusted LOOP frequency, representing the frequency of LOOPs that are not recovered in four hours by either restoring offsite power or recovering a failure of EDG 1
is 3.59E-2(0.1566)(0.695) = 3.91E-3/yr.
: For the base case, the adjusted LOOP
frequency considers that both EDGs are hypothetically recoverable.
: Therefore the base case LOOP frequency is 3.59E-2(0.1566)(0.4835) = 2.72E-3/yr.
: 3A3-4Resetting event time t=0 to four hours following the LOOP event requires that therecovery factors for offsite power and the EDGs be adjusted.
: For example, in two hour sequences in SPAR, the basic event for non-recovery of offsite power should be adjusted to the non-recovery at 6 hours, given that recovery has failed at four hours.
: An adjustment to account for the diminishment of decay heat must be considered.
: Thisis because the magnitude of decay heat at four hours following shutdown is less than in the early moments following a reactor trip, and the timing of core damage sequences is affected by this fact.
: In the SPAR model, recovery times for either offsite power, EDGs, or both are set at the intervals of 30 minutes, two hours, four hours, eight hours, and ten hours.
: The analyst determined that the average decay heat level in the first 30 minutes is approximately two times the average level that exists between four and five hours following shutdown.
: Therefore, baseline 30-minute SPAR model sequences, that essentially account for boil-off to fuel uncovery were adjusted to one hour sequences.
: The two hour sequences model safety relief valve failures to close, and are based more on inventory control than core heat production.
: Therefore, no adjustment was made for these sequences.
: The analyst determined that decay heat rates leveled out quickly following shutdown and could find no basis for adjusting the times associated with the four, eight, and ten hour sequences.The following table presents the adjusted offsite power non-recovery factors for theevent times that are relevant in the SPAR core damage cutsets:SPARrecoverytimeSPAR baseoffsite powernon-recoverySPAR baseoffsite powernon-recovery at4 hours SPAR baseoffsite powernon-recovery at4 hours + SPARrecovery time inColumn 1ModifiedSPARrecovery(Column 4divided byColumn 3)30 min.0.73140.15660.120510.7692 hours0.31810.15660.096370.615
hours0.15660.15660.067180.429
hours0.067180.15660.040400.25810 hours0.050700.15660.033460.2141.
: A SPAR recovery time of 1.0 hours is used, as discussed above, to account for the lessening of decay heat.The following table presents the analogous non-recovery factor adjustments for EDG 1recovery times for the current case (it is assumed that EDG 2 is not recoverable):
: 3A3-5SPARrecoverytimeSPAR base non-recovery for twoEDGsSPAR baseEDG non-recovery at 4hours for 1 EDG(square root of0.4835)SPAR baseEDG non-recovery at 4hours + SPARrecovery time inColumn 1(square root)ModifiedSPARrecovery(Column 4divided byColumn 3)30 min.0.85700.6950.65110.9372 hours0.64820.6950.6120.881
hours0.48350.6950.5440.783
hours0.29590.6950.4390.63210 hours0.23740.6950.3970.5711.
: A SPAR recovery time of 1.0 hours is used, as discussed above, to account for the lessening of decay heat.The following table presents the EDG non-recoveries used for the base case (bothEDGs are assumed available for recovery in the base case):SPARrecoverytimeSPAR base non-recovery for twoEDGsSPAR baseEDG non-recovery at 4hoursSPAR baseEDG non-recovery at 4hours + SPARrecovery time inColumn 1 ModifiedSPARrecovery(Column 4divided byColumn 3)30 min.0.8570.48350.424010.8772 hours0.6482.48350.37420.774
hours0.4835.48350.29590.612
hours0.2959.48350.19260.39810 hours0.2374.48350.15760.3261.
: A SPAR recovery time of 1.0 hours is used, as discussed above, to account for the lessening of decay heat.The SPAR base case was updated to reflect the new LOOP frequency and non-recovery times for offsite power and EDGs (column 5 figures).The SPAR base case update result, after applying the applicable revised LOOPfrequency and offsite power and EDG recovery figures, was 1.039E-5/yr.
: The current case result, with the EDG 2 fail-to-run set to one and the flag set house event set to
: TRUE, and the changed recoveries inserted for offsite power and the EDGs was
: 5.373E-5/yr.
: 3A3-6Therefore, the estimated ICCDP of the 37-day period during which EDG 2 was assumedto be in a condition that guaranteed its failure at four hours is (5.373E-5/yr. - 1.039E-
: 5/yr.) (37 days/365 days/yr.) = 4.4E-6/yr. B.Risk Estimate for the 24-day period between November 18, 2006 andDecember 12, 2006:During this exposure period, EDG 2 is assumed to have been capable of running for 10hours.
: The LOOP frequency used in the analysis was adjusted to reflect the situation that only LOOPs with durations greater than 10 hours would result in a risk increase attributable to the voltage regulator failure.
: The base LOOP frequency is 3.59E-2/yr.
: The 10-hour non-recovery of offsite power is 5.070E-2.
: The 10 hour non-recovery of diesel generators is 0.2374.
: To account for having only one EDG to recover during the first 10 hours (since recovery in this analysis only applies to the postulated failure of
: EDG 1), the EDG non-recovery factor was adjusted to the square root of the base non-
recovery factor.
: This adjusts the recovery from a one out of two EDG recovery to a one out of one recovery.
: This factor is (0.2374)1/2 = 0.487.
: Therefore, the adjusted LOOPfrequency, representing the frequency of LOOPs that are not recovered in ten hours by either restoring offsite power or recovering a failure of EDG 1, is 3.59E-2(5.070E-
: 2)(0.487) = 8.86E-4/yr.
: For the base case, the adjusted LOOP frequency considers that both EDGs are hypothetically recoverable.
: Therefore, the base case LOOP frequency is 3.59E-2(5.070E-2)(0.2374) = 4.32E-4/yr. The analyst considered an adjustment to account for the diminishment of decay heat asin the four hour case above.
: The analyst determined that the average decay heat level in the first 30 minutes is approximately three times the average level that exists between and 11 hours following shutdown.
: Therefore, the baseline 30 minute SPAR models, that essentially account for boil-off to fuel uncovery were adjusted to 1.5 hour sequences.
: The two hour sequences model safety relief valve failures to close, and are based more on inventory control than core heat production.
: Therefore, no adjustment was made for these sequences.
: Sequences of four and eight hours were increased by minutes each, but no change was made to the 10 hour sequences.The following table presents the adjusted offsite power non-recovery factors for theevent times that are relevant in the SPAR core damage cutsets:
: 3A3-7SPARrecoverytimeSPAR baseoffsite powernon-recoverySPAR baseoffsite powernon-recovery at10 hours SPAR baseoffsite powernon-recovery at10 hours +SPAR recoverytime in Column1ModifiedSPARrecovery(Column 4divided byColumn 3)30 min.0.73140.05070.042710.8422 hours0.31810.05070.04040.797
hours0.15660.05070.032120.6338 hours0.067180.05070.024120.47510 hours0.050700.05070.02200.4341.
: A SPAR recovery time of 1.5 hours is added to 10 hours, as discussed above, to account for the falloff of decay heat.2.
: The SPAR recovery time is increased by 30 minutes, as discussed above. The following table presents the analogous non-recovery factor adjustments for EDG 1recovery times:SPARrecoverytimeSPAR base non-recovery for twoEDGsSPAR baseEDG non-recovery at 10hours for 1 EDG(square root of0.2374)SPAR baseEDG non-recovery at 10hours + SPARrecovery time inColumn 1(square root)ModifiedSPARrecovery(Column 4divided byColumn 3)30 min.0.85700.48720.45110.9262 hours0.64820.48720.4390.901
hours0.48350.48720.38820.7968 hours0.29590.48720.32120.65910 hours0.23740.48720.3000.6161.
: A SPAR recovery time of 1.5 hours is added to 10 hours, as discussed above, to account for the falloff of decay heat.2.
: The SPAR recovery time is increased by 30 minutes, as discussed above. The following table presents the EDG recoveries used for the base case (both EDGs areassumed available for recovery in the base case):
: 3A3-8SPARrecoverytimeSPAR base non-recovery for twoEDGsSPAR baseEDG non-recovery at 10hours for 1 EDG SPAR baseEDG non-recovery at 10hours + SPARrecovery time inColumn 1 ModifiedSPARrecovery(Column 4divided byColumn 3)30 min.0.85700.23740.203010.8552 hours0.64820.23740.19260.811
hours0.48350.23740.150220.6338 hours0.29590.23740.103020.43410 hours0.23740.23740.08980.3781.
: A SPAR recovery time of 1.5 hours is added to 10 hours, as discussed above, to account for the falloff of decay heat.2.
: The SPAR recovery time is increased by 30 minutes, as discussed above. The SPAR base case was updated to reflect the new LOOP frequency and non-recovery times for offsite power and EDGs (column 5 figures).The SPAR base case update result, after applying the applicable revised LOOPfrequency and offsite power and EDG recovery figures, was 1.008E-5/yr.
: The current case result, with the EDG 2 fail-to-run set to one and the flag set house event set to
: TRUE and the changed recoveries inserted for offsite power and the EDGs, was
: 2.830E-5/yr.Therefore, the estimated ICCDP of the 24-day period during which EDG 2 was assumedto be in a condition that guaranteed its failure at 4 hours, is (2.830E-5/yr. - 1.008E-5/yr.)
(24 days/365 days/yr.) = 1.2E-6/yr. Total Internal Events Result:Exposure Period37-Day (12/12/06 - 01/18/07)4.4E-6/yr.
: 24-Day (11/18/06 - 12/12/06)1.2E-6/yr.Total Internal Events Result5.6E-6/yr.Sensitivity of EDG 2 Recovery:In the analysis presented above, it was assumed that EDG 2 could not be recovered intime to lower the risk of the relevant core damage sequences.
: This was because the failed voltage regulator could not be repaired or replaced quickly and operation of the emergency diesel generator in a manual voltage regulation mode was not a subject of operator training and not explicitly expressed in plant procedures.
: As a sensitivity to this assumption, a low bounding (highest allowance for recovery) case for operating the
: 3A3-9EDG in manual voltage regulation mode was considered using the
: SPAR-Hmethodology.
: The results of this analysis are presented in the table below.PerformanceShaping FactorDiagnosis (0.01)Action (0.001)Available TimeExtra Time (0.1)>5 Times Required (0.1)StressHigh (2)High (2)ProceduresIncomplete (20)Incomplete (20)Experience/TrainingLow (10)Low (3)Total10.2880.012Overall Total
: HRA0.31.
: This reflects the result using the formula for cases where 3 or more negative PSFs are present.The nominal time for performing the actions was small compared to the minimum timeof four hours available to restore power following failure of EDG 2 at four hours into the event.
: The Class 1E batteries are capable of supplying eight hours of power.
: For the core damage sequences that comprise most of the risk relative to this finding, it is assumed that EDG 1 fails initially and the Division 1 battery begins to deplete.
: Division dc is necessary for control of the RCIC system.
: EDG 2 is assumed to fail after four hours of run time, and therefore the Division 1 batteries have four hours of remaining capacity at this time.
: Therefore, extra credit for time available was applied for both diagnosis and action.
: High stress was assumed because the station would be in a blackout condition.
===Procedures===
for manual operation were not available, but credit for incomplete procedures was applied as a bounding assumption.
: Low training and experience was assumed because the plant staff had not performed this mode of operation and had not received training. The result of the
: SPAR-H analysis was a failure probability of 0.3.
: Although there aresome short-term sequences in the SPAR results, corresponding to the failure of dc-
powered high pressure injection sources, their contribution to core damage was less than two percent of the total risk.
: Therefore, for the purposes of this sensitivity assessment, as an adequate first-order approximation, the non-recovery probability of
: 0.3 was applied to every core damage sequence.
: The result is presented in the following table:Exposure Period37-Day (12/12/06 - 01/18/07)1.3E-6/yr.
: 24-Day (11/18/06 - 12/12/06)3.6E-7/yr.Total Internal Events Result1.7E-6/yr.
: 3A3-10External Events:The risk increase from fire initiating events was reviewed and determined to have asmall impact on the risk of the finding.
: Only two fire scenarios were identified where equipment damage could cause a LOOP to occur.
: One was a control room fire that affected either Vertical Board F or Board C.
: The second was a fire in the Division 2
critical switchgear.
: For the control room fires, the scenario probabilities are remote because of the confined specificity of their locations and the fact that a combination of hot shorts of a specific polarity are needed to cause a LOOP.
: In addition, recovery from
a LOOP induced in this manner would be likely to succeed because a minimum of four hours would be available (based on an 8-hour battery capacity and a four hour depletion of the Division 1 battery that provides power to the reactor core isolation cooling system)
prior to the EDG 2 failure, power would presumably be available in the switchyard, and the breaker manipulations needed to complete this task would be possible and within the capability of an augmented plant staff that would respond to the event.The other class of fires that would result in a LOOP were those that require anevacuation of the control room.
: In this case, plant procedures require offsite power to be isolated from the vital buses and the preferred source of power, the Division 2 EDG, is used to power the plant.
: With the assumption that the Division 2 EDG will fail four hours into the event, a station blackout would occur at this time.
: The sequences that could lead to core damage would include a failure of the Division 1 EDG, such that ultimate success in averting core damage would rely on recovery of either EDG or of offsite power.
: A review of the onsite electrical distribution system did not reveal any particular difficulties in restoring switchyard power to the vital buses in this scenario, especially given that at least four hours are available to accomplish this task.In general, the fire risk importance for this finding is small compared to that associatedwith internal events because onsite fires do not remove the availability of offsite power in the switchyard, whereas, in the internal events scenarios, long-term unavailability of offsite power is presumed to occur as a consequence of such events as severe weather or significant electrical grid failures.The CNS IPEEE Internal Fire Analysis screened the fire zones that had a significantimpact on overall plant risk.
: When adjusted for the exposure period of this finding, the cumulative baseline core damage frequency for the zones that had the potential for a control room evacuation (and a procedure-induced LOOP) or an induced plant centered
: LOOP was approximately 3.6E-7/yr.
: The methods used to screen these areas were not rigorous and used several bounding assumptions.
: The analyst qualitatively assumed that the increase in risk from having EDG 2 in a status where it is assumed to fail at four hours would likely be somewhat less than one order of magnitude above the baseline, or
: 3.6E-6/yr.
: This is easily demonstrated by an assumption that failure to re-connect offsite power within a period of at least four hours is well less than 10 percent.
: Based on these considerations, the analyst concluded that the risk related to fires would not be sufficiently large to change the risk characterization of this finding.The seismicity at CNS is low and would likely have a small impact on risk for an EDGissue.
: 3A3-11As a sensitivity, data from the RASP External Events Handbook was used to estimatethe scope of the seismic risk particular to this finding.
: The generic median earthquake acceleration assumed to cause a loss of offsite power is 0.3g.
: The estimated frequency of earthquakes at CNS of this magnitude or greater is 9.828E-5/yr.
: The generic median earthquake frequency assumed to cause a loss of the diesel generators is 3.1g, though essential equipment powered by the EDGs would likely fail at approximately 2.0g.
: The seismic information for CNS is capped at a magnitude of 1.0g with a frequency of
: 8.187E-6.
: This would suggest that an earthquake could be expected to occur with an approximate frequency of 9.0E-5/yr that would remove offsite power but not damage other equipment important to safe shutdown.
: In the internal events discussion above, it was estimated that LOOPs that exceeded four hours duration would occur with a frequency of 3.91E-3/yr.
: Most LOOP events that exceed the four hour duration would likely have recovery characteristics closely matching that from an earthquake. The ratio between these two frequencies is 43.
: Based on this, the analyst qualitatively concluded that the risk associated with seismic events would be small compared to the internal result.Flooding could be a concern because of the proximity to the Missouri River.
: However,floods that would remove offsite power would also likely flood the EDG compartments and therefore not result in a significant change to the risk associated with the finding.
: The switchyard elevation is below that of the power block by several feet, but it is not likely that a slight inundation of the switchyard would cause a loss of offsite power.
: The low frequency of floods within the thin slice of water elevations that would remove offsite power for at least four hours, but not render the diesel generators inoperable, indicatesthat external flooding would not add appreciably to the risk of this finding.Based on the above, the analyst determined that external events did not addsignificantly to the risk of the finding.Large Early Release Frequency:In accordance with Manual Chapter 0609, Appendix A, Attachment 1, Step 2.6,"Screening for the Potential Risk Contribution Due to LERF," the analyst reviewed the core damage sequences to determine an estimate of the change in large early release frequency caused by the finding. The LERF consequences of this performance deficiency were similar to thosedocumented in a previous SDP Phase 3 evaluation regarding a misalignment of gland seal water to the service water pumps.
: The final determination letter was issued on March 31, 2005 and is located in ADAMS, Accession No. ML050910127.
: The following excerpt from this document addressed the LERF issue:The NRC reevaluated the portions of the preliminary significance determination relatedto the change in LERF.
: In the regulatory conference, the licensee argued that the dominant sequences were not contributors to the LERF.
: Therefore, there was no change in LERF resulting from the subject performance deficiency.
: Their argument was
: 3A3-12based on the longer than usual core damage sequences, providing for additional time tocore damage, and the relatively short time estimated to evacuate the close in population surrounding Cooper Nuclear Station.LERF is defined in NRC Inspection Manual Chapter 0609, Appendix H, "ContainmentIntegrity Significance Determination Process" as: "the frequency of those accidents leading to significant, unmitigated release from containment in a time frame prior to the effective evacuation of the close-in population such that there is a potential for early health effect."
: The NRC noted that the dominant core damage sequences documented in the preliminary significance determination were long sequences that took greater than hours to proceed to reactor pressure vessel breach.
: The shortest calculated interval from the time reactor conditions would have met the requirements for entry into a general emergency (requiring the evacuation) until the time of postulated containment rupture was 3.5 hours.
: The licensee stated that the average evacuation time for CNS, from the declaration of a General Emergency was 62 minutes.The NRC determined that, based on a 62-minute average evacuation time, effectiveevacuation of the close-in population could be achieved within 3.5 hours.
: Therefore, the dominant core damage sequences affected by the subject performance deficiency were not LERF contributors.
: As such, the NRC's best estimate determination of the change in LERF resulting from the performance deficiency was zero.In the current analysis, the total contribution of the 30-minute sequences to the currentcase CDF is only 0.17% of the total.
: For two hour sequences, the contribution is only
: 0.04 percent.
: That is, almost all of the risk associated with this performance deficiency involves sequences of duration four hours or longer following the loss of all ac power.
: Based on the average 62 minute evacuation time as documented above, the analyst determined that large early release did not contribute to the significance of the current finding.References:GE-NE-E1200141-04R2, Table 5-1, Shutdown Power at Cooper Nuclear Station(proprietary)Green Screen Source Data, External Events PRA model, Nine Mile Point, Unit 1
: NUREG/CR-6890, "Reevaluation of Station Blackout Risk at Nuclear Power Plants, Analysis of Loss of Offsite Power Events: 1986-2004"
}}
}}

Revision as of 02:54, 22 October 2018

Acceptance of White Finding and Notice of Violation - NRC Special IR 05000298-07-007 - Cooper Nuclear Station
ML072970900
Person / Time
Site: Cooper Entergy icon.png
Issue date: 10/23/2007
From: Hay M C
NRC/RGN-IV/DRP/RPB-C
To: Minahan S B
Nebraska Public Power District (NPPD)
References
EA-07-090 IR-07-007
Download: ML072970900 (3)
v • d • e


Text

October 23, 2007

EA 07-090Stewart B. Minahan, Vice President-Nuclear and CNO Nebraska Public Power District 72676 648A Avenue Brownville, NE 68321

SUBJECT: ACCEPTANCE OF WHITE FINDING AND NOTICE OF VIOLATION - NRCSPECIAL INSPECTION REPORT 05000298/2007007 - COOPER NUCLEAR STATION

Dear Mr. Minahan:

Thank you for your letter dated October 15, 2007, in response to our August 17, 2007, letterand Notice of Violation concerning failing to promptly identify and correct, to preclude recurrence, of a significant condition adverse to quality regarding a defective voltage regulator circuit board used in Emergency Diesel Generator 2. The significance of this finding was determined to be White, a finding with low to moderate safety significance. We have reviewed your reply and find it responsive to the concerns raised in our Notice of Violation. Based on the guidance in Inspection Manual Chapter (IMC) 0305, the NRC staff will perform a95001, "Supplemental Inspection for One or Two White Inputs in a Strategic Performance Area." This inspection is scheduled for the week of December 3, 2007. We will review the implementation of your corrective actions during this inspection to determine that full compliance has been achieved and will be maintained.

Sincerely,/RA/Michael C. Hay, Chief, Reactor Projects Branch CDocket: 50-298License: DPR-46 Nebraska Public Power District- 2 -cc w/

Enclosures:

Gene Mace Nuclear Asset Manager Nebraska Public Power District P.O. Box 98 Brownville, NE 68321John C. McClure, Vice President and General Counsel Nebraska Public Power District P.O. Box 499 Columbus, NE 68602-0499David Van Der Kamp Licensing Manager Nebraska Public Power District P.O. Box 98 Brownville, NE 68321Michael J. Linder, DirectorNebraska Department of Environmental Quality P.O. Box 98922 Lincoln, NE 68509-8922ChairmanNemaha County Board of Commissioners Nemaha County Courthouse 1824 N Street Auburn, NE 68305Julia Schmitt, ManagerRadiation Control Program Nebraska Health & Human Services Dept. of Regulation & Licensing Division of Public Health Assurance 301 Centennial Mall, South P.O. Box 95007 Lincoln, NE 68509-5007H. Floyd GilzowDeputy Director for Policy Missouri Department of Natural Resources P. O. Box 176 Jefferson City, MO 65102-0176Director, Missouri State Emergency Management Agency P.O. Box 116 Jefferson City, MO 65102-0116Chief, Radiation and Asbestos Control Section Kansas Department of Health and Environment Bureau of Air and Radiation 1000 SW Jackson, Suite 310 Topeka, KS 66612-1366Melanie Rasmussen, State LiaisonOfficer/Radiation Control Program Director Bureau of Radiological Health Iowa Department of Public Health Lucas State Office Building, 5th Floor 321 East 12th Street Des Moines, IA 50319John F. McCann, Director, LicensingEntergy Nuclear Northeast Entergy Nuclear Operations, Inc.

440 Hamilton Avenue White Plains, NY 10601-1813Keith G. Henke, PlannerDivision of Community and Public Health Office of Emergency Coordination 930 Wildwood, P.O. Box 570 Jefferson City, MO 65102Paul V. Fleming, Director of Nuclear Safety Assurance Nebraska Public Power District P.O. Box 98 Brownville, NE 68321 Nebraska Public Power District- 3 -Electronic distribution by RIV:Regional Administrator (EEC)Deputy Regional Administrator (TPG)DRP Director (ATH)DRP Deputy Director (AXV)DRS Director (DDC)DRS Deputy Director (RJC1)Senior Resident Inspector (NHT)Branch Chief, DRP/C (MCH2)Senior Project Engineer, DRP/C (WCW)Branch Chief, EB1, DRS (WBJ)Team Leader, DRP/TSS (CJP)RITS Coordinator (MSH3)State Liaison Officer (WAM)Public Affairs Officer (VLD)K.S. Fuller, RC/ACES (KSF)Enforcement Specialist (GMV)C.A. Carpenter, D:OE (CAC)OE EA File (RidsOeMailCenter)SUNSI Review Completed: MCH ADAMS: Yes G No Initials: MCH Publicly Available G Non-Publicly Available G Sensitive Non-SensitiveR:\_REACTORS\CNS\2007\CNS 2007-07_ThankYouLtr.wpd SPE:DRP/CC:DRP/CWCWalkerMCHay /RA/ /RA/10/23/0710/23/07OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax

Retrieved from "https://kanterella.com/w/index.php?title=IR_05000298/2007007&oldid=1122070"