IR 05000261/2011009

IR 05000261-11-009; 3/21/2011 - 3/25/2011; H.B. Robinson Steam Electric Plant, Unit 2; Supplemental Inspection for a White Performance Indicator (PI)

ML111090365
Person / Time
Site:	Robinson
Issue date:	04/19/2011
From:	Randy Musser NRC/RGN-II/DRP/RPB4
To:	Duncan R Carolina Power & Light Co
References
IR-11-009
Download: ML111090365 (19)

Text

April 19, 2011

SUBJECT:

H.B. ROBINSON STEAM ELECTRIC PLANT-NRC SUPPLEMTENTAL INSPECTION REPORT 05000261/2011009

Dear Mr. Duncan:

On March 25, 2011, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection pursuant to Inspection Procedure 95001, Inspection for One or Two White Inputs in a Strategic Performance Area, at your H.B. Robinson Steam Electric Plant. The enclosed inspection report documents the inspection results, which were discussed at the exit meeting on March 25, 2011, with you and other members of your staff.

As required by the NRC Reactor Oversight Process Action Matrix, this supplemental inspection was performed to examine the causes for and actions taken related to the Unplanned Scrams per 7000 Critical Hours performance indicator crossing the threshold from Green (very-low risk significance) to White (low-to-moderate risk significance) in the third quarter of 2010. This issue was documented previously in Inspection Report 2010004, Follow-up Assessment Letter, dated November 12, 2010. The NRC was informed on March 11, 2011, of your staffs readiness for this inspection.

The objectives of this supplemental inspection were to provide assurance that: (1) the root and contributing causes were understood; (2) the extent of condition and extent of cause were identified; and (3) corrective actions were sufficient to address the root and contributing causes and to preclude repetition. The inspection consisted of examination of activities conducted under your license as they related to safety, compliance with the commissions rules and regulations, and the conditions of your operating license.

Based on the results of this inspection, no findings of significance were identified. The inspectors determined that, in general: (1) the root and contributing causes were understood; (2)

the extent of condition and extent of cause were identified; and (3) corrective actions were sufficient to address the root and contributing causes and to preclude repetition.

CP&L

In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Randall A. Musser, Chief

Reactor Projects Branch 4

Division of Reactor Projects

Docket Nos. 50-261 License Nos. DPR-23

Enclosure:

Inspection Report 05000261/2011009

w/ Attachment: Supplemental Information

REGION II==

Docket No.:

50-261

License No.:

DPR-23

Report No:

05000261/2011009

Licensee:

Carolina Power & Light Company (CP&L)

Facility:

H. B. Robinson Steam Electric Plant, Unit 2

Location:

3581 West Entrance Road Hartsville, SC 29550

Dates:

March 21, 2011 through March 25, 2011

Inspectors:

P. Lessard, Resident Inspector, Shearon Harris Nuclear Plant (Lead)

J. Worosilo, Project Engineer

Approved by:

Randall A. Musser, Chief Reactor Projects Branch 4 Division of Reactor Projects

Enclosure

SUMMARY OF FINDINGS

Inspection Report (IR) 05000261/2011009; 3/21/2011 - 3/25/2011; H.B. Robinson Steam

Electric Plant, Unit 2; Supplemental Inspection for a White Performance Indicator (PI)

This inspection was conducted by a resident inspector and a project engineer. No findings were identified.

Cornerstone: Initiating Events

This supplemental inspection was performed in accordance with Inspection Procedure (IP)95001, Inspection for One or Two White Inputs in a Strategic Performance Area, to assess the licensees evaluation associated with a White Performance Indicator (PI) in the Initiating Events cornerstone associated with greater than three reactor trips in 7000 critical hours.

The Unplanned Scrams (Reactor Trips) per 7000 Critical Hours performance indicator crossed the threshold from Green (very-low risk significance) to White (low-to-moderate risk significance) in the third quarter of 2010. Specifically, the licensee experienced unplanned reactor trips on the following dates: November 6, 2009, March 28, 2010, September 9, 2010, and October 7, 2010.

The inspectors determined that the licensees problem identification, root cause, extent of condition evaluation, extent of cause evaluation, and the corrective actions for the four unplanned reactor trips were generally adequate and properly prioritized.

Given the licensees adequate evaluation addressing the White PI, it will no longer be considered an input in assessing plant performance since the PI for trips in 7000 critical hours has reverted to Green and the 95001 Inspection has been completed successfully in accordance with the guidance in Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program. The implementation and effectiveness of the licensees corrective actions will be reviewed during future inspections.

NRC-Identified and Self-Revealing Findings

No findings were identified.

Licensee-Identified Violations

No findings were identified.

REPORT DETAILS

4OA4 Supplemental Inspection

.1 Inspection Scope

The supplemental inspection was performed in accordance with IP 95001 to assess the licensees evaluation of a White PI which affected the Initiating Events cornerstone objective in the Reactor Safety strategic performance area. The White PI is associated with having greater than three reactor trips in 7000 critical hours. The inspection objectives were to:

• Provide assurance that the root and contributing causes were understood;

• Provide assurance that the extent of condition and extent of cause were identified; and

• Provide assurance that the licensees corrective actions were sufficient to address the root and contributing causes and to preclude repetition.

The licensee entered the Degraded Cornerstone Column of the NRCs Action Matrix based on the Unplanned Scrams per 7000 Critical Hours PI crossing the threshold from Green to White in the third quarter of 2010 and three inspection findings classified as having low safety significance (White). This inspection focused solely upon the White PI. The three White findings will be inspected in the 95002, Supplemental Inspection for One Degraded Cornerstone or Any Three White Inputs in a Strategic Performance Area scheduled to start on May 31, 2011. Additional information regarding these findings can be found in Inspection Reports 05000261/2010004, 2010006, 2010008, 2011013, and 2010014.

The licensee notified the NRC on March 11, 2011, that they were ready for this supplemental inspection. The four unplanned reactor trips reviewed were:

• November 6, 2009 - The Feedwater Regulating Valve (FRV) for A Steam Generator unexpectedly closed resulting in a rapidly lowering steam generator water level, which required a manual reactor trip. (Root Cause Evaluation (RCE)364853)

• March 28, 2010 - A cable failure to the supply breaker on 4kV Bus 5 caused an arc flash and a reactor trip on Reactor Coolant System (RCS) loop low flow from the B reactor coolant pump (RCP). (RCE 390095)

• September 9, 2010 - An automatic reactor trip occurred due to an over temperature / delta temperature (OTT) signal resulting from the inadvertent closure of all four main turbine governor valves. (RCE 420936)

• October 7, 2010 - An automatic reactor trip occurred due to a low loop flow condition caused by an electrical fault and trip of the C RCP. (RCE 425433)

In addition to the RCEs for each reactor trip, the inspectors reviewed common cause evaluation (CCE) 424975. This investigation evaluated all four reactor trips to identify potential common causes, and identify additional actions needed to reduce reactor trips.

The inspectors reviewed the root and contributing causes as well as the corrective actions taken or planned for all four reactor trips. The inspectors also held discussions with licensee personnel to ensure that the root and contributing causes and the contribution of safety culture components were understood and corrective actions taken or planned were appropriate to address the causes and preclude repetition.

Of note, the scope of this inspection covered up to the point of the reactor trip. Aspects of these events that occurred after the reactor trip have been or will be covered in different inspections.

.2 Evaluation of the Inspection Requirements

2.01 Problem Identification

a. Determine that the evaluation identifies who (i.e. licensee, self revealing, or NRC), and under what conditions the issue was identified

The inspectors determined that the licensees evaluations of these unplanned reactor trips appropriately determined who and under what conditions the issue was identified.

All four reactor trips were classified as self revealing events.

b. Determine that the evaluation documents how long the issue existed, and prior opportunities for identification

The first unplanned reactor trip was a manual reactor trip due to a rapidly lowering steam generator water level after the FRV for A Steam Generator unexpectedly closed. The licensee identified that the FRV shut as a result of the failure of the power supply for the flow error signal summator for the A FRV control loop. With vendor assistance, the licensee was able to determine that a ceramic capacitor had failed within the power supply circuit board. The licensee concluded that this issue existed since the power supply was manufactured and first released in August 2008. Additionally, the licensee determined that there were no prior opportunities for identification because they have neither the tools nor resources necessary to identify this sort of issue through circuit board analysis.

The second unplanned reactor trip, an automatic reactor trip on RCS loop low flow from the B RCP, resulted from the failure of the supply cable to the 4kV Bus 5. This caused frequency and voltage to degrade on 4kV Bus 4, which supplies power to the B RCP.

The licensee determined that the faulted cable that initiated this event was installed during a plant modification that occurred in 1986. When reviewing the modification package, the licensee determined that the installed cable was different from the specification, was inappropriate for the application and was installed contrary to the manufacturers recommendations. Additionally, the licensee determined that there were no prior opportunities for identification because their preventative maintenance procedures did not include inspecting cables for the specific attributes that could have resulted in identification of this issue.

The third unplanned reactor trip, an automatic reactor trip due to an OTT signal, was determined to most likely be the result of a circuit board in the Electro Hydraulic Control (EHC) cabinet found to have a degraded connection with the backplane connector.

Specifically, two connection pins were discovered to be bent and spread apart. The licensee determined this condition may have existed since the last maintenance activity more than eight years prior to the reactor trip. The licensee determined that there were no prior opportunities for identification.

The fourth unplanned reactor trip, an automatic reactor trip due to a low loop flow condition was caused by an electrical trip of the C RCP. The pump tripped due to a significant fault in the motor stator. The failure mechanism was determined to be the result of a turn to turn short related to age and vibration. This turn to turn short ultimately revealed itself as a phase to phase short causing the reactor trip. Due to the nature of this fault however, it is not possible to determine how long the condition existed.

Approximately six months prior to the failure, testing was performed to determine the health of the motor including: a Megger test, Bridge test, Stepped Voltage (DC Hi-Potential) test, and Surge Comparison test. All tests were determined to be satisfactory; leading the licensee to conclude the pump was reliable. After the failure, the licensee referenced this testing and past maintenance schedules to support their conclusion that there were no prior opportunities for identification. During subsequent inspection, the NRC determined that the licensees decision to not perform recommended five year maintenance on the motor constituted a missed opportunity for identification of the motor damage that ultimately resulted in failure. This aspect and additional details are discussed in inspection finding 05000261/2010005-01 within NRC Inspection Report 05000261/2010005.

The inspectors determined that the licensee had identified, as appropriate and when able, how long the above discussed conditions existed and any prior opportunities for identification.

c. Determine that the evaluation documents the plant risk specific consequences (as applicable) and compliance concerns associated with the issue

The inspectors reviewed the licensees assessment of the plant-specific risk consequences of the unplanned reactor trips with a senior risk analyst in the region and headquarters and determined them to be adequate.

The first and third unplanned reactor trips were determined to have low risk consequences.

The second unplanned reactor trip was determined to have low to moderate risk significance for the plant. Additionally, this unplanned reactor trip was reviewed in detail by an NRC Augmented Inspection Team and documented in Inspection Report 05000261/2010009. Fourteen unresolved items were issued as part of this inspection.

Five green findings and two white violations were issued as a result of these unresolved items.

The fourth unplanned reactor trip was determined to have low to moderate risk significance for the plant. This unplanned reactor trip was reviewed in detail by an NRC Special Inspection and documented in Inspection Report 05000261/2010012. Two green findings were issued as a result of this inspection.

d. Findings

No findings were identified.

2.02 Root Cause, Extent of Condition, and Extent of Cause Evaluation

a. Determine that the problem was evaluated using a systematic method(s) to identify root cause(s) and contributing cause(s)

The inspectors noted that the licensee used combinations of different systematic methods to identify root and contributing causes for the four unplanned reactor trips; equipment performance analysis, support/refute methodology, cause and effect analysis, barrier analysis, and human performance analysis. Additionally, the inspectors determined that the methods used were adequate for the root cause evaluations.

For the first unplanned reactor trip, the licensee utilized equipment performance analysis and support/refute methodology to identify the root and contributing causes for the flow error signal summator for the A FRV control loop failure.

For the second unplanned reactor trip, the licensee utilized support/refute methodology, cause and effect analysis, barrier analysis, and human performance analysis to identify the root and contributing causes for the cable failure.

For the third unplanned reactor trip, the licensee utilized support/refute methodology to identify the most probable and contributing causes for the OTT signal.

For the fourth unplanned reactor trip, the licensee utilized support/refute methodology to identify the most probable and contributing causes for the C RCP motor failure.

b. Determine that the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem

For the four unplanned reactor trips, the inspectors determined that the root cause evaluations were of sufficient detail to support the identified root, most probable and contributing causes and commensurate with the significance of the problem.

For the first unplanned reactor trip, the licensee determined that the root cause of the flow error signal summator failure for the A FRV control loop was the result of the vendors manufacturing process resulting in premature failure of the power supply. The licensee determined that a ceramic capacitor in the summators power supply module had failed, causing a short circuit which subsequently blew the power fuse. The vendor determined that the solid-type ceramic capacitors were susceptible to damage from board flexing that occurred during the manufacturing process, resulting in premature failure.

For the second unplanned reactor trip, the inspectors observed that the licensees root cause evaluation problem statement focused upon the plant damage and operator challenges which occurred after the reactor tripped. As a result, the licensee determined that there were two root causes for the event. The first was that the failure of 4kV Bus 5 to isolate from 4kV Bus 4 resulted from an inadequate impact review of a work request that identified that the breaker connecting those buses lacked control power indication.

The second was that the operator challenges resulted from poor fundamentals and the failure of the organization to correct previously identified weaknesses. During the performance of this inspection, the inspectors determined that the reactor would have tripped even if 4kV Bus 5 properly isolated itself from 4kV Bus 4. As a result, the inspectors concluded that the root cause of the event was the improper installation of the incorrect 4kV cable during a modification that occurred in 1986. Although the licensee did not identify this as the root cause of the event, they did identify it as one of the contributing causes. As a result, the corrective actions taken by the licensee were determined by the inspectors to be adequate to preclude recurrence.

For the third unplanned reactor trip, the licensee could not determine a root cause due to the nature of the event. However, the licensee identified the most probable cause and several contributing causes. The most probable cause was that a circuit board in the EHC cabinet was found with a degraded connection with the backplane connector in the cabinet. Specifically, two connection pins were discovered to be bent and spread apart resulting in the potential for an intermittently degraded signal.

The licensee could not determine a root cause for the fourth unplanned reactor trip.

However, the licensee identified the most probable cause and a contributing cause. The licensee determined that the most probable cause of the C RCP failure was the synergistic effect of excessive vibration due to inadequate end winding bracing of the motor stator combined with normal aging and operating conditions which caused the turn-to-turn insulation to fail.

c. Determine that the root cause evaluation included a consideration of prior occurrences of the problem and knowledge of prior operating experience

The inspectors determined that each of the root cause evaluations for the four unplanned reactor trips had adequately considered prior occurrences of the problem and knowledge of prior operating experience.

d. Determine that the root cause evaluation addressed the extent of condition and the extent of cause of the problem

The inspectors determined that the evaluations for each unplanned reactor trip adequately addressed extent of condition and extent of cause. The inspectors also noted that the licensee implemented corrective actions to address issues identified by the extent of condition or extent of cause analyses.

In addition, the inspectors reviewed the licensees common cause evaluation which was performed to evaluate commonalities between the four unplanned reactor trips. Within this evaluation, the licensee conducted another extent of condition and extent of cause considering all of these events in aggregate. For the common causes identified in this evaluation, the licensee developed corrective actions and improvement plans.

e. Determine that the root cause evaluation, extent of condition, and extent of cause appropriately considered the safety culture components as described in IMC 0305

The inspectors determined that the safety culture components were appropriately considered and reviewed for all four unplanned reactor trips.

f. Findings

No findings were identified.

2.03 Corrective Actions

a. Determine that appropriate corrective actions are specified for each root/contributing cause or that there is an evaluation that no actions are necessary

The inspectors determined that appropriate corrective actions were established to address each of the root and contributing causes for all four unplanned reactor trips evaluations.

For the first unplanned reactor trip, the licensee replaced all potentially affected power supply modules in critical applications (where a failure could result in a plant transient or loss of function). In addition, all the potentially effected power supply modules that were in-stock were returned to the vendor.

For the second unplanned reactor trip, the licensee replaced all of the 4kV cables that were installed at the plant as part of the 1986 modification. In addition the licensee replaced conduit and other equipment damaged as a result of the cable failure. The control power fuse in the breaker which failed to isolate 4kV Bus 4 from 4kV Bus 5 was replaced and the breaker was refurbished prior to being returned to service.

For the third unplanned reactor trip, the licensee replaced the faulted circuit board and revised the existing preventive maintenance program to require testing to validate proper circuit card seating of any EH system circuit cards that are replaced. This revision also included checking circuit boards for bent pins prior to installation as well as verifying all installed and surrounding circuit boards are seated properly.

For the fourth unplanned reactor trip, the licensee replaced the faulted RCP motor with the spare motor and scheduled to rewind the faulted motor using a design that provides for proper securing of all winding end turns to reduce vibration and improve long-term reliability. The spare motor that was installed in the plant is scheduled to be rewound with the updated design in 2012. In addition, the licensee implemented a preventive maintenance task to rewind each of the RCP motors on a 20 year frequency.

b. Determine that the corrective actions have been prioritized with consideration of the risk significance and regulatory compliance

The inspectors determined that the corrective actions for the events were appropriately prioritized relative to their risk significance and regulatory compliance.

c. Determine that a schedule has been established for implementing and completing the corrective actions

The inspectors determined that the corrective actions for the events have been completed or reasonably scheduled.

d. Determine that quantitative or qualitative measures of success have been developed for determining the effectiveness of the corrective actions to prevent recurrence

The inspectors determined that effectiveness reviews had been completed or were scheduled for the causes of the four unplanned reactor trips. Additionally, the inspectors determined that each effectiveness review had quantitative or qualitative criteria established to measure success.

e. Determine that the corrective actions planned or taken adequately address a Notice of Violation (NOV) that was the basis for the supplemental inspection, if applicable

The NRC did not issue an NOV to the licensee: therefore this inspection requirement is not applicable.

f. Findings

No findings were identified.

4OA6 Meetings, Including Exit

Exit Meeting Summary

On March 25, 2011, the inspectors presented the inspection results to Mr. Duncan and other members of the staff. The inspectors confirmed that no proprietary information was obtained during the course of the inspection.

ATTACHMENT: SUPPPLEMENTAL INFORMATION

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee personnel

J. Caves, Acting Supervisor-Licensing & Regulatory Programs

T. Cosgrove, Plant General Manager

H. Curry, Training Manager

R. Duncan, Site Vice President

G. Kilpatrick, Operations Manager

A. Pope, Brunswick Nuclear Plant, Supervisor-Licensing & Regulatory Programs

G. Sanders, Licensing & Regulatory Programs, Senior Engineer

T. Zimmerman, Corporate Nuclear Operations, Lead Engineer

NRC personnel

J. Hickey, Senior Resident Inspector - Robinson

R. Musser, Chief, Reactor Projects Branch 4, Division of Reactor Projects Region II

LIST OF ITEMS

OPENED, CLOSED AND DISCUSSED

None