IR 05000255/2012011

Supplemental IR 05000255-12-011 and Assessment Follow-Up Letter, on 09/17 Trhough 09/28/12, Palisades Nuclear Plant

ML12314A304
Person / Time
Site:	Palisades
Issue date:	11/09/2012
From:	Stephanie West Division Reactor Projects III
To:	Vitale A Entergy Nuclear Operations
References
EA-11-241, EA-11-243 IR-12-011
Download: ML12314A304 (46)

Text

ber 9, 2012

SUBJECT:

PALISADES NUCLEAR PLANT - NRC SUPPLEMENTAL INSPECTION REPORT 05000255/2012011; AND ASSESSMENT FOLLOW-UP LETTER

Dear Mr. Vitale:

On September 28, 2012, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection pursuant to Inspection Procedure 95002, "Inspection for One Degraded Cornerstone or Any Three White Performance Inputs in a Strategic Performance Area," at your Palisades Nuclear Plant. The enclosed report documents the results of this inspection, which were discussed on September 28, 2012, with you and other members of your staff. In accordance with the NRC Reactor Oversight Process Action Matrix, this supplemental inspection was performed to follow up on a Yellow finding with substantial safety significance which occurred on September 25, 2011. This issue, a failure to have adequate work instructions for work performed on panel ED-11-2, was previously documented and assessed in NRC Inspection Report 05000255/2011014.

This supplemental inspection was also performed to follow up on a White finding with low-to-moderate safety significance which occurred on August 9, 2011. This issue, a failure to prevent recurrence of a significant

condition adverse to quality, was previously documented and assessed in NRC Inspection Report 05000255/2011016. The NRC was informed on July 17, 2012, of your staff's readiness for this inspection.

The objectives of this supplemental inspection were to provide assurance that: (1) the root causes and the contributing causes for the risk-significant issues were understood; (2) the extent of condition and extent of cause of the issues were identified; and (3) corrective actions were or will be sufficient to address and preclude repetition of the root and contributing causes.

This inspection also included an independent NRC review of the extent of condition and extent of cause for the Yellow and White findings and an assessment of whether any safety culture component caused or significantly contributed to the performance issue. The NRC determined that the staff at Palisades Nuclear Plant performed an acceptable evaluation of the Yellow finding. The evaluation identified the primary root cause of the issue to be an organizational issue where senior Entergy management had not established a sufficiently sensitive culture of risk recognition and management, which resulted in the plant's managers, supervisors, and workers not recognizing, accounting for, or preparing for the industrial safety risk and plant operation nuclear risk involved with the panel ED-11-2 breaker inspection and replacement maintenance. To correct this issue and prevent recurrence, the Palisades Nuclear Plant implemented corrective actions including reinforcing Entergy standards for procedure compliance, accountability, and unacceptable behavior via face to face communications from senior managers to individual contributor levels as well as implementing, and ensuring compliance with, Entergy risk management procedures.

The NRC determined that the staff at Palisades Nuclear Plant performed an acceptable evaluation of the White finding. The evaluation identified the primary root cause of the failure of the service water pump P-7C coupling was due to Intergranular Stress Corrosion Cracking and was a design failure, because Palisades Nuclear Plant failed to specify the correct material and to recognize that the pump coupling procurement specification did not ensure all critical material testing requirements for use in the service water operating environment. To correct this issue and prevent recurrence, the Palisades Nuclear Plant created a limited distribution Engineering Standard for Palisades that clearly identified station requirements and expectations for material changes affecting installed plant equipment, as well as replaced the 416 Stainless Steel (SS)

service water line shaft couplings with 17-4PH SS couplings.

After reviewing Palisades Nuclear Plant's performance in addressing the Yellow and White findings using Inspection Procedure 95002, "Inspection for One Degraded Cornerstone or Any

Three White Performance Inputs in a Strategic Performance Area," the NRC concluded your actions met the inspection objectives with no significant weaknesses. Therefore, in accordance with the guidance in Inspection Manual Chapter 0305, "Operating Reactor Assessment Program," the Yellow and White findings will only be considered in assessing plant performance for a total of four quarters, ending the third quarter of 2012. As a result, the NRC determined the performance at Palisades Nuclear Plant to be in the Licensee Response Column of the Reactor Oversight Process Action Matrix as of October 1, 2012.

Although the NRC has transitioned the Palisades plant to the Licensee Response Column (Column I), the NRC has determined that addition al inspection is warranted and deviation from the Reactor Oversight Process is appropriate (ML12306A367). The additional inspections will focus on two areas. The first area for inspection is related to follow-up on your actions to address the finding and observations in this report. Although the NRC did not find any significant weaknesses in the areas inspected, some of your corrective actions to improve the organization and strengthen the safety culture at the site have not been fully implemented to date. As noted in the report, we found the safety culture adequate to support safe operations, but the NRC will inspect future site activities to ensure that you are implementing appropriate corrective actions to improve the organization and strengthen the safety culture on site, as well as assessing the sustainability of these actions. Part of this inspection will focus on items which are currently Substantive Cross Cutting Issues in the Human Performance Areas of oversight and conservative assumptions. The second area additional inspection is needed to review several ongoing technical issues at the site that need follow-up, which include portions of Primary Coolant Pump 'P-50C' impeller being susceptible to tearing under certain operational conditions (Inspection Report 05000255/2012003) and through wall leaks in three areas: Control Rod Drive Mechanism, Safety Injection Refueling Water Tank, and Service Water.

Although these issues, thus far, do not appear to have other than very low safety significance, it is imperative that the causes of these issues, and your planned corrective actions, are understood to provide reasonable assurance that these issues will not lead to more significant safety concerns.

Based on the results of this inspection, one NRC-identified finding of very low safety significance was identified which also involved a violation of NRC requirements. However, because of the very low safety significance, and because the issue was entered into your corrective action program, the NRC is treating the issue as a non-cited violation (NCV) in accordance with Section 2.3.2 of the NRC Enforcement Policy.

If you contest the subject or severity of this NCV, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with a copy to the Regional Administrator, U.S. Nuclear Regulatory Commission - Region III, 2443 Warrenville Road, Suite 210, Lisle, IL 60532-4352; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001; and the Resident Inspector Office at the Palisades Nuclear Plant. In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records System (PARS) component of NRC's Agencywide Document Access and Management System (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA by Kenneth O'Brien For/

Steven West, Director

Division of Reactor Projects Docket No. 50-255 License No. DPR-20

Enclosure:

Inspection Report 05000255/2012011 w/Attachment: Supplemental Information

REGION III Docket No: 50-255 License No: DPR-20 Report No: 05000255/2012011 Licensee: Entergy Nuclear Operations, Inc. Facility: Palisades Nuclear Plant Location: Covert, MI Dates: September 17 through 28, 2012 Inspectors: Robert Orlikowski, Project Engineer (Team Lead) Tom Bilik, Senior Reactor Inspector Brian Cushman, Resident Inspector Diana Betancourt-Roldan, Reactor Engineer Jasmine Gilliam, Reactor Engineer Molly Keefe, Human Factors Specialist Kamishan Martin, Human Factors Specialist Stephanie Morrow, Human Factors Analyst (observer) Gregory Hansen, Physical Security Inspector (observer)

Approved by: Steven West, Director Division of Reactor Projects

Enclosure

SUMMARY OF FINDINGS

Inspection Report (IR) 05000255/2012011, 09/17/2012 - 09/28/2012; Palisades Nuclear Plant; Supplemental Inspection 95002, Supplemental Inspection for One Degraded Cornerstone or any Three White Inputs in a Strategic Performance Area. This report documents a supplemental inspection by NRC inspectors. The inspectors identified one Green finding. The finding was considered a Non-Cited Violation (NCV) of NRC regulations. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609, "Significance Determination Process" (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRC's pr ogram for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 4, dated December 2006. The NRC performed this supplemental inspection to assess the licensee's evaluation associated with the loss of left train of direct current (DC) power due to the failure to ensure that the work instructions on safety-related 125-Volt DC Distribution Panel ED-11-2 through Work Orders (WO) 291194-01, 291210-01, and 291123-03, all activities that affected quality, were adequate for the scheduled work; and the failure to ensure the work instructions were followed by licensee staff for the affected activity. This performance issue was previously characterized as having substantial safety significance (Yellow) in NRC Inspection Report 05000305/2011019. During this supplemental inspection, performed in accordance with Inspection Procedure 95002, "Supplemental Inspection for One Degraded Cornerstone or Any Three White Inputs in a

Strategic Performance Area," the inspectors determined that the root cause evaluation appeared adequate, and the evaluation appropriately evaluated the root and contributing causes, addressed the extent of condition/cause, assessed safety culture, and established corrective actions for risk significant performance issues that were sufficient to address the causes and prevent recurrence. The root causes identified by the licensee was that senior leaders had not established a sufficiently sensitive culture of risk recognition and management, which resulted in the plant's managers, supervisors and workers not recognizing, accounting for, or preparing for the industrial safety risk and plant operational nuclear risk, involved with the panel ED-11-2 breaker inspection and replacement maintenance.

In addition, the inspectors assessed the licensee's evaluation associated with the licensee's failure to prevent recurrence of a significant condition adverse to quality when, on August 9, 2011, coupling #6 on service water pump P-7C failed due to intergranular stress corrosion cracking (IGSCC). This performance issue was previously characterized as having low to moderate safety significance (White) in NRC Inspection Report 05000305/2011020. During this supplemental inspection, the inspectors determined that the root cause evaluation appeared thorough, and the evaluation appropriately evaluated the root and contributing causes, addressed the extent of condition/cause, assessed safety culture, and established corrective actions for risk significant performance issues that were sufficient to address the causes and prevent recurrence. The root causes identified by the licensee were: (1) The 2009 and 2011 line shaft coupling failures were due to IGSCC. The coupling material was a quenched and tempered 416 martensitic Stainless Steel (SS) with low toughness properties that made it particularly susceptible to IGSCC when subjected to a tensile stress and a corrosive environment (due to the presence of chlorides).

(2) Palisade's Engineering specified the wrong SS alloy for use in Palisades Service Water operating environment. The choice of 416 SS was based on historical data by personnel who did not have sufficient metallurgic knowledge. Given the licensee's acceptable performance in addressing the loss of left train of DC power, and the lapse of four calendar quarters since the issue was identified, the Yellow inspection finding associated with it will no longer be considered in the assessment process after the 3rd quarter of 2012. Also, given the licensee's acceptable performance in addressing the service water pump P-7C coupling failure, and the lapse of four calendar quarters since the issue was identified, the White inspection finding associated with it will no longer be considered in the assessment process after the 3rd quarter of 2012.

A. NRC-Identified

and Self-Revealed Findings

Cornerstone: Initiating Events

Green.

A finding of very low safety significance and associated non-cited violation of 10 CFR Part 50, Appendix B, Criterion II, "Quality Assurance Program," was identified by the inspectors for the failure to complete the Previous Occurrence Evaluation as required by Palisade's procedure EN-LI-118, "Root Cause Evaluation Process."

Palisades Nuclear Plant Procedure EN-LI-118 requires that a Previous Occurrence

Evaluation be performed "to determine whether the same or similar Conditions have occurred, either at your site or within the industry, and if so, why associated corrective actions for Conditions having the same causes were unsuccessful in preventing occurrence of this Condition." The inspectors identified two examples of Root Cause Evaluations that did not include a complete Previous Occurrence Evaluation as required by EN-LI-118. This issue was entered into the licensee's corrective action program as CR-PLP-2012-06419 for further evaluation. The inspectors determined the finding was more than minor because the finding, if left uncorrected, would become a more significant safety concern. Specifically, the failure to complete the Previous Occurrence Evaluation as required by station procedure EN-LI-118 could result in the station not identifying repeat events and previous corrective actions that may have been inadequate. This could result in those previous inadequate corrective actions being implemented for the current deficiency resulting in future failures. Because the inspectors identified two examples of Palisades' failure to perform the Previous Occurrence Evaluation, the inspectors determined that this was a programmatic issue and therefore more than minor. The inspectors determined the finding could be evaluated using the SDP in accordance with IMC 0609, "Significance Determination Process," Attachment 0609.04, "Initial Characterization of Findings," and

IMC 0609, Appendix A, "The Significance Determination Process for Findings at-Power," for the Initiating Events and Mitigating Systems Cornerstone. The most important impacts are in the Initiating Event cornerstone, but the inspectors reviewed screening questions for both to be conservative. The inspectors answered all of the Initiating Events and Mitigating Systems screening questions as "no," therefore the finding screened as very low safety significance or

Green.

The inspectors reviewed IMC 0310, "Components Within Cross-Cutting Areas," and determined that none of the components reflected the performance characteristics that contributed to the finding. Therefore, the inspectors did not identify a cross-cutting aspect associated with this finding.

(Section 4OA4.02.01.d(1))

B. Licensee-Identified Violations

No violations were identified.

REPORT DETAILS

4OA4 Supplemental Inspection

.01 Inspection Scope

This inspection was conducted in accordance with Inspection Procedure (IP) 95002, "Supplemental Inspection for One Degraded Cornerstone or Any Three White Inputs in a Strategic Performance Area," to assess the licensee's evaluation of one White inspection finding and one Yellow inspection finding in the Mitigating Systems Cornerstone. The inspection objectives were to:

• Provide assurance that the root causes and contributing causes of risk significant performance issues are understood for individual and collective risk significant performance issues;

• Independently assess and provide assurance that the extent of condition and the extent of cause for individual and collective risk significant performance issues are identified;

• Independently determine if safety culture components caused or significantly contributed to the individual or collective risk significant performance issues; and

• Provide assurance that the licensee's corrective actions to risk significant performance issues are sufficient to address the root causes and contributing causes, and to prevent recurrence. Palisades Nuclear Plant entered the Degr aded Cornerstone column of NRC's Action Matrix in the fourth quarter of 2011 due to a Yellow finding in the Initiating Events Cornerstone as documented in Inspection Report 05000255/2011019 and 05000255/2011020. The finding was associated with the loss of left train of direct current (DC) power on September 25, 2011, due to the failure to ensure that the work instructions on safety-related 125-Volt DC Distribution Panel ED-11-2 through work order (WO) 291194-01, 291210-01, and 291123-03, all activities that affected quality, were adequate for the scheduled work; and the failure to ensure the work instructions were followed by licensee staff for the affected activity. The finding was characterized as being of substantial safety significance (Yellow) based on the results of a Phase 3 risk analysis performed by a region-based senior reactor analyst (SRA), as discussed in NRC Inspection report (IR) 05000255/2011014 and finalized in NRC IR 05000255/2011019 and 05000255/2011020. As part of the corrective actions the licensee repaired the damage to Panel ED-11-2 to restore it to service and addressed the operability and effect of the transient on other components.

Palisades also had an additional White finding from the fourth quarter of 2011 in the Initiating Events Cornerstone that was associated with the failure of a coupling on service water (SW) pump 'P-7C' on August 9, 2011, during routine operation. The failure was determined to be due to IGSCC. The finding was characterized as being of 5 Enclosure low to moderate safety significance (White) based on the results of a Phase 3 risk analysis performed by a region-based SRA, as discussed in NRC IR 05000255/2011016

and finalized in NRC IR 05000255/2011019 and 05000255/2011020. The failure was determined to be a repeat of a 2009 event in which the same pump failed due to the same cause. As part of their corrective actions the licensee replaced the couplings in all three SW pumps with couplings of a new material 17-4PH SS, which is not susceptible to IGSCC. By letter dated July 17, 2012, the licensee notified the NRC that it had completed its evaluation of the circumstances surrounding the degraded performance and was ready for the NRC to assess the licensee's evaluation and subsequent corrective actions. The inspectors reviewed the licensee's Root Cause Evaluations (RCEs) in addition to other evaluations conducted in support and as a result of the RCEs. The inspectors reviewed corrective actions that were taken or planned to address the identified causes. The inspectors also held discussions with licensee personnel to ensure that the root and contributing causes were understood and corrective actions taken or planned were appropriate to address the causes and preclude repetition. The inspectors also independently assessed the extent of condition and extent of cause of the identified issues. In addition, the inspectors performed an assessment of whether any safety culture components caused or significantly contributed to the issues.

.02 Evaluation of Inspection Requirements

02.01 Problem Identification a. Determine whether the licensee's root cause evaluation specified who identified the issue and under what conditions the issue was identified.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The licensee's RCE concluded that the event was self-revealed on September 25, 2011, when, during work on the energized panel ED-11-2, a positive horizontal bus bar rotated and contacted a negative bus bar. This caused an electrical fault in panel ED-11-2 and a complete loss of the left train 125-Volt DC safety-related system coincident with both 120 Volt preferred alternating current (AC) power sources, busses Y-10 and Y-30. The licensee's RCE determined that the loss of left train of DC power was the result of the failure to ensure that the work instructions on safety-related 125-Volt DC Distribution Panel ED-11-2 through WO 291194-01, 291210-01, and 291123-03, all activities that affected quality, were adequate for the scheduled work; and the failure to ensure the work instructions were followed by licensee staff for the affected activity. The RCE was completed under condition report CR-PLP-2011-04822 and the corrective actions taken to address the failure are documented under the same condition report. The inspectors determined that the RCE adequately discussed the identification of the issue. This was accomplished by reviewing the information contained in the condition reports, as well as written statements and interviews from those individuals involved in the event.

6 Enclosure

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The licensee's RCE concluded that the event was self-revealed on August 9, 2011, when SW pump 'P-7C' unexpectedly failed to exhibit discharge pressure. During disassembly it was determined that line shaft coupling #6 had failed. Subsequent metallurgical analysis determined that the failure of coupling #6 was due to IGSCC, which was the same cause of the 2009 failure of coupling #7 in the same pump. The root cause evaluation was completed under condition report CR-PLP-2011-03902 and the corrective actions taken to address the failure are documented under the same condition report. The inspectors determined that the RCE adequately discussed the identification of the issue. This was accomplished by reviewing the narrative logs for August 9, 2011, and by reviewing the excerpts from the metallurgical report and information contained in condition reports. b. Determine whether the licensee's root cause evaluation documented how long the issue existed, and whether there were any prior opportunities for identification.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The RCE concluded the Root Cause of the event was that senior leaders had not established a sufficiently sensitive culture of risk recognition and management, which resulted in the plant's managers, supervisors, and workers not recognizing, accounting for, or preparing for the industrial safety risk and plant operational nuclear risk, involved with the panel ED-11-2 breaker inspection and replacement maintenance. Additionally, the licensee identified that the breaker and fuse coordination for the 125-Volt DC system left train was insufficient to prevent a reactor trip under the short circuit conditions experienced during the ED-11-2 maintenance on September 25, 2011.

The inspectors determined that the licensee's evaluation was adequate with respect to identifying how long the issue existed and whether there were any prior opportunities for identification. Breaker 72-01, "Isolation breaker to DC battery ED-1," was a shunt trip breaker that was used in conjunction with a trip switch to isolate the balance of the left channel DC circuit from panel ED-11A for a fire in the cable spreading room. When this breaker was inspected during receipt on October 21, 1980, the inspector failed to recognize that the breaker included an unspecified magnetic auto-trip feature. This breaker was subsequently installed in the plant in 1981 and contributed to the insufficient breaker and fuse coordination for the 125-Volt DC system that resulted in the event on September 25, 2011. The licensee has taken corrective actions including performing an extent of condition to determine that there were no other breakers installed in the 125-Volt DC system that did not meet their design specifications, and replacing the affected breakers with new breakers that met the plant design requirements. The licensee performed a Previous Occurrence Evaluation to determine if there was prior opportunity for identification and did not identify any opportunities that would have led to the identification and correction of the breaker issue that may have prevented the event on September 25, 2011.

7 Enclosure

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The RCE concluded that the susceptibility to IGSCC arose from a material change that was introduced in June 2009 when the line shaft couplings for 'P-7C' were changed from carbon steel to 416 SS. The specifications for the design change occurred under engineering change (EC) 5000121762 in December 2007. According to the EC, 416 SS was chosen due to its strength, corrosion resistance and wear resistance.

On September 29, 2009, SW pump 'P-7C' failed during routine service. Disassembly of the pump revealed that coupling #7 had fractured due to IGSCC. At the time the failed coupling was determined to have been improperly heat-treated based on high hardness. However, the RCE performed for the 2011 failure determined that the 2009 RCE (CR-PLP-2009-04519) failed to sufficiently investigate the material properties of 416 SS.

It also determined that the toughness properties of the coupling and the effect of the corrosive nature of the water from Lake Michigan were not investigated. The 2011 RCE provided an evaluation of prior opportunities to identify the increased susceptibility to IGSCC due to the use of 416 SS. The RCE references multiple industry

Operating Experiences (OEs) which could have been used to provide insights into 416 SS being prone to temper embrittlement and IGSCC. The OEs referenced included the Perry repeat failures in 2003 and 2004 as well as a 2010 Prairie Island failure. In addition, an NRC Information Notice (IN-2007-05) provided additional information regarding pump failures due to IGSCC. The licensee's evaluation concluded that the OEs mentioned were missed opportunities to identify that the material selected for use in the SW pumps was not the most suitable for the application and could have prevented these failures. The inspectors determined that the licensee's evaluation was adequate with respect to identifying how long the issue existed. However, the inspectors identified a prior opportunity to identify that the couplings that had been installed in pump 'P-7C' following the 2009 event had been improperly tempered. This was identified while reviewing Attachment VI - Bodycote Trip Report of the 2009 RCE. This attachment documented an October 12, 2009, shop visit to the sub-contractor who performed the coupling's heat treatment for the vendor in charge of fabricating the couplings. In this report it is documented that the vendor provided information to the licensee regarding their heat treatment process: "ASTM 416 require heat treat at 1875 0F max range (range 1775 - 1875 0 F) Preheat 1450 to 1550 0F to stabilize temp throughout the part then continue heat to max - Bodycote metallurgist uses ASM [American Society for Metals] books for basis of requirements." However, as was evidenced in the 2011 RCE, the range provided by the vendor did not conform to approved ASM guidelines: "Furthermore, the heat treatment did not conform to the recommendation in the ASM guides for heat treating Type 416 SS as prescribed- The austenizing temperature of 1870 0F exceeded the maximum recommending range of 1695 0 F to 1850 0F and was on the wrong side of the range when considering the tempering temperatures in the range of 1050 0F to 1080 0 F."

8 Enclosure The inspectors determined that the trip to the sub-contractor was a missed opportunity to identify that the couplings that were installed after the 2009 event were improperly heat-treated. The consequence of improper heat treatment is that it can lead to an increase in susceptibility to IGSCC. The licensee captured this in condition report, CR-PLP-2012-06388. The inspectors determined that this omission did not impact the effectiveness of the RCE since the licensee had already established corrective actions (CAs) to deal with the control of vendors. See Section 02.01.d for the associated finding.

c. Determine whether the licensee's root cause evaluation documented the plant specific risk consequences and compliance concerns associated with the issue(s) both individually and collectively.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The RCE included a discussion of nuclear, radiological and environmental safety significance and stated that the event posed no actual environmental safety or radiological risk consequences. The licensee's risk assessment was available in study EA-PSA-SDP-D11-2-11-07R2. The study concluded that the change in core damage frequency was less than 4.3E-06/yr, or Whit e. The differences between the NRC's risk evaluation (Yellow) and the licensee's risk evaluation are discussed in IR 05000255/2011019 and 05000255/2011020.

The inspectors concluded that the licensee's RCE did document the risk consequences and compliance concerns associated with the issue.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The RCE included a discussion of nuclear, radiological and environmental safety significance and stated that no actual consequences resulted from the inoperability of SW pump 'P-7C'. The licensee's risk assessment was available in study EA-PSA-SDP-P7C-11-06. The study concluded that the change in core damage frequency was less than 1E-06/yr, or Green. The differences between the NRC's risk evaluation (White) and the licensee's risk evaluation are discussed in IR 05000255/2011019 and 05000255/2012020. The inspectors concluded that the licensee's RCE did document the risk consequences and compliance concerns associated with the issue.

d. Findings

(1) Failure to Complete Previous Occurrence Evaluation for Root Cause Evaluations

Introduction:

A finding of very low safety significance and associated NCV of 10 CFR Part 50, Appendix B, Criterion II, "Quality Assurance Program," was identified by the inspectors for the failure to co mplete the Previous Occurrence Evaluation as required by Palisade's procedure EN-LI-118, "Root Cause Evaluation Process."

Description:

Palisades Nuclear Plant Procedure EN-LI-118 requires that a Previous Occurrence Evaluation be performed "to deter mine whether the same or similar Conditions have occurred, either at your site or within the industry, and if so, why associated corrective actions for Conditions having the same causes were unsuccessful 9 Enclosure in preventing occurrence of this Condition." Attachment 9.8, step 1.2, requires that the Evaluation "Assess any repeat or similar Conditions for causes similar to those identified in the present CR.

IF the same or similar causes existed, THEN assess why previous corrective actions failed to preclude the present Conditions." Section 5.5.5.2 states, "

IF there was a previous occurrence of the condition, THEN document the cause/contributing cause evaluation for recurrence." Additionally, Section 5.5.5.7.4 states, "Document the basis for any determinations made, including whether or not previous similar events were actually credible opportunities to have prevented this event (rather than missed opportunities)." While reviewing RCE CR-PLP-2012-03873, "Ground Connected to DC Circuit on component cooling water (CCW) Tank Level Switch," the inspectors reviewed the Previous Occurrence Evaluation performed as part of the RCE. The inspectors noted that the evaluation listed CR-PLP-2011-04822, "Plant Trip during Panel ED-11-2 Maintenance," as relevant and also lists seven corrective actions from CR-PLP-2011-04822 that were relevant to the CR-PLP-2012-03873 RCE. However, the Previous Occurrence Evaluation did not include an evaluation of the causes from the CR-PLP-2011-04822 to assess why previous corrective actions failed to preclude the issue documented in CR-PLP-2012-03873. Additionally, the Previous Occurrence Evaluation did not document the basis for any determination made of whether or not the event listed in CR-PLP-2011-04822 was a credible opportunity to have prevented the issue listed in CR-PLP-2012-03873. While reviewing RCE CR-PLP-2011-03902, "Service Water Pump P-7C Line Shaft Coupling Failure," the inspectors reviewed the Previous Occurrence Evaluation performed as part of the RCE. The Previous Occurrence Evaluation did not discuss a previous failure of the Service Water Pump P-7C shaft coupling that occurred in 2009, but the failure was mentioned in Attachment IV , Operating Experience. Attachment IV does list the 2009 failure, documented in CR-PLP-2009-04519; however there is no assessment of why previous corrective actions taken after the 2009 event failed to preclude the 2011 P-7C coupling failure or discussion of whether the 2009 event was a credible opportunity to have prevented the 2011 failure.

Analysis:

The inspectors determined that the failure to complete the Previous Occurrence Evaluation for two root cause evaluations was contrary to Procedure EN-LI-118, a quality procedure, and was a performance deficiency. The performance deficiency was determined to be more than minor because if left uncorrected, would become a more significant safety concern. Specifically, the failure to complete the Previous Occurrence Evaluation as required by Station Procedure EN-LI-118 could result in the station not identifying repeat events and previous corrective actions that may have been inadequate. This could result in those previous inadequate corrective actions being implemented for the current deficiency resulting in future failures. Because the inspectors identified two examples of Palisades' failure to

perform the Previous Occurrence Evaluation, the inspectors determined that this was a programmatic issue and therefore more than minor. The inspectors concluded this finding was associated with the Initiating Events and Mitigating Systems Cornerstone. The most important impacts are in the Initiating Event Cornerstone, but the inspectors reviewed screening questions for both to be conservative.

10 Enclosure The inspectors determined the finding could be evaluated using the Significance Determination Process (SDP) in accordance with Inspection Manual Chapter (IMC) 0609, "Significance Determination Process," Attachment 0609.04, "Initial Characterization of Findings," issued on June 19, 2012,and IMC 0609, Appendix A, "The Significance Determination Process for Findings at-Power," issued on June 19, 2012, for the Initiating Events and Mitigating Systems Cornerstone. The inspectors answered "no" to all of the Exhibit 1 - Initiating Events Screening Questions and Exhibit 2 - Mitigating Systems Screening Questions, therefore the finding screened as very low safety significance or Green. The inspectors reviewed IMC 0310, "Components Within Cross-Cutting Areas," and determined that none of the components reflected the performance characteristics that contributed to the finding. Therefore, the inspectors did not identify a cross-cutting aspect associated with this finding.

Enforcement

10 CFR Part 50, Appendix B, Criterion II, "Quality Assurance Program," requires, in part, that licensees "shall establish at the earliest practicable time, consistent with the schedule for accomplishing the activities, a quality assurance program which complies with the requirements of this appendix. This program shall be documented by written policies, procedures, or instructions and shall be carried out throughout plant life in accordance with those policies, procedures, or instructions." Palisades' Procedure EN-LI-118, "Root Cause Evaluation Process," implements requirements of the Palisades Nuclear Plant's Quality Assurance Program Manual Section A.6, "Corrective Action."

EN-LI-118 states that a Previous Occurrence Evaluation is performed "to determine whether the same or similar Conditions have occurred, either at your site or within the industry, and if so, why associated corrective actions for Conditions having the same causes were unsuccessful in preventing occurrence of this Condition."

Contrary to the above, licensee personnel did not follow the quality program procedural requirements when performing RCE's for CR-PLP-2011-03902 and CR-PLP-2012-03873 on February 28, 2012 and August 15, 2012, respectively. Specifically, the licensee did not adequately evaluate previous occurrences to determine if previous corrective actions failed to preclude the present Conditions. Because this violation was of very low safety significance and it was entered into the licensee's corrective action program as CR-PLP-2012-06419, this violation is being treated as an NCV, consistent with Section 2.3.2 of the NRC Enforcement Policy (NCV 05000255/2012011-01, Failure to Complete Previous Occurrence Evaluation for Root Cause Evaluations). 02.02 Root Cause, Extent of Condition, and Extent of Cause Evaluation a. Determine whether the licensee's root cause evaluation applied systematic methods in evaluating the issue in order to identify root causes and contributing causes.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

In its root cause analysis, the licensee used a Failure Mode Analysis to identify significant factors which led up to the plant trip during maintenance on Panel ED-11-2.

The licensee also used a Barrier and Change Analysis in order to identify and classify the actual causes of the trip. Based upon this, the inspectors determined that the licensee evaluated the issue using a systematic methodology to identify root and contributing causes.

11 Enclosure

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

In its root cause analysis, the licensee used the following systematic methods to complete the root cause evaluation: Event and Casual Factor Charting, Failure Mode Analysis and Metallurgical Analysis (performed by Lucius Pitkin, Inc. testing).

Based upon this, the inspectors determined that the licensee evaluated the issue using a systematic methodology to identify root and contributing causes. b. Determine whether the licensee's root cause evaluation was conducted to a level of detail commensurate with the significance of the problem.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

In its root cause analysis, the licensee used an evaluation team with broad knowledge

and expertise. Team members had the following backgrounds: Radiation Protection, Electrical Maintenance, Training, System Engineering, Human Performance, Operations, and Maintenance. The licensee's Failur e Mode Analysis evaluated each identified failure mode by presenting supporting or refuting evidence for each failure mode. The licensee used this analysis to identify the failure mode of Panel ED-11-2. The Barrier Analysis was then used to evaluate the barriers that were in place which could have stopped the evolution and how they failed. The licensee used this analysis to identify one root cause and four contributing causes. The root cause, as identified by the licensee, was "Senior leaders have not established a sufficiently sensitive culture of risk recognition and management, which resulted in the plant's managers, supervisors and workers not recognizing, accounting for, or preparing for the industrial safety risk and plant operational nuclear risk, involved with the panel ED-11-2 breaker inspection and replacement maintenance." The contributing causes, as stated in the licensee's root cause evaluation, were: - Breaker and fuse coordination for the 125-Volt DC system left train was insufficient to prevent a reactor trip under the short circuit experienced during ED-11-2 maintenance on 09/25/11. - WOs used for removal and inspection of breakers 72-119, 72-120, 72-121, and 72-123 did not include details appropriate for maintenance on energized, high critical electrical equipment with the Plant on line. - Oversight by managers and supervisors did not result in identification and correction of the human performance errors and weaknesses in the work involving the inspection and replacement of the breakers in the ED-11-2 panel. - Managers, supervisors and workers did not consistently follow approved procedures for job preparation, job execution and risk management.

Based upon the work performed for this root cause, the inspectors concluded that the root cause evaluation was conducted to a level commensurate with the significance of the problem. The licensee's evaluation team and analysis techniques used were sufficient to identify the root and contributing causes of the September 25, 2011, event.

12 Enclosure

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

In its RCE, the licensee used a root cause evaluation team that consisted of members of mechanical maintenance, operations, training, engineering and consultants. This team performed a detailed evaluation using Event and Casual Factor Charting, Failure Mode Analysis, and Metallurgical Analysis and identified two root causes and three contributing causes. The two root causes for the event were determined to be IGSCC induced failure and specification of the wrong material as it relates to its service environment.

In addition to the root causes, the RCE team identified three contributing causes of deficiencies associated with: 1) increases susceptibility to IGSCC caused by tempering embrittlement, 2) insufficient use of qualified metallurgical expertise and 3) ineffective use of OE. Based on the comprehensive evaluation performed for this RCE, the inspectors concluded that the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem. c. Determine whether the licensee's root cause evaluation included consideration of prior occurrences of the problem and knowledge of prior operating experience.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

In its RCE, the licensee did a search of prior internal and external OE. The search included Palisades and Entergy Fleet's Condition Reports, the Institute of Nuclear Power Operations (INPO) website, and Entergy's OE Database. The search identified a

number of previously identified management oversight, short circuit, and risk assessment examples both internally and externally. Based upon the considerations described in the analysis, the inspectors concluded that the licensee's RCE included a consideration of prior occurrences of the problem and knowledge of prior OE.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

In its RCE, the licensee included an evaluation of internal and external OE. This OE evaluation included previous similar events involving coupling failures of 416 SS. Various combinations of words including but not limited to "embrittlement" and "pump failure" were included. This search yielded examples of relevant OE that could have been used to take action before the failures at Palisades occurred. Noteworthy examples include Perry repeat failures in 2003 and 2004 and a Prairie Island failure in 2010. It was for this reason that the ineffective use of OE was identified as a contributing cause. In response to this deficiency, the licensee initiated a corrective action to provide additional training to its employee on the effective use of OE and their use in engineering changes. Based on the licensee's evaluation the inspectors determined that the licensee's RCE included a consideration of prior occurrences of the problem and knowledge of prior OE.

13 Enclosure d. Determine whether the licensee's root cause evaluation addressed extent of condition and extent of cause of the problem.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

In its root cause analysis, the licensee addressed both extent of condition and extent of cause. The licensee's extent of condition looked into all organizations and the use of the risk process. The licensee identified that the root cause did not just affect one group or organization but all organizations involved in the maintenance evolution of ED-11-2. The licensee also recognized that "site leadership at all levels was not sufficiently intrusive into work on panel ED-11-2." The licensee's extent of cause review searched for examples of previous condition reports, procedures, and change request that were related to the risk process, management oversight, 125 - Volt DC coordination issues, inadequate work instructions, and procedure adherence. During that review the licensee identified that a number of their maintenance procedures have not been revised since 2007, there are currently over 400 change requests for permanent Maintenance procedures, and there was a need to address potential coordination issues. The licensee created corrective actions to address these identified deficiencies in both the extent of condition and extent of cause.

The inspectors concluded that the licensee's analysis appropriately addressed extent of condition and extent of cause concerns.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

In its root cause analysis, the licensee performed an extent of cause/extent of condition evaluation that included changing the material of all three SW pumps to couplings made of 17-4PH SS, which is not susceptible to IGSCC. Since the SW pumps were the only pumps using tempered 416 SS the extent of cause was limited to these pumps. As it related to the second root cause of specifying the wrong material, the licensee performed a review of a sample size of 13 ECs developed since 2007 under the new procedure EN-DC-115, "Engineering Change Process," that included material changes.

The result of this review was that all the material changes were properly managed.

However, the review also identified that the use of OE was weak. Specifically, 6 of 13 EC's did not perform an OE review relating to the material change the modification entailed. The licensee's training in the effective use of OE was used to address this weakness. The licensee also performed a generic implication review of the human performance issues as they related to the implementation of the engineering change process. The review determined that the procedure EN-DC-115 has a more thorough review and approval process than the procedure that was used in 2007. The inspectors concluded that the licensee's analysis appropriately addressed extent of condition and extent of cause concerns.

e. Findings

No findings were identified.

14 Enclosure 02.03 Corrective Actions a. Determine whether the licensee specified appropriate corrective actions for each root/contributing cause or that the licensee evaluated why no actions were necessary.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The licensee's root cause evaluation concluded that there was one Root Cause and four Contributing Causes for the plant trip during maintenance on electrical panel ED-11-2.

The root cause for the event was determined to be the failure of senior leaders to establish a sufficiently sensitive culture of risk recognition and management which resulted in the plant's managers, supervisors, and workers not recognizing, accounting for, or preparing for the industrial safety risk and plant operational nuclear risk associated with the maintenance activity. The four contributing causes were determined to be: 1) Breaker and fuse coordination for the 125-Volt DC system left train was insufficient to prevent a reactor trip under the short circuit conditions experienced during ED-11-2 maintenance; 2) Work orders used for inspection and removal of breakers did not include details appropriate for maintenance on energized, high critical electrical equipment with the Plant online; 3) Oversight by managers and supervisors did not result in identification and correction of the human performance errors and weaknesses in the work involving the inspection and replacement of breakers in the ED-11-2 panel; and 4) Managers, Supervisors, and workers did not consistently follow approved procedures for job preparation, job execution, and risk management. To address the root cause, the licensee instituted Corrective Actions to Prevent Recurrence (CAPR) in the form of 1) reinforcing and institutionalize Entergy standards for procedure compliance, accountability, and unacceptable behaviors via face to face

communications from the Chief Operating Officer (COO) through individual contributor levels, and 2) Implementing and ensuring compliance with Entergy Risk Management Procedures. To address the contributing cause of the breaker and fuse coordination for the 125-Volt DC system the licensee's initial action was to increase the trip setting of the affected breakers (72-01 & 72-02) and notify Operations of the change in settings. In parallel the licensee had a corrective action to analyze the coordination between the breakers and the panels. This analysis led to the licensee replacing the breakers. To address the contributing cause of the inadequate work orders, the licensee took corrective actions that included training individuals on the use of risk procedures and also quarantining and correcting maintenance procedures used for electrical work activities. To address the contributing cause of poor oversight resulting in human performance errors and weaknesses in the planned work, the licensee implemented routing training as well as implementing a station monitoring program of work planning and work execution at Palisades. To address the contributing cause of procedure adherence the licensee initiated a number of corrective actions. These actions included information sharing, site training, reinforcement of management expectations of procedure adherence, and management observations of staff during use of work instructions/ procedures.

15 Enclosure The inspectors concluded that the corrective actions in place were appropriate to prevent recurrence as long as they were completed as stated in the root cause evaluation.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The licensee's RCE concluded that there were two root causes and three contributing causes for the failure of SW pump 'P-7C'. The two root causes for the event were determined to be IGSCC induced failure for the 2009 and 2011 events and the specification of the wrong material as it relates to its service environment. The three contributing causes were deficiencies associated with increased susceptibility to IGSCC caused by tempering embrittlement, insufficient use of qualified metallurgical expertise, and ineffective use of OE. To address each root cause the licensee instituted two CAPRs in the form of replacing the SW pump line shaft couplings on all three service water pumps with a couplings of a new material (17-4PH), which was completed on October 26, 2011, and creating an engineering standard that identifies station requirements and expectations for material changes, which was completed on December 22, 2011. To address the contributing causes of IGSSC susceptibility, insufficient use of qualified metallurgical expertise and ineffective use of OE, the licensee instituted corrective actions that including training and guidance of when to obtain external expertise to address an issue.

The inspectors determined that the contributing cause relating to increased IGSCC susceptibility had not been adequately addressed in the RCE. Specifically, EN-LI-118, "Root Cause Evaluation Process," requires that each root cause and contributing cause have an associated corrective action or an explanation as to why a corrective action is not needed. Contrary to the above, the corrective action taken to address this contributing cause stated that since the material was changed no further actions were necessary. However, the reason why the coupling had been susceptible to temper embrittlement, in this case poor vendor performance, was not discussed. The vendor that performed the tempering had applied the ASM guidelines incorrectly. No corrective actions addressed this issue. The licensee was able to provide documentation in separate condition reports showing that they had taken actions to address the vendor control issue. The licensee wrote CR-PLP2012-06319 to document this issue. Due to the fact that the corrective actions were implemented, although not documented in the RCE, this is a minor violation of 10 CFR 50 Appendix B, Criterion V, "Instructions, Procedures and Drawings." For the remaining corrective actions specified the inspectors concluded that they were appropriate and addressed each root and contributing cause and were appropriate to prevent recurrence. b. Determine whether the licensee prioritized the corrective actions with consideration of the risk significance and regulatory compliance.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The licensee's corrective actions to address the root and contributing causes were prioritized in accordance with procedure EN-LI-102, "Corrective Action Process." The inspectors reviewed the licensee's plans for accomplishing the corrective actions and noted that the risk significance of the equipment was being appropriately considered. Based on the guidance provided in the licensee's procedures and the prioritization of the 16 Enclosure corrective actions in accordance with these procedures, the inspectors determined that the corrective actions were prioritized with consideration of the risk significance and regulatory performance.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The licensee's corrective actions to address the root and contributing causes were prioritized in accordance with procedure EN-LI-102, "Corrective Action Process." The inspectors reviewed the licensee's plans for accomplishing the corrective actions and noted that the risk significance of the equipment was being appropriately considered.

Based on the guidance provided in the licensee's procedures and the prioritization of the corrective actions in accordance with these procedures, the inspectors determined that the corrective actions were prioritized with consideration of the risk significance and regulatory performance. c. Determine whether the licensee established a schedule for implementing and completing the corrective actions.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The licensee established due dates for the corrective actions in accordance with procedure EN-LI-102, "Corrective Action Process," requirements for timeliness. The majority of the due dates were captured in CR-PLP-2011-04822. The inspectors determined that the licensee adequately established a schedule for implementing and completing the corrective actions. At the time of this inspection all CAPRs had been completed. A number of corrective actions were outstanding at the end of this inspection and the inspectors verified that due dates were assigned for each in accordance with their significance. The inspectors considered the timeline for completion of corrective actions to be appropriate.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The licensee established due dates for the corrective actions in accordance with procedure EN-LI-102, "Corrective Action Process," requirements for timeliness. The majority of the due dates were captured in CR-PLP-2011-03902; however, many of the due dates for the action items were contained throughout the licensee's corrective actions program in various condition reports. The inspectors determined that the licensee adequately established a schedule for implementing and completing the corrective actions. At the time of this inspection all CAPRs had been completed. A number of corrective actions were outstanding at the end of this inspection and the inspectors verified that due dates were assigned for each in accordance with their significance. The inspectors considered the timeline for completion of corrective actions

to be appropriate. d. Determine whether the licensee developed quantitative or qualitative measures of success for determining effectiveness of the corrective actions to prevent recurrence.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The licensee established an effectiveness review plan that originally included performing focused self assessments of Palisades' procedure adherence with respect to promoting 17 Enclosure a sensitive risk culture. This assessm ent was to be performed by a non-Palisades Entergy employee approximately every 6 months for 2 years.

After the first assessment was completed, the station identified that the assessment did not follow Palisades' procedural requirements for a focused self assessment and documented the deficiency in CR-PLP-2012-05116. As part of the corrective actions for this CR, the effectiveness review requirem ents for the Root Cause were changed to perform a snapshot assessment every 6 months for 2 years. After four snapshot assessments are complete, the results of the four snapshot assessments were to be used to complete a focused self assessment. Additionally, the snapshot self assessments were to be performed utilizing the fleet's RISK meeting What it Looks Like (WILL) sheets, whereas the original focused self assessment did not specify to utilize the WILL sheets.

The inspectors reviewed the change to the effectiveness review plan and concluded that the change was done in accordance with Palisades' requirements by sending the change to the Corrective Action Review Board (CARB) for final approval. The inspectors questioned whether this was a reduction in the level effort put forth to perform the effectiveness review and if this was a change in scope of the effectiveness review by limiting the assessment to utilizing the WILL sheets. The inspectors met with the CARB Chairman to verify that the CARB chairman understood that the change approved by CARB was a change to the effectiveness review associated with CAPRs for root cause evaluation CR-PLP-2011-04822 and to determine if this was a change in scope of the effectiveness review. The CARB chairman stated that he understood the change to the effectiveness review was for root cause CR-PLP-2011-04822. Additionally, the CARB Chairman did not feel that this change was a significant change to the scope of the effectiveness review plan. The inspectors determined that quantitative and qualitative measures of success had been developed for determining the effectiveness of the corrective actions to preclude repetition. The inspectors noted that the effectiveness review was narrowly focused since it focused primarily on the CAPR to implement, and ensure compliance with, Entergy Risk Management Procedures and focused less on the CAPR to reinforce and institutionalize Entergy standards for procedure compliance, accountability, and unacceptable behavior. This could be significant as procedure compliance issues was a key deficiency in the September event. Additionally, the station relies on personnel behavior to 'stop when unsure' to prevent workers from going forward in the face of uncertainty. This behavior is often times the last barrier to preventing errors when station programs and processes do not work properly. By not measuring the effectiveness of the CAPR to reinforce and institutionalize Entergy standards as part of the effectiveness review for this RCE, the station could miss an opportunity to identify deficiencies in its previous corrective actions. However, the inspectors did identify that the station's improved behavior observation program, which was implemented as part of the site recovery plan but not credited as part of the root cause evaluation, would provide Palisades with a quantitative and qualitative measure of success of the CAPR to reinforce and institutionalize Entergy standards for procedure compliance, accountability, and unacceptable behavior. Based on the implementation of the recovery plan, the inspectors determined that there were appropriate tools to measure site's effectiveness in the areas of concern.

18 Enclosure

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The licensee established measures for determining the effectiveness CAPRs as part of the RCE. These CAPRs included:

• "Creating a limited distribution Engineering Standards for Palisades that clearly identifies station requirements and expectations for material changes affecting installed plant equipment. Consideration should be given to specifying required analysis, guidance on obtaining outside assistance, specification of deliverables, analysis of the service environment and documenting analysis of relevant OE-"

and; * "Replace 416 SS Service Water Line Shaft Couplings with 17-4PH couplings per EC 31337-" For each of these CAPRs the licensee established an effectiveness review to determine the effectiveness of these actions. For the first CAPR the licensee performed an analysis of EC packages of material changes to review that 416 SS (tempered) was not specified in ECs where the service environment is corrosive to 400 series SS. For the second CAPR the licensee reviewed the work order packages that performed the change of the SW pump line shaft couplings. At the time of the inspection the

effectiveness reviews were complete.

The inspectors determined that quantitative and qualitative measures of success had been developed for determining the effectiveness of the corrective actions to preclude repetition. However, the inspectors noted that the effectiveness review for the engineering standard, although it followed station procedures for its implementation, was narrowly focused since by only looking at 416 SS the station missed an opportunity to gauge how the station is implementing the new engineering standard. e. Determine whether the corrective actions planned or taken have adequately addressed the Notice of Violation that was the basis for the supplemental inspection.

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The NRC issued a Notice of Violation (N OV) (EA-11-243) to the licensee on February 14, 2012. The NOV associated with the Yellow finding that was the subject of this IP 95002 inspection identified one violation of NRC requirements. In particular, a violation of Title 10 of the Code of Federal Regulations (10 CFR), Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," occurred on September 25, 2011, when the licensee failed to ensure that the work performed on safety related bus ED-11-2 through work orders 291194-01, 291210-01, and 291123-03, all activities affecting quality, was prescribed by documented instructions or procedures of a type appropriate to the circumstances and accomplished in accordance with the instructions or procedures. The NRC has concluded that information regar ding the reasons for the violation, the corrective actions taken and planned to be taken to correct the violation and prevent recurrence, and the date when full compliance was achieved, is already adequately addressed on the docket in NRC IR No. 05000255/2011014, and during the regulatory conference held on January 11, 2012. During this inspection, the inspectors confirmed that the licensee's RCE and planned and taken corrective actions addressed the NOV.

19 Enclosure

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The NRC issued an NOV (EA-11-241) to the licensee on February 14, 2012. The NOV associated with the White finding that was the subject of this IP 95002 inspection identified two violations of NRC requirements. In particular, a violation of 10 CFR 50, Appendix B, Criterion III, "Design Control" due to the failure to verify the adequacy of the design when the P-7C coupling material was changed from carbon steel to 416 SS. In addition, there was a violation of 10 CFR 50, Appendix B, Criterion XVI, "Corrective Actions" for the failure to prevent recurrence of a significant condition adverse to quality.

The NRC has concluded that information regar ding the reasons for the violation, the corrective actions taken and planned to be taken to correct the violation and prevent recurrence, and the date when full compliance was achieved, is already adequately addressed on the docket in NRC IR No. 05000255/2011016, and during the regulatory conference held on January 11, 2012. During this inspection, the inspectors confirmed that the licensee's RCE and planned and taken corrective actions addressed the NOV.

f. Findings

No findings were identified. 02.04 Independent Assessment of Extent of Condition and Extent of Cause

a. Inspection Scope

IP 95002 requires that the inspectors perform a focused inspection to independently assess the validity of the licensee's conclusions regarding the extent of condition and extent of cause of the issue. The objective of this requirement is to independently sample performance, as necessary, within the key attributes of the cornerstone that is related to the subject issue to ensure that the licensee's evaluation regarding the extent of condition and extent of cause is sufficiently comprehensive.

The inspectors independently assessed the validity of the licensee's conclusions regarding the extent of condition and extent of cause of the issues.

The inspectors then reviewed licensee records, procedures, and documents; conducted detailed interviews; and reviewed plant evolutions in progress to assess and evaluate the extent of condition and extent of cause of the issues. The inspectors observed plant evolutions including a planned containment entry, diving activities in the cooling tower basin, and the plant response to a through-wall leak on a service water valve. The method of analysis included the use of IP 71841, "Human Performance," for reviewing human performance, IP 62709, "Configuration Risk Management Assessment and Risk Management Process," IP 93805, "Maintenance Program," and IP 71111.13, "Maintenance Risk and Assessments and Emergent Work" for evaluating Palisades' maintenance and risk management programs. Finally, the results were compared to the licensee's assessments and any differences noted were discussed with the licensee and evaluated using the SDP process. The inspectors reviewed these events based issues and conducted an independent assessment for these issues that led to the performance deficiency associated with this issue. Specific documents reviewed during this inspection are listed in the attachment.

20 Enclosure

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The inspection team conducted an independent extent of condition and extent of cause review of the issues associated with the Yellow finding. The Yellow finding ultimately revealed significant and broad organizational issues associated with the station's commitment to achieving a high level of human performance with nuclear safety as the highest priority. The organization failed to recognize or understand the significance of risk identification and risk mitigation in the work control process. The RCE revealed inadequacies of the work control process for the identification and mitigation of risk and assigning appropriate levels of management supervision for daily and emergent work activities. The inspection team's independent review focused on the primary root causes associated with the yellow finding in addition to the licensee's identified contributing causes that involved more specific aspects of the broader root causes. The inspectors assessed whether the licensee's extent of condition and extent of cause evaluations sufficiently identified and bounded all organizational issues. The team also assessed whether the licensee's extent of condition and extent of cause evaluations sufficiently determined the actual extent of similar organizational issues that potentially existed in other departments, programs, and processes. In conducting this independent review, the inspector interviewed station management and personnel, reviewed program and process documentation, reviewed existing station program monitoring and improvement efforts, and attended various licensee work control and daily planning

meetings.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The inspection team conducted an independent extent of condition and extent of cause review of the issues associated with the White finding. The White finding revealed that engineering personnel had inappropriately specified a material for use in the Palisades SW operating environment that was susceptible to IGSCC. This event also revealed that Palisades did not use an engineer with sufficient metallurgical expertise and also made ineffective use of operating experience. The inspection team's independent review focused on the primary root causes associated with the white finding in addition to the licensee's identified contributing causes that involved more specific aspects of the broader root causes. The inspectors assessed whether the licensee's extent of condition and extent of cause evaluations sufficiently identified and bounded all organizational issues. The team also assessed whether the licensee's extent of condition and extent of cause evaluations sufficiently determined the actual extent of similar organizational issues that potentially existed in other departments, programs, and processes. In conducting this independent review, the inspector interviewed station management and personnel, reviewed program and process documentation, reviewed existing station program monitoring and improvement efforts, and attended various licensee work control and daily planning

meetings.

21 Enclosure b. Assessment

.1 Failure to Have Adequate Work Instructions (Inspection Report 05000255/2011014)

The inspection team determined that the licensee conducted a comprehensive extent of condition and extent of cause review that sufficiently identified most relevant areas. The team did not identify any substantive extent of condition and extent of cause issues that the licensee had not already identified with either corrective actions from the RCE or actions from other licensee programs, such as the Palisades Recovery Plan. The inspection team focused inspection efforts on reviewing RCE CR-PLP-2012-03873, "Ground Connected to DC Circuit on CCW [Component Cooling Water] Tank Level Switch." This RCE evaluated an occurrence when, on May 14, 2012, workers erroneously placed a wire jumper between 115 Volt AC and 125 Volt DC circuits, causing multiple, unexpected control room alarms. The workers in the field performed this jumper installation without a work plan. This action could have led to a loss of DC power on the circuit, which would have caused a loss of non-critical Service Water and a subsequent reactor trip. The inspectors examined the details of this event because many of the actions and behaviors exhibited by the workers involved were similar in nature to the loss of DC bus event that occurred in September 2011. The inspectors evaluated this near-miss to determine if it was similar enough to the September 2011 event to be considered a repeat occurrence. Those similar behaviors included the lack of a pre-job brief and discussion regarding the limitations of the work scope, workers taking action outside of the scope allowed by 'toolpouch maintenance,' supervisors failing to adequately challenge the workers, and workers proceeding in the face of uncertainty when unexpected conditions arose.

The RCE concluded the root cause of the event was that "a field team did not follow work management processes which resulted in an inappropriate action resulting in a momentary DC ground." The contributing causes were determined to be: (1) "FIN [Fix it Now], WCC SRO [Work Control Center Senior Reactor Operator], and CRS [Control Room Supervisor] had a lack of knowledge of EN-MA-130 FIN Team Process and EN-WM-100, Work Request (WR) Generation, Screening and Classification;" and (2) "Two individual workers did not internalize the cause and corrective actions from the September 2011 DC bus (stop when unsure), and made an in-field decision to continue to work." The inspectors conducted interviews with those individuals involved in the May 2012 near-miss as well as senior management as part of the evaluation to determine if the inspection team should consider this a repeat occurrence of the September 2011 event.

Many of the supervisors and managers the inspection team interviewed stated that the May 2012 near-miss was not a repeat event of the September 2011 event because the May 2012 near-miss involved only a handful of individuals, whereas the September 2011 occurrence involved multiple individuals across multiple organizations at Palisades. The inspectors agreed that the May 2012 near-miss involved fewer individuals, but there were individuals from several organizations involved in the near-miss. The inspectors concluded that the RCE assessment was narrow in that it stated only the field work team failed to internalize the cause and corrective actions from the September 2011 DC bus event. The inspectors concluded that other individuals, including the WCC SRO, CRS, 22 Enclosure and a non-licensed plant operator also exhibited behaviors similar to those of the September 2011 DC bus event. Additionally, the inspectors reviewed the stations' corrective actions taken as part of RCE CR-PLP-2012-03873 to determine if they addressed weaknesses with any of the corrective actions from the RCE for the DC bus event of September 2011. The inspectors concluded that there were several corrective actions contained in RCE CR-PLP-2012-03873 that addressed those behav iors that were similar to behaviors exhibited during the September 2011 event. The inspectors concluded that the May 2012 near-miss did not meet the IP 95002 definition of a Repeat Occurrence, which is "two or more independent conditions which are the result of the same basic cause(s)." The inspectors determined that, while the May 2012 near-miss shared some commonalities with the September 2011 event, the two conditions were not the result of the same basic causes. The inspectors reached this conclusion because the May 2012 near-miss did not result in a significant plant transient and also did not exhibit the same site wide, organizational breakdowns in risk recognition and management that led to the September 2011 event. Finally, the inspectors determined that RCE CR-PLP-2012-03873 included appropriate corrective actions to address those weaknesses that were similar to those exhibited in the September 2011 event.

.2 Service Water Pump Coupling Failure (Inspection Report 05000255/2011016)

The inspection team determined that the licensee conducted a comprehensive extent of condition and extent of cause review that sufficiently identified most relevant areas. The team did not identify any substantive extent of condition and extent of cause issues that the licensee had not already identified with either corrective actions from the RCE or actions from other licensee programs, such as the Palisades Recovery Plan.

c. Findings

No findings were identified. 02.05 Safety Cultural Consideration

a. Inspection Scope

The inspectors performed a focused inspection to independently determine that the Palisades RCEs appropriately considered whether any safety culture component caused or significantly contributed to any risk significant performance issue. The inspectors conducted focus groups and interviews with 92 individual contributors from Operations, Operations Support, Maintenance, Radiation Protection, Chemistry, Engineering, Security, and Material Purchasing and Control Departments. The inspectors also conducted focus groups with 14 first line supervisors from engineering and maintenance, and interviewed 14 managers including the Employee Concerns Program (ECP) Manager, the Site Vice President and the General Manager of Plant Operations (GMPO). Inspectors also reviewed the licensee's RCE reports and associated procedures and documents listed in the attachment.

23 Enclosure

b. Findings and Observations

Palisades evaluated the identified root and contributing causes against the safety culture components that could have contributed to the events on September 25, 2011, and August 9, 2011. The inspectors did independently determine that safety culture components significantly contributed to risk performance issues. The inspectors also noted that Palisades identified appropriate safety culture components as contributors to

the events. The safety culture evaluation of the September 25, 2011, event and associated yellow finding identified decision making, resources, work control, work practices, accountability, and continuous learning environment as applicable components. The inspectors concluded that the assessment done in IR 05000255/2011014, which indicated that safety culture components possibly caused or significantly contributed to the performance issue, was accurate. The safety culture evaluation of the August 9, 2011, event and associated white finding identified resources, work control, work practices, corrective action program, operating experience, and continuous learning as applicable components.

The inspectors identified resources, communication challenges, alignment and coordination across departments, and trust of department level managers as the biggest challenges facing the site. Inspectors also determined that the site had many processes in place that weren't fully engrained in the organization. The site relied heavily on individuals in their respective positions instead of processes to drive key programs. The inspection team determined that a safety conscious work environment does exist at Palisades. Employees at all levels of the organization stated they felt comfortable raising nuclear safety concerns through multiple avenues.

However, there were some concerns with trust and communication at the department manager level. The site has replaced much of the senior leadership team, and a number of previous supervisors who were disrespectful to their employees have been removed from their positions. For an extended period of time, the majority of the senior leadership positions were filled by acting managers, which contributed to a lack of direction, accountability, and focus for the site. Most of those positions have since been filled permanently, and employees felt that Palisades had a solid leadership team in place at the time of this inspection. Generally, employees felt like the site was moving in the right direction, they trusted the site vice president, but there were lingering communication issues and concerns about the effectiveness of the CAP. One pervasive issue seemed to be with regard to change management, specifically with position "churn" (people moving around or out of Palisades) and knowledge transfer.

During interviews the inspectors heard that there were concerns about staffing levels in multiple departments, but the site was aware and was actively working with Entergy corporate management to post and fill positions. Within the security department inspectors determined, from focus groups and document reviews that the scheduling protocols included the routine use of waivers from the fatigue management provisions in Part 26 Subpart I. Entergy Corporate was perceived by many on the site to be stifling progress in filling positions.

The many issues at Palisades and staffing problems have contributed to the organization becoming more reactive to addressing maintenance and equipment reliability issues versus being proactive in addressing possible problems.

24 Enclosure The inspectors determined that the ECP was effective. Palisades' personnel knew that the program exists as an alternative method for raising concerns, though most did not feel the need to use it because issues get resolved through their management. The ECP program manager was visible and was often in the field engaging with the workforce. There were no issues with breaches of confidentiality identified by the inspectors. The team observed that the Entergy ECP process had two different methods for organizing concerns. Issues were filtered into either a Rapid Response category or into the normal ECP investigation process. Issues in the Rapid Response category were often addressed quickly and with less documentation than regular concerns. The ECP procedure contained criteria for screening issues into either category, but it was largely up to the discretion of the ECP manager to decide which method was best for resolving the issue. This process appeared to be contributing to a perception that the ECP was primarily for nuclear safety concerns, rather than any employee concern. The inspection team concluded the safety culture was adequate and improving. 02.06 Evaluation of Inspection Manual Chapter 0305 Criteria for Treatment of Old Design Issues The licensee did not request credit for self identification of an old design issue. Therefore, the subject risk significant issues were not evaluated against the IMC 0305 criteria for treatment of an old design issue.

4OA6 Exit Meeting

.1 Exit Meeting Summary

The inspectors presented the inspection results to Mr. A. Vitale and other members of licensee management on September 28, 2012. The licensee representatives acknowledged the findings presented. The inspectors asked licensee management whether any materials examined during the inspection should be considered proprietary.

They did identify several documents provided to the NRC inspectors that contained proprietary information. None of this proprietary information was included in this inspection report.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Tony Vitale, Site Vice President

Charlie Arnone, NSA Director

Barry Davis, Engineering Director

Otto Gustafson, Licensing Manager

Dave Mannai, Senior Manager, Nuclear Safety & Licensing

Jody Maumersen, System Engineering Manager Jim Miksa, Programs Engineering Manager Pat Rusell, PS&O Manager

Bart Nixon, Training Manager

Darrell Corbin, Assistant Operations Manager

Alan Blind, Assistant to the Site Vice President

Bob Bees, Information Technology Manager Lisa Marvin, Human Resources Manager

Andrew Notbohm, CA&A Manager

Chuck Sherman, Radiation Protection Manager

John Dills, Operations Manager Chris Plachta, Quality Assurance Manager Michael Sicard, Site Inspection Lead

Bret Baker, Assistant Maintenance Manager

Barb Dotson, Licensing Specialist

Dennis Fitzgibbon, Design Engineering Manager

Todd Mulford, Assistant Operations Manager Dave Berkenpas, Security Manager

Ernie Chatfield, ECP Manager

Dan Malone, Emergency Preparedness Manager

Mark Savage, Communications Manager

Pete Sabo, Finance Manager Ryan Prescott, Industrial Human Performance Superintendant

Nuclear Regulatory Commission

Gary Shear, Deputy Division Director, Division of Reactor Projects John

B. Giessner, Chief, Reactor Projects Branch 4 Robert Orlikowski, Project Engineer (Team Lead)

Tom Bilik, Senior Reactor Inspector

Brian Cushman, Resident Inspector

Diana Betancourt-Roldan, Reactor Engineer Jasmine Gilliam, Reactor Engineer Molly Keefe, Human Factors Specialist

Kamishan Martin, Human Factors Specialist

Stephanie Morrow, Human Factors Analyst

Gregory Hansen, Physical Security Inspector

Attachment

LIST OF ITEMS

Opened

05000255/2012011-01 NCV Failure to Complete Previous Occurrence Evaluation for

Root Cause Evaluations (Section 4OA4.02.01.d(1))

Closed

05000255/2012011-01 NCV Failure to Complete Previous Occurrence Evaluation for

Root Cause Evaluations (Section 4OA4.02.01.d(1))

05000255/2011014-02 VIO Failure to Have Adequate Work Instructions for Work Performed on Panel D11-2

05000255/2011016-01 VIO Failure to Prevent Recurrence of a Significant Condition

Adverse to Quality

Attachment

LIST OF DOCUMENTS REVIEWED

The following is a list of documents reviewed during the inspection.

Inclusion on this list does not imply that the NRC inspectors reviewed the documents in their entirety, but