View source for PNP 2011-080, Reply to Apparent Violation EA-11-227 in Inspection Report 2011013

{{Adams
| number = ML11339A099
| issue date = 11/28/2011
| title = Reply to Apparent Violation EA-11-227 in Inspection Report 2011013
| author name = Vitale A
| author affiliation = Entergy Nuclear Operations, Inc
| addressee name = 
| addressee affiliation = NRC/Document Control Desk, NRC/NRR
| docket = 05000255
| license number = DPR-020
| contact person = 
| case reference number = EA-11-227, PNP 2011-080
| document report number = IR-11-013
| document type = Inspection Report, Letter type:PNP, Notice of Violation
| page count = 10
| project = 
| stage = Other
}}

=Text=
{{#Wiki_filter:Entergy Nuclear Operations, Inc.
Enfarei-y                                                   Palisades Nuclear Plant 27780 Blue Star Memorial Highway Covert, MI 49043 Tel 269 764 2000 Anthony J. Vitale Site Vice President PNP 2011-080                                                                     EA-1 1-227 November 28, 2011 U.S. Nuclear Regulatory Commission ATTN: Document Control Desk Washington, DC 20555

==Subject:==
Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 Palisades Nuclear Plant Docket 50-255 License No. DPR-20

==References:==
: 1. Letter from Nuclear Regulatory Commission to Entergy Nuclear Operations, Inc. dated October 28, 2011, "Palisades Nuclear Plant, NRC Inspection Report 05000255/2011013 Preliminary White Finding"
: 2. Letter from Entergy Nuclear Operations, Inc. dated November 7, 2011, "10-Day Response to IR 2011013 Preliminary White Finding - Auxiliary Feedwater Pump"

==Dear Sir or Madam:==

On October 28, 2011, the Nuclear Regulatory Commission (NRC) issued Inspection Report 2011013 to Entergy Nuclear Operations, Inc (ENO). The inspection report identified a preliminary finding defined as an Apparent Violation (AV) of 10 CFR Part 50 Appendix B, Criterion V, "Instructions, Procedures and Drawings." The AV occurred at the Palisades Nuclear Plant (PNP), and is related to the failure to follow procedures for lubrication of linkages on the turbine driven auxiliary feedwater (TDAFW) pump, P-8B, overspeed trip device.
The inspection report provided ENO the option to attend a Regulatory Conference, or submit the ENO position on the finding, in writing, within 30 days. In addition, the letter-required a 10-day response to notify the NRC of the intended response.
On November 7, 2011, ENO submitted the required 10-day response to notify the NRC that a 30-day written response would be submitted to provide the ENO position on the finding. The 30-day response is provided in Attachment 1.

Document Control Desk PNP 2011-080 Page 2 ENO recognizes that a performance deficiency occurred, and does not dispute the AV.
However, ENO has assessed the safety significance, and has concluded that the finding is of very low safety significance. The very low safety significance is based upon additional plant-specific factors beyond those included in the NRC assessment.
ENO did not identify any single cause for the trip of the TDAFW pump. A root cause analysis (RCA) concluded that the preparation and execution of the overhaul work on the TDAFW pump turbine during the 2010 refueling outage did not maintain the turbine and turbine governor in a way that ensured overspeed trip reliability. The Failure Modes Analysis, performed as part of the RCA, considered many different failure modes.
However, no single failure mode could be identified that would have caused the TDAFW pump to trip.
On behalf of ENO, MPR Associates, Inc. performed a calculation of the overspeed trip mechanism linkage forces. The calculation concluded that the geometry of the linkage is such that large accelerations are needed to cause it to trip, the actual measured accelerations are small relative to those required to cause a trip, and the measured displacements of the linkage, on the order of 0.007 inches, are too small to move the hand trip lever the approximate 0.125 inches required to cause it to trip. The calculation is provided in Attachment 2.
ENO's safety significance assessment notes that the failure of pump P-8B was determined to be a spurious trip. Pump performance from 2005 to the present has resulted in no additional failures other than that observed on May 10, 2011. Using this information, sensitivity calculations show that the delta core damage frequency (ACDF) is less than 1E-06/yr. This ACDF for a spurious trip results in a significance determination of very low safety significance. The risk assessment supporting this conclusion is provided in Attachment 3.
ENO requests that the NRC re-evaluate the safety significance of this preliminary finding considering the information provided in the attachments to this letter.
This letter contains no new commitments and no revisions to existing commitments.
Sincerely, AJV/bed

Document Control Desk PNP 2011-080 Page 3 Attachments: 1. Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013
: 2. Calculation of Overspeed Trip Mechanism Linkage Forces
: 3. Risk Assessment of Steam Driven Auxiliary Feedwater P-8B Trip CC Administrator, Region III, USNRC Project Manager, Palisades, USNRC Resident Inspector, Palisades, USNRC

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 Abstract During Nuclear Regulatory Commission (NRC) review of a failure of a turbine driven auxiliary feedwater (TDAFW) pump at the Palisades Nuclear Plant (PNP),
the inspectors identified one apparent violation (AV) with a preliminary significance of white. The AV is related to the failure to follow procedures for lubrication of linkages on the TDAFW pump overspeed trip device. Specifically, during TDAFW maintenance, the knife edge on the trip mechanism was greased.
The greasing of the knife edge contributed to a trip of the pump on May 10, 2011, as well as rendering the pump inoperable for a period of time in excess of what is allowed by Technical Specifications.
Entergy Nuclear Operations, Inc. (ENO) recognizes that a performance deficiency occurred, and does not dispute the AV. However, ENO has assessed the safety significance, and has concluded that the finding is of very low safety significance. The very low safety significance is based upon additional plant-specific factors beyond those included in the NRC assessment. The additional information is described below.

===Background===
The PNP auxiliary feedwater (AFW) system is designed to provide a supply of water to the steam generators during start-up operations and to remove primary system sensible and decay heat during initial stages of shutdown operations.
The AFW system is also used to remove decay heat during emergency shutdown operations.
The AFW system consists of two independent motor-driven pumps (P-8A and P-8C) and a steam turbine-driven AFW pump (P-8B). Any one of the three pumps can provide 100% of the required feedwater flow to both steam generators. The piping configuration allows each AFW pump to supply both steam generators simultaneously. P-8B will auto-start if P-8A and P-8C fail to start.
As part of surveillance testing, valve alignments are made to recirculate the water in the condensate storage tank or use the path directly to the steam generators. These tests demonstrate that the AFW system can perform its function.
In October 2010, during a refueling outage, several maintenance activities were performed on the turbine-driver, and the associated mechanical overspeed trip linkage, of P-8B. The work was performed by supplemental personnel with ENO oversight. The turbine rotor and the overspeed trip pin were replaced as part of this work. After this maintenance, RO-1 45B, "AFW Comprehensive Pump Test,"
was completed successfully.
Page 1 of 7

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 Technical Specification Surveillance Test, QO-21 B, "P-8B IST Auxiliary Feedwater System," was successfully completed on February 15, 2011.
I1l. Event Description On May 10, 2011, during performance of surveillance procedure RO-97, "Auxiliary Feedwater System Automatic Initiation Test Procedure," to test auxiliary feedwater system automatic initiation, P-8B tripped due to the unexpected actuation of the mechanical overspeed trip linkage. An examination revealed that the knife edge of the trip resetting lever had disengaged from the hand trip lever latch allowing the trip valve assembly to isolate steam flow to the turbine driver.
IV. Corrective Actions An apparent cause evaluation (ACE) was initiated. As part of the ACE, a Kepner Tregoe (KT) problem analysis was performed. In parallel, on May 11, 2011, troubleshooting of the pump was performed, including the performance of QO-21 B, "P-8B IST Auxiliary Feedwater System." The test was completed satisfactorily, prior to the KT problem analysis being completed. During the troubleshooting, the TDAFW pump trip could not be replicated.
The KT problem analysis concluded that during the 2010 refueling outage, the overspeed trip device was greased at the point where the trip lever comes into contact with the trip actuator. This area is not normally greased. The ACE determined that the greasing occurred due to an unapproved revision to the procedure by a supplemental worker, who changed the procedure verbiage from "pin" to "knife edge." The supplemental worker perceived the procedure to be incorrect, and chose to make the procedure change without bringing the issue to the attention of a supervisor, thus bypassing the proper procedure change process. Based on the information available at the time the ACE was performed, the greasing of the overspeed trip device was considered to be the sole cause of the P-8B trip.
The grease was cleaned from the knife edge, and Technical Specification Surveillance RO-97, "Auxiliary Feedwater System Automatic Initiation Test Procedure," was performed satisfactorily, and the pump was declared operable.
A MPR Associates, Inc. (MPR) calculation of overspeed trip mechanism linkage forces was performed on behalf of ENO (ENO Calculation No. EA-EC31177-01).
The calculation concluded that the greasing alone would not account for the spurious pump trip. Based on that conclusion, additional investigations were performed. On October 25, 2011, P-8B was removed from service to evaluate the possible causes from the ACE failure modes analysis, as well as possible Page 2 of 7

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 causes identified in Elliott Service Bulletin ETSL-T-2012, Revision 0. During this investigation, field data was obtained for input to the MPR calculation. After incorporating the field data into the MPR calculation in November 2011, it was concluded that no additional failure mechanisms, either singularly, or in combination with the greasing, would be of sufficient magnitude to cause a failure of the overspeed trip device.
Based on these findings, a root cause analysis (RCA) was performed. The RCA did not identify any single cause for the trip of the TDAFW pump. The RCA concluded that the preparation and execution of the overhaul work on the TDAFW pump turbine during the 2010 refueling outage did not maintain the turbine and turbine governor in a way that ensured overspeed trip reliability. The Failure Modes Analysis, performed as part of the RCA, considered many different failure modes. However, no single failure mode could be identified that would have caused the TDAFW pump to trip. Corrective actions specific to the performance deficiency are as follows:
    " Grease was cleaned from knife edge and the trip linkage was checked on May 11, 2011.
* Associated steam traps were cleaned on May 12, 2011.
    " The knife edge was rotated and the latch plate was replaced; the reset lever springs were replaced; the governor was verified to contain no foreign material; the trip mechanism backseat was adjusted, and the overspeed gap was verified to be correct; the overspeed trip was set to 3850 rpm per T-1 86, "Auxiliary Feedwater turbine K-8 Overspeed Trip Test," on October 26, 2011.
* ENO plans to revise FWS-M-6, "AFW Turbine Maintenance," to incorporate recommendations from the Elliott Technical Service Letter ETSL-2012, industry experience, and the results of the RCA. This includes identification of critical steps, and checks of linkage and trip pin clearances with as-found data being recorded for review. This also includes requirements for periodic replacement of all trip mechanism springs and latch parts as well as the trip pin and plunger. In addition, ENO plans to review all other aspects beyond those related to overspeed trip for completeness and correct detail for use by supplemental workers.
Finally, the post-maintenance testing requirements are being revised to remove "consider" and "should." Those terms are being replaced with requirements for conditional testing, to ensure that ENO procedural requirements for post-maintenance testing are incorporated.
    " ENO plans to complete challenge boards prior to the established spring 2012 refueling outage milestone for all maintenance superintendents and project managers to verify that critical maintenance activities have been identified, planned and scheduled. These activities will require direct ENO oversight of supplemental workers in accordance with EN-MA-1 01-01, Page 3 of 7

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 "Critical Maintenance Identification and Oversight." EN-MA-101-01 applies to work being performed by supplemental workers/contractors.
       " Verification is planned to ensure that the common cause evaluations performed for condition report CR-PLP-2011-5099 have adequately addressed supplemental worker performance during the fall 2010 refueling outage. Additional actions will be initiated, if necessary, based on the verification. (Note: CR-PLP-2011-5099 was initiated to evaluate the quality of maintenance activities associated with multiple equipment performance issues identified resulting from work performed during the fall 2010 refueling outage).
* A review of high critical maintenance activities performed during the fall 2010 refueling outage is being conducted to verify that requirements for critical maintenance were met.
       " ENO plans to verify that maintenance supervisor and project manager job familiarization guides (JFGs) include critical maintenance control. If required, JFGs will be revised to include these requirements.
       " ENO plans to provide information sharing to maintenance supervisors and project managers concerning control of critical maintenance requirements and applications.
In addition, a performance recovery plan was developed to address underlying issues from this event, as well as from other recent events at PNP. The performance recovery plan is discussed below.
V. Palisades Performance Recovery Plan In order to address a negative trend in human performance, procedure use and adherence, as well as other issues at PNP, ENO has developed a performance recovery plan. The plan is a living document in which many actions have been completed, many actions remain, and actions will continue to be added, as warranted.
The recovery plan addresses the following areas:
: 1.      Leadership Effectiveness Problem Statement: Leaders are not sufficiently engaged and intrusive to identify and correct behavior and performance gaps at all levels of the organization.
Vision Statement: Leaders are engaged and intrusive to identify and correct behavior and performance gaps. Leaders routinely spend time in the field with eyes on the problems to provide oversight and reinforce standards. Leaders continually reinforce the principles for a strong nuclear safety culture.
Page 4 of 7

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013
: 2. Safety Culture Problem Statement: Degradation of safety culture principles has led to site performance deficiencies.
Vision Statement: All station personnel exhibit proper risk sensitivity and awareness. Mitigating strategies are consistently implemented.
: 3. Corrective Action Program Problem Statement: The corrective action program (CAP) is not always valued or effectively used by PNP leadership and personnel to identify, investigate, and correct problems in a timely manner to improve performance.
Vision Statement: Success in the implementation of a strong corrective action program is evidenced by PNP personnel identifying and correcting issues at low levels.
: 4. Equipment Reliability Problem Statement: Critical equipment failures have led to plant derates, forced outages, schedule perturbations, increased radiological dose, unplanned limiting conditions for operation (LCO), rework and failed post maintenance tests.
Vision Statement: Success in equipment reliability is evidenced by PNP achieving top decile performance in unit capability factor (currently
       >94.38%), low rework rates, lower collective radiation exposure and no unplanned LCOs.
: 5. Refueling Outages Problem Statement: Ownership of the outage preparation and execution process must be improved to support station operation and maintenance.
Work must be scheduled where appropriate, planned effectively, and completed when scheduled.
Vision Statement: Provide improvement in the implementation of the outage management process at PNP. Expected results are:
* Work package quality is in accordance with the ENO process.
* Problems in the field are quickly communicated.
Page 5 of 7

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 All activities are started and completed per the approved outage schedule.
* All walkdowns are completed per the ENO standard and process.
* All owners of outage milestones are accountable for their performance.
The Outage Accountability Model is driving outage readiness and ownership to the right level.
The site understands the reasons for decisions such that alignment with all workers is quickly achieved.
The outage is completed safely and error-free, on schedule, and on budget.
: 6.      Communication Plan Problem Statement: Communicate effectively to employees in order to ensure engagement in the PNP recovery.
Vision Statement: ENO employees are well informed of the causes and contributors to the current performance issues, as well as the corrective actions necessary to drive recovery. Additionally, employees are engaged and empowered in the execution of the plan. Employee ownership is evident.
A complete copy of the PNP performance recovery plan has been provided to the site NRC Resident Inspectors, as well as NRC Region III personnel.
VI. MPR Analysis MPR performed a calculation, on behalf of ENO, of the overspeed trip mechanism linkage forces. The calculation concluded that the geometry of the linkage is such that large accelerations are needed to cause it to trip, the actual measured accelerations are small relative to those required to cause a trip, and the measured displacements on the order of 0.007 inches are too small to move the hand trip lever the approximate 0.125 inches required to cause it to trip. The calculation is provided in Attachment 2.
VII. Safety Significance Assessment ENO's safety significance assessment notes that the failure of pump P-8B was determined to be a spurious trip. Pump performance from 2005 to the present, resulted in no additional failures other than that observed on May 10, 2011.
Using this information, sensitivity calculations show that the delta core damage frequency (ACDF) is less than 1 E-06/yr. This ACDF for a spurious trip results in a significance determination of very low safety significance. The risk assessment is provided in Attachment 3.
Page 6 of 7

Attachment 1 Reply to Apparent Violation EA-1 1-227 in Inspection Report 2011013 VIII. Conclusions Based on calculations, ENO has determined that the ACDF is less than 1E-06/yr for a spurious trip of TDAFW pump P-8B; therefore, this event is of very low safety significance.
Page 7 of 7}}