IR 05000348/2007009
| ML072200446 | |
| Person / Time | |
|---|---|
| Site: | Farley  |
| Issue date: | 08/02/2007 |
| From: | James Shea Division of Reactor Safety II |
| To: | Jerrica Johnson Southern Nuclear Operating Co |
| References | |
| EA-07-173, FOIA/PA-2008-0016 IR-07-009 | |
| Download: ML072200446 (27) | |
Text
ust 2, 2007
SUBJECT:
JOSEPH M. FARLEY NUCLEAR PLANT - NRC SPECIAL INSPECTION REPORT 05000348/2007009 AND 05000364/2007009; PRELIMINARY YELLOW FINDING
Dear Mr. Johnson:
On May 4, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed the onsite portion of a special inspection at your Joseph M. Farley Nuclear Plant. The inspection reviewed the circumstances surrounding the Unit 2 residual heat removal (RHR) containment sump suction valve failures during testing on April 29, 2006, and again on January 5, 2007. A special inspection was warranted based on the risk and deterministic criteria specified in Management Directive 8.3, NRC Incident Investigation Program. The determination that the inspection would be conducted was made by the NRC on April 25, 2007, and the inspection started on April 30, 2007. The preliminary inspection results were discussed with you and members of your staff on May 4, 2007. Subsequently, additional in-office reviews were conducted and the enclosed inspection report documents the final inspection results and preliminary significance determination which were discussed by telephone with you and members of your staff on July 3, 2007.
This inspection was performed in accordance with Inspection Procedure 93812, Special Inspection, and focused on the areas discussed in the inspection charter described in the report. The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your licenses. The team reviewed selected procedures and records, conducted field walkdowns, observed activities, and interviewed personnel.
The report documents one NRC-identified finding involving failure to promptly identify and correct a condition adverse to quality which resulted in the Unit 2 RHR train A containment sump suction valve failing to stroke full open on April 29, 2006, and January 5, 2007. This finding was assessed based on the best available information, including influential assumptions, using the applicable Significance Determination Process (SDP) and was preliminarily determined to be a Yellow finding (i.e., having substantial safety significance). This finding was also characterized as an Apparent Violation (AV) of 10 CFR Part 50, Appendix B, Criterion XVI ATTACHMENT 2 SNC 2 ATTACHMENT 2 requirements and is being considered for escalated enforcement action in accordance with the NRC Enforcement Policy. The current Enforcement Policy is included on the NRCs Web site at www.nrc.gov/about-nrc/regulatory/enforcement/enforce-pol.html.
The NRC determined that this condition does not present a current immediate safety concern because, following the valve failure on January 5, 2007, your staff completed corrective actions during the spring 2007 Unit 2 refueling outage to ensure valve operability.
Additionally, because equivalent RHR and containment spray sump suction valves for Unit 1 are located in encapsulations similar to the Unit 2 valves, and the humidity conditions for the Unit 1 encapsulated valves are similar, the inspectors determined that the Unit 1 encapsulated valves may be susceptible to the type of failure experienced on the Unit 2 valve. Accordingly, you are requested to provide the NRC with information regarding your evaluation of the effect of the condition on Unit 1 equipment.
Before we make a final decision on this matter, we are providing you an opportunity (1) to present to the NRC your perspectives on the facts and assumptions used by the NRC to arrive at the finding and its significance at a Regulatory Conference or (2) submit your position on the finding to the NRC in writing. If you request a Regulatory Conference, it should be held within 30 days of the receipt of this letter and we encourage you to submit supporting documentation at least one week prior to the conference in an effort to make the conference more efficient and effective. If a Regulatory Conference is held, it will be open for public observation and the NRC will issue a press release to announce the conference. If you decide to submit only a written response, such submittal should be sent to the NRC within 30 days of the receipt of this letter.
Please contact Mr. D. Charles Payne at (404) 562-4669 within seven days of the date of this letter to notify the NRC of your intentions regarding the regulatory conference for the preliminary Yellow finding. If we have not heard from you within 10 days, we will continue with our significance determination and associated enforcement processes on this finding, and you will be advised by separate correspondence of the results of our deliberations on this matter.
Because the NRC has not made a final determination in this matter, no Notice of Violation is being issued for the inspection finding at this time. Additionally, please be advised that the number and characterization of any apparent violations may change as a result of further NRC review.
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, portions of its enclosure and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS)
component of NRCs document system (ADAMS). However, the NRC is continuing to review the appropriate classification of the SDP Phase 3 Risk Analysis (Attachment 2) within our records management program, considering changes in our practices following the events of September 11, 2001. Attachment 2 contains Official Use Only - Security Related Information and its disclosure to unauthorized individuals could present a security vulnerability. Therefore, ATTACHMENT 2 SNC 3 ATTACHMENT 2 in accordance with 10 CFR 2.390(d)(1), Attachment 2 will not be made available electronically for public inspection in the NRC Public Document Room or from the NRCs document system (ADAMS) accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html.
Sincerely,
/RA: Kriss Kennedy for/
Joseph W. Shea, Director Division of Reactor Safety Docket Nos.: 50-348, 50-364 License Nos.: NPF-2, NPF-8
Enclosure:
Inspection Report 05000348/2007009 and 05000364/2007009 w/Attachments 1. Supplemental Information 2. Phase 3 SDP Risk Analysis (Official Use Only - Security-Related Information)
REGION II==
Docket Nos.: 50-348, 50-364 License Nos.: NPF-2, NPF-8 Report Nos.: 05000348/2007009 and 05000364/2007009 Licensee: Southern Nuclear Operating Company, Inc.
Facility: Joseph M. Farley Nuclear Plant Location: Columbia, AL 36319 Dates: April 30 - July 3, 2007 Inspectors: N. Merriweather, Senior Reactor Inspector (In-office Review)
P. OBryan, Senior Resident Inspector - Harris Nuclear Plant M. Thomas, Senior Reactor Inspector, Team Lead Accompanying Personnel: G. Gardner, Reactor Inspector Approved by: D. Charles Payne, Chief Engineering Branch 2 Division of Reactor Safety ATTACHMENT 2 Enclosure
ATTACHMENT 2
SUMMARY OF FINDINGS
IR 05000348/2007-009, 05000364/2007-009; 04/30/2007 - 07/03/2007; Joseph M. Farley
Nuclear Plant, Units 1 and 2; Special Inspection.
This report documents special inspection activities performed onsite and in the Region II office by two senior reactor inspectors and a senior resident inspector to review the Farley Unit 2 residual heat removal (RHR) train A containment sump suction valve failure to stroke fully open on April 29, 2006, and again on January 5, 2007. The inspectors identified one preliminary Yellow finding which was determined to be an apparent violation. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609, Significance Determination Process (SDP). The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649,
Reactor Oversight Process, Revision 4, dated December 2006.
NRC-Identified and Self-Revealing Findings
Cornerstone: Mitigating Systems
- TBD. The inspectors identified an apparent violation (AV) of 10 CFR 50,
Appendix B, Criterion XVI, Corrective Action, for the licensees failure to promptly identify and correct a condition adverse to quality (CAQ) which resulted in a Unit residual heat removal (RHR) containment sump suction valve failing to stroke full open during testing on April 29, 2006, and again on January 5, 2007. The licensee did not take corrective actions to address the high humidity condition inside the valve encapsulation which caused rust/corrosion accumulation on valve components and adversely impacted valve performance. After the valve failure on January 5, 2007, the licensee implemented interim corrective actions to support valve operability until long-term corrective actions were completed.
This finding is more than minor because failure of a RHR containment sump suction valve to fully open impacts long-term core decay heat removal (emergency core cooling system sump recirculation) and therefore, affects the mitigating systems cornerstone to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Phase 1 and Phase 2 significance determination process worksheets from NRC Inspection Manual Chapter 0609, the finding was determined to have potential safety significance greater than
- Green.
A regional Senior Reactor Analyst, with peer review from other qualified regional and headquarters personnel, performed a Phase 3 significance determination with a preliminary result of substantial safety significance. This finding has a cross-cutting aspect in the area of problem identification and resolution because the licensee failed to thoroughly evaluate the condition adverse to quality such that the resolution addressed the cause. (Section 4OA5.02.b)
ATTACHMENT 2 3 ATTACHMENT 2
Licensee-Identified Violations
None.
ATTACHMENT 2 ATTACHMENT 2
REPORT DETAILS
Summary of Plant Events On April 29, 2006, Unit 2 RHR motor operated valve (MOV) Q2E11MOV8811A (containment sump suction to RHR pump 2A) did not fully open during a quarterly surveillance test. The valve was reclosed and, based on the closing stroke time, the licensee determined that the valve had opened approximately 20%. The torque switch for the MOV was bypassed for the first 20% of valve travel in the open direction. The licensee cycled the valve two additional times. The valve stopped in the intermediate position on the second attempt, but successfully stroked full open on the third attempt. This same failure sequence (i.e., failure to stroke full open on the first two attempts and successfully stroking full open on the third attempt) repeated itself on January 5, 2007. The licensee attributed the cause of the April 29, 2006, failure to dirty contacts of the open direction torque switch. It was believed that exercising the torque switch contacts cleaned them which resulted in a successful valve opening on the third attempt.
Because the valve was encapsulated, the licensee decided to not open the encapsulation at power and the valve was not subjected to any further inspection or troubleshooting at that time.
After the January 5, 2007, failure, the licensee opened the valve encapsulation and found that the open torque switch contact internal spring and the contact support contained rust/corrosion.
The licensees subsequent root cause evaluation concluded that the cause of both valve failures was most likely the result of interrupted continuity of the open torque switch contact due to rust/corrosion accumulation, causing the retaining spring to hang up and the contacts to not come together firmly. The rust/corrosion accumulation was due to a high humidity environment inside the valve encapsulation. The inspectors noted that there were condition reports (CRs)dating back to 2001 which documented the high humidity environment inside the valve encapsulation and the rust/corrosion accumulation on valve components. However, there was no evidence that the licensee took corrective actions to address the high humidity and rust/corrosion issues. The high humidity and rust/corrosion conditions resulted in the failures of the RHR containment sump suction valve during testing on April 29, 2006, and again on January 5, 2007.
Inspection Scope Based on the deterministic and conditional risk criteria specified in Management Directive 8.3, NRC Incident Investigation Program, a Special Inspection was initiated in accordance with NRC Inspection Procedure 93812, Special Inspection Team. The inspection focus areas included the following special inspection charter items:
1. Develop a complete description of the RHR containment sump recirculation isolation
valve failure on January 5, 2007, and a complete sequence of events related to the valve failure. The sequence of events should also include a time line of previous RHR containment sump recirculation isolation valve failures and the circumstances surrounding the failures.
ATTACHMENT 2 5 ATTACHMENT 2 2.
Review licensee documents to determine the environmental qualification (EQ) status for the valves or if historical conditions warrant EQ for either routine or accident conditions.
Similar reviews should be performed to determine the design/licensing basis for the valve encapsulation enclosure and determine any adverse affect of the encapsulation enclosure on the safety-related RHR and containment spray (CS) sump recirculation isolation valves.
3. Review the maintenance program for the encapsulated RHR and CS sump recirculation
isolation valves.
4. Review the corrective actions (CAs) and maintenance work order databases to
determine the failure history of the encapsulated valves for both units RHR and CS systems.
5. Review the licensees root cause analysis and extent of condition. Assess the adequacy
of the licensees implemented and/or planned CAs to address the root cause and the time line for completing the CAs on both units.
6. Assess if any common failure modes have been established, whether they are being
addressed by the licensee, and what generic implications may exist.
7. Assess the adequacy of the licensees operability determination for the sump
recirculation valves on Unit 1 for both the RHR and CS systems.
operating experience items.
9. Collect data necessary to develop and assess the safety significance of any findings in
accordance with IMC 0609, Significance Determination Process.
10. Identify any potential generic safety issues and make recommendations for appropriate follow-up actions (e.g., Information Notices, Generic Letters, Bulletins).
OTHER ACTIVITIES
4OA5 Other Activities - Special Inspection
.01 Description of valve Q2E11MOV8811A failure to fully open on January 5, 2007, and a
complete sequence of events/timeline to include previous failures (Charter Item 1)
Background Information Valve Q2E11MOV8811A is the Unit 2 containment sump suction MOV closest to containment for the RHR pump 2A. The valve is enclosed in a steel encapsulation vessel located outside containment. The encapsulation vessel was bolted together and ATTACHMENT 2 6 ATTACHMENT 2 was designed to contain any stem packing or body-to-bonnet leakage that may occur during RHR operation in long term recirculation from the containment sump during post-accident conditions. The four sump suction MOVs closest to containment for each unit (one each for the A and B trains of RHR and the A and B trains of CS) are encapsulated. The encapsulations are routinely opened during refueling outages to perform MOV limit switch inspections and lubrication, and at 5-year intervals to inspect valve components and perform various preventive maintenance (PM) activities. The valves are stroked open and timed during quarterly surveillance testing to demonstrate valve operability between refueling outages. The team reviewed licensee work orders (WO) and CRs to develop the following time line summarizing the events leading to the April 29, 2006, and January 5, 2007, test failures of valve Q2E11MOV8811A.
Time Line February 23, 1987 Unit 2 valve Q2E11MOV8811B (containment sump suction to RHR pump 2B) would not stroke open. Operations continued attempts to stroke the valve and the valve finally opened. No work was done. This issue was documented in WO 144961.
March 23, 1987 Five stroke attempts were required for Unit 2 valve Q2E11MOV8811B to fully stroke open. Initiated WO 154003 which measured the valve stroke, checked the torque switch, and set the closed bypass limit.
April 1987 to March 2001 There were no documented test failures associated with the RHR and CS encapsulated valves during this time period.
March 20, 2001 During a refueling outage, Unit 2 valve Q2E11MOV8811A was re-packed. Part of the maintenance procedure had personnel open and close the valve using the manual hand wheel. While performing post-maintenance testing, the valve would not operate electrically. The problem was determined to be that the manual declutch lever had rusted to the spring cartridge cap and would not allow free movement of the declutch lever. The condition was corrected by cleaning the components. The valve was tested satisfactorily and returned to service. This was documented in CR 2001000682.
ATTACHMENT 2 7 ATTACHMENT 2 March 2001 to March 2003 There were no documented test failures associated with the RHR and CS encapsulated valves during this time period.
March 11, 2003 During routine surveillance testing, Unit 1 valve Q1E11MOV8811A (containment sump suction to RHR pump 1A) would not stroke open. The contacts on the valve hand switch in the control room were cleaned, but this did not fix the problem. The valve was stroked by momentarily bypassing the limit switch in the motor control center. By stroking the valve, it was believed that the limit switch contacts subsequently swiped off any dirt or oxidation present. Valve Q1E11MOV8811A was stroked open successfully and returned to service. This was documented in CR 2003000510 and WO 3001467.
March 2003 to November 2005 Unit 2 valve Q2E11MOV8811A passed all surveillance tests during this period.
November 11, 2005 The licensee performed the five-year preventive maintenance on Unit 2 valve Q2E11MOV8811A which required the encapsulation to be opened for inspection of the valve limit switch contacts, lubrication of the limit switches, and full diagnostic testing.
This was documented in WO 2051121601. This was the last time the encapsulation was opened prior to January 2007.
February 3, 2006 Unit 2 valve Q2E11MOV8811A passed the quarterly surveillance test STP-11.6. This was the last successful test before the April 29, 2006, failure.
April 29, 2006 Unit 2 valve Q2E11MOV8811A did not stroke full open on the first two attempts during performance of quarterly surveillance test STP-11.6. On the third attempt, the valve operated properly. The cause was suspected to be dirty torque switch contacts which were believed to have been cleaned by cycling the valve. No further investigation was done at that time. Part of the CR resolution was to increase the frequency of the surveillance stroke testing. The licensee commenced planning to replace the torque switch during the spring 2007 Unit 2 refueling outage. This was documented in CR 2006104125.
ATTACHMENT 2 8 ATTACHMENT 2 April 30, 2006 to May 21, 2006 Unit 2 valve Q2E11MOV8811A stroke tested satisfactory three times. Documented in STP-11.6.
May 22, 2006 Initiated CR 2006104961 (follow on to CR 2006104125) which documented the decision to test the valve on about June 21, 2006, and then return to quarterly testing in July 2006.
July 22, 2006 Unit 2 valve Q2E11MOV8811A stroke tested satisfactory. Documented in STP-11.6.
October 5, 2006 Documented in the final close out of CR 2006104961 were proposed corrective actions to replace the torque switches on all the encapsulated valves on both units during each units next refueling outage.
October 13, 2006 Unit 2 valve Q2E11MOV8811A stroke tested satisfactory. Documented in STP-11.6.
This was the last successful test before the January 5, 2007, failure.
January 5, 2007 Unit 2 valve Q2E11MOV8811A did not stroke full open on the first two attempts during routine surveillance test STP-11.6. The valve did stroke full open on the third attempt.
This was documented in CR 2007100142 and a root cause team was formed to evaluate this repeat failure. The licensee opened the encapsulation and found the encapsulation gasket was missing, rust/corrosion was found on the torque switch open contacts, moisture was found inside the torque switch and limit switch compartment, and water was found in the encapsulation. Work orders 207012851 & 2070129101 and CR action items 2007200538 through 2007200555 were generated. The licensee implemented a temporary modification to bypass the open torque switch. This modification would allow the valve to travel nearly full open if similar failures of the open torque switch were to occur in the future due to corrosion issues. Additionally, the licensee increased the surveillance test frequency for the valve.
ATTACHMENT 2 9 ATTACHMENT 2
.02 Determine the EQ status of the valves and the design/licensing basis for the valve
encapsulation (Charter Item 2)
a. Inspection Scope
The inspectors reviewed selected sections of the FNP Updated Final Safety Analysis Report (UFSAR) and EQ Documentation Package 0023C to determine the EQ status of the encapsulated RHR containment sump suction valves and associated components (e.g., actuator, motor, etc.). Additionally, the inspectors reviewed the RHR functional system description A181002 and engineering evaluation DOEJ-SM-2070129101-003 to determine the design/licensing basis for the valve encapsulation.
b. Findings
From review of the UFSAR and EQ documentation, the inspectors determined that the Unit 2 RHR containment sump suction valves and associated components (e.g.,
actuator, motor, etc.) were qualified for the expected post-accident environmental conditions. The UFSAR and EQ documentation stated that the maximum relative humidity expected in areas outside containment during normal plant operation was 50%.
The inspectors concluded that the normal operating environmental condition for the encapsulated RHR valves exceeded the 50% relative humidity design/licensing basis value specified in the UFSAR and EQ documentation, as evidenced by water collection and rust/corrosion accumulation on valve components inside the encapsulations. The licensees root cause evaluation concluded that the rust/corrosion on Unit 2 valve Q2E11MOV8811A components was due to the chronic high humidity environment inside the valve encapsulation. The inspectors noted that CRs dating back to 2001 documented the high humidity condition and rust/corrosion accumulation on valve components inside the valve encapsulation.
During discussions with the inspectors, licensee personnel stated that there was a longstanding belief that the valve encapsulations were part of the primary containment boundary and the policy was do not remove the encapsulations during power operation.
The licensees corporate engineering support group sent a letter to the plant (Letter PS-05-0622 dated April 1, 2005) which stated that the General Design Criteria did not allow the permanent removal of the encapsulations without additional modifications to collect valve stem or body-to-bonnet leakage. The inspectors determined that the design/licensing basis of the valve encapsulations was to contain stem packing or body-to-bonnet leakage that may occur during RHR operation in recirculation from the containment sump during post-accident conditions. The encapsulations were located outside containment and were not credited as part of the containment boundary.
ATTACHMENT 2 10 ATTACHMENT 2 Failure to Identify and Correct a Condition Adverse to Quality
Introduction:
The inspectors identified a preliminary Yellow finding involving an apparent violation (AV) of 10 CFR 50, Appendix B, Criterion XVI, Corrective Action, for failure to promptly identify and correct a condition adverse to quality (CAQ) which resulted in Unit 2 valve Q2E11MOV8811A failing to stroke full open during testing on April 29, 2006, and again on January 5, 2007.
Description:
On April 29, 2006, Unit 2 valve Q2E11MOV8811A did not fully open during a routine quarterly surveillance test. The valve stopped in an intermediate position, showing dual indications on the main control board. The valve was reclosed and, based on the closing stroke time, the licensee determined that the valve traveled approximately 20% open. The valve torque switch was bypassed for the first 20% of valve travel in the open direction. The licensee cycled the valve two additional times. The valve stopped in the intermediate position on the second attempt, but successfully stroked full open on the third attempt. The valve failure was documented in CR 2006104125. The licensee attributed the cause of the failure to dirty contacts of the open direction torque switch. It was believed that exercising the torque switch contacts cleaned them which resulted in a successful valve opening stroke on the third attempt. No substantive effort was made to determine the cause of the valve failure. Part of the CR resolution was to increase the frequency of the surveillance stroke testing. Because the valve was encapsulated, the licensee made the decision to not open the encapsulation at power and the valve was not subjected to any further inspection or troubleshooting at that time. The licensee commenced planning to replace the torque switch during the spring 2007 Unit 2 refueling outage. The licensee decided to return to quarterly surveillance testing after completing several successful tests during the increased test frequency in April - June 2006. The valve performed successfully during quarterly surveillance testing on July 22, 2006, and October 13, 2006.
On January 5, 2007, the same failure scenario as that of April 29, 2006, repeated itself during quarterly surveillance testing of valve Q2E11MOV8811A (i.e., failure to stroke full open on the first two attempts and successfully stroking full open on the third attempt).
The licensee documented the failure in CR 2007100142 and formed a root cause team to investigate this repeat failure. The licensee opened the encapsulation and found that the encapsulation gasket was missing and water was in the encapsulation.
Rust/corrosion was also found on the torque switch open contacts and moisture was inside the torque switch and limit switch compartment. The licensees subsequent root cause evaluation concluded that the cause of both valve failures was most likely the result of interrupted continuity of the open torque switch contacts due to corrosion buildup, causing the retaining spring to hang up and the contacts to not come together firmly. The rust/corrosion on the valve components was due to a chronic high humidity environment in the valve encapsulation. The licensee implemented a temporary modification to bypass the valves open torque switch until long term corrective actions were completed during the spring 2007 Unit 2 refueling outage. The licensee also increased the surveillance test frequency of the valve.
ATTACHMENT 2 11 ATTACHMENT 2 The licensee performed an initial past operability assessment of Unit 2 encapsulated valve Q2E11MOV8811A (based on a valve stroke of approximately 20% open), which concluded that the system would have remained functional with the valve 20% open.
During review of the operability assessment, the inspectors questioned the containment overpressure input used to determine the available net positive suction head (NPSH) for the 2A RHR pump. As a result, the licensee revised the operability assessment. The inspectors reviewed the revised operability assessment and determined that the licensee was not able to demonstrate that there was adequate NPSH available to ensure operability of the 2A RHR pump with valve Q2E11MOV8811A approximately 20% open during post-accident containment sump recirculation operation. The licensee concluded that the three remaining Unit 2 encapsulated RHR and CS containment sump suction valves were operable, based on the valves having been successfully stroked full open during quarterly surveillance testing.
Analysis:
The licensees failure to promptly identify and correct a CAQ which resulted in Unit 2 valve Q2E11MOV8811A not stroking full open during testing on April 29, 2006, and again on January 5, 2007, is a performance deficiency. This finding is more than minor because failure of the valve to fully open impacts long-term core decay heat removal (emergency core cooling system sump recirculation operation) and therefore, affects the reactor safety mitigating systems cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. This finding was evaluated in accordance with NRC Inspection Manual Chapter 0609, Appendix A, Determining the Significance of Reactor Inspection Findings for At-Power Situations. Using the Phase 1 and Phase 2 significance determination process (SDP) worksheets, the finding was determined to have potential safety significance greater than Green.
A regional Senior Reactor Analyst, with peer review from other qualified regional and headquarters personnel, performed a Phase 3 significance determination with a preliminary result of substantial safety significance (Yellow). The analysis used the licensees Probabilistic Risk Assessment (PRA) model with the basic event for valve Q2E11MOV8811A (RHR-MOV-CC-8811A) set to always fail. The exposure time was set at 851/2 days (Note: An exposure time of one-half of the time period (T/2) since the last successful demonstration of the valve function was used because the inception of the condition was unknown. Valve Q2E11MOV8811A passed its quarterly surveillance test on February 3, 2006, failed its next test on April 29, 2006, and was returned to service on April 29, 2006. T/2 for this period was 85 days ÷ 2 = 421/2 days. The valve passed its quarterly surveillance test on October 13, 2006, failed its next test on January 5, 2007, and was returned to service on January 7, 2007. T/2 for this period was 86 days ÷ 2 = 43 days. Total exposure time was 421/2 days + 43 days = 851/2 days).
Because the MOV was encapsulated, remote manual operation was not credible. Due to the possible common cause failure mechanism of corrosion for encapsulated valves, a new common cause term for valves Q2E11MOV8811A&B was developed and substituted in the model. Given that the plant could, with equal probability, be in one of two different configurations associated with either A or B Service Water/Component ATTACHMENT 2 12 ATTACHMENT 2 Cooling trains in service at the beginning of a postulated accident, two different numerical core damage frequency (CDF) results were generated. By appropriately partitioning and summing the A and B train on-service contributions with the common cause contribution, a total delta CDF for the exposure time from internal events of approximately 3E-5 was generated. The resulting dominant accident sequences involved:
C A Small or Medium Break Loss of Coolant Accident (LOCA) followed by the common cause failure of Emergency Core Cooling System (ECCS) recirculation when the containment sump valves failed to open at the new failure rate. Core damage ensued.
C Multiple mechanisms that failed the B in-service Service Water train followed by operators failing to trip the Reactor Coolant Pumps (RCPs) whose motors were no longer being cooled which led to pump failure and Reactor Coolant System boundary breach. Also, a failure to place the non-operating train service water system into service to provide RCP seal cooling with an ensuing RCS boundary breach. Both conditions led to the need for ECCS injection and recirculation.
However, the compounded failure of the B side cooling trains and the A side containment sump valve caused a loss of the ECCS recirculation.
Consequently, core damage ensued.
These accident sequences were validated by using the NRCs PRA model for Farley with similar assumptions used as inputs.
The Large Early Release Frequency (LERF) risk metric was considered for determining significance but the dominant sequences did not involve the dominant LERF initiators (Steam Generator Tube Rupture or Inter-System LOCA). Therefore, the CDF metric was selected.
An external events contribution was developed with fire being the most important. For the fire initiator, the dominant cutsets from the internal events result were used as a template and fire frequencies from Table 4A.1 of NRC Manual Chapter 0609, Appendix F were substituted for the initiating events. The resulting delta CDF for the exposure period was approximately 3E-6. The other external event initiators were considered but, were very minor risk contributors.
Three sensitivity studies were performed; considering recovery by repeatedly pushing the valves open push button, excluding common cause and altering certain initiating event frequencies consistent with a new study, NUREG/CR-6928, Industry-Average Performance of Components and Initiating Events at U.S. Commercial Nuclear Power Plants, (yet to be incorporated into PRA models). Considering recovery and excluding common cause reduced the numerical result but, even when combined, the delta CDF remained above 1E-5 for the exposure period. Using the new information of ATTACHMENT 2 13 ATTACHMENT 2 NUREG/CR-6928 increased the delta CDF for the exposure period. For the complete analysis, including sensitivity studies, see Attachment 2 to this report.
The inspectors determined that the finding does not present a current immediate safety concern because, following the valve failure on January 5, 2007, the licensee implemented long-term corrective measures for the RHR and CS containment sump suction valves and valve encapsulations during the spring 2007 Unit 2 refueling outage.
As of the report, the licensee had implemented similar corrective actions for the Unit 1 encapsulated RHR and CS valves.
Enforcement:
10 CFR 50, Appendix B, Criterion XVI, Corrective Action, requires in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, and non-conformances are promptly identified and corrected.
Contrary to the above, the licensee did not promptly identify and correct a CAQ which resulted in Unit 2 encapsulated valve Q2E11MOV8811A (containment sump suction to RHR pump 2A) failing to stroke full open during testing on April 29, 2006, and again on January 5, 2007. Specifically, the licensee did not take actions to address the high humidity condition which caused rust/corrosion accumulation on valve components in the valve encapsulation. There was licensee documentation dating back to 2001 which indicated that high humidity in the valve encapsulation was causing rust/corrosion accumulation on valve components and could adversely impact valve performance.
This finding has a cross-cutting aspect in the area of problem identification and resolution, specifically the corrective action program component, because the licensee failed to address the aspect to thoroughly evaluate the condition adverse to quality such that the resolution addressed the cause and extent of condition. This violation will be tracked as AV 05000364/2007009-01, Failure to Promptly Identify and Correct a Condition Adverse to Quality for RHR Pump 2A Containment Sump Suction Valve.
.03 Review the maintenance program for the encapsulated RHR and CS sump recirculation
isolation valves (Charter Item 3)
a. Inspection Scope
The inspectors reviewed maintenance procedures and activities for the encapsulated RHR and CS containment sump suction valves to verify that PM activities were consistent with vendor recommendations and licensing/design documents for normal operating environmental conditions. This review included the UFSAR, EQ Documentation Package 0023C, electrical maintenance procedures, and RHR and CS functional system descriptions.
ATTACHMENT 2 14 ATTACHMENT 2
b. Findings
No findings of significance were identified. The inspectors determined that the licensee was performing various PM activities on the encapsulated RHR and CS valves during refueling outages and at 5-year frequencies. The PM activities included removal of the encapsulations to inspect the torque and limit switches and associated wiring and inspect the grease inside the valve actuator compartment. Some WOs documented the presence of varying amounts of water inside the valve encapsulations. The inspectors did not find documentation which indicated that the licensee had evaluated the humidity and rust/corrosion conditions for potential impact on valve components. Maintenance procedures and practices were not changed to address the humidity in the encapsulations and the rust/corrosion on valve components.
.04 Review the CA and maintenance WO databases to determine the failure history of the
encapsulated valves for both units RHR and CS systems (Charter Item 4)
a. Inspection Scope
The inspectors reviewed the CR and WO databases to determine failure history of the encapsulated RHR and CS valves.
b. Findings
No findings of significance were identified. The failure history for the encapsulated RHR and CS valves is discussed in the time line in Section 4OA5.01 of this report.
.05 Review the licensees root cause analysis and extent of condition. Assess adequacy of
implemented and/or planned CAs to address root cause (Charter Item 5)
a. Inspection Scope
The inspectors reviewed the licensees root cause and extent of condition analyses to assess the adequacy of the licensees implemented and/or planned CAs to address the root cause and the time frame for completing the CAs on both units.
b. Findings
No findings of significance were identified. The root cause evaluation concluded that the cause of valve Q2E11MOV8811A failures during testing on April 29, 2006, and again on January 5, 2007, was most likely the result of interrupted continuity of the open torque switch contacts due to rust/corrosion accumulation, causing the retaining spring to hang up and the contacts to not come together firmly. The rust/corrosion accumulation on the valve components was attributed to a chronic high humidity environment inside the valve encapsulation. The licensee replaced the torque switch on valve Q2E11MOV8811A during the spring 2007 Unit 2 refueling outage. Some of the ATTACHMENT 2 15 ATTACHMENT 2 root cause teams recommendations to address the apparent causes of the valve failures included: 1) evaluate the effects of the water/moisture intrusion into the encapsulations and the rust/corrosion accumulation on valve components; 2)performance of the 5-year PM during the next refueling outage on each unit; 3) revise the 5-year PM procedure to monitor for adverse effects resulting from rust/corrosion; and 4) ensure proper gasket installed on the encapsulations to preclude leakage through the flange. The inspectors noted that licensee efforts were ongoing to identify the source of the moisture/water inside the encapsulations. The root cause evaluation indicated that some evidence suggested that the most likely source for the RHR containment sump suction valves was condensation from the RHR room coolers leaking through the encapsulation flange. The inspectors noted that the licensee replaced the actuator motors on the two Unit 2 encapsulated RHR MOVs (Q2E11MOV8811A & B)during the spring 2007 Unit 2 refueling outage after rust/corrosion was observed on motor components. The root cause evaluation also identified a contributing cause related to design, in that, the valve torque switch bypass was set to open at approximately 20% of valve travel in the open direction. The licensee addressed this contributing cause by implementing a modification during the spring 2007 Unit 2 refueling outage to change the torque switch bypass setting to approximately 90% of valve travel in the open direction for the Unit 2 encapsulated MOVs. This modification was scheduled to be implemented for the encapsulated Unit 1 valve torque switches during the fall 2007 Unit 1 refueling outage.
The root cause evaluation identified that the environmental conditions were applicable to all eight encapsulated MOVs at FNP, and to valves in valve boxes that were subject to similar chronic high humidity conditions. The licensees extent of condition review concluded that no additional actions were needed for the valves in valve boxes because the valves were typically butterfly valves. The design for the butterfly valves had the torque switches bypassed for approximately 90% - 95% of valve travel in both directions.
The inspectors noted that the root cause evaluation contained inconsistent statements in its assessment of whether the licensee effectively implemented its corrective action program following the April 29, 2006, valve failure. One root cause conclusion stated that the corrective actions for the April 29, 2006, valve failure were not effective in preventing the valve failure on January 5, 2007, because: 1) repair actions had not been completed; 2) mitigating actions in the form of increased surveillance testing were ineffective because the cause was not understood; 3) the cause was not understood because it was not investigated by any troubleshooting of the valve; and 4) repair actions were not completed and troubleshooting of the valve did not take place because of the licensees decision to not open the valve encapsulation at power. Yet, this same root cause conclusion also stated that the decisions and level of investigation for the April 29, 2006, valve failure were appropriate, based on the existing corrective action program severity level determination. The inspectors reviewed CR 2006104125 (which documented the April 29, 2006, valve failure) and concluded that the CRs Severity ATTACHMENT 2 16 ATTACHMENT 2 Level 4 designation was not consistent with licensee corrective action program procedures, and, the licensees level of investigation was not appropriate.
The inspectors concluded that the corrective actions implemented or planned by the licensee following the January 5, 2007, valve failure were addressing the causes identified for valve Q2E11MOV8811A failures on April 29, 2006, and again on January 5, 2007.
.06 Assess if common failure modes have been established, whether they are being
addressed by the licensee, and what generic implications may exist (Charter Item 6)
a. Inspection Scope
The inspectors reviewed the licensees root cause evaluation, CRs, and WOs to assess whether there were common failure modes and if the failure modes were being addressed by the licensee.
b. Findings
The inspectors determined that potential common cause failure modes may exist for the encapsulated valves due to the high humidity conditions causing rust/corrosion accumulation on valve components and the adverse impact of the rust/corrosion on valve component performance. The common cause failure modes are considered in the NRCs Phase 3 significance determination for the finding discussed in Section 4OA5.02.b of this inspection report.
The licensees root cause attributed the rust/corrosion on valve components to the high humidity environment inside the valve encapsulations. The inspectors noted that the licensee had not performed a common mode failure assessment during the time that the inspectors were on site. Subsequent to the onsite inspection, the licensee performed engineering evaluation DOEJ-SM-2070129101-002, Unit 2 Low Head Safety Injection Containment Sump Suction Isolation Valve Common Mode Failure Assessment. The licensees common mode failure assessment concluded that there were differences between Unit 2 RHR valve Q2E11MOV8811A and the other Unit 2 and Unit 1 RHR encapsulated valves which made the risk of a common mode failure extremely small.
For example, valve Q2E11MOV8811A was made by a different manufacturer and had a different size actuator than the other valves. Also, valve Q2E11MOV8811A was a flexible wedge gate valve while the other three RHR encapsulated valves were solid wedge gate valves. The licensees common failure mode assessment further stated that, except for Unit 2 valve Q2E11MOV8811A, the CS encapsulated valves performed in a manner similar to the other RHR valves. Thus, the licensees conclusion regarding common mode failure was applicable to the four Unit 1 and Unit 2 CS encapsulated valves.
ATTACHMENT 2 17 ATTACHMENT 2 The inspectors reviewed the licensees common mode failure assessment and noted that the assessment did not address other components exposed to the high humidity environment in the encapsulations such as the MOV actuator motor. Although the licensees failure history data did not identify any encapsulated valve failures related to the actuator motors, industry and NRC operating experience information have documented several failures of MOV actuators where high humidity conditions were a contributor. NRC Information Notice (IN) 2006-26, Failure of Magnesium Rotors in Motor Operated Valve Actuators, described recent failures of MOV actuators that were attributed to the oxidation and corrosion of the magnesium motor rotor fan blades and shorting ring resulting from exposure to high humidity and temperatures. The IN stated that magnesium rotors were susceptible to three main failure mechanisms (galvanic corrosion, general corrosion, and thermally-induced stress). The IN further stated that the rate of general corrosion increased in a higher humidity operating environment. The inspectors noted that the licensees root cause evaluation addressed the effects of corrosion and/or oxidation on valve actuator components. The root cause stated that, although valve Q2E11MOV8811A actuator motor had a magnesium rotor and was susceptible to the motor rotor failure mechanism, subsequent testing during troubleshooting did not identify degradation. However, the root cause also stated that the motor rotor was not inspected to determine if magnesium hydroxide buildup was present, even though the ambient and humid conditions inside the encapsulation created the conditions for this to be a potential issue. The inspectors noted that the licensee replaced the actuator motors on the two Unit 2 encapsulated RHR MOVs (Q2E11MOV8811A & B) during the spring 2007 Unit 2 refueling outage after the licensee observed corrosion on motor components.
The inspectors concluded that the licensees common mode failure assessment would have been more thorough if the actuator motors had been included within its scope.
.07 Assess adequacy of the licensees operability determination for the sump recirculation
valves on Unit 1 for both the RHR and CS systems (Charter Item 7)
a. Inspection Scope
The inspectors reviewed completed surveillance test procedures, CRs, WOs, and related engineering evaluations to assess the licensees operability determination for the Unit 1 RHR and CS containment sump suction valves.
b. Findings
No findings of significance were identified. The licensees operability determination concluded that the Unit 1 encapsulated RHR and CS containment sump suction valves were operable based on the valves having been successfully stroked full open during quarterly Technical Specification surveillance testing. During discussions regarding the Unit 1 operability determination, licensee personnel stated that, beginning in May 2007, the surveillance stroke test frequency was increased from quarterly to monthly for the ATTACHMENT 2 18 ATTACHMENT 2 Unit 1 encapsulated RHR and CS valves. In response to the inspectors questions regarding the environmental and rust/corrosion conditions inside the Unit 1 RHR and CS valve encapsulations, the licensee decided to open the Unit 1 encapsulations in June 2007, to inspect the material condition of the valve components prior to the fall 2007 Unit 1 refueling outage. The inspectors concluded that the additional measures implemented by the licensee provided further support for the licensees Unit 1 RHR and CS sump suction valve operability determination.
operating experience items (Charter Item 8)
a. Inspection Scope
The inspectors reviewed the licensees internal operating experience database, root cause evaluation, and the NRC operating experience (OpE) database to assess the licensees actions to address related operating experience items.
b. Findings
No findings of significance were identified. The licensees operating experience review indicated that the particular failure mechanism experienced for valve Q2E11MOV8811A was new for the industry. The inspectors review of the NRC OpE database did not identify any previous failures of the type experienced for the April 29, 2006, and the January 5, 2007 failures of valve Q2E11MOV8811A.
.09 Collect data necessary to develop and assess the safety significance of any findings in
accordance with IMC 0609, Significance Determination Process (Charter Item 9)
a. Inspection Scope
The inspectors reviewed licensee surveillance test procedures, CRs, WOs, root cause evaluation, operability assessment, engineering evaluations, maintenance procedures, and operating experience information to gather data necessary to develop and assess the safety significance of any findings.
b. Findings
The finding identified during this inspection is discussed in Section 4OA5.02.b of this inspection report.
ATTACHMENT 2 19 ATTACHMENT 2
.10 Identify any potential generic safety issues and make recommendations for appropriate
follow-up actions (e.g., information notices, generic letters, bulletins) (Charter Item 10)
a. Inspection Scope
The inspectors reviewed the licensees internal operating experience database, root cause evaluation, CRs, WOs, and the NRC OpE database to determine the potential for generic safety issues related to the failures of Unit 2 RHR valve Q2E11MOV8811A.
b. Findings
No findings of significance were identified. Based on the information reviewed, the inspectors did not identify any generic safety issues related to the failures of valve Q2E11MOV8811A. The identified failure mechanism of rust/corrosion accumulation on the MOV torque switch due to high humidity conditions inside the valve encapsulations appeared to be specific to the FNP encapsulated RHR valve configurations.
4OA6 Meetings, Including Exit
On May 4, 2007, the special inspection team leader presented the preliminary inspection results to Mr. J. Randy Johnson, Farley Plant Vice President, and members of his staff.
Subsequently, additional in-office reviews were conducted and the final inspection results and preliminary significance determination were discussed by telephone with Mr.
Johnson and members of his staff on July 3, 2007. The licensee acknowledged the inspection findings. No proprietary information is included in this inspection report.
ATTACHMENT 2 ATTACHMENT 2
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee Personnel
- W. Bargeron, Plant Manager
- W. Bayne, Performance Analysis Supervisor
- W. Burmeister, Plant Support Manager
- S. Chestnut, Engineering Support Manager
- B. George, Nuclear Licensing Manager
- J. Horn, Training and Emergency Preparedness Manager
- J. Hunter, Operations Support Superintendent
- J. Jerkins, Performance Analysis Engineer
- J. Johnson, Plant Vice President
- T. Livingston, Chemistry Manager
- H. Mahan, Farley Licensing Engineer
- B. McKinney, Farley Licensing Supervisor
- B. Moore, Maintenance Manager
- W. Oldfield, Quality Assurance Supervisor
- R. Rogers, Engineering Supervisor - Reactor/NSSS
- S. Soper, Mechanical/Civil Engineering Supervisor - Farley Plant Support
- R. Still, Site Corrective Action Program Coordinator
- C. Thornell, Daily Operations Superintendent
- R. Wells, Operations Manager
NRC Personnel
- J. Baptist, Resident Inspector
- E. Crowe, Senior Resident Inspector
- K. Kennedy, Deputy Director, Division of Reactor Safety, Region II
- S. Sandal, Resident Inspector
- S. Shaeffer, Branch Chief, Division of Reactor Projects, Region II
LIST OF ITEMS OPENED
Opened
05000364/2007009-01 AV Failure to Promptly Identify and Correct a Condition
Adverse to Quality for RHR Pump 2A Containment Sump
Suction Valve (Section 4OA5.02.b)
ATTACHMENT 2 Attachment 1
ATTACHMENT 2 Documents Reviewed
Procedures
FNP-2-ECP-1.1, "Loss of Emergency Coolant Recirculation"
FNP-2-ECP-1.3, "Loss of Emergency Coolant Recirculation Caused By Sump Blockage"
FNP-2-ESP-1.3, "Transfer to Cold Leg Recirculation"
FNP-0-EMP-1501.05, MOV Preventive Maintenance Flowpath
FNP-0-EMP-1501.18, Acquiring and Analyzing MC2 Data Using the Crane MOVATS MC2 TM or
the Crane MOVATS Universal TM Diagnostic System
FNP-0-EMP-1501.20, MOV Inspection and Lubrication for Models SMB and SB Operators
FNP-0-SOP-0.6, "Limitorque MOV Lubrication"
FNP-2-STP-11.6, Residual Heat Removal Valves Inservice Test
FNP-2-STP-16.7, Containment Spray System Valve Inservice Test
Miscellaneous Documents
FNP UFSAR Sections 3.11, 6.2.2, 6.3
Technical Specification Section 3.5.2
EQ Package No. 0023C, Environmental Qualification of Limitorque MOV Actuators Outside
Containment and Outside MSR, Rev. 12
DOEJ-SM-2070129101-001, "Verify Flow in Safety Injection System A Train with Partially Open
Gate Valve Q2E11V025A (Q2E11MOV8811A)"
DOEJ-SM-2070129101-002, Unit 2 Low Head Safety Injection Containment Sump Suction
Isolation Valve Common Mode Failure Assessment
DOEJ-SM-2070129101-003, "Unit 2 Low Head Safety Injection Containment Sump Suction
Isolation Valve Encapsulation Configuration"
DOEJ-SM-2070129101-004, Containment Overpressure for Evaluation of RHR NPSHa with
2HV-8811A Partially Open
DOEJ-SM-2070129101-005, Containment Sump Level for Evaluation of RHR NPSHa with
Q2E11MOV8811A Partially Open and Minimum Containment Overpressure
DOEJ-SM-2070129101-006, Throttled Position for Q2E11MOV8811A Containment Sump to
Low Head Safety Pump 2A
Operability Assessment for Encapsulated Valve Q2E11MOV8811A from April 29, 2006 to
January 5, 2007, Rev. 1
Operability Determination 07-04, Unit 1 Valves Q1E11MOV8811A & B, Q1E13MOV8826A & B
A-181002, Residual Heat Removal/Low Head Safety Injection Functional System Description,
Rev. 27
A-181003, Containment Isolation System Functional System Description, Version 17.0
A-181008, Containment Spray System Functional System Description, Version 15.0
NMP-GM-002, Corrective Action Program
NMP-GM-002-001, Corrective Action Program Instructions
Information Notice 2006-26, Failure of Magnesium Rotors in Motor Operated Valve Actuators
Drawing D-207132, "Elementary Diagram 575V Motor Operated Valves"
Drawing D-207100, Elementary Diagram Main Control Board Annunciators"
Licensee Internal Letter PS-05-0622 dated April 1, 2005, Corporate engineering support group
response to FNP
ATTACHMENT 2 Attachment 1
ATTACHMENT 2 Conditions Reports
CR 2001000682, CR 2003000510, CR 2006104125, CR 2006104961, CR 2007100142 (Root
Cause Evaluation), CR 2007100176, CR 2007100279, CR 2007103217, CR 2007103233,
CR 2007103501, CR 2007200538, CR 2007200545, CR 2007200546, CR 2007200548
Condition Reports Generated During Inspection
CR 2007104432, CR 2007104433
Work Orders
WO 00489142, WO 00622930, WO 00646640, WO 00646641, WO 00703105, WO 00707534,
WO 144961, WO 154003, WO 2051121601, WO 2061110701, WO 2070508901,
LIST OF ACRONYMS
AV apparent violation
CA corrective action
CAQ condition adverse to quality
CDF core damage frequency
CFR Code of Federal Regulations
CR condition report
ECCS emergency core cooling system
EQ environmental qualification
FNP Farley Nuclear Plant
LERF large early release frequency
LOCA loss of coolant accident
MOV motor operated valve
NPSH net positive suction head
NRC U.S. Nuclear Regulatory Commission
OpE NRC operating experience database
PRA probabilistic risk assessment
RCP reactor coolant pump
SDP significance determination process
UFSAR Updated Final Safety Analysis Report
WO work order
ATTACHMENT 2 Attachment 1