IR 05000354/2005010
| ML052090319 | |
| Person / Time | |
|---|---|
| Site: | Hope Creek  |
| Issue date: | 07/28/2005 |
| From: | Jason White NRC/RGN-I/DRS/PSB2 |
| To: | Levis W Public Service Enterprise Group |
| References | |
| IR-05-010 | |
| Download: ML052090319 (13) | |
Text
uly 28, 2005
SUBJECT:
HOPE CREEK NUCLEAR GENERATING STATION - NRC SUPPLEMENTAL INSPECTION REPORT 05000354/2005010
Dear Mr. Levis:
On June 23, 2005, the U. S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Hope Creek Station. The enclosed report documents the inspection results which were discussed on June 23, 2005, with Mr. George Barnes and other members of your staff.
The NRC performed this supplemental inspection to assess your evaluation of a low to moderate (White) safety significant finding involving a degraded level control valve for the A moisture separator drain tank which malfunctioned and caused the moisture separator drain system to operate outside its design resulting in a pipe failure on October 10, 2004. This was a self-revealing event which involved inadequate evaluation and corrective action for a degraded level control valve 25 days prior to the event, with further reference to a similar event that occurred in 1988. The supplemental inspection was conducted to determine if the root and contributing causes of the White finding were understood, to assess the extent of the condition review, and to determine if the corrective actions were sufficient to address causes and prevent recurrence. The inspection was conducted in accordance with Inspection Procedure 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area, and examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.
Based on the results of this inspection, we concluded that you have adequately completed a root cause analysis of the performance deficiencies surrounding the event and have identified appropriate corrective actions. No findings of significance were identified concerning the root cause evaluation and corrective actions. Given your acceptable performance in addressing the moisture separator level control valve failure, the White finding associated with this issue will only be considered in assessing plant performance for a total of four quarters in accordance with the guidance in Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program.
Mr. William Levis 2 In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/
John R. White, Chief Plant Support Branch 2 Division of Reactor Safety Docket No: 50-354 License No: NPF-57
Enclosure:
Inspection Report 05000354/2005010 w/Attachment: Supplemental Information
REGION I==
Docket No: 05000354 License No: NPF-57 Report No: 05000354/2005010 Licensee: PSEG LLC Facility: Hope Creek Nuclear Generating Station Location: P.O. Box 236 Hancocks Bridge, NJ 08038 Dates: June 21 - 23, 2005 Inspector: J. Noggle, Senior Reactor Inspector, DRS Approved by: J. White, Chief Plant Support Branch 2 Division of Reactor Safety
SUMMARY OF FINDINGS
IR 05000354/2005010; 06/21/2005 - 06/23/2005; Public Service Electric Gas Nuclear LLC,
Hope Creek Generating Station; Supplemental Inspection; IP 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area.
The inspection was conducted by one regional inspector. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649,
Reactor Oversight Process, Revision 3, dated July 2000.
Cornerstone: Initiating Events
The NRC performed this supplemental inspection to assess PSEGs evaluation of a low to moderate (White) safety significant finding where engineering staff did not properly evaluate and correct a degraded level control valve for the A moisture separator drain tank. In addition, engineers did not properly consider a similar occurrence from 1988. The level control valve failed open 25 days prior to the event and caused the moisture separator drain system to operate in a condition outside its design (unstable two-phase flow). As a result, an 8-inch diameter drain pipe in that system ruptured and caused the initiating event on October 10, 2004. This performance issue was previously characterized as having low to moderate risk significance (White) in NRC Inspection Report 05000354/2004013. During this supplemental inspection, performed in accordance with Inspection Procedure 95001, the inspector determined that the licensee performed a comprehensive evaluation of the degraded level control valve condition and prior occurrence.
PSEGs evaluation of the issue included a formal, structured root cause evaluation to identify the root and contributing causes associated with the level control valve failure. The root cause evaluation was thorough and documented weaknesses associated with the MS drain line rupture. PSEG identified two primary root causes. One was inadequate decisions by engineering and management to continue operating the MS system with the drain valve failed open. Specifically, PSEG did not have a rigorous process to apply effective decision-making principles to management and engineering decisions in response to plant conditions that fall below licensing thresholds and/or were not clearly defined by existing procedures. The second root cause was that operating procedures for the MS level control system were inadequate to prevent extended operation of the system in the condition of unstable two-phase flow.
PSEGs actions to address the root causes included incorporation of engineering and management technical decision making for degraded equipment and provide a program for monitoring degraded plant conditions. In addition, operating procedures were revised to limit the operation of moisture separators with empty moisture separator drain tanks. Based on the results of the inspection, the inspectors concluded that PSEG had adequately completed a root cause evaluation of the performance deficiencies associated with the event, and planned and completed corrective actions that were reasonable to address the root and contributing causes.
Given PSEGs acceptable performance in addressing the moisture separator drain tank level control valve failure, the White finding associated with this issue will only be considered in assessing plant performance for a total of four quarters in accordance with the guidance in IMC 0305, Operating Reactor Assessment Program.
ii
REPORT DETAILS
INSPECTION SCOPE
The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to review and assess PSEG Nuclears evaluation associated with the A moisture separator drain tank level control valve failure and subsequent drain line rupture. This performance issue was previously characterized as being White in NRC Inspection Report No. 05000354/2004013 and is related to the initiating events cornerstone in the reactor safety strategic performance area.
EVALUATION OF INSPECTION REQUIREMENTS 02.01 Problem Identification a.
Determination of who identified the issue and under what conditions The A moisture separator drain tank level control valve (LV-1039A) had been open for approximately 25 days and was the direct cause of the A moisture drain tank pipe rupture on October 10, 2004. In addition, a spring can pipe hanger (H25), designed to provide support for an upstream portion of the failed pipe, was found to have been disconnected. The disconnected extension rod, associated with hanger H25, had worn a hole in the air supply line to LV-1039A due to vibration over an extended period of time (several years) which caused the LV-1039A valve to fail open. This was a self-revealing issue.
Notification 20203784 was written on September 16, 2004, which identified that the MS low level alarm was received and the A MS dump valve, LV-1039A, was noted on the Control Room Information Display System (CRIDS) (computer display) to be about 10%
open while the associated valve controller was receiving an air signal to fully close the valve. Engineering responded on September 20, 2004, stating that there was no immediate safety concern. However, an operator, not satisfied with the September 20 notification response, initiated another notification (No. 20204256) that same day, stating that the prior engineering response addressed only flow accelerated corrosion concerns. Specifically, it did not address potential impact to the condenser penetration which had cracked on an earlier occasion (1988) when this same dump valve had failed open for an extended period of time. Again, a formal engineering response, completed on September 22, did not fully address the concern.
Neither evaluation considered that two-phase flow could be present from the MS drain tank to the main condenser. The piping from the MS drain tank to the condenser was not designed for the dynamic loading that would accompany two-phase flow. The disconnected hanger (H25), while likewise unknown at the time, was not available to mitigate the dynamic loading of the lines. The inspectors concluded that engineerings evaluations associated with the two notifications were inadequate because the associated MWe reduction due to the leakage, the loss of water level in MS A and the difference in operating pressures in the MS drain tank and the main condenser, should have led to the recognition that there was two-phase flow in the line upstream of LV-1039A.
After about 25 days (September 16 to October 10, 2004) of operation beyond the design loading capacity of the MS drain tank piping, the 8-inch pipe failed near the condenser penetration, resulting in a steam leak, manual reactor scram, and loss of condenser vacuum.
b.
Determination of how long the issue existed, and prior opportunities for identification PSEGs root cause evaluation (RCE) determined that the degraded level control valve condition existed for 25 days and the licensee appropriately recognized that a missed opportunity to identify the degraded condition had occurred on September 16, 2004, when it was determined that level control valve LV-1039A was opening. The RCE also identified that there was a prior event that occurred in 1988.
c.
Determination of the plant-specific risk consequences and compliance concerns associated with the issue PSEGs evaluation referenced that the performance deficiencies associated with the were assessed as having a low to moderate importance to safety (White), in accordance with the NRC significance determination process. The PSEG evaluation referenced the NRC risk assessment specified in Inspection Report No. 05000354/2004013, issued on February 4, 2005. The inspectors concluded that PSEGs evaluation appropriately documented the risk consequences associated with the issue.
02.02 Root Cause, Extent of Condition, and Extent of Condition Evaluation a.
Evaluation of methods used to identify root causes and contributing causes PSEG utilized an event and causal factors chart to identify the events and conditions that led up to the event. PSEG developed a fault tree analysis diagram to identify relationships among events and the probability of event occurrence.
PSEG identified two root causes and three contributing causes. The inspectors reviewed the root cause analysis methods employed and concluded that a formal, structured approach was utilized to identify root and contributing causes.
b.
Level of detail of the root cause evaluation The inspectors determined that the root cause evaluation was thorough and identified the appropriate root and contributing causes to a sufficient level of detail. PSEGs evaluation was self critical and identified weaknesses in engineering rigor and management engagement with degraded plant conditions.
.
c.
Consideration of prior occurrences of the problem and knowledge of prior operating experience The licensees evaluation determined there were prior opportunities to potentially prevent the October 10, 2004, pipe failure. Specifically, the 1988 encapsulation repair did not address the root cause for the pipe crack, which appeared to be related to operating the drain line outside its design. Rather, the encapsulation sealed the vacuum leak but moved the flex point slightly upstream of the repair.
Another missed opportunity was when the encapsulation was installed in 1988. The original scope of the controlling modification discussed a request to engineering to evaluate the need for additional supports on the MS drain line. The associated installation plan recommended vibration monitors on the piping. However, these recommendations were not implemented, and represent missed opportunities to determine whether vibration and line movement were acceptable for the piping configuration and operation.
Vendor Instruction GEK-37949A, MS and Reheater Drain Systems, stated that the check valve in the normal drain path (to the feedwater heaters) should be located close to the branch point for the dump line to minimize the amount of saturated water upstream of the check valve. The GEK-37949 instruction also stated that two-phase flow anywhere in the lines upstream of the level control valves can produce pressure pulsations and uncontrollable level oscillations in the drain tank; and that it was important that only single-phase liquid exist upstream of the level control valves. While PSEG had previously planned a detailed review of this information to determine whether system modifications were necessary, no such review was completed.
Each prior opportunity was reviewed by the licensee and systematically captured in the corrective actions. The inspectors determined that PSEG included sufficient consideration of prior occurrences of similar problems and other operating experience to prevent future recurrence.
d. Consideration of potential common causes and extent of condition of the problem Pipe Evaluation The selection of locations to be non-destructively examined was made to include all piping that is connected to main condenser nozzles that have a potential for nozzle and/or piping damage as a result of two-phase flow. This included piping to the condenser associated with valves that leaked in the past. PSEG reviewed notifications and work orders that included the following systems: main steam, condenser/feedwater, extraction steam, and heater drains. As a result of this review, 14 additional condenser penetrations were identified for inspection. The sample plan also included visual and magnetic particle examination on all welds on the A and B MS drain lines.
Because the October 10, 2004, pipe failure occurred at the location of a pipe attachment intended to contain a leak (1988 weld failure - encapsulation), PSEG performed a search to identify any other encapsulation devices that may have been used to contain leaks in BOP systems. The team noted that as a result of this examination effort, an additional encapsulation was identified on the steam seal evaporator relief valve piping.
No indications were identified in the vicinity of this encapsulation, or at any other location in the inspection sample. These lines were examined inside and outside the condenser.
During these inspections and examinations, some indications were identified, for which PSEG removed/repaired them as necessary. Defects were removed by grinding as necessary, and were verified as eliminated with the appropriate nondestructive examination technique before repair activities were completed. Repairs were made to within the original design specification requirements, and the weld repair locations were non-destructively tested to verify weld soundness.
Pipe Hangers PSEG formulated a plan to select and evaluate pipe hangers and supports in a large sample of BOP steam and high-energy water systems at Hope Creek and Salem. The selection was based on those systems with similar design, materials, operating parameters, and were believed to potentially have been exposed to two-phase flow (but designed for single phase flow). Also, systems were selected which similarly had a known history of valve leakage, either periodically or continuously, where such operation would be outside the piping design and potentially result in the application of unanalyzed forces (static or cyclic) to system components. PSEG performed system and component corrective action document searches, reviewed industry operating experience, and conducted interviews to aid in selecting the inspection sample.
Field inspections encompassed validation of integrity of over 5000 hangers in Hope Creek and Salem. Of this sample, 206 deficiencies were identified. Deficiencies were identified in the following broad categories; bent rod or support, bottomed-out spring can, loose support components, signs of excess vibration, or loose jam nuts on pipe hangers. The majority of the deficiencies were screened as having negligible impact because the discrepancies were minor and would not affect the function of the support.
These deficiencies were entered into the corrective action process. Of the 206 deficiencies identified, less than ten were considered more than minor and none were evaluated as having an immediate impact on structural integrity of the associated system.
The inspectors found the extent of condition reviews to be acceptable both in scope and detail. The original scope of the reviews were appropriately expanded as new information became available.
02.03 Corrective Actions a.
Appropriateness of corrective actions PSEG staff initiated Notification 20206626 in response to the moisture separator drain tank pipe rupture on October 10, 2004. Immediate corrective actions included reinstalling hanger 25, repairing the LV-1039A valve air supply line and replacement of the ruptured pipe section with thicker piping and an improved weld configuration. The plant returned to service on January 12, 2005, when baseline vibration monitoring of the subject piping was performed. These corrective actions effectively addressed the restoration of the ruptured drain line.
PSEG staff performed a root cause evaluation, Order 70041898. This analysis documented two root causes and three contributing causes. Corrective actions were reviewed associated with each identified cause.
Root causes and associated corrective actions 1) Inadequate decision by engineering and management allowed continued operation of the moisture separator system with the drain valve failed open. The corrective actions implemented by PSEG associated with this issue included development of two guidance standards addressing plant conditions or operations not defined by procedures.
Operational and Technical Decision Making Process Desk Guide establishes required peer and supervisory review and includes escalation criteria for additional management review for engineering responses to degraded conditions below action thresholds established in procedures. Preexisting degraded equipment conditions were also researched and the identified issues were evaluated using this new procedure. A second guidance standard, Adverse Condition Monitoring and Contingency Planning was established to provide clarification of shift operations responsibility for establishing additional monitoring requirements for degraded equipment or for reduced operating margin conditions.
2) Operating procedures for moisture separator level control were inadequate to prevent extended operation of the system in unstable two-phase flow. The corrective actions implemented for this issue included revision of plant operating procedures for moisture separator drain lines at both Salem and Hope Creek to prohibit operation of the moisture separator drain tanks with level empty for greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />.
Contributing causes and associated corrective actions 1) The disconnected hangar (H25) was not discovered allowing it to abrade and puncture the instrument air line causing LV-1039A to fail open. Corrective actions implemented by PSEG associated with this issue included: inspection of existing hangars and correction of deficiencies at Hope Creek and Salem; establishing an ongoing hangar inspection program for non-safety related systems at both Hope Creek and Salem; and addition of specific hangar inspection criteria in system engineer walkdown checklists.
2) The condition of LV-1039A was not monitored after failing open to detect degradation.
Corrective actions implemented by PSEG associated with this issued included:
development of procedure, Adverse Condition Monitoring and Contingency Planning, to provide clarification of shift operations responsibility for establishing additional monitoring requirements for degraded equipment.
3) Not enough rigor was applied to an engineering evaluation after operations raised a concern about prior pipe failure caused by operating LV-1039A in an open condition.
Corrective actions implemented by PSEG associated with this concern included:
Director of Engineering review of engineering evaluation protocol documents and issuance of guidance to engineering personnel to ensure adequate peer and supervisory review is provided for informal requests for assistance; and the criteria for appropriate engineering engagement and requirement for a thorough review was provided in Operational and Technical Decision Making Process Desk Guide for engineering responses to degraded conditions.
The inspectors determined that initial corrective actions had addressed the equipment concerns and the corrective actions from the root cause evaluation addressed the root and contributing causes of the issue. The inspectors found the completed and proposed corrective actions to be reasonable with regard to addressing the performance deficiencies identified with respect to the issue.
b. Prioritization of corrective actions Prioritization of the corrective actions was not directly based on risk perspectives or analysis, but rather based on a deterministic approach considering the significance of damage to non-safety systems and components. The inspectors reviewed the prioritization of the corrective actions and verified that actions of a generally higher priority were scheduled for completion in a reasonable time-frame.
c. Establishment of schedule for implementing and completing the corrective actions PSEGs corrective actions and proposed corrective action plan provided dates for completion of corrective actions described in their root cause evaluation. The inspectors reviewed the proposed schedule and determined that most corrective actions have been completed, while most remaining actions are scheduled for completion during the next available refueling and maintenance outage. The inspectors considered the schedule of completing these remaining corrective actions was appropriate.
D. Establishment of quantitative and qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence The inspectors determined that the root cause evaluation included actions with regard to effectiveness reviews for completed and proposed corrective actions. The actions consisted of requirements to assess the effectiveness of corrective actions at a later date in order to determine if additional actions may be necessary.
03 MANAGEMENT MEETINGS
Exit Meeting Summary
The results of this inspection were discussed with Mr. G. Barnes and other members of their staff at the conclusion of the inspection on June 23, 2005. The meeting was considered a Regulatory Performance Meeting in accordance with Manual Chapter 0305, Operating Reactor Assessment Program, and focused on discussion involving the performance deficiencies associated with the issue and proposed corrective actions.
No proprietary information was discussed.
ATTACHMENT 1
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee Personnel
- A. Johnson, Hope Creek Design Engineering Supervisor
- J. Morrison, Hope Creek Engineering, Root Cause Analysis
- B. Thomas, Licensing Engineer
- J. Williams, Director Engineering
LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED
Opened
None
Closed
- 05000354/2004013-04 FIN Failure to adequately evaluate and correct a failed open level control valve in the moisture separator drain system.