IR 05000410/2009009

IR 05000410-09-009; 08/24-28/2009 and 10/15-16/2009; Nine Mile Point Nuclear Station, Unit 2; Supplemental Inspection for White Performance Indicator - Inspection Procedure (IP) 95001

ML093280672
Person / Time
Site:	Nine Mile Point
Issue date:	11/24/2009
From:	Collins S Region 1 Administrator
To:	Belcher S Nine Mile Point
Dentel, G RGN-I/DRP/BR1/610-337-5233
References
EA-09-262 IR-09-009
Download: ML093280672 (25)

Text

UNITED STATES NUCLEAR REGULATORY COMMISSION ber 24, 2009

SUBJECT:

NINE MILE POINT NUCLEAR STATION, UNIT 2 - SUPPLEMENTAL INSPECTION REPORT 05000410/2009009 WITH ASSESSMENT FOLLOW UP

Dear Mr. Belcher:

On October 16, 2009, the U.S. Nuclear Regulatory Commission (NRC) completed a supplem,ental inspection pursuant to Inspection Procedure (IP) 95001, "Inspection for One or Two White Inputs in a Strategic Performance Area," at your Nine Mile Point Nuclear Station, Unit 2. The enclosed inspection report documents the inspection results, which were discussed at the exit meeting on October 16, 2009, with you and members of your staff.

As required by the NRC Reactor Oversight Process Action Matrix, this supplemental inspection was performed because the Mitigating Systems Performance Index for the cooling water systems crossed the Green to White performance indicator safety threshold in the 4th quarter of 2008. This performance indicator (PI) change occurred due to a combination of: 1) high unavailability of service water (SW) pumps due to maintenance activities and 2) SW pump reliability challenges caused by the introduction of foreign material into the C and F SW pumps.

The events leading to the C and F SW pump intrusion events were documented previously in NRC Inspection Report No. 05000220 and 05000410/2008005. The NRC staff was informed on July 20,2009, of your staff's readiness for this inspection.

The objedives of this supplemental inspection were to provide assurance that: 1) the root causes and the contributing causes for the risk-significant issues were understood; 2) the extent of condition and extent of cause of the issues were identified; and 3) corrective actions were or will be sufficient to address and preclude repetition of the root and contributing causes.

The inspection consisted of examination of activities conducted under your license as they relate to safety, compliance with the Commission's rules and regulations, and the conditions of your operating license. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

Although portions of Constellation's problem identification, root cause evaluations, and corrective actions for the cooling water systems White PI were adequate, the inspectors identified significant weaknesses regarding Constellation's root and contributing causes for the C and F SW pump intrusion events. The events occurred when portions of a suction hose became entrained in each pump's intake during the conduct of diving operations in the SW fore bay area on November 4,2008. The inspectors identified that the initial work controls put in place for the SW fore bay diving evolutions, and the changes made to the diving work scope on November 4, 2008, were inadequate and directly contributed to the C SW pump foreign material intrusion events. In addition, the inspectors identified that Constellation staff failed to follow the governing dive procedure and foreign material exclusion guidance after the SW pump was fouled on November 4, 2008. This failure to follow procedures directly contributed to the F SW pump fouling one hour later. The resulting SW pump reliability problems, coupled with the already high unavailability of the SW system, caused the cooling water systems PI to cross the Green to White threshold. Constellation's failure to identify the inadequate work control and procedural adherence causal factors that led to the reliability challenges with the pumps, which was just one contributor to the White PI, itself represents a performance deficiency and an NRC-identified finding. The finding screened as very low safety significance (Green) because the failure of the C and F SW pumps did not represent an actual loss of safety function of a single train of the SW system for greater than its Technical Specification Allowed Outage Time.

In accordance with IP 95001 and Inspection Manual Chapter (IMC) 0305, if there are significant weaknesses in the licensee's evaluation of a performance issue associated with a PI, a para"el PI inspeetion finding will be opened and given the same color as the PI. The licensee wi" remain in the same Response Column of the Action Matrix until a supplemental inspection has been completed successfully. The finding will not be double-counted in the assessment process. Based upon this guidance and the NRC-identified finding, a para"el White PI inspection finding is also being opened. Accordingly, Nine Mile Point Unit 2 will remain in the Regulatory Response Column of the Action Matrix until the NRC verifies through the successful completion of a supplemental inspection that Constellation has taken actions to address the identified procedural adequacy and adherence issues and why these procedural issues were not previously identified by Constellation.

The NRC-identified Green finding supports the basis for the NRC issuing a parallel White inspection finding and also involves a violation of NRC requirements. Because of the very low safety si!~nificance and because it is entered into your corrective action program, the NRC is dispositioning the violation as a non-cited violation (NCV) consistent with Section VI.A.1 of the NRC Enforcement Policy. If you contest the NCV, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN.: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, Region I; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Senior Resident Inspector at Nine Mile Point Nuclear Station. In addition, if you disagree with the characterization of any finding in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the Regional Administrator, Region I. and the NRC Senior Resident Inspector at Nine Mile Point Nuclear Station. The information you provide will be considered in accordance with IMC 0305.

In accordance with 10 CFR Part 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRC's document system (ADAMS). ADAMS is accessible from the NRC Website at http://W\IVW.nrc.gov/reading-rm/adams.html(the Public Electronic Reading Room).

Sincerely, IRA!

Samuel J. Collins Regional Administrator Region I Docket No.: 50-410 License No.: NPF-69

Enclosure:

Inspection Report 05000410/2009009 w/Attachment: Supplemental Information

REGION I==

Docket No.: 50-410 License No.: NPF-69 Report 1\10.: 05000410/2009009 Licensee: Nine Mile Point Nuclear Station, LLC (NMPNS)

Facility: Nine Mile Point, Unit 2 Location: Oswego, NY Dates: August 24-28, 2009 and October 15-16, 2009 Inspectors: W. Cook, Senior Reactor Analyst, Division of Reactor Safety J. Brand, Resident Inspector, Three Mile Island, Division of Reactor Projects Approved By: Glenn T. Dentel, Chief Projects Branch 1 Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000410/2009009; 08/24-28/2009 and 10/15-16/2009; Nine Mile Point Nuclear Station, Unit 2; Supplemental Inspection for White Performance Indicator - Inspection Procedure (IP) 95001 The report covered on site inspection and in-office reviews by a region-based inspector and a resident inspector. The inspectors identified a parallel White Performance Indicator inspection finding and a Green non-cited violation (NCV). The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609,

"Significance Determination Process (SOP)." Cross-cutting aspects are determined using IMC 0305, "Operating Reactor Assessment Program." Findings for which the SOP does not apply may be Green or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 4, dated December 2006.

Cornerstone: Mitigating Systems

The NRC performed this supplemental inspection in accordance with IP 95001, "Inspection for One or Two White Inputs in a Strategic Performance Area," to assess Constellation's evaluations and corrE~ctive actions associated with the Green to White cooling water systems Mitigating System Performance Index (MSPI) change reported in the 4th quarter of 2008. The inspectors identified significant weaknesses in Constellation's causal evaluation and corrective actions associatE~d with the White Performance Indicator (PI) change. Specifically, the inspectors identified that the initial work controls put in place for the service water (SW) fore bay diving evolutions, and the changes made to the diving work scope on November 4, 2008, were inadequate and directly contributed to the C SW pump foreign material intrusion event. In addition, the inspectors identified that the Constellation staff failed to follow the governing dive procedure and foreign material exclusion guidance after the C SW pump was fouled on November 4, 2008. This failure to follow procedures directly contributed to the F SW pump fouling one hour later. These performance deficiencies were not identified and corrected by Constellation. Consequently, a parallel White PI inspection finding is being opened and will remain in effect until a follow-up supplemental inspection (IP 95001) has been satisfactorily completed.

White.

The NRC identified a parallel White Performance Indicator (PI) inspection finding involving significant weaknesses identified in Constellation's causal evaluation and corrective actions for a White Cooling Water Systems PI.

In accordance with Inspection Procedure 95001 and NRC Inspection Manual Chapter 0305, "Operating Reactor Assessment Program," a parallel PI inspection finding is assigned the same safety significance as the initiating PI. This parallel PI inspection finding provides for additional NRC review of Constellation's actions to address the weaknesses identified in this report and to demonstrate appropriate progress in reversing the adverse trend in cooling water systems performance as evidenced by the White PI.

This finding takes the color (White) of the PI. Constellation entered this parallel finding into its Corrective Action Program (Condition Report No. 2009-007201). (Section .02.03.f)

Green.

The NRC identified a non-cited violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," involving two examples of conditions adverse to quality not properly identified and corrected. For the first example, Constellation failed to identify that rElsponsible station personnel did not adhere to the governing procedures for diving operations and foreign material intrusion for the C service water (SW) pump fouling event, and as a consequence, this led to the fouling of the F SW pump on November 4, 2008.

For the second example, Constellation failed to identify that inadequate work controls and procedural guidance contributed to the November 4, 2008, SW pump foreign material intrusion events. The failure of Constellation to have identified these procedural non compliance and adequacy issues is considered a significant weakness in Constellation's causal analysis and associated corrective actions for the White PI. This NRC-identified finding supports the basis for a parallel White finding. Constellation entered this finding into their Corrective Action Program (Condition Report No. 2008-08430).

This corrective action finding is more than minor because it adversely impacted the equipment performance attribute and the Mitigating Systems Cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The inspectors evaluated the significance of this finding using IMC 0609, Attachment 4, "Phase 1 - Initial Screening and Characterization of Findings." The finding screened as very low safety significance (Green) because the failure of the C and F SW pumps did not represent an actual loss of safety function of a single train of SW system for greater than its Technical Specification Allowed Outage Time. This finding has a cross-cutting aspect of P.1(c), referring to the area of Problem Identification and Resolution - Corrective Action Program, involving the thoroughness and effectiveness of evaluations. (Sections .02.01.d and .02.02.f)

Other Findings

None.

REPORT DETAILS

OTHER ACTIVITIES

40A4 Supplemental Inspection (95001)

.01 Inspection Scope

The NRC conducted this supplemental inspection in accordance with Inspection Procedure (IP) 95001, "Inspection for One or Two White Inputs into a Strategic Performance Area," to assess Constellation's causal evaluations and corrective actions associated with a White Mitigating Systems Performance Index (MSPI) for Cooling Water Systems reported in the fourth quarter of 2008. The cooling water systems performance indicator (PI) is based upon the sum of the service water (SW) system's unavailability and unreliability indices, over the previous 12 quarters. During a time-frame spanning approximately five months between October 2007 and February 2008, the Unit 2 E SW pump was removed from service (unavailable) for preventive maintenance. This unplanned out-of-service time contributed to the PI unavailability index and caused the PI to approach, but not cross the GreenIWhite PI threshold. On November 7,2008, the F service water pump was started and failed to achieve sufficient flow output because of a November 4, 2009, intrusion of foreign material into the pump suction. The F SW pump was promptly secured and declared inoperable (a demand failure). This demand failure, which contributed to the unreliability index in combination with the existing high unavailability index value for the COOling water systems, caused the PI to cross the Green/White threshold value of >1.0E-6. In January 2009, Constellation reported this threshold change in their fourth quarter 2008 performance indicator data to the NRC.

As a result of the White MSPI for cooling water systems, Nine Mile Point Unit 2 entered the Regulatory Response Column of the NRC's Action Matrix in the fourth quarter of 2008.

Constellation completed its causal evaluations and independent reviews by July 20,2009, and informed the NRC staff of its readiness for the supplemental inspection. To address the White PI, Constellation initiated two Category 1 (root cause evaluation) Condition Reports (CRs). CR 2009-000080 addressed the specific root and contributing causes for the performance indicator change from Green to White. CR 2008-008492 addressed the organizational response to the Unit 2 service water system foreign material intrusion event. A third CR of particular interest to the team was CR 2008-008330, "Sediment hose sucked into two service water pumps." This CR was aSSigned Category 2, which required an apparent cause evaluation. The team noted that numerous CRs were initiated based upon the issues and concerns developed via the two Category 1 and one Category 2 CRs.

A list of all CRs reviewed by the team is summarized in Attachment A to this report. In addition to the Constellation staff reviews, independent assessments of the casual evaluations were performed by the Quality and Performance Assurance (QPA)department and outside contractors.

The inspection objectives were as follows:

• Provide assurance that Constellation understood the root and contributing causes of the risk significant performance issues associated with the cooling water systems White PI;

• Provide assurance that Constellation identified the extent of condition and extent of cause of the White PI; and,

• Provide assurance that Constellation has taken or planned corrective actions that were sufficient to address the root causes and contributing causes and to prevent recurrence of the associated station performance issues .

.01.0 1 Timeline of Significant Events

• 7/1/06 - Industry-wide MSPI monitoring commenced

• 6/29/07 - E SW pump planned unavailability baseline hours increased to 750 hours0.00868 days <br />0.208 hours <br />0.00124 weeks <br />2.85375e-4 months <br /> based upon earlier D SW pump maintenance taking 785.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />

• 2/7/08 - Completed extended maintenance window on E SW pump (1450 hours0.0168 days <br />0.403 hours <br />0.0024 weeks <br />5.51725e-4 months <br /> over the planned baseline unavailability); PI approaches Green/White threshold

• 11/4/08 - Foreign material intrusion event impacting service water pumps C and F occurs during SW fore bay diving/cleaning activities

• 11/07/08 - F service water pump started in preparation for securing B SW pump, resulting in a demand failure of the F pump; 5-foot section of vacuum hose discovered in pump suction and impeller; demand failure results in PI change to White

• 11/08/08 - C SW pump fails surveillance test; 5-inch section of vacuum hose found in pump impeller.

.02.0 1 Problem Identification

a.

Determination of who identified the issue and under what conditions.

1) Cooling Water System White Performance Indicator Constellation reported the Cooling Water Systems - Mitigating System Performance Index (MSPI) -fourth quarter 2008 PI crossed the Green to White threshold in January 2009, per Nuclear Energy Institute 99-02, "Regulatory Assessment Performance Indicator Guideline." The inspection team verified Constellation was cognizant of the cooling water systems unavailability performance trend, as reflected in the earlier quarterly PI submittals and internal regulatory performance reports. Likewise, Constellation was mindful (as documented in an internal report, Tier 2 - Regulatory Performance, MSPI Cooling Water

- Unit 2," dated February 2008) that one SW pump demand/start failure would drive the PI to White. The team concluded that the self-revealing failure of the F SW pump on November 7,2009, was properly categorized and processed by Constellation, with respect to the cooling water systems PI and associated reporting requirements.

2) November 4, 2008 - Foreign Material Intrusion Event Diving operations to clean the SW fore bay commenced in early October 2008. This was the first time that the fore bay was cleaned since initial plant operations. Debris (compacted silt and sludge) removal and cleaning was performed in the fore bay area such that the diver was always facing the SW pump intakes, working the suction hose from west to east. In this fashion, the equipment lines were strung behind the diver with the suction hose immediately to the side of the diver and behind. This configuration assured positive control of the suction hose by the diver and the dive tender. The dive tender is responsible for feeding out and retrieving the diver's air line, communications cables and tether line to ensure they do not tangle and that there is not too much tension or slack.

On November 4, 2008, due to lowering seasonal temperatures and the debris removal filtration and collection equipment being exposed to the outside elements, Constellation decided to change the work scope. As the NRC inspection team later learned, this diving work scope change was developed based upon informal discussions and limited review.

The diving activities that commenced on the morning of November 4 involved the diver removing debris in the 4-foot area immediately in front of the service water intakes for the C, A and E SW pumps, working parallel (north to south) along the east wall of the fore bay. The diver controlled the suction hose in the same fashion as earlier dives, with the hose to his side and trailing behind. The major difference in this approach was that the suction hose now trailed behind the diver, but looped across from the west wall of the fore bay, down and along the east wall behind the diver. The dive tender did not maintain positive control of this long loop or "belly" of suction hose, but did properly tend the diver's support lines. Consequently, as the trailing 6-inch diameter flexible (corrugated plastic)suction hose passed in front of the operating C SW pump, the 9000 gpm intake flow entrained the suction hose and pulled it through the diver protection grating and into the pump suction piping and impeller. The licensee started the E SW pump and secured the C pump following this intrusion event.

The diver was directed by the Dive Manager and Shift Manager to remove the sections of hose sucked into the C SW pump intake. Approximately 50 minutes after these recovery actions were initiated, the diving crew alerted the control room that the B SW pump was in jeopardy of being fouled by the free end of the suction hose. Operators started the D SW pump and secured the B pump. Approximately ten minutes later, the loose end of the sLlction hose that had threatened the B SW pump was entrained in the intake flow of the F SW pump. Control room operators restarted the C SW pump (having been informed by the personnel in the screen house that they believed all of the hose sections had been removed from the pump suction piping) and secured the F SW pump.

The inspectors concluded that the Constellation staff failed to follow procedures when the C SW pump was fouled on November 4, 2008. The failure to follow the governing dive procedure and foreign material exclusion guidance directly contributed to the F service water pump fouling by the free end of the suction hose, one hour after the C pump hose intrusion event. (See Section

.02.0 1.d)

3) November 7, 2008 - F Service Water Pump Demand Failure Event The F SW pump was started on November 7, 2008, in preparation for scheduled maintenance on the B SW pump. When started, plant operators heard unusual noises and observed abnormal pump performance parameters causing them to secure the pump and declare it inoperable. Upon disassembly, a five-foot length of 6-inch diameter vacuum hose was discovered in the suction and impeller of the pump.

4) November 8,2008 - C Service Water Pump Degraded Operability On November 8, 2008, an extent of condition review, consisting of running a surveillance test on the C SW pump, resulted in the in-service testing acceptance criteria not being satisfied and the C SW pump being declared inoperable. Disassembly of the C SW pump identified a five-inch length of vacuum hose in the pump impeller.

b. Determination of how long the issue existed and prior opportunities for identification

.

Organizationally, Constellation was cognizant of the cooling water systems PI and associated trend since the 1st quarter of 2008. Quarterly PI submittals, preceding the fourth quarter 2008 White PI change, were accurate and properly reflected cooling water/service water system performance. As documented in Constellation's root cause analysis (reference CR 2009-000080), management direction and expectations were less than adequate with respect to work planning for MSPI related systems. Additionally, the station-specific performance targets were the same as regulatory performance targets and did not assist Constellation management in being proactive and sensitive to approaching regulatory safety system performance thresholds.

c. Determination of the plant-specific risk consequences and compliance concerns associated with the issue.

The team concluded that Constellation's causal evaluation of the White PI change was thorough with respect to evaluating the station management practices and available performance monitoring tools in place prior to the November 7,2008 PI change. The associated corrective actions taken or planned were appropriate for these causal factors.

However, the team concluded that the causal evaluation for the SW pump foreign material intrusion event mischaracterized procedural compliance issues (see Section

.02.0 1.d,

below) and overlooked a significant contributing cause with respect to control of work activities (see Section

.02.0 2.f below).

d. Findings

Introduction:

The inspectors identified a violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," (Example 1) for the licensee's failure to identify that the Constellation staff involved with the service water fore bay diving activities on November 4,2008, failed to follow procedures when the C SW pump was fouled. The failure to follow the governing dive procedure and foreign material exclusion guidance directly contributed to the F SW pump fouling by the free end of the suction hose, one hour after the C pump hose intrusion event. This NRC-identified violation was of very low safety significance (Green) and satisfied the criteria for a non-cited violation (NCV).

Description:

At 11 :43 am, on November 4, control room operators became aware of a SW system problem, just prior to notification by the dive team of the hose intrusion, when the "Service Water Pump 1A11C/1E Suction Pressure Low" alarm annunciated. The C SW pump discharge flow was noted to have degraded to -6000 gpm (normal discharge flow -9000 gpm). Constellation's root cause evaluation (reference CR 2008-008492)identified that this degraded SW system condition presented an opportunity for the shift crew to have entered N2-S0P-11, "Loss or Degraded Service Water System." Interviews by the Constellation root cause investigation team confirmed that the shift crew discussed entry into N2-S0P-11, but deemed it inappropriate. Approximately 50 minutes after the C SW pump intrusion event, it was reported to the Shift Manager (by the diver via the Dive Manager) that the B SW pump was in jeopardy of being fouled by the free end of the suction hose. To address this developing SW system threat, the D SW pump was started and the B SW pump secured. Constellation's root cause evaluation concluded that this was another opportunity for the shift crew to have entered N2-S0P-11. Approximately 10 minutes later, the F SW pump was fouled by the suction hose. Control room operators reacted to this event by restarting the C SW pump (all pieces of suction hose were presumed to have been removed from the pump's suction) and securing the F SW pump.

The control room staff again did not enter N2-S0P-11. This third opportunity was viewed by the Constellation investigation team as another "missed opportunity" to have entered the off-normal procedure. Constellation's root cause evaluation stated that had the shift crew entered into procedure N2-S0P-11, a plant public address system announcement would have been made and additional station staff assistance would have been focused on the SW fore bay diving operations and SW system pumps being fouled.

The inspectors credit Constellation's identification of the "missed opportunities" associated with the repeated failure of the shift crew to enter into N2-S0P-11. However, the inspectors view the "missed opportunity" characterization as understating the procedural compliance aspect of the crew's actions. Constellation's root cause evaluation (CR 2008 008492) acknowledges these missed opportunities as symptomatic of the organization "under responding" to the SW system intrusion event. The failure of Constellation to have more appropriately characterized and taken corrective action to address the procedural non-compliance aspects is considered a weakness in Constellation's causal analysis and associated corrective actions for the White PI. This inspector observation provides additional basis for the issuance of a parallel White finding.

The inspectors identified two distinct performance deficiencies that contributed to the fouling of the F SW pump on November 4 and its subsequent demand failure on November 7, 2008. One of the performance deficiencies was previously identified by the resident staff and documented as an NCV (reference NRC Integrated Inspection Report No. 05000410/2008005, dated January 27,2009). As documented in the January 27, 21009 report, the resident staff concluded that Constellation had not taken appropriate corrective action to address the operability impact of the ingestion of suction hose on both the C and F SW pumps on November 4, 2008. The failure to properly address pump operability on November 4, 2008, contributed to the demand failure of the F SW pump on November 7 and the failed surveillance test of the C SW pump on November 8, 2008. On November 4, following the removal of pieces of suction hose from the intakes of both C and F SW pumps, neither pump was tested to ensure it was fully functional and that no additional pieces of hose remained in the suction piping and impellers.

The second performance deficiency, identified by the inspectors, involved procedural non compliance. The inspectors concluded that personnel involved with the immediate actions in response to the C SW pump intrusion event, failed to follow the governing dive procedure, S-MAP-MAI-010B, "Control of Non-RCA Diving Activities," step 3.1.6, which states, "The dive activity shall be immediately terminated for any event or condition with safety implication: Examples include: Any deviation from the pre-dive briefing that could cause the diver to take action outside the parameters of the briefed dive area." In addition to immediately terminating the dive activities (stop work), this dive procedure and Fleet Administrative Procedure CHG-MN-1.01-1001, "Foreign Material Exclusion," call for prompt notification of station management. development of a recovery plan by key stakeholders, and event documentation via a Condition Report. Contrary to this procedure guidance, diving activities were not immediately stopped, station management was not notified, and no recovery plan was developed by key stakeholders, prior to suction hose recovery actions being taken by the diver, under the direction of the Shift Manager and Dive Manager.

The inspectors acknowledge that the individuals directly involved with the November 4 intrusion event (Dive Manager, diver, Shift Manager, Shift Technical Advisor, and on-shift operators) attempted to stabilize the SW system following the C SW pump hose intrusion.

Their initial actions included starting the E SW pump and securing of the C SW pump.

However, recovery actions were initiated to remove the severed pieces of suction hose from the diver protection grating and suction piping of the C SW pump, prior to securing both free ends of the severed suction hose. Consequently, an unsecured free end of the suction hose fouled the running F SW pump, one hour after the C SW pump intrusion event occurred. The inspectors viewed the C pump hose recovery actions taken by the diver, under the direction of the Shift and Dive Managers, to have been contrary to the requirements of procedures S-MAP-MAI-010B and CHG-MN-1.01-1001. The inspectors concluded that this was the cause for the F SW pump intrusion event. Constellation did not identify this procedural noncompliance issue as a part of their causal evaluation of the events of November 4. In addition, the inspectors noted that Constellation's causal evaluation did not individually assess the C and F SW pump fouling events.

Analysis:

The failure to adhere to the procedural guidance of S-MAP-MAI-010B and CHG MN-1.01-1001 was a performance deficiency and contributed to the fouling of the F SW pump. This finding is more than minor because it adversely impacted the equipment performance attribute and the Mitigating Systems Cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The inspectors evaluated the significance of this finding using IMC 0609, Attachment 4, "Phase 1 Initial Screening and Characterization of Findings." The finding screened as very low safety significance (Green) because the failure of the C and F SW pumps did not represent an actual loss of safety function of a single train of SW system for greater than its Technical Specification Allowed Outage Time. The failure of Constellation to identify this procedural non-compliance issue is considered a significant weakness in Constellation's causal analysis and associated corrective actions for the White PI. This NRC-identified finding supports the basis for a parallel White finding. This finding has a cross-cutting aspect of P.1 (c), referring to the area of Problem Identification and Resolution - Corrective Action Program, involving the thoroughness and effectiveness of evaluations.

Enforcement:

10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," states that measures shall be established to assure that conditions adverse to quality such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances are promptly identified and corrected. Contrary to the above, conditions adverse to quality were not promptly identified and corrected. For this first example (refer to Section

.02.0 2.f for the second example), Constellation failed to identify

and correct that responsible station personnel did not adhere to the governing procedures for diving operations and foreign material intrusion for the C SW pump on November 4, 2008. Because this finding is of very low safety significance and was entered into Constellation's Corrective Action Program (CAP) under CR 2008-08430, it is considered an NCV, consistent with Section VI.A.1 of the NRC Enforcement Policy. (NCV 05000410/2009009-01, Failure to identify procedural inadequacies and non compliances that contributed to the November 4,2008 SW pumps foreign material intrusion events.)

.02.0 2 Root Cause, Extent of Condition and Extent of Cause Evaluation

a.

Evaluation of methods used to identify the root and contributing causes.

The inspectors reviewed Constellation's root cause evaluation methodology and concluded it was adequately applied. Constellation used a combination of "Staircase" and "Root Cause Road Map" methods to assist their evaluation team in developing and validating their results. However, the NRC inspectors identified two causal factors involving procedural adequacy and compliance that Constellation did not identify.

Following the inspectors' debrief on August 28,2009, Constellation initiated a Condition Report (CR 2008-08430) to examine the procedural issues raised by the inspectors. This CR and associated evaluations and corrective actions will be examined during the follow up IP 95001 inspection.

b.

Level of detail of the root cause evaluations.

To evaluate the circumstances involving the White cooling water systems PI, Constellation initiated two Category 1 Condition Reports (CRs). CR 2009-000080 addressed the specific root and contributing causes for the performance indicator change from Green to White. CR 2008-008492 addressed the organizational response to the Unit 2 SW system foreign material intrusion event. A third CR reviewed by the inspectors was CR 2008-008330, "Sediment hose sucked into two service water pumps," and was a Category 2 - Apparent Cause Evaluation CR. The inspectors noted that several additional CRs were initiated based upon the issues and concerns developed via these three CRs.

A liist of all CRs reviewed by the inspectors is summarized in Attachment A to this report.

1) CR 2008-00080 Constellation concluded that the root cause for the cooling water system PI crossing the GreenlWhite threshold was "management direction and expectations were less than adequate" relative to the strategies and planning for preventive maintenance performed on the SW pumps. The inspectors concluded that Constellation's causal evaluation and associated corrective actions for the high SW system unavailability were generally comprehensive. The principle reason for the long unavailability time was the need to rebuild SW pump casings and the dependency on an outside contractor. The inspectors verified that Constellation plans to procure a spare SW pump casing and rotating assembly to ensure shorter pump out-of-service times and to minimize system un;3vailability. More broadly, Constellation took numerous steps to improve management oVf:lrsight and visibility of all MSPI-related work activities. Included in these types of enihancements were revised (more conservative) thresholds for internal and external performance indicators. Constellation expects that heightened sensitivity and awareness of all systems and human performance trends will provide more opportunity for early intervention prior to issues becoming a safety or regulatory concern.

2) CR 2008-00842 The inspectors concluded that Constellation's evaluation of its organizational response to the hose intrusion events and subsequent actions associated with identifying and resolving pump operability issues adequately addressed some of the causes. Constellation's causal evaluation identified that the broad root cause for their poor handling of the intrusion event was that leadership behaviors did not reflect a "healthy skepticism" or "trust but verify" type bias towards the SW system operating parameters and information gathered during the hose recovery activities. These behaviors lead to wrong conclusions and poor decision making. Constellation concluded that this principle root cause bounded over 23 "missed opportunities" associated with the intrusion event. Two noteworthy contributing causes were also identified by Constellation. These contributing causes were "poor operator and engineering fundamental and questioning attitude" and "senior management job assignments during and after the event resulted in reduced teaming, support, and self challenging among the management team."

However, the inspectors concluded there was a significant shortcoming in Constellation's review of their organizational response to the hose intrusion event. Specifically, the inspectors considered Constellation's identification of a number of "missed opportunities" regarding procedural noncompliance issues as a mischaracterization of the associated human performance and overall organizational response. (The supporting details for this conclusion are documented in Section

.02.0 1.d. above and Section .02.02.f below.)

3) CR 2008-008330 This CR is a Category 2 (Apparent Cause Evaluation) and titled "Sediment hose sucked into two service water pumps." The purpose of this apparent cause evaluation was to determine why the foreign material intrusion event occurred. Constellation's evaluation concluded the apparent cause was "faulty mental model" by personnel developing, reviewing, and approving the work package. Specifically, Constellation identified that the only risk identified during the planning of the sediment removal work was the "high industrial risk" associated with the diving activity. The risk assessment did not consider nuclear, radiological, or corporate risk as outlined in the governing Fleet Administrative Procedure, CHG-OP-4.01-1000, "Integrated Risk Management."

Thl~ inspectors' approach to reviewing the hose intrusion event was to first look at the causal factors for the hose being sucked into the C SW pump and then the causal factors associated with the F SW pump hose intrusion. As previously discussed, this approach appeared to have differed from Constellation's evaluation of the events. As discussed in SHction

.02.0 1.d above, the inspectors concluded that procedural non-compliance was the

primary cause for the F SW pump hose intrusion event. With respect to the C SW pump hose intrusion and based upon the review of available work control documentation, the licensee's apparent cause evaluation, and interviews with responsible station personnel, the inspectors concluded the primary cause was Constellation's failure to properly control thl3 diving operations in the service water fore bay. Specifically, the governing Work Order and procedure S-MAP-MAI-01 08 lacked sufficient detail and guidance, commensurate with the scope of diving activities in the SW fore bay. (Additional details are documented in Section

.02.0 2.f. below.)

c.

Consideration of prior occurrences of the problem and knowledge of prior operating experience.

The inspectors noted that diving operations in the SW system fore bay area was a "first time activity." Consequently, no prior opportunities for identifying problem areas or using in-plant operating experience were available to the station staff. The inspectors determined that Constellation had compared their diving controls with other nuclear stations, but concluded there were no significant differences. These facts notwithstanding, Constellation's causal evaluation identified that this activity was a "first time" station evolution and should have been afforded additional management and administrative oversight, per station procedure CNG-OP-4.01-1 000. Refer to Section

.02.0 2.f below for additional details.

d. Determination of the extent of condition and the extent of cause of the problem.

The inspectors concluded that Constellation's extent of condition/cause review, with respect to trending and monitoring other regulatory Pis and the establishment of appropriate internal PI thresholds prior to exceeding regulatory PI thresholds, was comprehensive. The inspectors noted that Constellation also conducted a thorough examination of all internal (non-regulatory) Pis to ensure appropriate and timely focus on declining system, plant, and operator performance.

With respect to the inspectors' conclusions regarding the adequacy of work controls and procedural adherence, Constellation had not completed their reviews (including extent of condition/cause) in these areas prior to the conclusion of the inspection. This area will be reexamined during the follow-up IP 95001 inspection.

e. Determine that the root cause evaluation. extent of condition. and extent of cause appropriately considered the safety culture components as described in Inspection Manual Chapter 0305. "Operating Reactor Assessment Program."

The inspectors reviewed the results of a July 2009 Nuclear Safety Culture Assessment (final report, dated August 2009). No noteworthy concerns were identified by the independent safety culture assessment team. The inspectors noted that the one area for improvement identified in the July 2009 assessment (establishing a questioning attitude and making conservative decisions) highlighted the SW pump foreign material intrusion event as an illustration of where improvement was warranted.

The inspectors also observed that Category 1 CR Nos. 2009-000080 and 2008-008492 included an Appendix A which summarized a Safety Culture Impact Review. The Safety Culture Impact Reviews examined the broad areas of human performance, problem idemtification and resolution, safety conscious work environment, and other safety culture components (Le., accountability, organizational change management, safety policies, and continuous learning environment). No significant concerns were identified by the inspectors associated with these Constellation safety culture reviews.

Introduction:

The inspectors identified a violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," (Example 2) for the licensee's failure to identify that the work controls put in place for the SW fore bay diving evolutions in early October 2008 and the changes made to the diving work scope on November 4, were inadequate and directly contributed to the C SW pump foreign material intrusion event. This NRC-identified violation was of very low safety significance (Green) and satisfied the criteria for a non cited violation (NCV).

Deiscription: Constellation's apparent cause for the intrusion event was that the personnel responsible for developing, planning, and approving the risk assessment for this non-RCA diving evolution (Work Order No. 06-16744) had a "faulty mental model." The basis for this determination was that the planning department identified the diving activity to clean the fore bay as high risk industrial only (nuclear risk was not considered). Consequently, the focus of the procedure CNG-OP-4.01-100, "Integrated Risk Management," Attachment 9, "High Risk Activity Plan" was on personnel (diver) safety only. Constellation's evaluation did identify a "lack of rigor" in the implementation of procedure CNG-OP-4.01 1000, Attachment 2, "Risk Assessment Worksheet." Constellation's evaluation concluded that this diving activity should have been screened out as high risk because the silt and sludge removal activities potentially introduce "debris around FME Zone 1 plant equipment or systems important to nuclear safety," (the SW pumps). Constellation also concluded thi:lt the diving in the SW fore bay should have screened as high risk because it was a "First Time Maintenance" activity. Constellation identified that procedure CNG-OP-4.01 1000, Attachment 3, "Managing and Approving Risk Significant Activities," states that a Management Oversight Board (MOB) should be considered if the activity is a "first of its kind activity" or is classified as "First Time Maintenance." Constellation's causal evaluation clearly identified that this was the first time that sediment was removed from the SW fore bay, but no MOB was performed. Constellation's evaluation also acknowledged that when the change in work scope was made, no MOB was held to review this change against the existing High Risk Activity Plan. Constellation identified these procedural issues as "missed opportunities" and addressed them in their intrusion event case studies as lessons learned and/or areas for improvement with respect to conservative decision making.

The inspectors concluded that these procedure CNG-OP-4.01-1000 issues represented additional examples of poor/inadequate procedural compliance. The failure of Constellation to have more appropriately characterized and taken corrective action to address the procedural non-compliance aspects is a weakness in their causal analysis and associated corrective actions for the White PI. This inspector observation provides additional basis for the issuance of a parallel White finding.

The inspectors observed that the level of detail in Work Order (WO) No. 06-16744 was limited to a simple step that instructed Maintenance Department staff to support the dive team. The plant impact summary was similarly brief and non-informative. Constellation used S-MAP-MAI-0108, "Control of Non-RCA Diving Activities," as the overall guidance document for the SW fore bay diving activities. Based upon the interviews conducted by the inspectors, it was evident that Constellation had purposefully changed the established diving work scope on November 4 and introduced the hazard (unattended loop of suction hose) that adversely impacted the SW system. Based upon the diving events of November 4,2008, the inspectors concluded that WO No. 06-16744 and procedure S-MAP-MAI-0108 lacked sufficient detail and guidance to prevent a challenge to the safe operation of the unit. Of particular concern to the inspectors was the informal work scope change made by Constellation on November 4, 2008, that directly contributed to the intrusion event. The inspectors also observed that no protective tagging was used during the October to November 2008 diving evolutions, including the one-hour time period during removal of suction hose from the C and F SW pumps by the diver on November 4.

The securing and tagging of operating equipment, when practicable, minimizes the risk to personnel and plant equipment.

Analysis:

The failure to have adequate procedures for the control of safety related diving operations in the SW fore bay was a performance deficiency and contributed to the fouling of the C and F SW pumps. This finding is more than minor because it adversely impacted the equipment performance attribute and the Mitigating Systems Cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The inspectors evaluated the si~Jnificance of this finding using IMC 0609, Attachment 4, "Phase 1 - Initial Screening and Characterization of Findings." The finding screened as very low safety Significance (Green) because the failure of the C and F SW pumps did not represent an actual loss of safety function of a single train of SW system for greater than its Technical Specification Allowed Outage Time. The failure of Constellation to have identified this procedural inadequacy is considered a significant weakness in Constellation's causal analysis and associated corrective actions for the White PI. This NRC-identified finding supports the basis for a parallel White finding. This finding has a cross-cutting aspect of P.1 (c),referring to the area of Problem Identification and Resolution Corrective Action Program, involving the thoroughness and effectiveness of evaluations.

Enforcement:

10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," states that measures shall be established to assure that conditions adverse to quality such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances are promptly identified and corrected. Contrary to the above, conditions adverse to quality were not promptly identified and corrected. For this second example (refer to Section

.02.0 1.d for the first example), Constellation failed to identify

that inadequate work controls and procedural guidance contributed to the November 4, 2008, SW pump foreign material intrusion events. Because this finding is of very low safety significance and was entered into Constellation's CAP under CR 2008-08430, it is considered a non-cited violation (NCV), consistent with Section VI.A.1 of the NRC Enforcement Policy. (NCV 05000410/2009009*01, Failure to identify procedural inadequacies and non-compliances that contributed to the November 4, 2008 SW pumps foreign material intrusion events.)

.02.0 3 Corrective Actions

a.

8Q.propriateness of corrective actions.

Based upon the identified root and contributing causes developed by Constellation, to date, the inspectors concluded that the corrective actions taken or planned were appropriate. However, based upon the inspectors having identified weaknesses in Constellation's causal analysis, as documented in this report, Constellation committed to conduct additional reviews. Additional corrective actions are anticipated and will be examined during the follow-up IP95001 inspection (reference Sections

.02.0 2.f.)

The inspectors noted that a contributing factor for the delayed start of the 95001 inspection (approximately nine months following the intrusion event and PI change) was the impact of the independent review by the Constellation Quality and Performance Assurance (QPA) Department. Following the completion of the Category 1 CRs associated with the White PI change, QPA performed an assessment of the adequacy of the root cause evaluations and associated corrective actions. QPA identified that the station did not sufficiently disposition numerous "missed opportunities" identified in CR 2008-008492, "Organizational Response to the NMP2 Service Water Foreign Material Intrusion Event." Consequently, Constellation requested a postponement of the NRC inspection until later in the year, after they had completed the additional reviews recommended by QPA.

Appendix B to CR 2008-008492, summarizes the Missed Opportunity Analysis/Safety Culture Attribute Review. QPA review of Appendix B, as documented in CR-2009-002477, dated April 23, 2009, identified that "specific missed opportunities and errors (gaps in performance) identified in CR-2008-008492 were not evaluated to determine if separate CRs were needed to address those deficiencies." In response, Constellation evaluated each "missed opportunity" and where appropriate initiated a Category 3 CR (improve performance) or Category 4 CR (capture for trend analysis). In some instances, the "missed opportunity" was evaluated by the Constellation staff and found to be captured by an existing CR or bounded by another root or contributing cause and associated corrective action.

Although this independent review by QPA and the additional evaluations of staff performance raised concerns with respect to adherence to programs, procedures, and management expectations, no procedural compliance theme was identified by Constellation. Based upon the inspectors' review of CR-2009-002477 and the multiple CRs initiated for the "missed opportunities" they concluded that Constellation and the QPA organization did not focus on the pattern of procedural non-compliance associated with station response to the November 4, 2008, intrusion event. The specific "missed opportunities" which constitute the NRC-identified pattern of procedural non-compliance are listed in Appendix B to CR 2008-008492, and include, but are not limited to:

• No.4 - Station Policy #7 was not referred to or used as a guide as intended

• No.5 - Shift Manager failed to enter N2-S0P-11, even though multiple SW pumps were impacted

• No.1 0 - Shift Manager failed to declare C and F SW pumps inoperable and did not implement GAI-OPS-17

• No. 16 Operations personnel failed to initiate a CR for the C service water pump discharge flow indication change (normal flow of 9000 gpm dropped to 7800 gpm following the intrusion)

• No. 17 Human Performance Review Board not completed as required by procedure and as expected by Site Vice President

• No. 19 - Dayshift understood that two pumps were initially impacted by the suction hose, and for unclear reasons, dayshift failed to record the pumps as inoperable in the control room log.

The inspectors consider this observation a shortcoming in Constellation's overall evaluation of the November 4, 2008 service water system intrusion event. This observation also supports the inspectors' basis for issuing the parallel White finding.

b.

Prioritization of corrective actions.

Overall, based on Constellation's identified root and contributing causes for the White PI, the inspectors concluded that the corrective actions, taken or planned to date, were prioritized commensurate with their significance.

c. Schedule for implementing and completing the corrective actions.

The inspectors observed that Constellation was generally timely in recognizing the significance of the performance issues associated with the MSPI change. A "case study" was prepared by each department involved (operations, maintenance, and engineering)with the intrusion event and communicated to department staff by the responsible manager. This corrective action demonstrated a clear emphasis on the importance of learning from past mistakes, maintaining a questioning attitude, conservative decision making, and improving individual and organizational response to events and adverse safety conditions. As discussed above, corrective actions for the NRC-identified issues of procedural adequacy and adherence have not yet been developed by Constellation and will be reviewed in a follow-up IP 95001 inspection.

d. ME3asures of success for determining the effectiveness of the corrective actions to prevent recurrence.

The inspectors reviewed Constellation's corrective action effectiveness evaluations, completed and planned, which are an integral part (Section 5, titled "Effectiveness Review") of each Category 1 CR. The licensee's planned effectiveness reviews for CR 2009-000080, "White NRC ROP MSPI for NMP2 Cooling Water Systems," were considered appropriately scoped and timely, with respect to evaluating the effectiveness of both internal and external performance indicators. Interim and longer term effectiveness reviews planned for the review of corrective actions associated with CR 2008-008492, "Organizational Response to the NMP2 Service Water Foreign material Intrusion Event,"

were also considered appropriate by the inspectors.

e. Determine the adequacy of the licensee's completed or planned corrective actions associated with the Notice of Violation (NOV) which was the basis for the supplemental inspection, if applicable.

The NRC staff did not issue an NOV to the licensee; therefore, this inspection requirement was not applicable.

Introduction:

The NRC identified a parallel White Performance Indicator inspection finding involving significant weaknesses identified in Constellation's causal evaluation and corrective actions for a White cooling water systems PI.

Description:

The inspectors identified that, although Constellation acknowledged a change in diving work scope directly contributed to the hose intrusion events, no inadequacies in station work control processes were identified via Constellation's root cause evaluation. In addition. the inspectors concluded that Constellation either did not identify or mischaracterized a number of procedural non-compliance issues as "missed opportunities." The inspectors acknowledged that Constellation reviewed and identified several procedural issues associated with the planning, execution, and organizational response to the service water fore bay diving activities of November 4,2008. However, the inspectors did not identify specific corrective actions taken or planned by Constellation that address procedural non-compliance.

The NRC concluded that Constellation's characterization of numerous procedural issues as "missed opportunities," the failure to identify work control process deficiencies associated with the fore bay diving activities, and the absence of corrective actions to address procedural adequacy and compliance issues were significant weaknesses in Constellation's causal evaluation. A Significance and Enforcement Review Panel (SERP)conducted on October 27,2009, concurred in this inspection conclusion.

Analysis:

In accordance with NRC Inspection Manual Chapter 0305, "Operating Reactor Assessment Program," a parallel Performance Indicator inspection finding is assigned the same safety significance as the initiating PI. This parallel PI inspection finding provides for additional NRC review of Constellation's actions to address the weaknesses identified in this report and to demonstrate appropriate progress in reversing the adverse trend in cooling water systems performance as evidenced by the White PI. This finding takes the color (White) of the PI.

Enforcement:

No violation of regulatory requirements is associated with this specific finding. This finding is tracked as FIN 05000410/2009009-02, Parallel Performance Indicator White Finding. This parallel White inspection finding is applied to the NRC Action Matrix until the inadequacies in Constellation's efforts to address the issues have been corrected and the NRC closes the finding via a follow-up IP 95001 review. This finding is not double-counted with the White PI in the assessment process.

.03.0 1 Inspector Debrief Meeting

The inspectors and Mr. Glenn Dentel, Chief, Reactor Projects Branch 1, met with Mr. Thomas Lynch and other members of his staff on August 28, 2009, to discuss the pre~liminary observations and issues identified during the on site inspection. The inspectors informed Constellation that addition review of the material provided and gathered during the on site inspection would be reviewed in office (Region I) and discussed with regional management prior to the completion of the inspection .

.03.0 2 Exit Meeting

The inspectors presented the inspection results to Mr. Sam Belcher and other members of his staff on October 16, 2009. The inspectors asked the licensee whether any materials examined during the inspection should be considered proprietary. No proprietary information was identified.

ATTACHMENT: SliPPLEMENTAL INFORMATION

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

S. Belcher, Vice President

W. Byrne, Manager, Nuclear Safety and Security

T. Chwalek, Director, Quality Performance and Assurance

J. Dean, Quality Performance and Assurance

R. Deane, Manager, Training

S. Doty, Manager, Maintenance

M. Eron, System Engineering

J. Evans, Long Island Power Associates

J. Gerber, Manager, IWM

J. Krakuszeski, Manager, Operations

T. Lynch, Plant General Manager

J. Pacher, Engineering

R. Slade, IWM

W. Smith, Performance Improvement

A. Sterio, Engineering

T. Syrell, Director, Licensing

D. Wolniak, Performance Improvement

J. Yoe, FIE~et Operations

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000410/2009009-01 NCV Failure to identify procedural inadequacies and non compliances that contributed to the November 4, 2008 SW pumps foreign material intrusion events.

05000410/2009009-02 FIN Parallel Performance Indicator White Finding