IR 05000220/2013007

IR 05000220-13-007, 05000410-13-007; 08/05/2013 - 08/23/2013; Nine Mile Point Nuclear Station (Nmpns), Units 1 and 2; Biennial Baseline Inspection of Problem Identification and Resolution

ML13274A647
Person / Time
Site:	Nine Mile Point
Issue date:	10/01/2013
From:	Daniel Schroeder Reactor Projects Branch 1
To:	Costanzo C Constellation Energy Group
References
IR-13-007
Download: ML13274A647 (25)

Text

October 1, 2013

SUBJECT:

NINE MILE POINT NUCLEAR STATION UNITS 1 AND 2 - NRC PROBLEM IDENTIFICATION AND RESOLUTION INSPECTION REPORT 05000220/2013007 AND 05000410/2013007

Dear Mr. Costanzo:

On August 23, 2013, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at your Nine Mile Point Nuclear Station (NMPNS), Units 1 and 2. The enclosed report documents the inspection results, which were discussed on August 23, 2013, with you and other members of your staff.

This inspection examined activities conducted under your license as they relate to identification and resolution of problems and compliance with the Commissions rules and regulations and conditions of your license. Within these areas, the inspection involved examination of selected procedures and representative records, observations of activities, and interviews with personnel.

Based on the samples selected for review, the inspectors concluded that NMPNS was generally effective in identifying, evaluating, and resolving problems. NMPNS personnel identified problems and entered them into the corrective action program at a low threshold. NMPNS prioritized and evaluated issues commensurate with the safety significance of the problems and corrective actions were generally implemented in a timely manner.

This report documents one NRC-identified finding of very low safety significance (Green).

The inspectors determined that the finding also involved a violation of NRC requirements.

However, because of its very low safety significance and because it was entered into your corrective action program, the NRC is treating this finding as a non-cited violation, consistent with Section 2.3.2 of the NRC Enforcement Policy. If you contest the non-cited violation, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with copies to the Regional Administrator, Region I; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at NMPNS. In addition, if you disagree with the cross-cutting aspect assigned to the finding, you should provide a response, within 30 days of the date of this inspection report, with the basis for your disagreement, to the Regional Administrator, Region I, and the NRC Resident Inspector at NMPNS. In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs Agencywide Documents and Management System (ADAMS). ADAMS is accessible from the NRC website at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Daniel L. Schroeder, Chief Reactor Projects Branch 1 Division of Reactor Projects

Docket Nos:

50-220 and 50-410 License Nos: DPR-63 and NPF-69

Enclosure:

Inspection Report 05000220/2013007 and 05000410/2013007

w/Attachment: Supplementary Information

REGION I==

Docket Nos:

50-220 and 50-410

License Nos:

DPR-63 and NPF-69

Report Nos:

05000220/2013007 and 05000410/2013007

Licensee:

Constellation Energy Nuclear Group, LLC

Facility:

Nine Mile Point Nuclear Station, Units 1 and 2

Location:

Oswego, NY

Dates:

August 5 through August 23, 2013

Team Leader:

Leonard Cline, Senior Project Engineer

Inspectors:

Brian P. Smith, Resident Inspector

Eric Miller, Resident Inspector

Andrey Turilin, Project Engineer

Approved by:

Daniel L. Schroeder, Chief

Reactor Projects Branch 1

Division of Reactor Projects

Enclosure

SUMMARY

IR 05000220/2013007, 05000410/2013007; 08/05/2013 - 08/23/2013; Nine Mile Point Nuclear

Station (NMPNS), Units 1 and 2; Biennial Baseline Inspection of Problem Identification and Resolution. The inspectors identified one finding in the area of corrective action implementation.

This NRC team inspection was performed by two regional inspectors and two resident inspectors. The inspectors identified one finding of very low safety significance (Green) during this inspection and classified this finding as a non-cited violation (NCV). The significance of most findings is indicated by their color (i.e., greater than Green, or Green, White, Yellow, Red)and determined using Inspection Manual Chapter (IMC) 0609, Significance Determination Process, dated June 2, 2011. Cross-cutting aspects are determined using IMC 0310,

Components Within Cross-Cutting Areas, dated October 28, 2011. All violations of NRC requirements are dispositioned in accordance with the NRCs Enforcement Policy, dated January 28, 2013. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 4.

Problem Identification and Resolution

The inspectors concluded that NMPNS was generally effective in identifying, evaluating, and resolving problems. NMPNS personnel identified problems, entered them into the corrective action program (CAP) at a low threshold, and prioritized issues commensurate with their safety significance. In most cases, NMPNS appropriately screened issues for operability and reportability and performed causal analyses that appropriately considered extent of condition, generic issues, and previous occurrences. The inspectors also determined that NMPNS typically implemented corrective actions to address the problems identified in the CAP in a timely manner. However, the inspectors identified one violation of NRC requirements in the area of corrective action implementation.

The inspectors concluded that, in general, NMPNS adequately identified, reviewed, and applied relevant industry operating experience to NMPNS operations. In addition, based on those items selected for review, the inspectors determined that NMPNSs self-assessments and audits were thorough.

Based on the interviews the inspectors conducted over the course of the inspection, observations of plant activities, and reviews of individual CAP and employee concerns program (ECP) issues, the inspectors did not identify any indications that site personnel were unwilling to raise safety issues nor did they identify any conditions that could have had a negative impact on the sites safety conscious work environment.

Cornerstone: Mitigating Systems

Green: The inspectors identified an NCV of 10 CFR 50, Appendix B Criterion XVI,

Corrective Actions, because between November 5, 2012, and July 22, 2013, NMPNS did not promptly identify and correct a failed automatic de-watering system for the buried high pressure core spray (HPCS) medium voltage power supply cable duct bank. As a result, on July 22, 2013, NMPNS unexpectedly discovered significant water level in the two manholes that contained the buried HPCS cable duct bank. NMPNS subsequently determined that multiple level switches for the de-watering system had failed. In response, NMPNS pumped down the affected manholes, replaced the failed level switches and initiated weekly manual pump downs of the manholes until final corrective actions could be completed. NMPNS entered this performance deficiency into the NMPNS CAP under CR-2013-006992.

The inspectors determined that this performance deficiency was more than minor because if left uncorrected the failed automatic dewatering system would have become a more significant safety concern. Specifically, with no preventative maintenance (PM) task to inspect and test the dewatering system and no work order (WO) scheduled to investigate the cause of the MH-1 hi-hi level alarm, the inspectors determined that, based on NMPNS previous experience of rising level in this manhole and wetting of these cables, it was not likely that NMPNS would identify the failed de-watering system before the HPCS power supply cables were wetted. Wetted cables become a more significant concern because, in accordance with industry and NRC operating experience, the long term reliability of medium voltage cables is negatively affected when wetted. In accordance with IMC 0609.04, Initial Characterization of Findings, and Exhibit 2 of IMC 0609, Appendix A, The Significance Determination Process for Findings At-Power, issued June 19, 2012, the inspectors determined that the finding was of very low safety significance (Green), because the finding was not a design or qualification deficiency, did not represent a loss of safety system function, and did not screen as potentially risk significant due to external initiating events.

The inspectors determined that this finding had a cross-cutting aspect in the area of human performance, resources, because NMPNS did not maintain long term plant safety by maintenance of design margins, minimization of long-standing equipment issues, minimizing PM deferrals, and ensuring maintenance and engineering backlogs which are low enough to support safety. Specifically, an NMPNS planner changed the scope of a PM task to eliminate inspecting MH-1 and MH-3 cable ducts every six months, and as a result, PM activities were not performed in November 2012 and May 2013. This error prevented NMPNS from identifying the condition adverse to quality associated with the HPCS medium voltage power supply cable duct bank de-watering equipment. H.2(a) (4OA2.1.c)

REPORT DETAILS

4OA2 Problem Identification and Resolution

This inspection constitutes one biennial sample of problem identification and resolution as defined by Inspection Procedure 71152. All documents reviewed during this inspection are listed in the Attachment to this report.

.1 Assessment of Corrective Action Program Effectiveness

a. Inspection Scope

The inspectors reviewed the procedures that described NMPNSs CAP. To assess the effectiveness of the CAP at NMPNS, the inspectors reviewed performance in three primary areas: problem identification, prioritization and evaluation of issues, and corrective action implementation. The inspectors compared performance in these areas to the requirements and standards contained in 10 CFR 50, Appendix B, Criterion XVI, Corrective Action, and NMPNS procedure, CNG-CA-1.01-1000, Corrective Action Program, Revision 00801. For each of these areas, the inspectors considered risk insights from the stations risk analysis and reviewed condition reports selected across the seven cornerstones of safety in the NRCs Reactor Oversight Process. Additionally, the inspectors attended Plan-of-the-Day, Screening Committee, and Management Review Committee meetings. The inspectors selected items from the following functional areas for review: engineering, operations, maintenance, emergency preparedness, radiation protection, chemistry, security, and oversight programs.

(1) Effectiveness of Problem Identification

In addition to the items described above, the inspectors reviewed selected condition reports, system health reports, a sample of completed corrective and preventative maintenance WOs, completed surveillance test procedures, operator logs, and periodic trend reports. The inspectors also completed field walkdowns of various systems on site, such as the Unit 2 service water system; the Unit 1 primary containment vent, purge and pressure control system; and the Unit 2 emergency diesel generators (EDGs).

Additionally, the inspectors reviewed a sample of condition reports written to document issues identified through internal self-assessments, audits, emergency preparedness drills, and the operating experience program. The inspectors completed this review to verify that NMPNS entered conditions adverse to quality into their CAP as appropriate.

(2) Effectiveness of Prioritization and Evaluation of Issues

The inspectors reviewed the evaluation and prioritization of a sample of condition reports issued since the last NRC biennial problem identification and resolution inspection completed in October 2011. In this review, the inspectors reviewed root cause evaluations (RCEs) and apparent cause evaluations (ACEs). The inspectors also reviewed condition reports that were assigned lower levels of significance, i.e. category 3 and 4, that did not include cause evaluations to ensure that they were properly classified. The inspectors review included the appropriateness of the assigned significance, the scope and depth of the causal analysis, and the timeliness of resolution. The inspectors assessed whether the evaluations identified likely causes for the issues and developed appropriate corrective actions to address the identified causes. Further, the inspectors reviewed equipment operability and functionality determinations, reportability assessments, and extent-of-condition reviews for selected problems to verify these processes adequately addressed equipment operability/functionality, reporting of issues to the NRC, and the extent of the issues.

(3) Effectiveness of Corrective Actions

The inspectors reviewed NMPNSs completed corrective actions through documentation review and, in some cases, field walkdowns to determine whether the actions addressed the identified causes of the problems. The inspectors also reviewed condition reports for adverse trends and repetitive problems to determine whether corrective actions were effective in addressing the broader issues. The inspectors reviewed NMPNSs timeliness in implementing corrective actions and effectiveness in precluding recurrence for significant conditions adverse to quality. The inspectors also reviewed a sample of condition reports associated with selected NCVs and findings to verify that NMPNS personnel properly evaluated and resolved these issues. In addition, the inspectors expanded the corrective action review to five years to evaluate NMPNSs actions related to the Unit 1 EDG raw water system degradation, the Unit 1 feedwater isolation valve leakage, and the Unit 1 and 2 standby liquid control flow measurement issues.

b.

Assessment

(1) Effectiveness of Problem Identification

NMPNS staff initiated over 20,000 condition reports between October 2011 and July 2013. Based on the selected samples, plant walkdowns, and interviews of site personnel in multiple functional areas, the inspectors determined that in general, NMPNS identified problems and entered them into the CAP at a low threshold. The inspectors did, however, identify five examples where condition reports were not initiated for conditions adverse to quality and one example of inadequate maintenance rule system monitoring. These examples are discussed below. The inspectors also identified one NCV of 10 CFR 50, Appendix B, Criterion XVI, Corrective Action, because NMPNS did not identify and correct a condition adverse to quality associated with the HPCS system. This violation is described in section 4OA2.1.c.

The inspectors observed supervisors at the plan-of-the-day, screening committee, and management review committee meetings appropriately questioning and challenging condition reports to ensure clarification of the issues. Based on the samples reviewed, the inspectors determined that NMPNS trended equipment and programmatic issues, and concluded that personnel were identifying trends at low levels. In addition, in response to several questions and observations identified by the inspectors during the inspection, NMPNS personnel promptly initiated condition reports and took immediate action to address the issue.

a) Condition reports not issued when required by CAP procedure

The inspectors identified examples of degraded conditions, per NMPNSs CAP procedure, CNG-CA-1.01-1000, Revision 00801, Attachment 2, Condition Report Threshold Guidance, that were not entered into the CAP. Attachment 2 stated, in part, that condition reports should be written for nonconformances, departures from specified requirements or expectations, deviations, deficiencies, concerns, undesirable states, or near misses. The inspectors determined that the following examples met this definition, but did not have associated condition reports:

On April 19, 2013, while moving fuel during the Unit 1 refueling outage, source range monitors (SRMs) 12, 13, and 14 experienced spiking due to electrical interference caused by welding in the drywell. The inspectors concluded that not entering the issue into the CAP was not more than minor because the spiking was a single occurrence associated with welding and was not exhibited continuously or with amplitude that would have led to the inability for operators to determine a significant change in neutron flux during fuel moves. CR-2013-005355 was written to address this performance deficiency.

On June 22, 2013, Unit 1operators received an unexpected annunciator, A2-1-6 Turbine Performance Calc. Cabinet Trouble. Operators verified computer point B155 indicated vacuum pump trouble in cabinet 12. The inspectors concluded that not entering the issue into the CAP was not more than minor because operators later determined that the vacuum pump associated with cabinet 12 was a spurious alarm and that no abnormal indications existed. The alarm cleared on its own. CR-2013-005360 was written to address this performance deficiency.

On June 22, 2013, Unit 1 operators bypassed SRM 12 due to unexpected hi-hi alarm. The inspectors concluded that not entering the issue into the CAP was not more than minor because at the time SRMs were not required to be operable for core neutron flux monitoring per technical specifications (TS)due to the reactor being at full power. NMPNS investigated the problem and no issues were identified. CR-2013-007251 was written to address this performance deficiency.

On September 21, 2012, the Unit 2 division 1 EDG tripped on overspeed resulting in an incomplete sequence during testing after maintenance. The inspectors concluded that not entering the issue into the CAP was not more than minor because the trip occurred during post-maintenance testing when the EDG was already considered inoperable, and the concern was addressed before the EDG was returned to operable status. CR-2013-006694 was written to address this performance deficiency.

On December 3, 2012, the Unit 2 division 2 EDG fuel level dropped below its required level band after an endurance run. The inspectors concluded that not entering the issue into the CAP was not more than minor because the fuel oil storage tank was refilled and the tank level remained above the TS procedural limit of 50,000 gallons. CR-2013-006692 was written to address this performance deficiency.

b) Maintenance rule monitoring for redundant reactivity control system

The inspectors identified an example of inadequate maintenance rule monitoring for Unit 2 redundant reactivity control system (RRCS). Specifically, not all unavailability for RRCS was appropriately tracked in accordance with NMPNS maintenance rule procedures. For certain maintenance activities performed by the fix-it now team, NMPNS did not count unavailability hours because the activities did not exceed six hours, and NMPNS Unit 2 TS allowed an instrument to be placed in an inoperable status for up to 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> for required surveillance testing before entering the limiting condition of operation. However, the inspectors determined that NMPNSs maintenance rule procedures did not apply this same exemption to unavailability tracking. NMPNS wrote CR-2013-006683 to document this concern. The inspectors determined this performance deficiency was not more than minor because when the unavailability hours were correctly applied, it did not result in unavailability hours exceeding the a(2)/a(1) threshold, which would have required additional evaluation under the maintenance rule. In addition, NMPNS extent of condition did not identify other systems with similar unavailability tracking errors.

(2) Effectiveness of Prioritization and Evaluation of Issues

The inspectors determined that, in general, NMPNS appropriately prioritized and evaluated issues commensurate with the safety significance of the identified problem.

NMPNS screened condition reports for operability and reportability, categorized the condition reports by significance, and assigned actions to the appropriate department for evaluation and resolution. The condition report screening process considered human performance issues, radiological safety concerns, repetitiveness, adverse trends, and potential impact on the safety conscious work environment.

Based on the sample of condition reports reviewed, the inspectors noted that the guidance provided by NMPNSs CAP implementing procedures appeared sufficient to ensure consistency in categorization of issues. Operability and reportability determinations were generally performed when conditions warranted and in most cases, the evaluations supported the conclusion. However, there were two examples discussed below in which functionality assessments did not meet NMPNS procedural guidance. In the case of causal evaluations: causal evaluations appropriately considered the extent of condition or problem, generic issues, and previous occurrences of the issue. However, the inspectors did identify three examples where cause evaluations did not comply with NMPNS standards for the conduct of the evaluations.

These examples are also discussed below.

a) Operability/Functionality Assessments that did not comply with procedural guidance:

NMPNS procedure, CNG-OP-1.01-1002, Conduct of Operability Determinations/

Functionality Assessments, describes the process for addressing operability and functionality when a degraded, non-conforming or unanalyzed condition brings into question the analysis, design, or qualification of a structure, system or component (SSC). The procedure stated that functionality assessments are developed for conditions affecting SSCs that perform required support functions for TS SSCs, as well as for conditions affecting SSCs that are not described in TS, but warrant programmatic controls to ensure that SSC availability and reliability are maintained.

The procedure stated that when an identified degraded or non-conforming condition falls into this scope, the shift manager must assess functionality of the affected equipment or system. If the shift manager determined functionality exists, then the basis for functionality must be documented in the operability section of the condition report. From among the sample of condition reports reviewed, the inspectors identified two examples of inadequate functionality assessments:

CR-2011-011345 - On December 23, 2011, after performance of N2-OSP-FOF-M001, The Engine Driven Fire Pump Operability and Storage Tank Level Test, the Unit 1 diesel fire pump oil pressure below safe level alarm lit and then cleared after the pump control switch was returned to auto. NMPNS generated a WO to troubleshoot the alarm circuit and verify response of the low pressure signal, but no assessment of its capability to perform its function was documented. The Unit 1 diesel fire pump supports accident mitigation during fires and station blackout accident scenarios. The pump is used to provide water to the fire protection system or as a back-up source of water to the reactor pressure vessel, the emergency condensers, or the EDG cooling. Therefore, in accordance with CNG-OP-1.01-1002, the identified degraded condition required a functionality assessment for the Unit 1 diesel fire pump. The operability section of the condition report stated that operability was not applicable because the pump was not an SSC, but does impact fire and e-plan, but did not provide a basis for functionality. The inspectors determined that this did not meet the functionality assessment requirements of NMPNS procedure, CNG-OP-1.01-1002 and was a performance deficiency. The performance deficiency was not more than minor because, based on the indications described in other areas of the condition report - the pump completed the test run satisfactory, the alarm came in and reset during pump shutdown and oil level was checked after shutdown and found to be satisfactory - there was a reasonable expectation of functionality for the pump at the time that the degraded condition was identified.

CR-2011-009890 - On November 2, 2011, Unit 2 reactor building ventilation fan, 2HVR-F1C, failed to start in a timely manner during surveillance testing. The condition report was screened as a category 4 and NMPNS generated a WO to troubleshoot the fan. The 2HVR -F1C and 2HVR-F2C fans are used in emergency operating procedures to maintain the reactor building at a negative pressure relative to the atmosphere to prevent the release of airborne activity.

To maintain functionality, one of the two fans is required to remain functional.

Therefore, in accordance with NMPNS procedure, CNG-OP-1.01-1002, the identified degraded condition required a functionality assessment of the reactor building ventilation system be completed. The documented functionality assessment stated the condition does not involve a potential degraded or nonconforming condition impacting an SSC relied upon for TS; therefore there is no operability concern, but did not discuss functionality. The inspectors determined that this did not meet the functionality assessment requirements of NMPNS procedure, CNG-OP-1.01-1002. The performance deficiency was not more than minor because, based on the availability of the 2HVR-F2C fan and although the fan was slow to start it did eventually run and perform its function - there was a reasonable expectation of functionality for the reactor building ventilation system at the time that the degraded condition was identified.

b) Cause evaluations that did not comply with the fleet standard

For RCEs and ACEs, NMPNS procedures provide guidance regarding the use of analysis techniques. CNG-CA-1.01-1005 for ACEs states that one or more analysis techniques are to be used, while for RCEs, NMPNS procedure CNG-CA-1.01-1004 states that two or more analysis techniques are to be used. Guidance for available analysis techniques was provided in NMPNS fleet guideline CNG-CA-1.01-GL002, Cause Analysis Handbook. It listed the following techniques for use in establishing underlying causes: event and causal factor charting, comparative timeline, change analysis, Kepner-Tregoe, MORT, hazard barrier target analysis, the WHY staircase, common cause analysis, anatomy of an event, and task analysis. The handbook also stated that if the WHY staircase was selected as one of the analysis methods, it must be used in conjunction with another method. From among the cause analyses reviewed, the inspectors identified three examples where the standards provided by the fleet guidelines and NMPNS procedures were not followed:

CR 2012-011247 was written after operators declared Unit 1 primary containment inoperable on December 13, 2012, and CR 2013-003887 was written after primary containment isolation valves, IV-201-32 and IV-201-31, failed local leak rate testing on May 13, 2013. For both condition reports, the Management Review Committee directed that an ACE be performed to identify and correct the cause for each event. For each of these condition reports, NMPNS used two evaluation techniques, a WHY staircase and a failure modes and effects analysis (FMEA) to perform the cause analysis. However, the inspectors determined that for both cases, the ACE did not adhere to the NMPNS cause evaluation standard because an FMEA was not a technique described in the fleet guideline for cause analysis. In addition, the inspectors concluded that an FMEA cannot provide insight into programmatic and organizational issues that lead to an event as expected per NMPNSs guideline for ACEs. The inspectors determined that these performance deficiencies were not more than minor because, although the conduct of the cause evaluations did not adhere to the fleet standard for cause evaluations, the corrective actions identified and implemented in response to the events adequately addressed the conditions adverse to quality - inadequate preventative maintenance and incorrect flange bolt torque - that caused the events.

CR-2013-002926 was written on April 16, 2013, after Unit 1 experienced a loss of battery board 12, one of two safety-related direct current electrical sources, with a subsequent loss of shutdown cooling (SDC). To identify the causes that led to the loss of SDC, NMPNS performed an RCE. The evaluation techniques used in the RCE were a fault tree analysis and a comparative timeline. However, the RCE also included a discussion of the results for an event & causal factor analysis that concluded no other causes or contributors were identified that had not already been reviewed under the comparative timeline or fault tree analysis.

No documentation of the formal event & causal factor analysis was identified and the inspectors determined that this did not adhere to standards established in 5 of CNG-CA-1.01-GL002, Causal Analysis Handbook, which provided specific guidance for the development of an event and casual factor analysis chart. The inspectors determined that this performance deficiency was not more than minor because, although the conduct of the cause evaluation did not adhere to the NMPNS fleet standard for cause evaluations, the corrective actions identified and implemented in response to the event adequately addressed the condition adverse to quality - inadequate procedures - that caused the loss of SDC.

(3) Effectiveness of Corrective Actions

The inspectors concluded that corrective actions for identified deficiencies were generally timely and adequately implemented. For significant conditions adverse to quality, NMPNS identified actions to prevent recurrence. The inspectors concluded that corrective actions to address the sample of NRC NCVs and findings since the last problem identification and resolution inspection were timely and effective. The inspectors did, however, identify one NCV of 10 CFR 50, Appendix B, Criterion XVI, Corrective Action, because NMPNS did not identify and correct a condition adverse to quality associated with a support system for HPCS. The inspectors determined that NMPNS did not implement the corrective action intended to investigate an alarm condition for a non-safety-related support system for HPCS and that this would have identified the associated condition adverse to quality. This violation is described below.

c. Findings

Failure to Identify and Correct a Condition Adverse to Quality Associated With HPCS Medium Voltage Power Supply Cables

Introduction:

The inspectors identified a Green NCV of 10 CFR 50, Appendix B, Criterion XVI, Corrective Actions, because between November 5, 2012, and July 22, 2013, NMPNS did not promptly identify and correct a failed automatic de-watering system for the buried HPCS medium voltage power supply cable duct bank. As a result on July 22, 2013, NMPNS unexpectedly discovered significant water level in the two manholes that contained the buried HPCS cable duct bank. NMPNS subsequently determined that multiple level switches for the de-watering system had failed.

Description:

The Unit 2 HPCS system provides an injection source of high pressure water into the reactor vessel to restore and maintain inventory after accidents and transients. The system consists of a single pump with a dedicated standby EDG cooled by service water. Power supply cables from the standby EDG that supplies power to the pump are routed from the EDG located in the Unit 2 control building through an underground duct bank that passes through two manholes, identified as MH-1 and MH-3, and then to the pump located in the HPCS pump room in the reactor buildings lowest elevation.

On November 2, 2009, inspectors documented a violation of 10 CFR 50, Appendix B, Criterion III, Design Control, because NMPNS did not establish adequate measures to ensure that the Unit 2 HPCS pump power cables were maintained in an environment for which they were designed (NCV 05000410/2009004-01). In response to this violation, NMPNS generated a design change (ECP-11-000727) to install an automatic de-watering system in the two manholes that contain HPCS power supply cables, MH-1 and MH-3. NMPNS also revised PM procedure S-EPM-GEN-702, Associated Transformer and Switchyard PMs, to describe the functional testing and inspection requirements for the manholes and new de-watering equipment. The functional testing required included:

pumping down of the manhole sumps with the newly installed sump pumps if water level was present; confirming operation of the automatic sump pumps and associated automatic controls and alarms; and inspection of cables and other components located in the manholes.

On November 5, 2012, an operator conducting rounds discovered that the hi-hi water level alarm light was lit on the MH-1 sump pump control panel, 2DFM-IPNL103. The alarm light was very dim and when the operator conducted a lamp test the light became brighter. The operator concluded based on the results of the lamp test and the fact that the sump pump was not running, that the hi-hi level alarm light was not due to an actual hi-hi level in the sump. When the shift manager screened the condition report for this issue (2012-010137), he also concluded, based on indications, that it was likely that the automatic pumping system remained functional, but recommended troubleshooting the alarm to determine why the lamp was partially lit. In accordance with NMPNS procedure, CNG-CA-1.01-1000, Corrective Action Program, NMPNS screened this condition report as a category 3 significance condition report and on November 21, 2012, as allowed by the procedure, NMPNS closed the condition report to a priority 4 WO, C92081210.

The inspectors concluded based on discussions with NMPNS personnel and review of the associated condition report and WOs, that when operators identified the hi-hi level alarm they determined that manhole inspections would be completed during the last week of November 2012, and, as such, they expected that a priority 4 WO was sufficient to ensure that the recommended troubleshooting would be performed during that week.

The inspectors reviewed NMPNS work management procedures and determined that this expectation was reasonable. Priority 4 work items were defined in NMPNS procedure CNG-MN-4.01-1002, WO Initiation, Screening and Prioritization, as quarterly system maintenance items, such as surveillance tests, preventative maintenance, and maintenance conducted to restore degraded equipment to optimum operating condition.

On November 6, 2012, the electrical shop completed a work planning walkdown to prepare for performance of a switchyard and transformer PM that was scheduled to be completed the last week of November using S-EPM-GEN-702. This PM item included the MH-1 and MH-3 inspections. After the walkdown, electrical shop personnel questioned the need to perform the MH-1 and MH-3 inspections per attachment 2 of the procedure because the technicians knew an automatic de-watering system was recently installed in these manholes. The electrical shop provided this feedback to work planning and, contrary to site procedure, CNG-AM-1.01-1018, Preventative Maintenance Program, the planner removed the MH-1 and MH-3 (task 20 and 60 in the PM WO)inspections from the PM WO.

The inspectors determined based on discussion with NMPNS and a review of NMPNSs PM program procedures that in accordance with CNG-AM-1.01-1018, the changes suggested by the electrical shop personnel were PM scope changes and therefore required a preventative maintenance change request (PMCR). In accordance with the change process described by procedure CNG-AM-1.01-1018, the change request would have required system engineer/component specialist review to determine if the action was appropriate. NMPNS and the inspectors concluded, based on the changes engineering previously made to S-EPM-GEN-702 as part of ECP-11-000727, that the PMs would have not been deleted and would have been performed as scheduled if Engineering reviewed a PMCR for this proposed change. However, because the procedure for PM scope changes was not followed, the MH-1 and MH-3 inspections were removed from the work schedule and were not performed in November 2012 or May 2013, when they were originally scheduled. In addition, the inspectors determined that with the PMs deleted, the priority 4 WO issued for troubleshooting the hi-hi level alarm in MH-1, (C92081210), did not progress in planning and was not scheduled to be performed.

Once the PM work items for MH-1 and MH-3 were removed from the schedule, there was nothing to drive completion of the corrective action generated to follow-up on the MH-1 hi-hi level alarm identified during rounds on November 5, 2012. Therefore, on July 22, 2013, when performing WO C92018153, which removed several conduit seals in MH-1 and MH-3 (work that was included in the original de-watering system modification package but missed during installation in November 2012), NMPNS identified several feet of water in both MH-1 and MH-3 and determined that the automatic sump pumps were not de-watering the sumps as designed. NMPNSs subsequent investigation determined that multiple level switches for the automatic de-watering system had failed, and that water level in the manhole had risen to the bottom of the medium and low voltage cable duct bank. At the discovered water level several low voltage cables were wetted, but the HPCS medium voltage pump power supply cables were not in contact with the water and, therefore, remained operable. NMPNS replaced the failed level switches and initiated monthly manual pump downs for MH-1 and MH-3 until a more reliable level switch design could be installed. NMPNS documented the failed de-watering in the NMPNS CAP under CR-2013-006992.

10 CFR 50, Appendix B, Criterion XVI requires that measures shall be established to ensure that conditions adverse to quality are promptly identified and corrected. NMPNS procedure CNG-CA-1.01-1000, Corrective Action Program, implements this requirement at NMPNS. CNG-CA-1.01-1000, Attachment 4, Corrective Action Processing Summary, states that the due date guideline for Category 3 condition report corrective action item closure was 180 days from the date of discovery. The inspectors determined that the corrective action generated to investigate the cause of hi-hi level alarm in MH-1 was not completed in a timeframe that met this guideline, there were over 250 days between discovery of the hi-hi level alarm in MH-1 and discovery of the actual high level condition in MH-1 and MH-3. As stated above, based on inspector discussions with operations department personnel, the schedule for completion of the troubleshooting of the alarm failure was linked to the PM inspections for MH-1 and MH-3 that were intended to be performed every six months. These activities were originally scheduled to be completed in November 2012 and May 2013; however, because those PM tasks were inappropriately deleted by work planning on November 6, 2012, the degraded condition of the MH-1 and MH-3 dewatering systems was not identified and corrected.

Analysis:

The inspectors determined that not taking timely action in accordance with the NMPNS CAP to identify and correct a condition adverse to quality associated with the de-watering system for MH-1 and MH-3, which contained safety-related power supply cables for the HPCS pump, was a performance deficiency within NMPNSs ability to foresee and correct and should have been prevented. The inspectors determined that this performance deficiency was more than minor because if left uncorrected the failed automatic sump system would have become a more significant safety concern.

Specifically, with no PM to inspect and test the dewatering system and no WO scheduled to investigate the cause of the MH-1 hi-hi alarm, the inspectors determined that, based on NMPNSs previous experience of rising level in this manhole and wetting of these cables, it was not likely that NMPNS would identify the failed de-watering system before the HPCS power supply cables were wetted. Wetted cables become a more significant concern because, in accordance with industry and NRC operating experience, the long term reliability of medium voltage cables is negatively affected when wetted. In accordance with IMC 0609.04, Initial Characterization of Findings, and Exhibit 2 of IMC 0609, Appendix A, The Significance Determination Process for Findings At-Power, issued June 19, 2012, the inspectors determined that the finding was of very low safety significance (Green), because the finding was not a design or qualification deficiency, did not represent a loss of safety system function, and did not screen as potentially risk significant due to external initiating events.

The inspectors determined that this finding had a cross-cutting aspect in the area of human performance, resources, because NMPNS did not maintain long term plant safety by maintenance of design margins, minimization of long-standing equipment issues, minimizing PM deferrals, and ensuring maintenance and engineering backlogs which are low enough to support safety. Specifically, an NMPNS planner changed the scope of a PM task to eliminate inspecting MH-1 and MH-3 cable ducts every six months, and as a result, PM activities were not performed in November 2012 and May 2013. This error prevented NMPNS from identifying the condition adverse to quality associated with the HPCS medium voltage power supply cable duct bank de-watering equipment. H.2(a)

Enforcement:

10 CFR 50, Appendix B, Criterion XVI states that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances are promptly identified and corrected. Contrary to this between November 5, 2012, and July 22, 2013, NMPNSs CAP did not assure that a condition adverse to quality associated with the automatic de-watering system for MH-1, which contained safety-related medium voltage power supply cables for the HPCS pump, was promptly identified and corrected.

Specifically, on November 5, 2012, an operator on rounds identified that the hi-hi level alarm was partially lit on 2DFM-IPNL103, which provided indications for the de-watering system for MH-1. One week later a WO request was initiated to investigate the cause of the partially lit alarm and a priority 4 was assigned, however, between November 5, 2012, and July 22, 2013, no action was taken to plan or schedule the work. As a result on July 22, 2013, when NMPNS opened MH-1 and MH-3 to complete conduit seal removal in accordance with WO C92018153, water level was found in MH-1 and MH-3, and subsequent investigation determined that the multiple level switches for the system had failed. Because this violation was of very low safety significance (Green), and NMPNS has entered this performance deficiency into the CAP as CR-2013-006992, the NRC is treating this as an NCV in accordance with Section 2.3.2 of the NRC Enforcement Policy. (NCV 05000410/2013007-01; Failure to Identify and Correct a Condition Adverse To Quality Associated With HPCS Medium Voltage Power Supply Cables)

.2 Assessment of the Use of Operating Experience

a. Inspection Scope

The inspectors reviewed a sample of condition reports associated with review of industry operating experience to determine whether NMPNS appropriately evaluated the operating experience information for applicability to NMPNS and had taken appropriate actions, when warranted. The inspectors also reviewed evaluations of operating experience documents associated with a sample of NRC generic communications to ensure that NMPNS adequately considered the underlying problems associated with the issues for resolution via their CAP. In addition, the inspectors observed various plant activities to determine if the station considered industry operating experience during the performance of routine and infrequently performed activities.

b.

Assessment

The inspectors determined that NMPNS appropriately considered industry operating experience information for applicability, and used the information for corrective and preventive actions to identify and prevent similar issues when appropriate. The inspectors determined that operating experience was appropriately applied and lessons learned were communicated and incorporated into plant operations and procedures when applicable. The inspectors also observed that industry operating experience was routinely discussed and considered during the conduct of plan-of-the-day meetings and pre-job briefs.

c. Findings

No findings were identified.

.3 Assessment of Self-Assessments and Audits

a. Inspection Scope

The inspectors reviewed a sample of audits, including the most recent audit of the corrective action program, departmental self-assessments, and assessments performed by independent organizations. Inspectors performed these reviews to determine if NMPNS entered problems identified through these assessments into the corrective action program, when appropriate, and whether NMPNS initiated corrective actions to address identified deficiencies. The inspectors evaluated the effectiveness of the audits and assessments by comparing audit and assessment results against self-revealing and NRC-identified observations made during the inspection.

b.

Assessment

The inspectors concluded that self-assessments, audits, and other internal NMPNS assessments were generally critical, thorough, and effective in identifying issues. The inspectors observed that NMPNS personnel knowledgeable in the subject completed these audits and self-assessments in a methodical manner. NMPNS completed these audits and self-assessments to a sufficient depth to identify issues which were then entered into the corrective action program for evaluation. In general, the station implemented corrective actions associated with the identified issues commensurate with their safety significance.

c. Findings

No findings were identified.

.4 Assessment of Safety Conscious Work Environment

a. Inspection Scope

During interviews with station personnel, the inspectors assessed the safety conscious work environment at NMPNS. Specifically, the inspectors interviewed personnel to determine whether they were hesitant to raise safety concerns to their management and/or the NRC. The inspectors also interviewed the station ECP coordinator to determine what actions are implemented to ensure employees were aware of the program and its availability with regards to raising safety concerns. The inspectors reviewed the ECP files to ensure that NMPNS entered issues into the CAP when appropriate.

b.

Assessment

During interviews, NMPNS staff expressed a willingness to use the CAP to identify plant issues and deficiencies and stated that they were willing to raise safety issues. The inspectors noted that no one interviewed stated that they personally experienced or were aware of a situation in which an individual had been retaliated against for raising a safety issue. All persons interviewed demonstrated an adequate knowledge of the CAP and the ECP. Based on these limited interviews, the inspectors concluded that there was no evidence of an unacceptable safety conscious work environment and no significant challenges to the free flow of information.

c. Findings

No findings were identified.

4OA6 Meetings, Including Exit

On August 23, the inspectors presented the inspection results to Chris Costanzo, Site Vice President, and other members of the NMPNS staff. The inspectors verified that no proprietary information was retained by the inspectors or documented in this report.

ATTACHMENT:

SUPPLEMENTARY INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

C. Costanzo, Vice President

J. Stawley, Plant General Manager

K. Clark, Director of Security

J. Cole, General Supervisor of Radiation Protection

J. Dosa, Director of Licensing

G. Lozier, Director of Quality and Performance Assessment

F. Payne, Operations Manager

W. Rheaume, Director of Performance Improvement

M. Shanbhag, Senior Licensing Engineer

A. Sterio, Maintenance Manager

P. Swift, Manager of Engineering Services

T. Syrell, Manager of Nuclear Safety and Security

T. Verno, Director of Emergency Preparedness

LIST OF ITEMS OPENED, CLOSED, DISCUSSED, AND UPDATED

Opened and Closed

05000410/2013007-01 NCV Failure to Identify and Correct a Condition Adverse to Quality Associated With HPCS Medium Voltage Power Supply Cables (4OA2.1.c)