IR 05000263/2008009
ML083510254 | |
Person / Time | |
---|---|
Site: | Monticello |
Issue date: | 12/16/2008 |
From: | Pederson C Division Reactor Projects III |
To: | O'Connor T Northern States Power Co |
References | |
IR-08-009 | |
Download: ML083510254 (44) | |
Text
ber 16, 2008
SUBJECT:
MONTICELLO NUCLEAR GENERATING PLANT SPECIAL INSPECTION REPORT 05000263/2008009
Dear Mr. OConnor:
On November 3, 2008, the U.S. Nuclear Regulatory Commission (NRC) completed a special inspection at your Monticello Nuclear Generating Plant. The special inspection evaluated the facts and circumstances surrounding the loss of normal offsite power to non-safety buses and resultant reactor scram, and other complications that occurred on September 11, 2008. The scope of inspection was expanded to include the loss of normal offsite power that occurred on September 17, 2008 and the loss of shutdown cooling on September 20, 2008. The enclosed report documents the inspection findings, which were discussed on November 3, 2008, with B. Sawatzke and other members of your staff.
The inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.
The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.
Based on the results of this inspection, two NRC-identified and three self-revealed findings of very low safety significance were identified. Four of the findings involved violations of NRC requirements. However, because of their very low safety significance, and because the issues were entered into your corrective action program (CAP), the NRC is treating the issues as Non-Cited Violations (NCVs) in accordance with Section VI.A.1 of the NRC Enforcement Policy.
If you contest the subject or severity of these NCVs, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with a copy to the Regional Administrator, U.S. Nuclear Regulatory Commission - Region III, 2443 Warrenville Road, Suite 210, Lisle, IL 60532-4352; the Director, Office of Enforcement, U.S.
Nuclear Regulatory Commission, Washington, DC 20555-0001; and the Resident Inspector Office at the Monticello Nuclear Generating Plant. In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA by Gary Shear, Acting For/
Cynthia D. Pederson, Director Division of Reactor Projects Docket No. 50-263 License No. DPR-22 Enclosure: Inspection Report 05000263/2008009 w/Attachments:
1. Supplemental Information 2. Timeline (09/07/08 - 09/13/08)
3. Timeline (09/17/08)
cc w/encl: D. Koehl, Chief Nuclear Officer Manager, Nuclear Safety Assessment P. Glass, Assistant General Counsel Nuclear Asset Manager, Xcel Energy, Inc.
J. Stine, State Liaison Officer, Minnesota Department of Health R. Nelson, President Minnesota Environmental Control Citizens Association (MECCA)
Commissioner, Minnesota Pollution Control Agency R. Hiivala, Auditor/Treasurer, Wright County Government Center Commissioner, Minnesota Department of Commerce Manager - Environmental Protection Division Minnesota Attorney Generals Office
SUMMARY OF FINDINGS
This report covers a special inspection by four NRC Region III inspectors and one resident inspector during September through November of 2008. Five Green findings were identified by the inspectors. Four of the findings were considered Non-Cited Violations (NCVs) of NRC requirements. The significance of most findings is indicated by their color (Green, White,
Yellow, Red) using Inspection Manual Chapter (IMC) 0609, Significance Determination Process (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649,
Reactor Oversight Process, Revision 4, dated December 2006.
NRC-Identified
and Self-Revealed Findings
Cornerstone: Initiating Events
- Green.
A self-revealed finding of very low safety significance, associated with a NCV of 10 CFR 50, Appendix B, Criterion V, was identified for inadequate procedural guidance after operator response to the reactor scram and vessel isolation was complicated by rising water level. Specifically, the operating instructions for the control rod drive system were inadequate in that procedures did not provide direction to control the addition of water to the reactor pressure vessel when the operators determined it was necessary to restore level to the emergency operating procedure specified control band of +9 to
+48 inches. Additionally, the inspectors determined that the performance deficiency affected the cross-cutting area of Problem Identification and Resolution, having Corrective Action Program (CAP) components, and involving aspects associated with timely resolution of identified problems when safety concerns are raised under alternative processes (i.e., a procedure change request initiated as a result of a self-assessment)(P.1(e)). Operators took action to close the control rod drive valve that was causing water level to rise and used safety relief valve actuations to remove inventory to return water level to the band specified by the emergency procedure.
The finding was determined to be more than minor in accordance with IMC 0612,
Power Reactor Inspection Reports, Appendix B, Issue Screening, because it impacted the Initiating Events Cornerstone attribute of procedure quality with the objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown. The finding was determined to be of very low safety significance because it did not result in an actual impairment of mitigating systems or the reactor coolant system boundary. (Section 4OA3.2)
- Green.
The inspectors identified a finding of very low safety significance and NCV of 10 CFR 50.65 (a)(1) for the licensees failure to establish an effective monitoring and corrective action plan that included the 34.5kV underground feeder cables routed from 2RS to 2R transformer in the scope of a monitoring program that met the requirements of 10 CFR 50.65 (a)(1). The inspectors determined that the preventive maintenance and testing methodology implemented for the 34.5 kV cables was not sufficient to establish the condition of the cables and therefore the exemption of paragraph (a)(2) of 10 CFR 50.65 was not applicable. Additionally, the preventive maintenance and testing methodology implemented for the 34.5 kV cables did not provide the necessary information needed to ensure that the 2R transformer was capable of fulfilling its intended function and therefore the performance goal was not effectively assessed prior to the functional failures of the cables on September 11, 2008. The inspectors determined that the preventive maintenance and testing methodology implemented for the 34.5 kV cables, to identify deteriorating cable insulation conditions prior to failure, was inadequate and therefore the exemption of paragraph (a)(2) of 10 CFR 50.65 was not applicable. Additionally, the finding was determined to be cross-cutting in the area Human Performance, Work Practices, in that supervision and management oversight of work activities did not identify that the periodic maintenance and performance monitoring of the cables did not appropriately support the Maintenance Rule credited function during the periodic evaluations performed for the systems (H.4(c)).
This finding was determined to be more than minor in accordance with IMC 0612,
Power Reactor Inspection Reports, Appendix E, Example 7.d because a failure of the cables did occur resulting in a reactor scram and vessel isolation. In addition, it affected the Initiating Events Cornerstone attribute of equipment performance reliability.
Specifically, the failure to establish an effective monitoring and corrective action plan that included the 34.5kV underground feeder cables in the scope of a monitoring program that met the requirements of 10 CFR 50.65 (a)(1) contributed to lack of effective monitoring and early identification of degradation of these cables. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The 2RS to 2R transformer cable failures that occurred did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. Therefore, the finding screened as having very low safety significance. (Section 4OA3.4)
- Green.
The inspectors identified a finding of very low safety significance with no associated violation of regulatory requirements for the licensees failure to establish and implement an effective test control program that demonstrated that underground 34.5kV medium voltage cables subjected to submersion would perform satisfactorily in service.
Specifically, prior to the September 11, 2008, cable failure, the licensee failed to establish and implement an adequate test control program, and failed to ensure that appropriate cable testing was being periodically performed and that test results were trended to identify adverse trends prior to cable failures. In addition, NMC Corporate Directive CD 5.33, Underground Electrical Cable Management Program, dated April 6, 2006 required, in part, that Monticello Nuclear Generating Plant (MNGP) develop a site underground electrical cable management program to monitor and trend performance of underground electrical cables. The failure to conduct adequate cable testing potentially contributed to the failure of the underground submerged 34.5kV feeder cables routed from 2RS to 2R transformers. Additionally, the inspectors determined that the finding had a cross-cutting aspect in the area of Problem Identification and Resolution. Specifically, the licensee failed to incorporate known relevant internal and external operating experience related to numerous industry concerns and failures of similar underground submerged cables (P.2(a)).
This finding was determined to be more than minor because if left uncorrected the finding could become a more significant safety concern. The finding is of very low safety significance (Green) because it did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. An NCV was not issued because these cables were classified as non-safety related cables.
(Section 4OA3.4)
- Green.
A self-revealed finding of very low safety significance, associated with a NCV of 10 CFR 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, was identified following a loss of shutdown cooling on September 20, 2008. Specifically, operators failed to complete the shutdown checklist following the scram on September 11, 2008, and did not close the reference leg fill valve from the control rod drive system. When the control rod drive pump was started on September 20, the reference leg experienced a pressure spike and the resulting full RPS actuation and Group 2 isolation signals resulted in a loss of shutdown cooling. Additionally, the finding was determined to be cross-cutting in the area of Human Performance, Work Practices, in that the licensee failed to ensure supervisory and management oversight of work activities such that nuclear safety is supported. In this instance, operations shift management did not track implementation of the shutdown checklist to ensure completion (H.4(c)).
This finding was determined to be more than minor in accordance with IMC 0612,
Power Reactor Inspection Reports, Appendix B, Issue Screening, because it impacted the Initiating Events Cornerstone attribute of configuration control with the objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown. Inspectors determined that this finding was of very low safety significance using IMC 0609, Significance Determination Process,
Appendix G, Attachment 3, Phase 2 Significance Determination Process Template for BWR during Shutdown. (Section 4OA3.9)
Cornerstone: Mitigating Systems
- Green.
A self-revealed finding of very low safety significance, associated with a NCV of TS 5.4.1, was identified following a failure of the high pressure coolant injection (HPCI)
System to trip when reactor pressure vessel (RPV) water level reached the trip setpoint of +48 inches. Investigation revealed that the normally de-energized HPCI trip solenoid valve, SV-1, failed to trip promptly when actuated and was degraded due to improper reassembly of the solenoid valve after refurbishment in 1996 and degraded elastomers.
Follow-up investigation revealed that although a 2003 engineering evaluation recommended a periodic replacement of the elastomers in this valve as an enhancement action, no preventive maintenance activity was created or performed prior to the failure even though the recommended interval had been exceeded since the last overhaul. Additionally, the finding was determined to be cross-cutting in the area of Human Performance, Work Practices, in that supervision and management oversight of work activities did not identify that the preventive maintenance recommendation had not been resolved since 2003 (H.4(c)).
This finding was determined to be more than minor in accordance with IMC 0612,
Power Reactor Inspection Reports, Appendix B, Issue Screening, because it impacted the Mitigating Systems Cornerstone attribute of equipment performance with the objective of ensuring the availability, reliability and capability of systems to prevent undesirable consequences. Inspectors determined that this finding was of very low safety significance after completing a Phase 1 evaluation of the Mitigating System Cornerstone in accordance with IMC 0609, Attachment 4, Initial Screening and Characterization of Findings and answering all questions No in the Table 4a worksheet. (Section 4OA3.3)
REPORT DETAILS
Summary of Plant Event On September 11, 2008, at 10:50 p.m. CDT, Monticello Nuclear Generating Plant experienced a line fault on the supply line to the 2R transformer. The 1R transformer was out-of-service for planned maintenance when the event started. With both the 1R and 2R transformers unavailable, the offsite electrical power supply to the non-safety buses was lost, resulting in a reactor scram with loss of the normal heat sink (i.e., feed water, condensate, circulating water and recirculation systems were lost). The unit also experienced Group 1, 2 and 3 isolations of containment and the reactor pressure vessel. The 1AR transformer remained available and the safety-related buses automatically transferred to that source as designed. Both emergency diesel generators started and were running, but did not load as offsite power was available to the safety buses. Since the normal heat sink was lost as a result of main steam isolation valve closure and loss of electrical power to support equipment, operators used the reactor core isolation cooling system (RCIC), the high pressure coolant injection (HPCI), the safety relief valves (SRVs) and the torus cooling system for pressure and level control. All control rods fully inserted and the licensee decided to place the plant in Mode 4 (Cold Shutdown) pending assessment of the transient. Subsequently, the licensee restored the 1R transformer and returned power to the non-safety-related buses. As of September 15, the plant was in mode 4 (depressurized) with shutdown cooling in service.
On September 17, 2008 at 9:45 a.m. CDT, Monticello experienced a lockout of the 1R transformer after a self-powered man-lift struck a power line providing offsite power to the plant.
The 2R transformer was still out-of-service due to the September 11 partial loss of offsite power event. With both the 1R and 2R transformers unavailable, the offsite electrical power supply to the non-safety buses was lost, resulting in a loss of shutdown cooling. The 1AR transformer remained available and the safety-related buses again automatically transferred to that source.
Additionally, emergency diesel generators again started and were running, but did not load as offsite power was available to the safety-related buses. The licensee declared a Notice of Unusual Event (NOUE) at 10:30 a.m. CDT and the NRC subsequently entered Monitoring Mode. The licensee subsequently restored shutdown cooling.
On September 20, 2008, following restoration of the 1R transformer and realignment of electrical buses, operators restarted the first control rod drive pump. As the control rod drive hydraulic control units recharged, the primary instrumentation reference leg underwent a pressure surge and a Group 2 isolation occurred causing a loss of shutdown cooling. Operators were able to reset the isolation signal and restart the shutdown cooling system.
Following restoration of the 2R transformer, Monticello Nuclear Generating Plant exited the NOUE at 11:00 a.m. CDT on September 21, 2008. Based on the probabilistic risk and deterministic criteria specified in Management Directive 8.3, NRC Incident Investigation Program, and Inspection Procedure 71153, Event Followup, and due to concerns pertaining to equipment and operator performance during the event, a Special Inspection was initiated in accordance with Inspection Procedure 93812, Special Inspection, and Regional Procedure RP- 8.31, Special Inspections at Licensed Facility.
The Special Inspection focus areas included the following charter items:
- Establish a sequence of events of the September 11, 2008, event including: the lockout of the 2R transformer; the reactor scram; the automatic start of the high pressure coolant injection (HPCI) system, operation of the automatic depressurization system (ADS),operation of the emergency diesel generators (EDGs), manual isolation of the HPCI and ADS systems; and the shutdown of the 12 EDG.
- Monitor the licensees human performance investigation regarding operator performance during the September 11 event. Evaluate the actions taken by the operating crew to maintain reactor water level, including operators use and understanding of systems used for pressure/level control, instrumentation used to monitor reactor water level, and procedures/operating instructions used for event response.
- Evaluate the licensees cause evaluation regarding the HPCI failure to auto isolate upon a high reactor vessel water level during the September 11 event. Verify that for the licensee corrective actions, the post-maintenance testing will ensure HPCI operability, including whether the HPCI turbine will automatically and manually (i.e., via the push button) isolate upon high reactor water level (48 inches).
- Evaluate the licensees evaluation/troubleshooting of the cabling issues. Observe, if possible, the testing and as found condition of the cables and review the licensees determination of the cause of failure. In particular, review the licensees response to Generic Letter 2007-01 and determine whether the licensees program for monitoring inaccessible and underground cables was adequate. Verify that the licensee has appropriately considered extent of condition and that any cabling issues are appropriately addressed.
- Evaluate the licensees troubleshooting of the 12 EDG. Verify that the post-maintenance testing will validate that the speed control for the 12 EDG will operate as designed following a fast start and while running in the unloaded condition. Evaluate whether the licensee has determined the affect on the diesel reliability for the period in which it ran in an unloaded condition.
- Evaluate the licensees troubleshooting of the ADS system. Verify that the post-maintenance testing will validate that the timing and timer display work properly upon a low-low reactor water level signal.
- Establish a sequence of events for the September 17 partial loss of offsite power event; including, the lockout of the 1R transformer, auto start of the emergency diesel generators, and the loss and subsequent restoration of shutdown cooling.
- Evaluate the licensee cause evaluation and subsequent response for the September 17 partial loss of offsite power event.
- Evaluate the licensees cause evaluation regarding the loss of shutdown cooling that occurred on September 20, 2008.
4OA3 Special Inspection
.1 Establish the Sequence of Events for the September 11, 2008, Loss of Normal Offsite
Power
a. Inspection Scope
The Special Inspection charter charged the team with independently establishing the sequence of events for the September 11, 2008, event. To that end, inspectors reviewed operators logs, plant parameter recordings and trending information, and conducted interviews with operating crew members. In addition, the inspectors reviewed the sequence of events against the licensee generated sequence of events to ensure completeness and accuracy of both documents.
b. Findings
No findings of significance were identified. The inspector-generated sequence of events is included with this report as Attachment 1 and an overview narrative summary of the event was presented in this reports Summary of Plant Event.
.2 Monitor the Licensees Human Performance investigation and Evaluate the Actions
Taken by the Operating Crew to Maintain Reactor Water Level During the Event
a. Inspection Scope
The inspectors interviewed the licensed operators involved in the event. The inspectors also reviewed control room operator logs and licensee CAP documents to identify licensee procedures used during the event. Operators effectively responded to the immediate plant conditions and took action to stabilize reactor pressure vessel (RPV)pressure and level. Operators appropriately monitored the level and were cognizant of expected actions. Prompt actions were taken to control HPCI and RCIC injection rates to augment pressure control and control level within emergency operating procedures (EOP) guidelines. Operators restored water level to above 9 inches in order to clear isolation signals and appropriately realigned systems as water level in the RPV started to increase. The control room crew was aware of critical plant parameter changes and systems available to control those parameters. When RPV water level exceeded 48 inches, the crew recognized that the HPCI turbine did not trip as expected and inserted the Group 4 isolation. Water level continued to rise to the steady state water level of 80 inches until operators closed the control rod drive (CRD) charging water valve, CRD-14.
After valve closure, inventory loss through the SRV actuation lowered reactor water level.
b. Findings and Observations
- (1) Inadequate Procedural Guidance for Control Rod Drive System Valve Operation
Introduction:
A self-revealed finding of very low safety significance and an associated NCV of 10 CFR 50, Appendix B, Criterion V was identified for inadequate procedural guidance after the operator response to the reactor scram and vessel isolation was complicated by rising water level.
Description:
During reactor cooldown from the loss of normal offsite power (LONOP)and shutdown event in which all control rods inserted, the licensee implemented the reactor water level and pressure control portions of their EOP flowchart, C.5-1100, RPV Control. The normal means of reactor water level control was complicated by unavailability of a reactor water cleanup system flow path. The primary means of pressure control was achieved by cycling the SRVs. The high pressure coolant injection system was isolated and inoperable due to a failure to trip on high reactor water level, and RCIC was in service for pressure control. At 23:22 p.m. on September 11, 2008, one CRD pump was started. The lack of water rejection, operator deliberate action to raise water level to allow resetting of group isolation and scram signals, and the continued CRD pump injection eventually caused RPV water level to increase. The operators attempted to control reactor water level inventory and minimize pressure cycles by expanding the pressure band to 700-1056 psig in accordance with Flowchart C.5-1100. However, the water level eventually exceeded the upper allowed EOP water level band limit of +48 inches at approximately 03:15 a.m. on September 12. Interviews with the operating crew control room supervisor indicated that he was aware of the main steam line level of +108.5 inches and the operational concerns of water entering the HPCI and RCIC steam lines, but he allowed reactor water level to continue to increase above the EOP limit because he was unsure of the effect of isolating CRD charging flow and wanted to consult with the oncoming shift. Further interviews with the shift manager indicated that he was attempting to identify the procedural guidance needed to allow the crew to isolate CRD drive charging flow and continued to pursue this action until finally determining that neither the reactor scram procedure (C.4-A) nor the control rod drive procedure provided the necessary directions.
Although constantly monitored, the water level continued to increase for approximately three hours and reached a maximum indicated value of +110 inches from swell due to SRV actuation until the CRD accumulator charging water header manual isolation valve, CRD-14, was closed by the operators. The EOP provides guidance for maintaining reactor water level below +48 inches to preclude water entering the main steam lines.
The EOP bases document specifies the upper limit of +48 inches to preserve the availability of feedwater, HPCI and RCIC, and avoids moisture carryover into the main steam lines. Additionally, controlling water level to prevent water from entering the main steam lines is intended to preclude water hammer effects resulting from filling hot steam lines with comparatively cool water and thus adding additional thermal and hydrodynamic loading to the steam lines and the SRVs, potentially threatening their integrity.
Analysis:
The inspectors determined the operating procedures for the control rod drive system were inadequate following a reactor scram with the inability to reset the scram.
Procedures did not provide guidance to support the EOP direction to control the addition of water to the reactor pressure vessel from the CRD system when the operators determined it was necessary to restore level to the EOP specified control band of +9 to
+48 inches. The failure to establish adequate procedural guidance to support the EOP action is a performance deficiency. The finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because it impacted the Initiating Events Cornerstone attribute of procedure quality with the objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown. The finding was determined to be of very low safety significance, or Green, because it did not result in an actual impairment of mitigating systems or the reactor coolant system boundary.
The inspectors identified that the licensee had recognized the lack of procedure steps to isolate/throttle CRD charging water flow (CRD-14) following a self-assessment of the January 10, 2007, reactor scram. A procedure change request was issued to revise C.4-A, Reactor Scram, which is immediately performed following a reactor scram; however, the revised procedure was not issued prior to the September 11, 2008, scram.
The revision changes would have allowed operators to isolate/throttle CRD-14 when the scram could not be reset with control rods fully inserted. The inspectors determined that the untimely procedure changes to C.4-A following the January 2007 scram self-assessment was a contributing cause of the performance deficiency. The inspectors determined that the performance deficiency affected the cross-cutting area of Problem Identification and Resolution, having CAP components, and involving aspects associated with timely resolution of identified problems when safety concerns are raised under alternative processes (i.e., a procedure change request initiated as a result of a self-assessment). (P.1(e))
Enforcement:
Title 10 CFR 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings requires, in part, that Activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures or drawings. The emergency operating procedures provide directions for actions that are implemented by normal and abnormal plant operating procedures when the plant systems are used within their design functions and contain the necessary guidance to accomplish the EOP required direction. The Monticello EOP bases document and EOP C.5.1-1100, RPV Control, clearly address the need to control RPV injection to maintain level within +9 to +48 inches or to restore level to within those limits and supports the discussion of increasing water level transient.
Contrary to the above, the licensee failed to provide procedural guidance that would allow operators to throttle or shut the CRD-14 valve and thereby control RPV injection with the CRD system to maintain or restore water level within the EOP specified control band. Because this violation was determined to be of very low safety significance, and because this issue was entered into the CAP as CAP1150773, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 05000263/2008009-01). Operators took action to close the control rod drive valve that was causing water level to rise and used safety relief valve actuations to remove inventory to return water level to the band specified by the emergency procedure. The reactor scram abnormal procedure, C.4-A, was revised to include the appropriate procedural guidance for manipulation of the CRD-14 valve.
- (2) Operator Response to Rising RPV Water Level Not Consistent with EOP Bases During interviews, the operating crew control room supervisor stated that he was aware of the main steam line level of +108.5 inches and the operational concerns of water entering the HPCI and RCIC steam lines, but he allowed reactor water level to continue to increase above the EOP limit because he was unsure of the effect of isolating the CRD charging flow and wanted to consult with the oncoming shift. Further, the shift manager indicated that he was attempting to identify the procedural guidance to allow the crew to shut the control rod drive valve and continued to pursue this action until finally determining that neither the reactor scram procedure nor the control rod drive procedure provided the necessary directions. The shift manager stated that he had established +80 inches as the steady state water level limit where he was going to direct closure of the CRD-14 valve. Both of these decisions resulted in the shift management determination to delay stopping injection for over three hours after they had determined that action to be needed. The inspectors determined that the decision to continue injection until water level indication exceeded the level of the main steam lines did not support the conservative assumptions of the emergency operating procedures. The issue was determined to be minor since the injection was stopped before there was any impact to safety relief valves or RCIC. Additionally, the main steam isolation valves were closed and there was no indication of water carryover into the HPCI/RCIC steam supply lines or water hammer events as a result of the level increase.
.3 Evaluate the Licensees Cause Evaluation Regarding the High Pressure Coolant
Injection Failure to Auto Isolate on High Reactor Water Level
a. Inspection Scope
The inspectors reviewed operators logs, plant parameter recordings and computer trending information, and conducted interviews with operating crew members in clarifying the HPCI system response during the event. The licensees cause evaluation for the failure of the trip valve, SV-1, was reviewed, as well as vendor documentation for operation and maintenance of the valve assembly. In addition, the HPCI valve maintenance history was reviewed and interviews were conducted with engineering and maintenance staff members. The work package used to perform the repair and troubleshooting was reviewed in detail. The post-maintenance test was reviewed to ensure operability was established, and to ensure that the post-maintenance test verified automatic and manual trip functions on a high level.
b. Findings
- (1) Inadequate Maintenance on High Pressure Coolant Injection SV-1 Valve
Introduction:
A self-revealed finding of very low safety significance and an associated NCV of Technical Specification (TS) 5.4.1 was identified for an inadequate maintenance procedure after the HPCI turbine failed to trip on a valid high reactor water level signal.
Description:
Following the HPCI and RCIC systems auto start at -47 inches reactor water level, operators recovered vessel water level and attempted to stabilize level and pressure in accordance with the directions provided by the emergency operating procedures. Water level continued to rise and operators attempting to place HPCI in pressure control mode recognized that the turbine did not trip at +48 inches as designed.
The operator tried unsuccessfully to trip HPCI using the manual trip pushbutton and inserted a Group 4 isolation at the direction of the shift management to isolate HPCI.
Investigation revealed that the normally de-energized HPCI trip solenoid valve, SV-1, failed to trip promptly when actuated. SV-1 is designed to vent control oil from the hydraulic pilot valve on HO-7, HPCI Turbine Stop Valve, when actuated allowing HO-7 to close.
As-found bench testing of the solenoid valve after removal from the system identified that the solenoid failed to actuate at some minor degraded voltage (120 - 122 Vdc).
When the solenoid was disassembled, technicians found that the valve was not assembled correctly. The flux plate and washer coil were on top of the housing below the nameplate instead of under the coil. The solenoid was reassembled with the same components in the correct orientation and retested on the bench. When the solenoid valve was retested, it actuated with voltage as low as 102 Vdc. The valve was further disassembled and the diaphragm material was observed to be wrinkled in three places, but was not torn. The valve was refurbished using a vendor supplied rebuild kit and tested with the valve actuating at voltages as low as 98 Vdc.
Maintenance history review revealed that the valve was last refurbished in 1996 and is actuated on at least a quarterly basis when the system surveillance procedure is performed. There was no indication of a previous problem with the valve actuating during the quarterly testing, and the last successful test was performed on June 10, 2008. Follow-up investigation revealed that a 2003 engineering evaluation recommended a periodic replacement of the elastomers in this valve as an enhancement action based on the Electric Power Research Institute (EPRI) solenoid operated valve (SOV) preventive maintenance basis guideline. This guide specifies the elastomer replacement periodicity for this type valve (critical SOVs in mild environments)to be 10 years. Although the preventive maintenance task request had been initiated by the system engineer, no preventive maintenance activity was created or performed prior to the failure and prior to exceeding the recommended interval since the last overhaul.
Analysis:
Failure to develop and perform a preventive maintenance task to replace elastomers within the component lifetime identified by the industry standard is a performance deficiency. The finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because it impacted the Mitigating Systems Cornerstone attribute of Equipment Performance with the objective of ensuring the availability, reliability and capability of systems that respond to initiating events to prevent undesirable circumstances. The inspectors performed a Phase 1 SDP review of this finding using the guidance provided in IMC 0609, Significance Determination Process, and determined that the finding represented a loss of the HPCI system safety function and a Phase 2 SDP evaluation was warranted. The inspectors used the Monticello pre-solved SDP worksheet and determined that the duration of the finding was 47 days, which was half the time the solenoid functioned properly after the surveillance on June 10, 2008.
The duration used in the worksheet was one year and yielded a yellow risk result; however, the NRC benchmarking of the worksheet indicated that the risk result was over-estimated by one order of magnitude. Because the SDP pre-solved worksheet does not account for the actual 47 day duration and the Phase 2 risk result was over estimated, a Phase 3 evaluation was performed.
The senior reactor analyst (SRA) performed the risk evaluation using the Monticello Rev.
3P Standardized Plant Analysis Risk (SPAR) Model, Level 1, Change 3.45, created June 2008. To bound the risk, the SRA conservatively assumed that the HPCI pump started but failed to run (due to water intrusion in the HPCI steam line) with no operator recovery for the 47 day duration. The resulting risk significance was the low to mid E-7 range.
Accounting for operator action to isolate HPCI before water reaches the steam supply lines, which did occur and is detailed in the EOP, the risk significance is in the E-8 to E-9 range, which corresponded to a Green finding.
Additionally, the inspectors determined that the finding has a cross-cutting aspect in the area of Human Performance, Work Practices, in that supervision and management ongoing oversight of work processes and activities did not identify that the preventive maintenance recommendation had not been resolved since 2003 (H.4(c)).
Enforcement:
Technical Specification 5.4.1 requires in part that Written procedures shall be established, implemented, and maintained covering the applicable procedures recommended in Regulatory Guide 1.33, and Revision 2, Appendix A, February 1978.
Section 9 of Appendix A, Procedures for Performing Maintenance, paragraph (a)requires maintenance activities that can affect the performance of safety-related equipment to be performed using written procedures, and paragraph
- (b) specifies that maintenance schedules be developed to ensure replacement of parts that have a specific limited lifetime.
Contrary to the above, the licensee failed to develop and perform a preventive maintenance task to replace elastomers in the HPCI SV-1 valve within the 10 year periodicity specified in the EPRI SOV Preventative Maintenance Basis Guideline.
Because this violation was determined to be of very low safety significance, and because this issue was entered into the CAP as CAPs 1150495, 1150523, and 1150546, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 05000263/2008009-02). The valve was refurbished and tested satisfactorily with verification that the valve trips from both the manual pushbutton and the automatic level signal. A preventive maintenance task was created to replace solenoid valve elastomers on a 10 year periodicity.
.4 Licensees Evaluation and Troubleshooting of the Degraded 34.5kV Cables and
Adequacy of Licensees Monitoring Program for Inaccessible Underground Cables and Implementation of Related Information Provided in Generic Letter 2007-01.
a. Review of Licensee Response to Generic Letter (GL) 2007-01 and Adequacy of Licensees Monitoring Program for Inaccessible Underground Cables.
- (1) Inspection Scope:
The inspectors reviewed the licensees response to Generic Letter (GL) 2007-01, "Inaccessible or Underground Power Cable Failures that Disable Accident Mitigation Systems or Cause Plant Transients." In addition, the inspectors reviewed the licensee's implementation of the maintenance rule requirements with respect to the 34.5kV underground cables that failed on September 11, 2008, to verify that the cables were scoped into the maintenance rule and properly evaluated in accordance with 10 CFR 50.65.
- (2) Findings and Observations
Introduction:
While available documentation did not make it clear to the inspection team, the licensee stated that the 2R transformer function was being monitored to assess the condition of the 34.5kV cables instead of monitoring the performance and condition of the 34.5kV cables themselves to provide reasonable assurance that they were capable of fulfilling their intended function. The inspectors identified a (Green) NCV of 10 CFR 50.65 (a)(1) for licensees failure to establish an effective monitoring program against identified goals in a manner sufficient to provide reasonable assurance that the cables and supported transformer remained capable of performing their intended function.
Description:
The inspectors reviewed the licensees maintenance effectiveness implementation for the plant 34.5 kV cable system including system scoping.
Specifically, the inspectors reviewed, Monticello Maintenance Rule Program Document, Revision 11 and noted that Table 14.1, Monticello Maintenance Rule System Identification, did not include/list the 34.5kV AC system underground cables.
The licensee stated that these cables were being monitored as part of the 4.16 kV Station Auxiliary system, described in System Basis Document B.9.6, Revision 7, and their Unavailability and Maintenance Preventable Functional Failures (MPFFs) are tracked based on the 2R transformer set monitoring criteria. However, the inspectors noted that the 34.5 kV system cables were not listed in the scoping table on page 2 of system basis document B.9.6. The table specified individual performance criteria for transformers 1R, 2R, 1AR and for their associated 4.16 kV buses, but not for the 34.5 kV system cables which feed transformer 2R and which were classified as risk significant per Maintenance Rule requirements.
Concerning classification of these cables, MNGP fleet procedure FP-E-SE-02 defined critical components as components and structures that support maintenance rule important functions (i.e., nuclear safety and power generation). If a failure of the component or its structural support defeats or degrades any Maintenance Rule important function, then it is regarded as a critical component. Therefore, the 34.5 kV cables should have been classified as critical components, which support a Maintenance Rule important function.
NRC review of industry cable failure data prior to February, 2007 has indicated an adverse trend in unanticipated failures of underground/inaccessible cables that are important to safety. To address this adverse trend, on February 7, 2007, the NRC issued Generic Letter 2007-01, "Inaccessible or Underground Power Cable Failures that Disable Accident Mitigation Systems or Cause Plant Transients." The GL was issued to:
- (1) inform licensees that the failure of certain power cables can affect the functionality of multiple accident mitigation systems or cause plant transients,
- (2) inform licensees that in the absence of adequate monitoring of cable insulation, equipment could fail abruptly during service, causing plant transients or disabling accident mitigation systems, and
- (3) ask licensees to provide information on the monitoring of inaccessible or underground electrical cables. The GL required, in part, that licensees submit a written response, and
- (1) provide a history of inaccessible or underground power cable failures for all cables that are within the scope of 10 CFR 50.65 (the Maintenance Rule) and
- (2) describe inspection, testing and monitoring programs to detect the degradation of inaccessible or underground power cables that support EDGs, offsite power, emergency service water (ESW), service water, component cooling water and other systems that are within the scope of 10 CFR 50.65 (the Maintenance Rule).
The GL further stated that cable degradation can be detected prior to failures through techniques for measuring and trending the condition of cable insulation. Licensees can assess the condition of cable insulation with reasonable confidence using state of the art testing including one or more of the following testing techniques: partial discharge testing, time domain reflectometry, dissipation factor testing, and very low frequency AC testing. Licensees can replace faulty cables during scheduled refueling outages prior to cable failure that would challenge plant safety.
The MNGP provided the response to NRC Generic Letter 2007-01 on May 7, 2007. In the response, the licensee stated that a Cable Condition Monitoring Program was created at MNGP in 1998 in an attempt to identify cables, which require replacement prior to failure. Program guidance was provided in procedure EWI-08.19.01, Cable Condition Monitoring Program. A list of cables that were considered to be high risk for failure was generated periodically, which included a replacement and inspection plan for these cables. A cable risk factor (CRF) was assigned to each cable based on ampacity, insulation type, installed location, criticality, radiation exposure, temperature exposure, etc. Those cables with the highest risk factor were to receive priority for visual inspections, more frequent testing and replacement. In addition, MNGP had committed to implement additional programs for cable condition monitoring with the license renewal project. All cables required for continued safe plant operation were to be included in this program. However, during this inspection, the inspectors noted that only 480V and 4.16 kV cables were in the scope of this program. The 34.5 kV cables that failed on September 11, 2008, were omitted from the scope of this program.
The MNGP response to the GL stated that most electrical cables at MNGP are located in dry environments. However, some cables may be exposed to condensation and wetting in inaccessible locations such as conduits, cable trenches, cable troughs, duct banks, underground vaults or direct buried installations. When an energized medium-voltage cable is exposed to wet conditions for which it is not designed, water treeing or a decrease in the dielectric strength of the conductor insulation can occur and this can potentially lead to electrical failure. The response stated that to date, Butyl Rubber cables, either direct buried or installed in underground conduits, have the highest failure rate and that both 480 VAC and 4.16 kV rated cables are susceptible.
MNGP further stated that in the license renewal aging management program being developed, periodic actions will be taken to prevent cables from being exposed to significant moisture, such as inspecting for water collection in cable manholes and conduit, and draining water, as needed. The program also stated that in-scope, medium-voltage cables exposed to significant moisture and significant voltage are tested to provide an indication of the condition of the conductor insulation. Additionally, the program stated that the specific type of test performed would be determined prior to the initial test and would be a proven test for detecting deterioration of the insulation system due to wetting, such as power factor, partial discharge, polarization index, or other testing that is state-of-the-art at the time the test is performed.
All 10 cable failures reported by MNPG in response to GL 2007-01 were either direct buried or installed in underground conduits and had Butyl Rubber insulation. The cause of cable failures included high moisture within the cable, water in cable conduit, faulting in underground cable, brittleness and cracking of cable insulation due to aging, and grounds found during meggering. The inspectors noted that these cables were not designed nor qualified for continuous submergence.
The inspectors determined that inspection, monitoring, testing and trending of the submerged 34.5 kV cables routed from 2RS to 2R transformers lacked rigor and management attention and were not effectively implemented to identify deteriorating cable insulation conditions prior to failure. The inclusion of the 34.5 kV cables in the scope of the monitoring program was necessary because failure of these cables could potentially prevent safety-related components fed from the safety-related buses (#15 and #16) from fulfilling their safety-related functions. The inspectors noted that these cables were not designed nor qualified for submergence. Therefore, the licensees monitoring and testing methodology must ensure that inaccessible cables (e.g., routed underground) be capable of performing their function when subjected to anticipated environmental conditions such as moisture or continuous submergence. Cable failures that could disable risk-significant equipment are expected to have monitoring programs to demonstrate that the cables can perform their safety function when called upon.
Analysis:
The inspectors determined that the preventive maintenance and testing methodology implemented for the 34.5 kV cables was not sufficient to establish the condition of the cables and therefore could not support the necessary information needed to provide reasonable assurance that the 2R transformer was capable of fulfilling its intended function. The inspectors determined that the licensees failure to establish a monitoring plan and corrective action plan that included the 34.5 kV underground feeder cables routed from 2RS to 2R transformer in the scope of a monitoring program that met the requirements of 10 CFR 50.65 (a)(1), was a performance deficiency warranting a significance determination. The finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because if left uncorrected the finding would become a more significant safety concern. In addition, it affected the Initiating Events Cornerstone attribute of Equipment Performance Reliability as well as the cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. Therefore, the finding screened as having very low safety significance (Green). Additionally, the finding was determined to be cross-cutting in the area Human Performance, Work Practices, in that supervision and management oversight of work activities did not identify that the periodic maintenance and performance monitoring of the cables did not appropriately support the Maintenance Rule credited function (H.4(c)).
Enforcement:
Title 10 CFR Part 50.65 (a)(1) requires, in part, that licensees monitor the performance and condition of structures, systems, or components (SSCs), against licensee established goals, in a manner sufficient to provide reasonable assurance that such SSCs, as defined in paragraph
- (b) of the rule, are capable of fulfilling their intended function.
Paragraph (a)(2) of the rule provides an exemption from the monitoring of paragraph (a)(1) if the performance or condition of a structure, system, or component is being effectively controlled through the performance of appropriate preventive maintenance such that the component remains capable of performing its intended function.
Paragraph (a)(3) of the rule requires that periodic evaluation of the performance and condition monitoring activities and associated goals and preventive maintenance activities taking into account industry-wide operating experience.
Title 10 CFR 50.65 (b)(2) requires, in part, that the scope of the monitoring program specified in 10 CFR 50.65 (a)(1) include non-safety-related SSCs whose failure can prevent safety-related SSCs from fulfilling their safety-related function and that are relied upon to mitigate accidents or transients or are used in plant emergency operating procedures.
Contrary to the above, as of September 11, 2008, the licensee failed to establish a monitoring and corrective action plan that included the 34.5 kV underground feeder cables routed from 2RS to 2R transformer in the scope of a monitoring program that met the requirements of 10 CFR 50.65 (a)(1). Specifically, the inspectors determined that the preventive maintenance and testing methodology implemented for the 34.5 kV cables, to identify deteriorating cable insulation conditions prior to failure, was inadequate and therefore the exemption of paragraph (a)(2) of 10 CFR 50.65 was not applicable. Additionally, the preventive maintenance and testing methodology implemented for the 34.5 kV cables did not provide the necessary information needed to provide reasonable assurance that the 2R transformer was capable of fulfilling its intended function and therefore the performance goal was not effectively assessed prior to the functional failures of the cables on September 11, 2008. Because of the very low safety significance of this finding and because the finding was captured in the licensees CAP as CAPs 01151583, 01151315, 01151402 and 01152518, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 05000263/2008009-03).
b. Review of Licensees Evaluation, Troubleshooting and Corrective Actions to Address the Cabling Issues
- (1) Inspection Scope The inspectors reviewed corrective action documents, work orders, the licensees responses to the NRCs GL 2007-01, the cable condition monitoring program and implementation, industry standards and documents related to the testing of medium voltage cables and the condition of the failed cables. In addition, the inspectors observed ongoing vendor Tan-Delta (TD) and Partial-Discharge (PD) testing activities to identify existing degraded cable insulation and terminations.
- (2) Findings and Observations
Introduction:
The inspectors identified a finding of very low safety significance (Green)for the licensees failure to establish and implement an effective test control program that demonstrated that 34.5kV reliability related, non-safety-related underground medium voltage cables, subjected to submersion, would perform satisfactorily in-service.
Specifically, prior to the September 11, 2008, failures of the 34.5kV underground and submerged feeder cables routed from 2RS to 2R transformers, the licensee failed to establish and implement an adequate test program to ensure that 34.5kV cable testing is being periodically performed and test results trended to identify adverse trends prior to cable failure. These cables feed 4.16kV non-safety buses #13 and #14, which feed 4.16kV safety-related buses #15 and #16.
Description:
On September 11, 2008, a double phase-to-ground fault had occurred on the underground feeder cable routed from 2RS (345kV/34.5kV) transformer to the 2R (34.5kV/4.16kV) transformer. The 2R transformer feeds safety-related buses #15 and
- 16 via non-safety but reliability related and risk significant buses #13 and #14. Each of the three phase feeder circuits had two parallel XLPE (cross-linked-polyethylene)insulation cables rated at 35 kV, size 750 kcmil, aluminum conductor. The licensee determined that the splice on one of the A-phase, underground direct buried cables, had failed. In addition, the insulation on one of the B-phase underground cables, which was routed in conduit from a manhole to the 2R transformer J-box, had failed. This cable was found submerged and wet as the manhole was full of water and above the level of the conduits. These conditions can contribute to water treeing if left uncorrected. Water treeing phenomenon is a slow process by which a breakdown of the cable insulation properties may lead to cable failure. The inspectors noted that water treeing has been most often associated with XLPE insulation and certain polyethylene insulations.
MNGP DBD-B.09.06, Design Basis Document for 4160V AC System, Revision D, section 4.2.2, Non-Safety-Related Requirements, stated, in part, that the 4160V AC system shall provide adequate electrical power, protection, and control to 4160V AC loads with non-safety-related functions, required to support safe shutdown in an Appendix R event.
NMC Corporate Directive (CD) 5.33, Underground Electrical Cable Management Program, was issued on April 6, 2006. The directive required that MNGP, and all other fleet nuclear plants, develop a site specific process to comply with 10 CFR 50.65, Requirements for Monitoring the Effectiveness of Maintenance at Nuclear Plants, which requires that each nuclear plant monitor the performance or condition of systems, structures and components (SSCs), against licensee-established goals, in a manner sufficient to provide reasonable assurance that such SSCs are capable of fulfilling their intended functions. Corporate Directive 5.33 required that such goals shall be established commensurate with safety and, where practical, take into account industry-wide operating experience. Corporate Directive 5.33 further stated that an effective cable management program is an integral part of this strategy and that cables that are operationally important as well as cables that are safety-related and/or important to safety shall be within the scope of this standard.
The inspectors determined that as of September 29, 2008, no action had been initiated by MNGP to develop and implement the specific requirements delineated in NMC CD 5.33. An effective underground cable program (testing, inspecting, trending, etc.)
has not been established nor implemented to demonstrate that underground medium voltage cables, subjected to submersion, would perform satisfactorily in-service.
Consequently, the medium voltage cable testing program at MNGP was inadequate and failed to provide any indication of declining cable performance or imminent failure of the 2RS to 2R transformer cables before the actual failure on September 11, 2008. The licensee initiated CAPs 01151583 and 01155038 to document and address this issue.
In addition, the inspectors noted that Plant Excellence Action Item GAR 01094860 was initiated to formulate an action plan to implement cable testing. Corrective Action Program 01094860 was issued on June 1, 2007 which specified assignments and due dates in 2007 and 2008, to perform benchmarking, to determine best test methods to inspect/test buried cables to consider combining the license renewal program proposed testing with the existing cable condition monitoring program, to investigate procurement of TD test equipment, and to identify 480V and 4.16 kV cables (not 34.5 kV cables)susceptible to moisture intrusion whos failure would significantly impact plant operations. Benchmarking had been completed; however, the remaining items still needed to be accomplished. Document XPLA 01094860-06 delineated the buried cables strategy as noted in MNGP response to GL 2007-01.
The inspectors reviewed MNGP procedure EWI-08.19.01,Cable Condition Monitoring Program (CCMP), dated June 28, 2001, and determined that the licensee failed to include the 2RS to 2R transformer underground feeder cables in the scope of this program because the cables had no assigned cable numbers and therefore were not listed in the licensees Cable and Raceway Information System (CARIS) program.
Consequently, the cables were not evaluated by engineering for risk of failure and were not risk ranked. The CCMP was created at MNGP in 2000 as a result of a previous NRC commitment. The CCMP was initiated to identify high risk for failure cables which require replacement prior to failure. All cables required to support plant operating systems were required to be included in this program. Engineers have used guidance provided in EWI-08.19.01 to evaluate risk significance for failure of the listed cables based on cable ampacity, insulation type, installed location, criticality, radiation exposure, temperature exposure, etc. The cable data collected from the CCMP has been used to assign a CRF to each cable. Those cables with the highest risk factor were targeted to receive priority for visual inspections, more frequent testing and replacement. Following NRC questioning on September 21, 2008, the licensee initiated CAP 01151402 to document the failure to include the 2RS to 2R transformer feeder cables in the CCMP and to risk rank the cables.
The inspectors reviewed testing performed on the failed cables prior to September 2008 and noted that these cables had been previously meggered in 1991 and in 1999.
Work Order (WO) 9906223, dated June 10, 1999, which was performed following a 2R transformer CLIP fuse failure/replacement on B phase, documented that the megger test results were acceptable but showed a downward trend (3200 Meg-ohms to 1500 Meg-ohms) from the 1991 test results, on the B phase cable. Insulation resistance testing provides meaningful results when compared with historical test data on the same cable. However, the licensee failed to follow up on this concern raised by the system engineer. No other state of the art test methodologies have been adopted for testing these cables, until after their failure that resulted in the September 11, 2008, event. The inspectors also noted that these 34.5 kV cables were meggered in 1991 and 1999 at 5000 Vdc, for one minute. When questioned by the inspectors, the licensee could not provide vendor or industry standards that endorsed/confirmed adequacy of test results when testing 34.5 kV cables at 5000 Vdc.
The inspectors determined, based on industry experience and review of IEEE standards, that meggering tests alone of the 34.5kV cables would not provide reasonable assurance that cable degradation would be identified prior to cable failure, and as such, were inadequate to demonstrate that structures, systems, and components would perform satisfactorily in service.
In a related 2006 NRC inspection of the licensees program for inaccessible medium voltage cables, NRC License Renewal inspection report 05000263/2006006, Division of Reactor Safety (DRS), stated in section B2.1.21, Inaccessible Medium Voltage Cables Not Subject to EQ, that the inspectors concluded that the inaccessible medium-voltage cables not subject to 10 CFR 50.49 requirements program, when implemented as described in the required enhancements, will effectively manage aging effects, since it will incorporate periodic inspections and state of the art testing techniques. The report further stated that implementation of this program will provide reasonable assurance that the effects of aging will be managed such that components within the scope of the program will perform their intended functions consistent with the current licensing basis of the period of extended operations. In addition, MNGP had committed to implement additional programs for cable condition monitoring with the license renewal project.
However, the inspectors determined that these programs had not been scheduled for implementation until prior to the period of extended operation (2009). Therefore, from 2006 to 2008, no interim underground cable monitoring program had been implemented.
The inspectors concluded that the cable monitoring and testing program at Monticello for the 34.5kV cables was inappropriate based on meggering test results and given that the objective of the cable condition monitoring program was to periodically assess the condition of inaccessible underground power cables that have been submerged but had not been designed nor qualified for such an application.
Analysis:
The inspectors determined that the licensees failure to establish an appropriate testing program that adequately demonstrated that the medium-voltage cables subjected to submergence would perform satisfactorily in service was a performance deficiency and a finding. The finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because if left uncorrected the finding would become a more significant safety concern. In addition, it affected the Initiating Events Cornerstone attribute of Equipment Performance Reliability as well as the Initiating Events Cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations.
The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available, and the finding did not increase the likelihood of a fire or internal or external event. Therefore, the finding screened as having very low safety significance (Green).
Additionally, the inspectors determined that the finding had a cross-cutting aspect in the area of problem identification and resolution. Specifically, the team determined that the licensee had evaluated and was aware of numerous industry external and Monticellos internal (in the response to GL 2007-01) events where underground submerged cables faulted to ground causing a LOOP and forced shutdowns. However, the licensee has failed to incorporate this relevant internal and external operating experience information into plant processes (e.g., implement periodic inspection and draining of manholes and perform state of the art testing) on underground submerged cables to identify adverse trends before cable failure. (P.2(a)).
Enforcement:
The failure to establish and implement an effective test control program that demonstrated that underground 34.5kV non-safety-related cables subjected to submersion would perform satisfactorily in service was not an activity affecting quality subject to 10 CFR Part 50, Appendix B, because the cables were classified as non-safety-related. Therefore, while a performance deficiency existed, no violation of NRC regulatory requirements occurred. This is considered a finding of very low safety significance (FIN 05000263/2008009-04).
The licensee has implemented corrective actions following the cable failures and performed TD and PD testing on the 2R to 2RS cables. The testing identified inadequate splices and electrical pot head connections. The licensee replaced all cable splices and corrected the pot head connections. The licensee initiated CAPs 01151179 and 01151197 to address identified cable related testing issues. In addition, the licensee planned to add the failed cables to the cable monitoring program and select a suitable testing methodology for future cable testing.
c. Assess the Licensees Corrective Actions and Extent of Condition Evaluation Associated with the Underground Cable Failures
- (1) Inspection Scope The inspectors assessed the licensees ongoing corrective action activities, reviewed manhole and cable design documents and drawings, observed in-field cable testing activities, reviewed related CAPs and work orders, attended licensee debriefs, interviewed personnel involved in the licensees root cause evaluation and reviewed subsequent cable testing and manhole inspection results to assess the licensees extent of condition evaluation.
- (2) Findings and Observations No findings of significance were identified; however, the following observations were identified.
- (a) Review of Cable Manhole Monitoring Program The inspectors determined that scheduled periodic actions to prevent inaccessible (e.g., in conduits, cable trenches, cable troughs, duct banks, underground vaults, or direct buried) medium-voltage cables from being exposed to significant moisture, such as inspecting for water collection in cable manholes and conduit, and periodically draining water from manholes had not been implemented as of the September 11, 2008, event. The licensee stated that they planned to implement these actions prior to the start of the period of extended operation. Monticello Nuclear Generating Plant License Renewal Commitment MO5037A addresses medium voltage underground cable.
During the event follow-up inspection of the manhole that contained the 2RS to 2R cables, the licensee noted a large amount of standing water when the manhole cover was opened. The licensee did not establish a requirement to periodically conduct inspections for water in manholes containing medium voltage cables. Consequently, the licensees extent of condition analysis focused on similar underground inaccessible periodically submerged cables because the cause of the cable failures was potentially associated with continuously submerged underground cables in conduits or direct buried. All cables within the scope of the Maintenance Rule were considered; however, the licensee identified a total of 18 manholes that contained portions of similar medium voltage cables that were running underground and could have been periodically submerged in water. The licensee did not identify standing water in any of the additional manholes inspected. However, based on the condition of the manholes in pictures obtained during the inspection, it is possible that water had existed and had drained into the ground through the manholes gravel floor.
On September 17, 2008, the inspectors reviewed Monticello Nuclear Generating Plant (MNGP) drawing NF-74413-4, MNGP Underground Services Electrical Power, Revision 77, which showed the locations of some of the 18 manholes to be inspected.
The inspectors identified that some of the manholes scheduled for inspection had no assigned numbers and were not depicted on the drawing. The inspectors questioned whether all manholes containing such cables had been identified on the design drawings. On September 24, 2008, the licensee initiated CAP 01151963, Generate a CAP for Manholes without Identification on Drawings, to document and address this and other noted drawing discrepancies.
- (b) Review of Monitoring and Testing of Underground Cables Routed from 1AR Transformer to 4.16kV Safety-Related Buses 15 and 16 During review of MNGP drawing NF-36298-1, MNGP Electrical Load Flow One Line Diagram, Revision 79, the inspectors identified that inaccessible underground medium-voltage feeder cables A511-X04/1 and A610-X04/1, routed from 1AR (13.8/4.16kV)transformer to 4.16kV safety-related redundant buses #15 and #16, respectively, had no manholes that could be opened to inspect the condition of the buried cables. These inspectors determined that Monticello Maintenance Rule Program System Basis Document B.9.6, 4.16kV Station Auxiliary, Revision 7, designated the A511-X04/1 and A610-X04/1 cables as risk significant and provided a safety-related function. Updated Final Safety Analysis Report Appendix J and licensees CARIS program described the cables as Division I-Red and Division II-Green fire zones (associated with Appendix R requirements). These cables were not designed nor qualified for submergence. The inspectors noted that these 750 mcm cables had Butyl-Rubber insulation, are continuously energized, and had not been inspected nor tested since installation in 1970. The cables were originally scheduled for replacement in 2001, based on a licensees CRF ranking of 24 for this cable. This CRF number placed this cable in the higher risk for failure category. However, the replacement has been rescheduled a number of times due to lack of licensee follow-up action and funds.
The inspectors also noted that all 10 MNGP cable failures reported by the licensee in response to NRC GL 2007-01 had Butyl-Rubber insulation. The cables were included in the licensees cable condition monitoring program and were evaluated to have a relatively high risk CRF of 23 (highest CRF assigned to a cable was 27). Upon discovery, the licensee issued a plant restart mode restraint until the cables were tested and initiated WO 368889 to test these cables prior to plant restart. On September 24, 2008, the cables were megger tested and found acceptable. Also, as part of the MNGP power uprate project, the licensee considered replacement of these cables in 2009 (modification EC 12816).
- (c) Observation of Ongoing Cable Testing Activities As part of the extent of condition, on September 18, 2008, the inspectors observed the licensee perform testing of 2RS to 2R transformer feeder cable leads using WO 368428, Revision 0. The inspectors attended the pre-job briefing and observed portions of the Tan-Delta (TD) testing performed by an outside contractor to identify failed insulation on the 6 conductors tested (2 per phase). Partial Discharge (PD) testing was then performed to determine cable conductor insulation condition. Tan Delta test results indicated that the A2 phase conductor (which failed on September 11, 2008) had a severe fault at a second splice location and the splice needed to be replaced. The PD testing identified termination problems in the B2 and C2 cable conductor splices and at the stress cones. To address these deficiencies the licensee repaired/replaced the faulted cable terminations (CAP 01151179, dated September 19, 2008). The licensee managements decision to obtain outside contractor expertise that employed state of the art testing techniques (TD and PD testing) was appropriate and effective in identifying additional feeder cable conductor insulation/termination anomalies, which needed to be addressed prior to re-start.
Corrective Action Program document CAP 01151878 dated September 24, 2008, documented that samples from the A2 failed cable were evaluated by the licensee and a number of vented trees were identified. This indicated cable degradation. The licensee initiated weekly inspections for water in the manholes via WO 569332.
d. Assess Licensees Root Cause Evaluation of The Cable Failures As of the end of this inspection, the licensee had not completed the root cause evaluation (CAP 01150362) for the failed cables. The inspection team discussed the progress of the evaluation with licensee staff and evaluated much of the information gathered during the licensees investigation. This information was discussed in more detail under the previous paragraphs in section 40A3.4 of this report. The team determined that the scope of the licensees root cause was appropriate and that immediate and short term corrective actions taken following the extent of condition evaluation addressed the immediate regulatory concern.
.5 Evaluate the licensees troubleshooting and repair of the #12 Emergency Diesel
Generator
a. Inspection Scope
In addition to evaluating the licensees troubleshooting and repair of the 12 EDG, the charter required inspectors to verify that the post-maintenance testing validated that the speed control for the 12 EDG would operate as designed following a fast start and while running in the unloaded condition. Further, the charter required that the team evaluate whether the licensee determined the affect on the diesel reliability for the period in which it ran in an unloaded condition.
Inspectors reviewed licensee corrective action documents, operator logs, and maintenance work packages. Inspectors also interviewed operations, maintenance and engineering personnel involved with the 12 EDG operation, troubleshooting, and repair.
b. Findings and Observations
At 22:47 p.m. on September 11, 2008, the 2R transformer locked out and a reactor scram occurred. The EDGs automatically started on bus low voltage as designed and the safety buses automatically transferred to the 1AR transformer leaving the diesels running in standby with a fast start signal. At 00:40 a.m. on September 12, the 11 EDG was shut down using B.09.08-05, Emergency Diesel Generators. Operators attempted to perform the shutdown for the 12 EDG, but found that generator frequency was slightly higher than the band specified in step 5 of the diesel shutdown procedure. That step states, Using GSC-2/CS No. 12 DIESEL GEN SPEED ADJUST, adjust the speed of 12 EDG to between 60.4 and 60.6 Hz at Panel C-08. Since speed was 60.7 Hz, the operator attempted to lower the machine speed to within the band, but received no response. In accordance with site expectations for procedure execution, he stopped and reported the unexpected response to the supervisor. When the issue was reported to maintenance and engineering, the report stated that the EDG could not be shut down from the control room instead of more precisely reporting that the control room could not lower speed using the governor speed control. The machine continued to run unloaded while a troubleshooting plan was developed.
At 13:45 p.m. on September 12, personnel shut the machine down from the local panel and continued troubleshooting. Based on the incomplete information provided, the individuals developing the troubleshooting plan made some basic assumptions that were incorrect. For example, since they believed the control room had actually tried to stop the machine from the control room, they assumed a malfunction of the stop circuit was associated with the problem. They initially identified the cause to be the failure of the engine stop delay relay coil to energize, preventing the fast start lock-in from releasing.
The licensee isolated the 12 EDG to replace the coil on September 15, and returned the machine to standby on September 16, while awaiting the post-maintenance test run.
During the partial loss of offsite power on the morning of September 17, the 12 EDG automatically started but experienced a similar issue with the inability to either remotely or locally reduce speed. The system engineer was able to immediately identify that the machine was operating in accordance with the design. According to system drawings, on a fast start signal, the governors lower function is disabled until either the output breaker is closed or the machine is shut down. Operators did not encounter a similar problem on the 11 EDG because frequency did not have to be adjusted prior to shutdown. The procedure did not provide the system knowledge the operators lacked regarding operation of the EDG speed control with a fast start signal present.
Bench testing of the engine stop delay relay coil after it was removed from the system indicated that although it appeared visually degraded, the coil functioned correctly. The operating crew misidentification of the problem with the diesel control circuit contributed to running the diesel unloaded for an extended period of time as troubleshooting plans were prepared and subsequent removal from service for a relay replacement that was not required at that time. This is a performance deficiency that is evaluated as minor in the SDP as a non-consequential procedure non-compliance because the work was accomplished within TS requirements and was appropriately risk evaluated. The work did however result in unnecessarily removing the machine from service to replace that coil. In addition, engineering performed a technical evaluation of the machine after it ran unloaded and determined that the machine was still capable of performing its function (reliability was not impacted.) Operators started and loaded both diesels to remove any carbon buildup after the offsite electrical system was restored to a normal lineup.
.6 Evaluate the licensees troubleshooting and repair of the Automatic Depressurization
System
a. Inspection Scope
The charter assignment to evaluate the licensees troubleshooting and repair of the automatic depressurization system (ADS) was based on the initial reports of equipment malfunction. In addition, the charter required the team to verify that the post-maintenance testing validated that the timing and timer display would work properly upon a low-low reactor water level signal. This assignment was based primarily upon the equipment malfunction communicated to the NRC in an update to Event Notification 44484 on September 12 at 06:55 AM. That update stated, Additionally, the ADS timer showed erratic indication following the event. The ADS timer was inhibited to prevent automatic action. ADS is inoperable, but manual steam relief valve operation remains available."
The inspectors conducted personnel interviews to assess the malfunction and actions taken during the event on September 11, 2008, and during the subsequent replacement of the ADS timer. The inspectors also reviewed the associated work package and action request for the replacement of the timer, the procedures used during the event response, the past surveillance test results for the timer, and the electrical drawings of the ADS system.
b. Findings and Observations
No findings of significance were identified; however, the following observation was identified.
- (1) ADS Timer Erratic Indication During the event, the operators noted that the timer display showed
.8.8.8 and became
concerned that the control circuit was malfunctioning. The operators interpreted the malfunctioning display to mean that the ADS timer had started and inhibited the ADS system per the EOP procedure C.5-1100 (RPV Control), part D. The EOP bases document supports inhibiting the automatic function of ADS in these conditions.
The shift manager indicated that he thought there was a potential electrical short or other problem with the timer logic that could have resulted in an inadvertent actuation of ADS and therefore supported the decision to inhibit the system. Through interviews with the operators on shift at the time of the event, the inspectors noted that some confusion existed regarding the safety significance of the ADS timer that malfunctioned and the role of the timer within the ADS system. In one discussion, operators indicated that they thought the timer was safety-related.
The ADS timer that malfunctioned is not safety-related and initiates with a time buffer built in to allow the operators extra time to either initiate the ADS system or inhibit it.
Several questions arose during the inspection regarding the use and verification of this timer. Inspectors noted that some operators were unsure of how to validate the reading on this timer which aids in making decisions about using or inhibiting the ADS system as needed or even if validation was warranted since ADS is safety-related. Inspectors considered this a potential knowledge weakness on the part of individual operators, but evaluated the issue as minor since the bases for the EOP required action was supported and no misoperation resulted.
.7 Establish the Sequence of Events for the September 17, 2008, Partial Loss of Normal
Offsite Power
a. Inspection Scope
The inspectors observed operator response to the event directly in the main control room. Additionally, the inspectors reviewed operator logs, plant parameter recordings and computer trending information, and conducted interviews with operating crew members in developing the sequence of events. In addition the inspectors sequence of events was reviewed against the licensee generated sequence of events to ensure completeness and accuracy.
b. Findings
No findings of significance were identified. The inspector-generated sequence of events is included with this report as Attachment 2 and an overview narrative summary of the event was presented in this reports Summary of Plant Event.
.8 Evaluate the licensee cause evaluation and subsequent response to the
September 17, 2008, Partial Loss of Normal Offsite Power
a. Inspection Scope
The inspectors reviewed the licensees cause evaluation for the partial loss of normal offsite power. Additionally, the inspectors reviewed the licensees assessment of operator response including procedural issues encountered during the response.
b. Findings and Observations
No findings of significance were identified.
.9 Evaluate the Licensee Cause Evaluation and Subsequent Response to the
September 20, 2008, Loss of Shutdown Cooling
a. Inspection Scope
The inspectors reviewed the licensees cause evaluation for the loss of shutdown cooling. Additionally, the inspectors reviewed the licensees assessment of operator response including procedural issues encountered during the response.
b. Findings and Observations
Introduction:
A self-revealed finding of very low safety significance was identified after the operations department failed to fully complete the post scram checklist and close the reactor reference leg backfill valve. This performance deficiency resulted in a Group 2 isolation and loss of shutdown cooling following a start of the 12 CRD pump on September 20, 2008, when reference leg pressure spiked high.
Description:
On September 20, 2008, the licensee restored power to the 1R transformer following corrective actions from the September 17 event. The Operations department then transferred normal station service loads to the 1R transformer. After completing this evolution, Operations restarted the 12 CRD pump. When the control rod drive pump started, the reference leg experienced a pressure spike and the resulting full RPS actuation and Group 2 isolation signals resulted in a loss of shutdown cooling.
Investigation revealed that the operators performing the shutdown checklist following the scram on September 11, 2008, did not close the reference leg fill valve from the control rod drive system. The operator assigned that task found that the valve was in the overhead and would require a ladder or scaffold to operate. He then skipped that step with the intention of completing that task later. No one tracked completion of the task and, as a result, it was not completed.
On September 20, when the bus transfers were taking place, the CRD pumps were shut down allowing the control rod drive hydraulic control unit accumulators to depressurize.
After the pump was restarted, the system pressurized with flow to the hydraulic control units until the accumulators were pressurized on the water side. When the flow stopped, the system saw a pressure spike which translated through the reference leg fill valve to the reference leg. The instrumentation reacted to the pressure spike causing a full Group 2 isolation and RPS actuation. Operators were able to reset the scram and Group 2 isolation very quickly. Shutdown cooling was then restarted.
Analysis:
The finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because it impacted the Initiating Events Cornerstone attribute of Configuration Control with the objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown. Specifically, the finding resulted in a loss of reactor decay heat removal event while the reactor was shut down.
Since the plant was shut down, the inspectors evaluated this finding in accordance with IMC 0609, Appendix G, Shutdown Operations Significance Determination Process. As part of the SDP, the inspectors assessed conditions or events that represent a loss of control. Table 1 in Appendix G lists criteria for losses of control. If the criteria are met then the finding needs to be quantitatively assessed via a Phase 2 analysis. The inspectors concluded that the criterion for loss of thermal margin (PWRs and BWRs)was met since the margin to boil was estimated at 0.21 (criterion met if greater than 0.2).
However, this was considered very conservative because the residual heat removal (RHR) system was always available and only interrupted due to the Group 2 isolation signal. When the plant lost RHR shutdown cooling, water level remained at +64 inches throughout the event. The operator action necessary to restore shutdown cooling was clearing the Group 2 isolation and restarting the RHR shutdown cooling function. This restoration was accomplished within 90 minutes and could have been easily accomplished much sooner had plant conditions warranted faster recovery.
The plant maintained the five shutdown safety functions listed in NUMARC 91-06 and Appendix G, in that, the plant had appropriate water level indications, had all low pressure and high pressure emergency core cooling systems for inventory control, had all of the offsite AC power transmission network available, had proper containment controls, and met reactivity guidelines. The condenser was also available. Therefore, the plant had sufficient mitigating equipment available. The consequence of this event would not have necessarily resulted in boil-off of water until pressure control means, such as SRV actuation, were implemented. Based on mitigating systems available and only an interruption of RHR shutdown cooling, the finding was determined to have low risk significance. This is a Green finding.
Additionally, the finding was determined to be cross-cutting in the area of Human Performance, Work Practices, in that the licensee failed to ensure supervisory and management oversight of work activities such that nuclear safety is supported. In this instance, operations shift management did not track implementation of the shutdown checklist to ensure completion (H.4(c)).
Enforcement:
Title 10 CFR 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings requires in part that Activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures or drawings.
Contrary to the above, from September 11, 2008, until the evening of September 20, 2008, operators failed to complete the shutdown checklist required following the scram on September 11, 2008, and did not close the reference leg fill valve from the control rod drive system. Because of the very low safety significance of this finding and because the finding was captured in the licensees CAP as CAP 01151413, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 05000263/2008009-05).
4OA6 Management Meetings
.1 Exit Meeting Summary
On November 3, 2008, the inspectors presented the inspection results to B. Sawatzke and other members of the licensee staff. The licensee acknowledged the issues presented. The inspectors confirmed that none of the potential report input discussed was considered proprietary.
.2 Interim Exit Meetings
An Interim exit was conducted for the special inspection on October 7, 2008, discussing the ongoing issues and preliminary potential findings with T. OConnor and staff. The inspectors confirmed that none of the potential report input discussed was considered proprietary.
ATTACHMENTS: 1.
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee
- T. OConnor, Site Vice President
- B. Sawatzke, Plant Manager
- J. Grubb, Site Engineering Director
- K. Jepson, Business Support Manager
- S. Sharp, Operations Manager
- B. Cole, Radiation Protection/Chemistry Manager
- T. Blake, Regulatory Affairs Manager
Nuclear Regulatory Commission
- K. Riemer, Chief, Reactor Projects Branch 2
LIST OF ITEMS
OPENED, CLOSED AND DISCUSSED
Opened
- 05000263/2008009-01 NCV Inadequate Procedural Guidance for Control Rod Drive System Valve Operation (Section 4OA3.2)
- 05000263/2008009-02 NCV Inadequate/Improper Maintenance on HPCI SV-1 Valve (Section 4OA3.3)
- 05000263/2008009-03 NCV Failure to Scope into the Maintenance Rule Monitoring Program the 34.5kV Cables (Section 4OA3.4)
- 05000263/2008009-04 FIN Inadequate Medium Voltage Cable Testing Program (Section 4OA3.4)
- 05000263/2008009-05 NCV Failure to Correctly Implement the Post Scram Checklist (Section 4OA3.9)
Closed
- 05000263/2008009-01 NCV Inadequate Procedural Guidance for Control Rod Drive System Valve Operation (Section 4OA3.2)
- 05000263/2008009-02 NCV Inadequate/Improper Maintenance on HPCI SV-1 Valve (Section 4OA3.3)
- 05000263/2008009-03 NCV Failure to Scope into the Maintenance Rule Monitoring Program the 34.5kV Cables (Section 4OA3.4)
- 05000263/2008009-05 NCV Failure to Correctly Implement the Post Scram Checklist (Section 4OA3.9)
Attachment 1