IR 05000361/2007004

IR 05000361-07-004, IR 05000362-07-004; 06/27/07 - 09/26/07; San Onofre Nuclear Generating Station, Units 2 & 3; Integrated Resident and Regional Report; Fire Protection, Maintenance Risk Assessments and Emergent Work Control, and Postmaint

ML073180739
Person / Time
Site:	San Onofre
Issue date:	11/09/2007
From:	Clark J NRC/RGN-IV/DRP/RPB-E
To:	Rosenblum R Southern California Edison Co
References
FOIA/PA-2011-0157 IR-07-004
Download: ML073180739 (29)

Text

November 09, 2007

SUBJECT:

SAN ONOFRE NUCLEAR GENERATING STATION - NRC INTEGRATED INSPECTION REPORT 05000361/2007004; 05000362/2007004

Dear Mr. Rosenblum:

On September 26, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at your San Onofre Nuclear Generating Station, Units 2 and 3 facility. The enclosed integrated report documents the inspection findings, which were discussed on September 27, 2007, with Dr. R. Waldo and other members of your staff.

The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your licenses. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

This report documents three NRC identified findings of very low safety significance (Green).

These findings were determined to involve violations of NRC requirements; however, because of the very low safety significance and because they were entered into your corrective action program, the NRC is treating these findings as noncited violations (NCVs) consistent with Section VI.A of the NRC Enforcement Policy. If you contest this/these NCVs, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington DC 20555-0001; and the NRC Resident Inspector at San Onofre Nuclear Generating Station, Units 2 and 3, facility.

Southern California Edison Company-2-In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Jeffrey A. Clark, Chief Project Branch E Division of Reactor Projects Dockets: 50-361 50-362 Licenses: NPF-10 NPF-15

Enclosure:

NRC Inspection Report 05000361/2007004; 05000362/2007004 w/Attachment: Supplemental Information

REGION IV==

Docket:

50-361, 50-362 Licenses:

NPF-10, NPF-15 Report No.:

05000361/2007004 and 5000362/2007004 Licensee:

Southern California Edison Co. (SCE)

Facility:

San Onofre Nuclear Generating Station, Units 2 and 3 Location:

5000 S. Pacific Coast Hwy.

San Clemente, California Dates:

June 27 to September 26, 2007 Inspectors:

C. C. Osterholtz, Senior Resident Inspector, Project Branch E, DRP M. A. Sitek, Resident Inspector, Project Branch E, DRP J. R. Groom, Project Engineer, Project Branch E, DRP M. O. Miller, Resident Inspector, Project Branch C, DRP G. G. Warnick, Senior Resident Inspector, Project Branch D, DRP L. M. Willoughby, Resident Inspector, Project Branch E, DRP M. R. Young, Reactor Engineer, Branch E, DRP Approved By:

Jeffrey A. Clark, Chief Project Branch E Division of Reactor Projects

ENCLOSURE-2-

SUMMARY OF FINDINGS

IR05000361/2007004, 05000362/2007004; 06/27/07 - 09/26/07; San Onofre Nuclear

Generating Station, Units 2 & 3; Integrated Resident and Regional Report; Fire Protection,

Maintenance Risk Assessments and Emergent Work Control, and Postmaintenance Testing.

This report covered a 3-month period of inspection by resident inspectors and Regional office inspectors. The inspection identified four Green findings, all of which were noncited violations.

The significance of most findings is indicated by their color (Green, White, Yellow, or Red)using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management's review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.

Cornerstone: Mitigating Systems

Green.

The inspectors identified a noncited violation of 10 CFR 50, Appendix B,

Criterion V (Procedures) for the failure to follow requirements for evaluating applicable kV breaker operating experience. Engineers had inappropriately determined that operating experience related to loose and/or missing mechanism operated cell components, dated 2003, was not applicable to San Onofre. Subsequently, a safety related 4 kV breaker (offsite power to 4 kV safety bus) malfunctioned because of a missing nut. In addition, following the breaker malfunction, the licensees extent of condition review was initially untimely and lacked rigor. After NRC prompting, other loose and missing fastening nuts were identified. This issue was entered into the licensees corrective action program as Action Request 070601194.

The finding was more than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and it affected the cornerstone objective by reducing the availability, reliability, and capability of the electrical systems. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding had very low safety significance (Green)because it did not result in a loss of system safety function, a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding or severe weather initiating events. The cause of the finding had a cross-cutting aspect in the area of problem identification and resolution, corrective action program component (P.1(c))

because of the poor extent of condition evaluation (Section 1R13.2.1).

Green.

The inspectors identified a noncited violation of 10 CFR Part 50, Appendix B,

Criterion XVI (Corrective Actions) for the failure to prevent the recurrence of a significant condition adverse to quality, water intrusion into auxiliary feedwater electrical Cabinet 3MS4706. In the most recent event, the water rendered Valve 3HV4706 (auxiliary feedwater turbine driven pump to steam Generator 3E089 discharge valve)

inoperable. Several previous occurrences of water intrusion were identified in the 1990s, including one instance where a valve contactor was significantly corroded. This issue was entered into the licensees corrective action program as Action Request 070701029.

The finding was more than minor because it was associated with the equipment performance and external factors attributes of the mitigating systems cornerstone and it affected the cornerstone objective by reducing the availability, reliability, and capability of the auxiliary feedwater system. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding had very low safety significance (Green) because it did not result in a loss of system safety function, a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding or severe weather initiating events (Section 1R13.2.2).

Green.

The inspectors identified a Green noncited violation of Technical Specification 5.5.1.1 (Procedures) for the failure of maintenance personnel to have adequate procedures in place to ensure a feedwater isolation valve hydraulic pump discharge filter housing was properly installed, causing a hydraulic oil leak that had to be emergently repaired. This issue was entered into the licensees corrective action program as Action Request 070601194.

The finding was more than minor because it affected the procedure quality attribute of the mitigating systems cornerstone, and affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding had very low safety significance (Green) because it did not result in an actual loss of system safety function, a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding, or severe weather initiating events. The cause of the finding has a cross-cutting aspect in the area of human performance associated with resources (H.2.(c)) because the licensee failed to ensure complete and accurate procedures were available and to assure the safe operation of a main feedwater isolation valve (Section 1R19).

Licensee-Identified Violations

None

REPORT DETAILS

Summary of Plant Status

Unit 2 began the inspection period in a shutdown condition at normal operating pressure and temperature in Mode 3. Operations personnel commenced a reactor start-up on June 28, 2007. Unit 2 ended the inspection period at approximately 99 percent reactor power.

Unit 3 began the inspection period at approximately 100 percent reactor power and remained there throughout the inspection period.

REACTOR SAFETY

Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity

1R01 Adverse Weather Protection

.1 Readiness For Seasonal Susceptibilities

a. Inspection Scope

The inspectors completed a review of the licensee's readiness of seasonal susceptibilities involving brush fires that are likely to occur during high temperatures and high winds. The inspectors:

(1) reviewed plant procedures, the Updated Final Safety Analysis Report (UFSAR), and Technical Specifications (TS) to ensure that operator actions defined in adverse weather procedures maintained the readiness of essential systems;

(2) walked down portions of the below listed system to ensure that adverse weather protection features were sufficient to support operability, including the ability to perform safe shutdown functions;

(3) evaluated operator staffing levels to ensure the licensee could maintain the readiness of essential systems required by plant procedures; and

(4) reviewed the corrective action program (CAP) to determine if the licensee identified and corrected problems related to adverse weather conditions.

C July 5, 2007, Units 2 and 3, reserve auxiliary transformers and the electrical switchyard C

July 6, 2007, Units 2 and 3, auxiliary feedwater system Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

ENCLOSURE

1R04 Equipment Alignment

.1 Partial System Walkdowns

a. Inspection Scope

The inspectors:

(1) walked down portions of the two listed risk important systems and reviewed plant procedures and documents to verify that critical portions of the selected systems were correctly aligned; and

(2) compared deficiencies identified during the walk down to the licensee's UFSAR and CAP to ensure problems were being identified and corrected.

C August 10, 2007, Unit 3, Train B emergency diesel Generator 3G003 while Train A emergency diesel Generator 3G002 was out of service for planned maintenance C

August 23, 2007, Unit 3, Train A emergency diesel Generator 3G002 while Train B emergency diesel Generator 3G003 was out of service for planned maintenance Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

1R05 Fire Protection

a. Inspection Scope

Quarterly Inspection The inspectors walked down the six listed plant areas to assess the material condition of active and passive fire protection features and their operational lineup and readiness.

The inspectors:

(1) verified that transient combustibles and hot work activities were controlled in accordance with plant procedures;

(2) observed the condition of fire detection devices to verify they remained functional;

(3) observed fire suppression systems to verify they remained functional and that access to manual actuators was unobstructed;

(4) verified that fire extinguishers and hose stations were provided at their designated locations and that they were in a satisfactory condition;

(5) verified that passive fire protection features (electrical raceway barriers, fire doors, fire dampers, steel fire proofing, penetration seals, and oil collection systems) were in a satisfactory material condition;

(6) verified that adequate compensatory measures were established for degraded or inoperable fire protection features and that the compensatory measures were commensurate with the significance of the deficiency; and

(7) reviewed the UFSAR and Updated Fire Hazard Analysis (UFHA) to determine if the licensee identified and corrected fire protection problems.

ENCLOSURE C

July 11, 2007, Unit 3 saltwater cooling pump room C

July 11, 2007, Unit 2 charging Pump 2P191 room C

July 11, 2007, Unit 2 charging Pump 2P190 room C

July 13, 2007, Unit 2 saltwater cooling pump room C

July 13, 2007, Unit 3 charging Pump 2P191 room C

July 13, 2007, Unit 3 charging Pump 2P190 room Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed six samples.

b. Findings

No findings of significance were identified.

1R06 Flood Protection Measures

.1 External Flooding

a. Inspection Scope

The inspectors:

(1) reviewed the UFSAR, the flooding analysis, and plant procedures to assess seasonal susceptibilities involving external flooding;

(2) reviewed the UFSAR and CAP to determine if the licensee identified and corrected flooding problems;

(3) inspected underground bunkers/manholes to verify the adequacy of

(a) sump pumps,

(b) level alarm circuits,

(c) cable splices subject to submergence, and

(d) drainage for bunkers/manholes;

(4) verified that operator actions for coping with flooding can reasonably achieve the desired outcomes; and

(5) walked down the listed areas to verify the adequacy of:

(a) equipment seals located below the floodline,

(b) floor and wall penetration seals,

(c) watertight door seals,

(d) common drain lines and sumps,

(e) sump pumps, level alarms, and control circuits, and

(f) temporary or removable flood barriers.

C July 26, 2007, Units 2 and 3, seawall, berm, and underground electrical tunnels Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

.2 Internal Flooding

a. Inspection Scope

The inspectors:

(1) reviewed the UFSAR, the flooding analysis, and plant procedures to assess seasonal susceptibilities involving internal flooding;

(2) reviewed the UFSAR and

ENCLOSURE CAP to determine if the licensee identified and corrected flooding problems;

(3) inspected underground bunkers/manholes to verify the adequacy of

(a) sump pumps,

(b) level alarm circuits,

(c) cable splices subject to submergence, and

(d) drainage for bunkers/manholes;

(4) verified that operator actions for coping with flooding can reasonably achieve the desired outcomes; and

(5) walked down the two listed areas to verify the adequacy of:

(a) equipment seals located below the floodline,

(b) floor and wall penetration seals,

(c) watertight door seals,

(d) common drain lines and sumps,

(e) sump pumps, level alarms, and control circuits, and

(f) temporary or removable flood barriers.

C July 25, 2007, Units 2 and 3, safety equipment building C

July 26, 2007, Units 2 and 3, auxiliary feedwater pump rooms Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

1R11 Licensed Operator Requalification

a. Inspection Scope

The inspectors observed testing and training of senior reactor operators and reactor operators to identify deficiencies and discrepancies in the training, to assess operator performance, and to assess the evaluator's critique. The training scenarios involved:

• On July 23, 2007, an excess steam demand event and a pressurizer safety loss of coolant accident

• On September 20, 2007, raise RCS level to less than the top of the hot leg (42")

using the gravity feed method Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

ENCLOSURE

1R12 Maintenance Effectiveness

Maintenance Effectiveness Baseline Review

a. Inspection Scope

The inspectors reviewed the overall implementation of the requirements of the Maintenance Rule (10 CFR 50.65) to verify that the licensee had conducted appropriate evaluations of equipment functional failures, maintenance preventable functional failures, unplanned capacity loss factor, and system unavailability. The inspectors reviewed root causes and corrective action determinations for equipment failures and reviewed performance goals for ensuring corrective action effectiveness. The inspectors discussed the evaluations with the reliability engineering supervisor and the system engineers. In particular, the inspectors reviewed overall system performance and equipment deficiencies associated with the following systems for both Units 2 and 3:

• steam bypass control system

• main feedwater system The inspectors also reviewed the qualification requirements and records of maintenance foreman to ensure maintenance performed on equipment important to safety had proper supervisory oversight.

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

1R13 Maintenance Risk Assessments and Emergent Work Control

.1 Risk Assessment and Management of Risk

a. Inspection Scope

The inspectors reviewed the listed assessment activities to verify:

(1) performance of risk assessments when required by 10 CFR 50.65 (a)(4) and licensee procedures prior to changes in plant configuration for maintenance activities and plant operations;

(2) the accuracy, adequacy, and completeness of the information considered in the risk assessment;

(3) that the licensee recognizes, and/or enters as applicable, the appropriate licensee-established risk category according to the risk assessment results and licensee procedures; and

(4) the licensee identified and corrected problems related to maintenance risk assessments.

ENCLOSURE

• August 10, 2007, Unit 3, risk assessment and management during an unplanned outage of back-up nitrogen to instrument air, the Train A emergency diesel generator, and both trains of the normal chilled water system

• September 17, 2007 risk assessment and management during unplanned outage of reserve auxiliary transformers 2XR1, 2XR2, and 2XR3 to support emergent removal of a degraded metering current transformer.

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

.2 Emergent Work Control

a. Inspection Scope

The inspectors:

(1) verified that the licensee performed actions to minimize the probability of initiating events and maintained the functional capability of mitigating systems and barrier integrity systems;

(2) verified that emergent work-related activities such as troubleshooting, work planning/scheduling, establishing plant conditions, aligning equipment, tagging, temporary modifications, and equipment restoration did not place the plant in an unacceptable configuration; and

(3) reviewed the UFSAR to determine if the licensee identified and corrected risk assessment and emergent work control problems.

• July 21, 2007, Unit 3, missing nut on the mechanism operated contact of the Train B reserve auxiliary Transformer 3XR2 supply Breaker 3A0618

• July 24, 2007, Unit 3, auxiliary feedwater Pump 3P140 discharge Valve 3HV4706 to steam Generator 3E089 failure to open Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

.1 Missing Nuts on Mechanism Operated Contacts of 4 kV Breakers

Introduction.

The inspectors identified a Green NCV of 10 CFR 50, Appendix B, Criterion V (Procedures) for the failure to follow requirements for evaluating applicable 4 kV breaker operating experience. Engineers had inappropriately determined that operating experience related to loose and/or missing mechanism operated cell components, dated 2003, was not applicable to San Onofre. Subsequently, a safety

ENCLOSURE related 4 kV breaker (offsite power to 4 kV safety bus) malfunctioned because of a missing nut. In addition, following the breaker malfunction, the licensees extent of condition review was initially untimely and lacked rigor. After NRC prompting, other loose and missing fastening nuts were identified.

Description.

On July 21, 2007, operations personnel were performing a monthly loaded run of emergency diesel Generator (EDG) 3G003. At the completion of the test, operations personnel actuated Relay K401B in order to open the EDG output Breaker 3A0613 to separate the EDG from the electrical grid. The output breaker failed to open upon actuation of the relay and operations personnel had to manually open it in order to separate the EDG from the grid. Operations personnel declared EDG 3G003 inoperable while troubleshooting activities took place.

The troubleshooting effort revealed that the mechanism operated contact (MOC) for the Unit 3 Train B reserve auxiliary Transformer (RAT) 3XR2 supply Breaker 3A0618 was open. This contact is used in the EDG output breaker control circuit. With the contact open, the EDG breaker logic received a signal that the RAT breaker was open when it was actually (correctly) closed. As a result, the EDG breaker logic kept the EDG breaker in the closed position as it should have if the RAT breaker had been in the open position.

The correct positions of the equipment were for the RAT MOC to be closed which parallels the RAT breaker position of closed. With the RAT breaker closed, the EDG output breaker would then be allowed to open ensuring power to the 4 kV electrical bus.

Further examination of the RAT MOC assembly revealed that the retaining nut for the contact arm was missing which caused the contact misalignment. The licensee placed the nut back on the MOC assembly and declared the EDG operable.

Following discovery of the missing nut of the MOC assembly, the inspectors questioned the extent of the condition as it related to similar 4 kV breakers. The inspectors met with engineering personnel on July 26, 2007, and discovered that an inspection plan for the 61 similar safety-related breakers had not been developed or adequately discussed.

Following this discussion, the licensee formed a multi-disciplinary team to address the extent of the condition. This team developed criteria for performing the inspections and a schedule that included completing the inspections for the 61 safety-related breakers and 20 critical non-safety-related breakers by August 10, 2007. Those inspections resulted in the licensee finding additional problems with the MOC assemblies in other breakers.

Specifically, on August 2, 2007, the castle nut on the MOC assembly for the feeder breaker for Unit 3 high pressure safety injection Pump 3P019 was found 1.5 turns loose.

In addition, on August 7, 2007, the MOC nut for the feeder breaker for Unit 3 saltwater cooling Pump 3P307 was missing and found laying on a shelf in the breaker cubicle.

After each of these cases, the licensee stopped the inspections while they corrected the problems. As a result, the inspection schedule started to change from taking two weeks to over four weeks. The inspectors challenged the licensees process of slowing down the inspections as more problems were found. The licensee agreed that slowing the inspections was not prudent and completed the inspections on August 21, 2007.

The inspectors also reviewed applicable operating experience associated with the event and determined that a similar event occurred at another nuclear power plant in 2003.

ENCLOSURE The inspectors reviewed San Onofres response to this operating experience and determined that it was received, but the licensee determined that it was not applicable to their plant. As a result, the licensee did not take any actions to mitigate the potential for deficiencies in their own breakers.

Analysis.

The failure to properly evaluate operating experience associated with missing nuts in 4 kV breakers was a performance deficiency. The finding was more than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and it affected the cornerstone objective by reducing the availability, reliability, and capability of the electrical systems. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding had very low safety significance (Green) because it did not result in a loss of system safety function, a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding or severe weather initiating events. The finding had a crosscutting aspect in the area of problem identification and resolution, corrective action program component (P.1(c))

because of the poor extent of condition evaluation.

Enforcement.

The regulations in 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings, requires that activities affecting quality shall be prescribed by documented procedures of a type appropriate to the circumstance and shall be accomplished in accordance with these procedures. Procedure SO123-XII-2.21, Nuclear Safety Group Functions and Responsibilities, Revisions 6 and 7, implemented this requirement for the review of industry operating experience. Section 6.6.1 required that conditions adverse to safety and/or license conditions identified during the review of operating experience shall have been evaluated and necessary corrective actions requested and tracked in accordance with the corrective action program. In 2003, the licensee received operating experience from another nuclear plant that documented several cases of MOC nuts missing in 4 kV electrical breakers. Contrary to the above, the licensee evaluated this operating experience report and concluded that it was not applicable to San Onofre Units 2 and 3 and did not take necessary corrective actions On July 21 and August 7, 2007, nuts were discovered missing from MOCs in two safety-related 4 kV electrical breakers similar to the operating experience received in 2003.

Because the finding is of very low safety significance and has been entered into the licensees corrective action program as AR 070700909, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy: NCV 05000361; 362/2007004-01, Failure to Evaluate Operating Experience for Missing Nuts in 4 kV Electrical Breakers.

.2 Water Intrusion into Control Cabinet of Valve 3HV4706

Introduction.

The inspectors identified a Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI (Corrective Actions) for the failure to prevent the recurrence of a significant condition adverse to quality, water intrusion into auxiliary feedwater electrical Cabinet 3MS4706. In the most recent event, the water rendered Valve 3HV4706 (auxiliary feedwater turbine driven pump to steam Generator 3E089 discharge valve) inoperable.

Several previous occurrences of water intrusion were identified in the 1990's, including one instance where a valve contactor was significantly corroded.

ENCLOSURE

Description.

On July 24, 2007, the licensee was making preparations to perform a surveillance of main steam isolation signal Relay K-623B. As part of those preparations for the surveillance, but not part of the surveillance itself, operations personnel were directed by procedure to open motor operated Valve 3HV4706 (AFW turbine driven Pump 3P140 discharge valve to steam Generator 3E089). The valve, however, failed to open when operations personnel attempted to open it from the control room. The licensee promptly entered TS 3.7.5 Condition B which required repair of Valve 3HV4706 within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.

Maintenance electricians and engineering personnel examined control Cabinet 3MS4706 for Valve 3HV4706 to determine the cause of the valve failing to open. They determined that the normally closed auxiliary contactor for the valve was open and had signs of corrosion. The licensee subsequently removed the contactor and determined the root cause of the problem to be excessive corrosion of the screw and lock washer as a result of exposure to water laden chlorides. The licensee determined that the source of the water intrusion into the cabinet was from the electrical conduit leading into the cabinet.

Cabinet 3MS4706 is located directly below an air vent in the Unit 3 auxiliary feedwater pump room and during rain, fog, and other moisture causing conditions, water vapor collected on the conduit and flowed down into the panel.

Following the event on July 24, 2007, the inspectors questioned the status of other similar AFW valves because they are located in similar cabinets. The licensee verified immediate operability of the other similar valves by stroking them successfully on July 26, 2007, after the inspectors questioned why that activity had not already been performed.

The inspectors also questioned the status of the visual inspections of the cabinets associated with the similar valves. The licensee presented a schedule that started in September 2007 and finished in October 2007. After raising concerns with the timeliness of those inspections, the licensee completed the inspections in the first week of August 2007.

The inspectors reviewed the licensees internal operating experience with water intrusion into Cabinet 3MS4706. In 1995, the contactor for Valve 3HV4715 (in Cabinet 3MS4706),had to be replaced because it had corroded. The inspectors also noted that rain water had intruded into the cabinet in 1998 and again in 1999. After both incidents, the top of the cabinet was sealed in an attempt to prevent water intrusion. In both cases, the sealing was ineffective and led to further water intrusion events. As a result of the inadequate sealing in the late 1990's, the licensee developed an engineered seal to eliminate further water intrusion events. The licensee also inspected and sealed twenty electrical cabinets throughout the plant to ensure that the extent of condition of the original problem had been evaluated and that appropriate corrective actions were in place to prevent similar problems in other systems. The licensee completed that effort on August 24, 2007.

Analysis.

The failure to take appropriate corrective action to prevent the recurrence of water intrusion into electrical Cabinet 3MS4706, a significant condition adverse to quality, was a performance deficiency. The finding was more than minor because it was associated with the equipment performance and external factors attributes of the mitigating systems cornerstone and it affected the cornerstone objective by reducing the

ENCLOSURE availability, reliability, and capability of the auxiliary feedwater system. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding had very low safety significance (Green) because it did not result in a loss of system safety function, a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding or severe weather initiating events.

Enforcement.

The regulations in 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, state, in part, that in the case of significant conditions adverse to quality, measures shall be taken to ensure that the cause of the condition is determined and corrective action taken to preclude repetition. Multiple instances of water intrusion into Cabinet 3MS4706 was a significant condition adverse to quality, because it could render safety related equipment inoperable. Contrary to the above, the licensee failed to take effective corrective action to preclude subsequent water intrusion events into Cabinet 3MS4706, as evidenced by the July 24, 2007 water instrusion event. Because the finding was of very low safety significance and has been entered into the licensees corrective action program as AR 070701029, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy: NCV 05000362/2007004-02, Inadequate Corrective Actions to Prevent Water Intrusion into Auxiliary Feedwater Valve Cabinet 3MS4706.

1R15 Operability Evaluations

a. Inspection Scope

The inspectors:

(1) reviewed plant status documents such as operator shift logs, emergent work documentation, deferred modifications, and standing orders to determine if an operability evaluation was warranted for degraded components;

(2) referred to the UFSAR and design basis documents to review the technical adequacy of licensee operability evaluations;

(3) evaluated compensatory measures associated with operability evaluations;

(4) determined degraded component impact on any TSs;

(5) used the Significance Determination Process to evaluate the risk significance of degraded or inoperable equipment; and

(6) verified that the licensee has identified and implemented appropriate corrective actions associated with degraded components.

• July 22, 2007, Unit 3, reactor coolant Pump 3P002 operability with degraded seals

• August 13, 2007, Units 2 and 3, indications of failed fuel pins

• August 22, 2007, Unit 3, Train B component cooling water seismic make-up Pump 3P1019 indications of elevated temperatures on the shaft and bearing lip seal

• September 9, 2007, Units 2 and 3, degradation of plastic wire labels Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed four samples.

ENCLOSURE

b. Findings

No findings of significance were identified.

1R19 Postmaintenance Testing

a. Inspection Scope

The inspectors selected the four listed postmaintenance test activities of risk significant systems or components. For each item, the inspectors:

(1) reviewed the applicable licensing basis and/or design-basis documents to determine the safety functions;

(2) evaluated the safety functions that may have been affected by the maintenance activity; and

(3) reviewed the test procedure to ensure it adequately tested the safety function that may have been affected. The inspectors either witnessed or reviewed test data to verify that acceptance criteria were met, plant impacts were evaluated, test equipment was calibrated, procedures were followed, jumpers were properly controlled, the test data results were complete and accurate, the test equipment was removed, the system was properly re-aligned, and deficiencies during testing were documented. The inspectors also reviewed the UFSAR to determine if the licensee identified and corrected problems related to postmaintenance testing.

• August 11, 2007, Unit 3, Train A emergency diesel generator 3GOO2 following planned maintenance

• August 22, 2007, Units 2 and 3, Train A emergency Chiller ME336 following replacement of a CB1 control panel breaker

• September 14, 2007, Unit 3, Train A salt water cooling pump 3P112 following maintenance to correct failed inservice test

• June 28, 2007, Unit 2 main feedwater isolation Valve 2HV4048 following corrective maintenance Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed four samples.

b. Findings

Introduction.

The inspectors identified a Green noncited violation of TS 5.5.1.1 (Procedures) for the failure of maintenance personnel to have adequate procedures in place to ensure a feedwater isolation valve hydraulic pump discharge filter housing was properly installed, causing a hydraulic oil leak that had to be emergently repaired.

Description.

On June 28, 2007, after opening feedwater isolation Valve 2HV4048 for a Unit 2 start-up, the licensee discovered a hydraulic oil leak on the hydraulic pump discharge filter housing (S21305MF1205) to the manifold block joint. Unit 2 had been

ENCLOSURE shut down ten days earlier to repair a similar hydraulic oil leak at the same location. The Unit 2 startup was aborted and feedwater isolation Valve 2HV4048 was shut for rework.

The hydraulic pump discharge filter housing was bolted to the manifold block using four 5/16-18 x 2-3/4" allen head cap screws, and each housing had two ports with recesses to hold an o-ring seal. During the housing removal process, one of the cap screws for the filter was dropped and lost. There was an attempt to locate the cap screw, but when it could not be found maintenance personnel decided to replace it with a new cap screw from shop stock. The replacement was in-kind material and cut to length by comparison with one of the other cap screws. The new cap screw was installed with no verification that the length was correct. It was later determined that the replacement for the lost cap screw was cut

.012 " longer then the next longest original cap screw, and installed with a

washer slightly thinner then the other three.

With the cap screw 0.012" longer than the others, when combined with the use of a slightly thinner washer, it was enough to cause the cap screw to bottom out. This prevented the filter housing from being held tightly enough to the manifold block. This potential problem was not considered or checked during the installation of the replacement cap screw. The licensees corrective action program concluded that maintenance personnel failed to demonstrate proper task management for the change in plan when they did not verify that the replacement cap screw and washer would not bottom out in the threaded hole. The licensee also modified the filter housing to allow for some additional tolerance should replacement bolts be required in the future.

The inspectors considered that the corrective actions did not address the overall problem of procedure adequacy. The inspectors noted that there were no proposed enhancements to maintenance procedures to ensure replacement parts were appropriately procured and installed. The licensee indicated that their corrective actions would be reevaluated to ensure thorough and comprehensive resolution for this concern.

Analysis.

The failure of maintenance personnel to have adequate procedures in place to properly install a feedwater isolation valve hydraulic pump discharge filter housing was a performance deficiency. The finding was more than minor because it affected the procedure quality attribute of the mitigating systems cornerstone, and affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding had very low safety significance (Green) because it did not result in a loss of system safety function, a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding or severe weather initiating events. The cause of the finding has a cross-cutting aspect in the area of human performance associated with resources (H.2.(c)) because the licensee failed to ensure complete and accurate procedures were available and to assure the safe operation of a main feedwater isloation valve.

Enforcement.

TS 5.5.1.1 requires that written procedures be established, implemented, and maintained for activities specified in Appendix A, Typical Procedures for Pressurized Water Reactors and Boiling Water Reactors, of Regulatory Guide 1.33, Quality

ENCLOSURE Assurance Program Requirements (Operations), dated February 1978. Regulatory Guide 1.33, Appendix A, Section 9.c, requires that procedures for the repair or replacement of safety-related equipment should be prepared prior to beginning work.

Contrary to this requirement, on June 28, 2007, maintenance personnel had inadequate procedures in place to ensure a feedwater isolation valve hydraulic pump discharge filter housing was properly installed, causing a hydraulic oil leak that had to be emergently repaired. Because the finding is of very low safety significance and has been entered into the licensees corrective action program as AR 070601194, this violation is being treated as an NCV consistent with Section VI.A of the NRC Enforcement Policy:

NCV 05000361/2007004-03, Main Feedwater Isolation Valve Hydraulic Leak due to Inadequate Maintenance Procedure.

1R22 Surveillance Testing

a. Inspection Scope

The inspectors reviewed the UFSAR, procedure requirements, and TSs to ensure that the five listed surveillance activities demonstrated that the SSCs tested were capable of performing their intended safety functions. The inspectors either witnessed or reviewed test data to verify that the following significant surveillance test attributes were adequate:

(1) preconditioning;

(2) evaluation of testing impact on the plant;

(3) acceptance criteria;

(4) test equipment;

(5) procedures;

(6) jumper/lifted lead controls;

(7) test data;

(8) testing frequency and method demonstrated TS operability;

(9) test equipment removal;

(10) restoration of plant systems;

(11) fulfillment of ASME Code requirements;

(12) updating of performance indicator data;

(13) engineering evaluations, root causes, and bases for returning tested SSCs not meeting the test acceptance criteria were correct;

(14) reference setting data; and

(15) annunciators and alarms setpoints. The inspectors also verified that the licensee identified and implemented any needed corrective actions associated with the surveillance testing.

• July 16, 2007, Unit 2, charging Pump 2P191 biennial comprehensive inservice test

• July 18, 2007, Unit 3, turbine driven auxiliary feedwater Pump 3P140 inservice test

• July 21, 2007, Unit 3, emergency diesel Generator 3G003 semiannual surveillance

• August 14, 2007, Unit 3, Train A main steam isolation Relays K-623A and K-723A semiannual surveillance

• September 19, 2007, Unit 2, turbine driven auxiliary feedwater Pump 2P140 inservice test Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed five samples.

ENCLOSURE

b. Findings

No findings of significance were identified.

1EP6 Drill Evaluation

a. Inspection Scope

For the listed drill and simulator-based training evolution contributing to Drill/Exercise Performance and Emergency Response Organization Performance Indicators, the inspectors:

(1) observed the training evolution to identify any weaknesses and deficiencies in classification, notification, and Protective Action Recommendation development activities;

(2) compared the identified weaknesses and deficiencies against licensee identified findings to determine whether the licensee is properly identifying failures; and

(3) determined whether licensee performance is in accordance with the guidance of the NEI 99-02, "Voluntary Submission of Performance Indicator Data,"

acceptance criteria.

• July 18, 2007, Units 2 and 3, simulator, technical support center, emergency operations facility, and operations support center - main steam line break followed by a station blackout and steam generator tube rupture Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

4OA1 Performance Indicator (PI) Verification

Cornerstone: Mitigating Systems

The inspectors sampled licensee data for the Mitigating System Performance Index (MSPI)performance indicators (PI) listed below for Units 2 and 3 for the period from April 1, 2006, through June 30, 2007. The definitions and guidance of Nuclear Energy Institute 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 4, were used to verify the licensees basis for reporting unavailability and unreliability in order to verify the accuracy of PI data. The inspectors reviewed operating logs, Limiting Conditions for Operation logs, ARs, and the maintenance rule database to verify that the licensee properly accounted for planned and unplanned unavailability as part of the assessment. The

ENCLOSURE inspectors sampled data to verify that the licensee:

(1) accurately documented the actual unavailability hours for the MSPI systems; and

(2) accurately documented the actual unreliability information for each MSPI monitored component. In addition, the inspectors interviewed licensee personnel associated with PI data collection and evaluation.

• high pressure safety injection system

• emergency alternating current power system

• auxiliary feedwater system

• residual heat removal system

• support cooling water system The inspectors completed ten samples.

Documents reviewed by the inspectors are listed in the attachment.

b. Findings

No findings of significance were identified.

4OA2 Identification and Resolution of Problems

.1 Routine Review of Identification and Resolution of Problems

a. Inspection Scope

The inspectors performed a daily screening of items entered into the licensee's corrective action program. This assessment was accomplished by reviewing maintenance orders, action requests, the management focus list, and attending corrective action review and work control meetings. The inspectors:

(1) verified that equipment, human performance, and program issues were being identified by the licensee at an appropriate threshold and that the issues were entered into the corrective action program;

(2) verified that corrective actions were commensurate with the significance of the issue; and

(3) identified conditions that might warrant additional follow-up through other baseline inspection procedures.

b. Findings

No findings of significance were identified.

4OA6 Meetings, Including Exit

On September 28, 2007, the inspectors presented the inspection results to Dr. R. Waldo and others who acknowledged the findings. The inspectors confirmed that proprietary information was not provided or examined during the inspection.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

D. Axline, Technical Specialist, Nuclear Regulatory Affairs

D. Breig, Manager, Engineering Standards and Excellence

B. Corbett, Manager, Health Physics

J. Hirsch, Manager, Maintenance

K. Johnson, Manager, Design Engineering

B. Katz, Vice President, Nuclear Oversight and Regulatory Affairs

L. Kelly, Engineer, Nuclear Regulatory Affairs

C. McAndrews, Manager, Nuclear Oversight and Assessment

N. Quigley, Manager, Mechanical/Nuclear Maintenance Engineering

J. Reilly, Vice President, Engineering and Technical Services

A. Scherer, Manager, Nuclear Regulatory Affairs

R. St. Onge, Manager, Maintenance and Systems Engineering

T. Vogt, Manager, Special Projects

R. Waldo, Vice President, Nuclear Generation

D. Wilcockson, Manager, Plant Operations

C. Williams, Manager, Compliance

T. Yackle, Manager, Operations

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

None

Opened and Closed

05000361;362/2007004-01 NCV Failure to Evaluate Operating Experience for Missing Nuts in 4 kV Electrical Breakers (Section 1R13.2.1)

05000362/2007004-02 NCV Inadequate Corrective Actions to Prevent Water Intrusion into Auxiliary Feedwater Valve Cabinet 3MS4706 (Section 1R13.2.2)

05000361/2007004-03 NCV Main Feedwater Isolation Valve Hydraulic Leak due to Inadequate Maintenance Procedure (Section 1R19)

ATTACHMENT

Closed

None

Discussed

None