IR 05000361/2007003

IR 05000361-07-003, 05000362-07-003; 04/01/07 - 06/26/07; San Onofre Nuclear Generating Station, Units 2 & 3; Integrated Resident and Regional Report; Operability Evaluations and Identification and Resolution of Problems

ML072220153
Person / Time
Site:	San Onofre
Issue date:	08/09/2007
From:	Clark J NRC/RGN-IV/DRP/RPB-E
To:	Rosenblum R Southern California Edison Co
References
FOIA/PA-2011-0157 IR-07-003
Download: ML072220153 (35)

Text

ust 9, 2007

SUBJECT:

SAN ONOFRE NUCLEAR GENERATING STATION - NRC INTEGRATED INSPECTION REPORT 05000361/2007003; 05000362/2007003

Dear Mr. Rosenblum:

On June 26, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at your San Onofre Nuclear Generating Station, Units 2 and 3 facility. The enclosed integrated report documents the inspection findings, which were discussed on June 28, 2007, with Dr. R. Waldo and other members of your staff.

The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your licenses. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

This report documents two NRC identified findings of very low safety significance (Green).

Additionally, two licensee identified violations which were determined to be of very low safety significance are listed in the report. These findings were determined to involve violations of NRC requirements; however, because of the very low safety significance and because they were entered into your corrective action program, the NRC is treating these findings as noncited violations (NCVs) consistent with Section VI.A of the NRC Enforcement Policy. If you contest these NCVs, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, U.S.

Nuclear Regulatory Commission Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington DC 20555-0001; and the NRC Resident Inspector at San Onofre Nuclear Generating Station, Units 2 and 3, facility.

Southern California Edison Company -2-In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Jeffrey A. Clark, P.E., Chief Project Branch E Division of Reactor Projects Dockets: 50-361 50-362 Licenses: NPF-10 NPF-15

Enclosure:

NRC Inspection Report 05000361/2007003; 05000362/2007003 w/Attachment: Supplemental Information

REGION IV==

Docket: 50-361, 50-362 Licenses: NPF-10, NPF-15 Report No.: 05000361/2007003 and 5000362/2007003 Licensee: Southern California Edison Co. (SCE)

Facility: San Onofre Nuclear Generating Station, Units 2 and 3 Location: 5000 S. Pacific Coast Hwy.

San Clemente, California Dates: April 1 to June 26, 2007 Inspectors: C. C. Osterholtz, Senior Resident Inspector, Project Branch E, DRP M. A. Sitek, Resident Inspector, Project Branch E, DRP G. W. Apger, Operations Engineer, DRS P. J. Elkmann, Emergency Preparedness Inspector, DRS R. E. Lantz, Senior Emergency Preparedness Inspector, DRS T. O. McKernon, Senior Operations Engineer, DRS G. D. Replogle, Senior Project Engineer, Project Branch E, DRP Approved By: Jeffrey A. Clark, Chief Project Branch E Division of Reactor Projects-1- Enclosure

SUMMARY OF FINDINGS

IR05000361/2007003, 05000362/2007003; 04/01/07 - 06/26/07; San Onofre Nuclear

Generating Station, Units 2 & 3; Integrated Resident and Regional Report; Operability Evaluations and Identification and Resolution of Problems.

This report covered a 3-month period of inspection by resident inspectors and Regional office inspectors. The inspection identified two Green findings, both of which were noncited violations. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management's review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.

Cornerstone: Mitigating Systems

Green.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the failure of engineering personnel to promptly identify and correct the formation of gas pockets in the piping of the Units 2 and 3 component cooling water systems from March 8 to December 15, 2006. This deficiency resulted in the Unit 3 Train B component cooling water system being inoperable for approximately eight days from December 8 to 15, 2006. This issue was entered into the licensees corrective action program as Action Requests 061001379 and 070500468.

The finding was more than minor, because it was similar to the not minor if, conditions in Example 3.j of Manual Chapter 0612, Appendix E, Examples of Minor Issues, in that the performance deficiency resulted in a condition where there was reasonable doubt on the operability of the component cooling water system. The inspectors evaluated the issue using the Manual Chapter 0609, Appendix A, "Significance Determination of Reactor Inspection Findings for At-Power Situations," Phase 1 screening worksheet. The inspectors determined that a Phase 2 significance determination was required because both the mitigating systems and containment barriers cornerstones were affected. The inspectors performed a Phase 2 significance determination using the Risk-Informed Inspection Notebook for San Onofre Nuclear Generating Stations, Units 2 and 3,

Revision 2.1. The finding was potentially greater than Green using these worksheets.

The inspectors requested that a Region IV Senior Reactor Analyst perform a Phase 3 significance determination to provide a better estimation of overall risk significance.

Based on the results of the Phase 3 analysis, the finding was determined to have very low safety significance (Green). The cause of the finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program (P.1(c)) because the licensee failed to thoroughly evaluate the formation of gas in the Units 2 and 3 component cooling water systems to ensure that the cause and extent of condition were addressed in a timely manner (Section 1R15).

Green.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the failure of engineering personnel to promptly correct a common cause problem associated with debris intrusion into potentiometers in the automatic voltage regulators of the emergency diesel generators. The licensee identified the root cause of the problem in September 2006, but failed to correct the problem before an additional failure on March 3, 2007. This issue was entered into the licensees corrective action program as Action Requests 060800603 and 070300161.

The finding was determined to be more than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and it affected the cornerstone objective by reducing the availability, reliability, and capability of the emergency diesel generators. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding was determined to have very low safety significance (Green) because it did not result in an actual loss of safety function for the affected system. The cause of the finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program (P.1.(d)) because the licensee failed to take appropriate corrective action to address the deficient emergency diesel generator automatic voltage regulator potentiometers in a timely manner (Section 4OA2).

Licensee-Identified Violations

Violations of very low safety significance which were identified by the licensee have been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensee's corrective action program. These violations and their corrective actions are listed in Section 4OA7 of this report.

REPORT DETAILS

Summary of Plant Status

Unit 2 began the inspection period at approximately 99 percent reactor power. On May 5, 2007, operations personnel reduced reactor power to approximately 75 percent for one day in order to obtain an oil sample from a reactor coolant Pump 2P002. On June 15, 2007, operations personnel commenced a reactor shutdown in order to repair hydraulic leak on a main feedwater isolation Valve 2HV4048. Unit 2 returned to approximately 96 percent power on June 18, 2007. On June 20, 2007, operations personnel manually tripped the reactor in response to a decline of instrument air pressure caused by a pipe break in the system. Unit 2 ended the inspection period in a shutdown condition at normal operating pressure and temperature.

Unit 3 began the inspection period at approximately 100 percent reactor power. On May 11, 2007, operations personnel commenced lowering reactor power to approximately 15 percent.

Unit 3 was separated from the electrical grid on May 12, 2007, in order to allow maintenance personnel to repair an oil leak on the main electrical transformer. Unit 3 was reconnected to the electrical grid on May 14, 2007, and returned to approximately 100 percent reactor power on May 15, 2007. Unit 3 ended the inspection period at approximately 100 percent reactor power.

REACTOR SAFETY

Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity

1R01 Adverse Weather Protection

.1 Readiness For Impending Adverse Weather Conditions

a. Inspection Scope

On April 12, 2007, the inspectors completed a review of the licensee's readiness for impending adverse weather involving high surf and sea grass intrusion. The inspectors:

(1) reviewed plant procedures, the Updated Final Safety Analysis Report (UFSAR), and Technical Specifications (TS) to ensure that operator actions defined in adverse weather procedures maintained the readiness of essential systems;

(2) walked down portions of the listed system to ensure that adverse weather protection features (heat tracing, space heaters, weatherized enclosures, temporary chillers) were sufficient to support operability, including the ability to perform safe shutdown functions;

(3) reviewed maintenance records to determine that applicable surveillance requirements were current before the anticipated high surf and sea grass developed; and

(4) reviewed plant modifications, procedure revisions, and operator work arounds to determine if recent facility changes challenged plant operation.

C April 11 - 13, 2007, Units 2 and 3, intake structure and saltwater cooling system

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

1R04 Equipment Alignment

.1 Partial System Walkdowns

a. Inspection Scope

The inspectors:

(1) walked down portions of the three listed risk important systems and reviewed plant procedures and documents to verify that critical portions of the selected systems were correctly aligned; and

(2) compared deficiencies identified during the walk down to the licensee's UFSAR and corrective action program (CAP) to ensure problems were being identified and corrected.

C April 3, 2007, Unit 2, realignment of the emergency core cooling system following the filling of safety injection Tank 2T009 C April 10, 2007, Unit 2, Train B emergency diesel Generator (EDG) 2G003 while the Train A EDG 2G002 was out of service for planned maintenance C April 27, 2007, Unit 2, realignment of the chemical and volume control system following misalignment of the rad waste controls Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed three samples.

b. Findings

No findings of significance were identified.

1R05 Fire Protection

a. Inspection Scope

Quarterly Inspection The inspectors walked down the six listed plant areas to assess the material condition of active and passive fire protection features and their operational lineup and readiness.

The inspectors:

(1) verified that transient combustibles and hot work activities were controlled in accordance with plant procedures;

(2) observed the condition of fire detection devices to verify they remained functional;

(3) observed fire suppression

systems to verify they remained functional and that access to manual actuators was unobstructed;

(4) verified that fire extinguishers and hose stations were provided at their designated locations and that they were in a satisfactory condition;

(5) verified that passive fire protection features (electrical raceway barriers, fire doors, fire dampers, steel fire proofing, penetration seals, and oil collection systems) were in a satisfactory material condition;

(6) verified that adequate compensatory measures were established for degraded or inoperable fire protection features and that the compensatory measures were commensurate with the significance of the deficiency; and

(7) reviewed the UFSAR to determine if the licensee identified and corrected fire protection problems.

C May 15, 2007, Unit 2 auxiliary feedwater pump room C May 15, 2007, Units 2 and 3, technical support center C May 15, 2007, Unit 2 secondary switchgear room C June 22, 2007, Unit 3 auxiliary feedwater pump room C June 22, 2007, Unit 3 secondary switchgear room C June 22, 2007, Units 2 and 3, control room Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed six samples.

b. Findings

No findings of significance were identified.

1R12 Maintenance Effectiveness

Maintenance Effectiveness Baseline Review

a. Inspection Scope

The inspectors reviewed the three listed maintenance activities to:

(1) verify the appropriate handling of structure, system, and component (SSC) performance or condition problems;

(2) verify the appropriate handling of degraded SSC functional performance;

(3) evaluate the role of work practices and common cause problems; and

(4) evaluate the handling of SSC issues reviewed under the requirements of the maintenance rule, 10 CFR Part 50 Appendix B, and the TSs.

• May 15, 2007, Unit 2, Micro Switch RYCA20B latching relay performance and obsolescence replacement plan

• May 14-18, 2007, Units 2 and 3, motor operated valve testing capabilities

• May 19-20, 2007, Units 2 and 3, control room dust induced electrical switch malfunctions and corrective actions Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed three samples.

b. Findings

No findings of significance were identified.

1R13 Maintenance Risk Assessments and Emergent Work Control

.1 Emergent Work Control

a. Inspection Scope

The inspectors:

(1) verified that the licensee performed actions to minimize the probability of initiating events and maintained the functional capability of mitigating systems and barrier integrity systems;

(2) verified that emergent work-related activities such as troubleshooting, work planning/scheduling, establishing plant conditions, aligning equipment, tagging, temporary modifications, and equipment restoration did not place the plant in an unacceptable configuration; and

(3) reviewed the UFSAR to determine if the licensee identified and corrected risk assessment and emergent work control problems.

• April 4, 2007, Unit 3, atmospheric dump Valve 3HV8421 opened to approximately 18 percent versus the demanded 25 percent during surveillance testing

• April 17-20, 2007, Unit 2, Train A EDG 2G002 bent fuel injector shaft

• April 25 and May 12-14, 2007, Unit 3, main transformer oil preservation system flexible connection pipe leak

• May 5, 2007, Unit 2, reactor coolant Pump 2P002 increased upper bearing temperature

• June 4-18, 2007, Unit 2, main feedwater isolation Valve 2HV4048 hydraulic leak

• June 20-26, 2007, Units 2 and 3, instrument air header breach Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed six samples.

b. Findings

Findings associated with the June 20-26, 2007, instrument air header breach were being evaluated at the end of the inspection period by a special inspection team (see NRC Inspection Report 05000361/2007013). Otherwise, no findings of significance were identified.

1R15 Operability Evaluations

a. Inspection Scope

The inspectors:

(1) reviewed plant status documents such as operator shift logs, emergent work documentation, deferred modifications, and standing orders to determine if an operability evaluation was warranted for degraded components;

(2) referred to the UFSAR and design basis documents to review the technical adequacy of licensee operability evaluations;

(3) evaluated compensatory measures associated with operability evaluations;

(4) determined degraded component impact on any TSs;

(5) used the Significance Determination Process to evaluate the risk significance of degraded or inoperable equipment; and

(6) verified that the licensee has identified and implemented appropriate corrective actions associated with degraded components.

• March 28, 2007, Unit 3, suspected incorrect oil in the Train B component cooling water Pump 3P026

• April 9, 2007, Units 2 and 3, incorrect impedance value used in switchyard to reserve auxiliary transformer calculations

• April 24, 2007, Unit 2, incorrect oil added to governor actuator of the Train A EDG 2G002

• April 1 to June 26, 2007, Units 2 and 3, component cooling water systems long-term operability to address the formation of gas pockets in the critical and non-critical loops

• May 15, 2007, Unit 2, EDG 2G003 air compressor thermal overload failure Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed five samples.

b. Findings

Introduction.

The inspectors identified a Green noncited violation (NCV) of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the failure of engineering personnel to promptly identify and correct the formation of gas pockets in the piping of the Units 2 and 3 component cooling water (CCW) systems.

Description.

On March 8, 2006, the licensee generated action request (AR) 060300413 to document abnormal Unit 2 CCW surge tank level trends that suggested the possibility that gas was entrained in the system. Multiple concurrent issues with the Unit 2 CCW system made it difficult to conclusively determine if gas was entrained in the system.

The licensee continued to monitor the CCW surge tank level trends and generated an AR assignment on May 17, 2006, to vent portions of the Unit 2 Train B CCW system.

The licensee did not actually perform the venting until August 3, 2006, because of

competing priorities and scheduling conflicts. Based on the change in surge tank level, the licensee determined that the equivalent of approximately 100 gallons of gas (13.4 cubic feet) was vented from a dead leg in the CCW noncritical loop return line from the radioactive waste system. The licensee completed an operability assessment on August 23, 2006, after being prompted by the inspectors, which concluded that the gas would be isolated during an accident and would not adversely affect the operation of the CCW system. At the same time, the inspectors questioned the extent that gas was in other parts of the Units 2 and 3 CCW systems.

The licensee did not vent safety-related portions of the Unit 2 CCW system until September 12, 2006, approximately 40 days after finding approximately 13.4 cubic feet of gas in the noncritical loop of Unit 2. Up to that point, the licensee still had not vented any portions of the Unit 3 CCW system. The results of the venting on September 12 revealed that approximately 25 gallons of gas were found downstream of the Train A CCW return from the letdown heat exchanger throttle Valve 2TV0223. The licensee theorized that gas was coming out of solution after passing through Valve 2TV0223. The licensee indicated that they suspected that the same gas accumulation was likely to exist on Unit 3 because of the similar piping and valve configuration. The corresponding areas on Unit 3 were vented approximately 17 days later on September 29, 2006, where approximately 100 gallons of equivalent gas were discovered. Up to this point, the licensee still did not know the extent of the condition in other parts of the CCW system nor did they fully understand how the gas pockets were forming. An additional 15 days passed before the licensee vented a second time downstream of Valves 2/3TV0223 in the Units 2 and 3 Train A CCW systems on October 14, 2006, where they discovered 19 and 14 gallons of gas, respectively. The licensee waited an additional 10 days to vent a third time downstream of Valves 2/3TV0223, where on October 24, 2006, they discovered additional gas. At that point the licensee determined that the gas was reforming and in order to ensure operability of the Units 2 and 3 CCW system, they initiated daily system venting.

Approximately three months after the initial discovery of gas in the system, the licensee formed a multi-disciplinary team (tiger team) to evaluate the issue in a focused and methodical manner. Up to that point, licensee management had not been fully involved with this issue because of poor interdepartmental communication. As a result, the issue was not being addressed in a timely manner. Furthermore, as of October 25, 2006, the licensee had not fully evaluated other portions of the CCW system to ensure a complete understanding of the extent of the condition. In addition, the inspectors discovered that four separate operability assessments had been performed on the issue since August 2006, but those assessments addressed the potential impact of the individual gas pockets found as opposed to addressing the CCW system as a whole.

Daily venting of the Unit 3 CCW system was suspended during portions of the Unit 3 refueling outage from late October to mid-December 2006 because the system was drained for maintenance. Following the refueling outage, the licensee vented additional points of the Unit 3 CCW system in order to ensure that all of the gas had been removed.

On December 11, 2006, 106 gallons of gas was vented downstream of the Train A CCW return from the letdown heat exchanger throttle Valve 3TV0223. The licensee indicated that they had not expected gas at that location and did not fully understand what caused

the gas pocket to form. Despite venting other portions of the CCW system daily, the licensee waited four days to vent the location a second time, where on December 15, 2006, 120 gallons of gas were vented. The licensee contracted with a vendor to perform a fifth operability assessment of the issue. The licensee determined that, from November 24 through December 15, 2006, the Unit 3 Train B CCW system was inoperable. During that time frame, Unit 3 was being restarted following a refueling outage and entered Mode 4 on December 7, 2006. As a result, the Unit 3 Train B CCW system was inoperable for an eight day period when it was in Modes 1 through 4. A train of CCW may be inoperable for up to 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> according to Technical Specification 3.7.7.

The licensee submitted Licensee Event Report (LER) 2006-006 to the NRC on April 9, 2007, to document the issue.

The licensee continued to vent the Units 2 and 3 CCW systems on a daily basis in order to maintain system operability. In parallel, the tiger team was chartered to determine the source of the gas pockets and to develop solutions to prevent recurrence. In early March 2007 the inspectors reviewed the progress of the tiger team and determined that it had become relatively dormant and unfocused. Specifically, the inspectors noted that a standard vent procedure had not been created. In addition, the tiger team had not implemented nor even planned various tests to aid in determining the source of the gas.

As a result, the inspectors met weekly with the team in order to monitor their progress.

After implementing those meetings, the tiger team made significant progress to resolve the issue.

The licensee determined that the source of the gas in the CCW systems was from a combination of improperly venting the systems following draining them during outages and from the throttling effect created by Valves 2/3TV0223. The licensees planned corrective actions include: installing a by-pass line around Valves 2/3TV0223 in order to ensure that the pipes remain full of water; revising the CCW system fill-and-vent procedure; and continuing to vent the CCW systems at an appropriate frequency to ensure that the systems remain full of water.

Analysis.

The failure to promptly identify and correct the accumulation of gas in the safety-related portions of the Units 2 and 3 CCW systems, after identifying that gas accumulation was occurring, was a performance deficiency. The finding was more than minor, because it was similar to the not minor if, conditions in Example 3.j of Manual Chapter 0612, Appendix E, Examples of Minor Issues, in that the performance deficiency resulted in a condition where there was reasonable doubt on the operability of the component cooling water system. The inspectors evaluated the issue using the Manual Chapter 0609, Appendix A, "Significance Determination of Reactor Inspection Findings for At-Power Situations," Phase 1 screening worksheet. The inspectors determined that a Phase 2 significance determination was required because both the mitigating systems and containment barriers cornerstones were affected. The inspectors performed a Phase 2 significance determination using the Risk-Informed Inspection Notebook for San Onofre Nuclear Generating Stations, Units 2 and 3, Revision 2.1.

The finding was potentially greater than Green using these worksheets. The dominant initiator was a small break loss of coolant accident and the dominant core damage

sequences included: 1) containment heat removal; 2) high pressure recirculation; and 3)early inventory high pressure injection. The inspectors requested that a Region IV Senior Reactor Analyst perform a Phase 3 significance determination to provide a better estimation of the overall risk significance.

The analyst performed a calculation to bound the change in risk. The most significant assumptions used in the analysis included:

• The Unit 3, Train B CCW void identified would only affect CCW operability if Valve 3TV0223 (CCW temperature control valve) failed open concurrent with isolation of the non-critical loop. This limited the applicable sequences to loss of coolant accidents, steam generator tube rupture and the loss of offsite power.

• Valve 3TV0223 will go fully open upon isolation of the non-critical loop.

• Common cause failure rates were nominal.

• The letdown heat exchanger was supplied with CCW from Train B about half the time (PTrainB).

• The exposure period was December 8 to 15, 2006. The exposure time (EXP) was 8 days.

The analyst quantified the change in risk by using the failure of the Train B heat exchanger as a surrogate for the complete failure of the train. The resulting conditional core damage frequency was 1.987 x 10-5/yr. The plant-specific standardized plant analysis risk has a baseline core damage frequency of 1.381 x 10-5/yr. Therefore, the change in core damage probability over one year (CCDP) for a failure of CCW Train B, given that the specific failure would not impact Train A was: 6.06 x 10-6. The analyst determined that 89.55% of this probability was the result of the initiating events (Pinitiators)discussed in Assumption 1. The analyst then determined that the change in core damage frequency (CDF) as defined by the significance determination process was the product of the conditional core damage probability, the probability that the cutsets were affected by the performance deficiency, the probability that Train B was serving the letdown heat exchanger, and the exposure period for the subject performance deficiency.

Therefore, the analyst concluded that the risk related to the subject performance deficiency could be no greater than the value calculated as follows:

CDF = CCDP

• Pinitiators

• PTrainB

• EXP

= (6.06 x 10-6) * (0.8955) * (0.5) * (7 days/365 days)

= 5.2 x 10-8 Based on the results of the Phase 3 analysis, the finding is determined to have very low safety significance (Green).

The cause of the finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program (P.1(c)) because the licensee failed to thoroughly evaluate the formation of gas in the Units 2 and 3 CCW systems to ensure that the cause and extent of condition were addressed in a timely manner.

Enforcement.

The regulations in 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, state, in part, that measures shall be established to ensure that conditions adverse to quality, such as deviations and nonconformances are promptly identified and corrected. Contrary to this requirement, the licensee failed to promptly identify and correct the formation of gas pockets in the Units 2 and 3 CCW systems from March 8 to December 15, 2006. The formation of gas pockets deviated from, and was not in conformance with, the design of the CCW system. Because the finding is of very low safety significance and has been entered into the licensees corrective action program as ARs 061001379 and 070500468, this violation is being treated as an NCV consistent with Section VI.A of the NRC Enforcement Policy: NCV 05000361; 362/2007003-01, Failure to Promptly Identify and Correct Gas Accumulation in the Units 2 and 3 Component Cooling Water Systems.

1R19 Postmaintenance Testing

a. Inspection Scope

The inspectors selected the five listed postmaintenance test activities of risk significant systems or components. For each item, the inspectors:

(1) reviewed the applicable licensing basis and/or design-basis documents to determine the safety functions;

(2) evaluated the safety functions that may have been affected by the maintenance activity; and

(3) reviewed the test procedure to ensure it adequately tested the safety function that may have been affected. The inspectors either witnessed or reviewed test data to verify that acceptance criteria were met, plant impacts were evaluated, test equipment was calibrated, procedures were followed, jumpers were properly controlled, the test data results were complete and accurate, the test equipment was removed, the system was properly re-aligned, and deficiencies during testing were documented. The inspectors also reviewed the UFSAR to determine if the licensee identified and corrected problems related to postmaintenance testing.

• April 2, 2007, Unit 3, control element Assembly CEA 86 following corrective maintenance

• April 20-21, 2007, Unit 2, Train A EDG 2G002 following planned maintenance

• May 4, 2007, Unit 2, component cooling water Pump 2P025 following planned maintenance

• May 10, 2007, Units 2 and 3, Train B emergency chilled water system following corrective maintenance on emergency Chiller ME335

• May 12, 2007, Units 2 and 3, diesel driven firewater Pump 2P220 following failed surveillance test and corrective maintenance Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed five samples.

b. Findings

No findings of significance were identified.

1R22 Surveillance Testing

a. Inspection Scope

The inspectors reviewed the UFSAR, procedure requirements, and TSs to ensure that the five listed surveillance activities demonstrated that the SSCs tested were capable of performing their intended safety functions. The inspectors either witnessed or reviewed test data to verify that the following significant surveillance test attributes were adequate:

(1) preconditioning;

(2) evaluation of testing impact on the plant;

(3) acceptance criteria;

(4) test equipment;

(5) procedures;

(6) jumper/lifted lead controls;

(7) test data;

(8) testing frequency and method demonstrated TS operability;

(9) test equipment removal;

(10) restoration of plant systems;

(11) fulfillment of ASME Code requirements;

(12) updating of performance indicator data;

(13) engineering evaluations, root causes, and bases for returning tested SSCs not meeting the test acceptance criteria were correct;

(14) reference setting data; and

(15) annunciators and alarms setpoints. The inspectors also verified that the licensee identified and implemented any needed corrective actions associated with the surveillance testing.

• November 7, 2006, Unit 3, reactor coolant Pump 3P004 oil reservoir level instrument calibration check

• April 21-22, 2007, Unit 2, Train B EDG 2G003 monthly surveillance

• April 23, 2007, Unit 3, atmospheric dump Valve 3HV8419 inservice test

• April 24, 2007, Unit 2, Train B high pressure safety injection Pump 2P019 inservice test

• May 11, 2007, Unit 3, Train A EDG 3G002 semi-annual fast start and loaded run Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed five samples.

b. Findings

No findings of significance were identified.

1R23 Temporary Plant Modifications

a. Inspection Scope

The inspectors reviewed the UFSAR, plant drawings, procedure requirements, and TSs to ensure that the listed temporary modification was properly implemented. The inspectors:

(1) verified that the modifications did not have an affect on system operability/availability;

(2) verified that the installation was consistent with modification documents;

(3) ensured that the post-installation test results were satisfactory and that the impact of the temporary modifications on permanently installed SSCs were supported by the test;

(4) verified that the modifications were identified on control room drawings and that appropriate identification tags were placed on the affected drawings; and

(5) verified that appropriate safety evaluations were completed. The inspectors verified that licensee identified and implemented any needed corrective actions associated with temporary modifications.

• April 24, 2007, Unit 3, installation of a temporary leak sealing device on auxiliary feedwater vent Valve 3MR333 in order to mitigate a body-to-bonnet leak Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance was identified.

1EP1 Exercise Evaluation

a. Inspection Scope

The inspectors reviewed the objectives and scenario for the 2007 biennial emergency plan exercise to determine if the exercise would acceptably test major elements of the licensee's capabilities to implement the emergency plan. The scenario simulated a transformer failure, a failure of the reactor protection system to automatically shut down the reactor, a failed reactor coolant pump seal with loss of coolant, a medical response to an injured person, fission product barrier failures, core damage and a radiological release to the environment through the containment mini-purge line.

The inspectors evaluated exercise performance by focusing on the risk-significant activities of event classification, offsite notification, recognition of offsite dose consequences, and development of protective action recommendations, in the simulator control room and the following emergency response facilities:

• Technical support center

• Operations support center

• Emergency operations facility The inspectors also assessed recognition of and response to abnormal and emergency plant conditions, the transfer of decision making authority and emergency function responsibilities between facilities, onsite and offsite communications, protection of emergency workers, emergency repair evaluation and capability, and the overall implementation of the emergency plan to protect public health and safety and the environment. The inspectors reviewed the current revision of the facility emergency plan, and emergency plan implementing procedures associated with operation of the above facilities and performance of the associated emergency functions. These procedures are listed in the Attachment to this report.

The inspectors compared the observed exercise performance with the requirements in the facility emergency plan; 10 CFR 50.47(b); 10 CFR Part 50, Appendix E; and with the guidance in the emergency plan implementing procedures and other federal guidance.

The inspectors attended the post-exercise critiques in each of the above facilities to evaluate the initial licensee self-assessment of exercise performance. The inspectors also attended a subsequent formal presentation of critique items to plant management.

The inspectors completed one sample during the inspection.

b. Findings

No findings of significance were identified.

1EP4 Emergency Action Level and Emergency Plan Changes

.1 Emergency Plan Implementing Procedure Units Revision

a. Inspection Scope

The inspector performed an in-office review of Revision 25 to the San Onofre Nuclear Generating Station Emergency Plan Implementing Procedure SO123-VIII-1 Recognition and Classification of Emergencies, submitted November 11, 2006. This revision changed the units of measurement in one emergency action level from mR/h to mrem/h.

The revision was compared to the previous revision, to the criteria of NUREG-0654, Criteria for Preparation and Evaluation of Radiological Emergency Response Plans and Preparedness in Support of Nuclear Power Plants, Revision 1, and to the standards in 10 CFR 50.47(b) to determine if the revision was adequately conducted following the requirements of 10 CFR 50.54(q). This review was not documented in a Safety Evaluation Report and did not constitute approval of licensee changes, therefore this revision is subject to future inspection.

The inspectors completed one sample during the inspection.

b. Findings

No findings of significance were identified.

.2 Emergency Plan Protective Action Recommendation Revision

a. Inspection Scope

The inspector performed an in-office review of Revision 20 to Section 6, "Emergency Measures," of the San Onofre Nuclear Generating Station Emergency Plan, submitted in February 2007. This revision changed the protective action recommendation methodology to divide the emergency planning zone into five distinct protective action zones, with protective action recommendations for each zone based on a risk-informed evaluation of that zone. This revision was submitted for pre-approval to the NRC by letter dated November 17, 2006, and was approved in a subsequent Safety Evaluation Report dated December 6, 2006.

The revision was compared to the previous revision, to the criteria of NUREG-0654, Criteria for Preparation and Evaluation of Radiological Emergency Response Plans and Preparedness in Support of Nuclear Power Plants, Revision 1, to the standards in 10 CFR 50.47(b), and to the December 6, 2006, approval letter, to determine if the revision was adequately conducted following the requirements of 10 CFR 50.54(q). This revision was documented in a Safety Evaluation Report and is therefore the new approved revision against which future revisions will be evaluated.

The inspectors completed one sample during the inspection.

b. Findings

No findings of significance were identified.

4OA1 Performance Indicator (PI) Verification

Cornerstone: Emergency Preparedness

a. Inspection Scope

The inspectors reviewed licensee evaluations for the three emergency preparedness cornerstone performance indicators of Drill and Exercise Performance, Emergency Response Organization Participation, and Alert and Notification System Reliability, for the period January 1, 2006, through March 31, 2007. The definitions and guidance of Nuclear Energy Institute 99-02, Regulatory Assessment Indicator Guideline, Revisions 2 through 4, and licensee Performance Indicator Procedures SO123-VIII-0.401 Emergency Preparedness Performance Indicators, Revision 1, and

SSSPG-SO123-G-8, "Offsite Emergency Planning - Alert and Notification System Performance Indicator," Revision 2, were used to verify the accuracy of the licensees evaluations for each performance indicator reported during the assessment period.

The inspectors reviewed a 100 percent sample of drill and exercise scenarios and licensed operator simulator training sessions, notification forms, and attendance and critique records associated with training sessions, drills, and exercises conducted during the verification period. The inspectors reviewed 16 selected emergency responder qualification, training, and drill participation records. The inspectors reviewed alert and notification system testing procedures, maintenance records, and a 100 percent sample of siren test records. The inspector also reviewed other documents as listed in the to this report.

The inspectors completed one sample during the inspection.

b. Findings

No findings of significance were identified.

4OA2 Identification and Resolution of Problems

.1 Routine Review of Identification and Resolution of Problems

a. Inspection Scope

The inspectors performed a daily screening of items entered into the licensee's corrective action program. This assessment was accomplished by reviewing maintenance orders, action requests, the management focus list, and attending corrective action review and work control meetings. The inspectors:

(1) verified that equipment, human performance, and program issues were being identified by the licensee at an appropriate threshold and that the issues were entered into the corrective action program;

(2) verified that corrective actions were commensurate with the significance of the issue; and (3)identified conditions that might warrant additional follow-up through other baseline inspection procedures.

b. Findings

No findings of significance were identified.

.2 Selected Issue Follow-up Inspection

a. Inspection Scope

In addition to the routine review, the inspectors selected the listed issue for a more in-depth review. The inspectors considered the following during the review of the licensee's actions:

(1) complete and accurate identification of the problem in a timely manner;

(2) evaluation and disposition of operability/reportability issues;

(3) consideration of extent of condition, generic implications, common cause, and previous occurrences;

(4) classification and prioritization of the resolution of the problem;

(5) identification of root and contributing causes of the problem;

(6) identification of corrective actions; and

(7) completion of corrective actions in a timely manner.

C May 17-24, 2007, Units 2 and 3, emergency diesel generator automatic voltage regulator potentiometer failures Documents reviewed by the inspectors are listed in the attachment.

b. Findings

Introduction.

The inspectors identified a Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the failure of engineering personnel to promptly correct a common cause problem associated with debris intrusion into potentiometers in the automatic voltage regulators (AVR) of the emergency EDG.

Description.

On March 3, 2007, the Unit 3 Train B EDG 3G003 was operated for a scheduled monthly surveillance test. During the test, operations personnel noted that the EDGs volt-amperes reactive (VAR) output was oscillating unexpectedly while on AVR Channel A. Operations personnel successfully completed the test, but declared the EDG inoperable because of the erratic VAR indications associated with AVR Channel A.

Operations personnel switched to AVR Channel B and successfully performed the surveillance test. The Unit 3 Train B EDG 3G003 was subsequently declared operable approximately two hours after the initial AVR Channel A surveillance problem.

In response to the AVR Channel A failure, the licensee performed a common cause evaluation of the other AVRs in the Units 2 and 3 EDGs. This evaluation was done, in part, to satisfy Condition B of TS 3.8.1, AC Sources - Operating, which required either a cause evaluation or a successful surveillance test of the Unit 3 Train A EDG within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. The licensee satisfied the TS requirements by successfully performing the surveillance test on AVR Channel B.

The inspectors reviewed the common cause evaluation and determined that it was inadequate. The evaluation report indicated that the exact cause cannot be determined at this time because no troubleshooting has been performed yet. At the time, the licensee had not physically inspected the cabinet area that housed the AVR or the AVR itself in order to make a meaningful common cause assessment. The licensee waited approximately eight days after the event before they removed and physically inspected the AVR. The licensee sent the erratic AVR to a vendor who determined that the most likely cause was a foreign substance inside the voltage adjustment potentiometer that caused an intermittent open connection between the potentiometers wiper arm and the resistor pad.

The inspectors reviewed the licensees previous operating experience with these AVR potentiometers and determined that the same type of failure had occurred three previous times. Specifically, in July 2006, the Unit 3 Train B EDG AVR was erratic during postmaintenance testing. In August 2006, the Unit 2 Train B EDG AVR was erratic during a monthly surveillance test. Finally, in September 2006, the Unit 3 Train A EDG

AVR was erratic during postmaintenance testing. The inspectors also reviewed AR 060800603 which documented a direct cause evaluation of these failures that was performed in September 2006. The licensee determined that the erratic behavior of the AVRs was caused by the open type housing of the potentiometers that allowed dirt and dust to interfere with the operation of the potentiometers wiper. The corrective action for this common cause issue was to replace the open type housing potentiometers with ones of a sealed design in order to prevent the introduction of foreign substances into the potentiometer. The inspectors noted, however, that the engineering design change assignments were not generated until January 12, 2007. In addition, at the time of the March 3, 2007, AVR failure, the replacement of the potentiometers still had not been scheduled by maintenance personnel. The inspectors determined that the corrective actions were untimely. The licensee subsequently took appropriate corrective actions and scheduled the replacements to be completed by the end of June 2007.

Analysis.

The failure of the licensee to take timely corrective action to correct a common cause problem associated with debris intrusion into EDG AVR potentiometers was determined to be a performance deficiency. The finding was determined to be more than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and it affected the cornerstone objective by reducing the availability, reliability, and capability of the emergency diesel generators. Using the Manual Chapter 0609, Significance Determination Process, Phase 1 worksheet, the finding was determined to have very low safety significance (Green) because it did not result in an actual loss of safety function for the affected system.

The cause of the finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program (P.1.(d)) because the licensee failed to take appropriate corrective action to address the deficient EDG AVR potentiometers in a timely manner.

Enforcement.

The regulations in 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, state, in part, that measures shall be established to ensure that conditions adverse to quality, such as deficiencies are promptly identified and corrected. Contrary to this requirement, the licensee failed to promptly correct a deficient EDG AVR potentiometer design from September 2006 to June 2007. Because the finding is of very low safety significance and has been entered into the licensees corrective action program as ARs 060800603 and 070300161, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy: NCV 05000361; 362/2007003-02, Failure to Promptly Correct Deficient Emergency Diesel Generator Potentiometers.

.3 Semiannual Trend Review

a. Inspection Scope

The inspectors completed a semi-annual trend review of repetitive or closely related issues that were documented to identify trends that might indicate the existence of more safety significant issues, specifically in the areas of procedural compliance and human performance. The inspectors review consisted of the six month period from January 1, 2007, through June 25, 2007. When warranted, some of the samples expanded beyond

those dates to fully assess the issue. The inspectors also reviewed corrective action program items associated with human performance improvement, and met with representatives from the San Onofre human performance improvement team at regular intervals. Corrective actions associated with a sample of the issues identified in the licensee's trend report were reviewed for adequacy. Documents reviewed by the inspectors are listed in the attachment.

b. Findings

No findings of significance were identified. However, the inspectors noted five additional instances where deficiencies in procedural compliance and human performance contributed to inadvertent and unexpected problems. On February 11, 2007, an inadvertent boration of approximately 20 gallons occurred on Unit 3 because a planned boration on February 10 was secured without flushing the volume control tank makeup line as required by procedure (AR 070200484). On March 17, 2007, the Unit 3 Train A emergency diesel Generator 3G003 was inadvertently started during testing due to miscommunications between test technicians and control room operators (AR 070300896). On April 2, 2007, the Unit 2 Train A charging Pump 2P190 was inadvertently tripped when a technician bumped a breaker relay while performing motor monitoring testing (AR 070400082). On April 24, 2007, an equipment operator added a tablespoon of incorrect oil to the Unit 2 Train A emergency diesel Generator 2G002 (AR 070401313). On April 27, 2007, Unit 2 letdown was inadvertently isolated when a radioactive waste operator depressed the wrong control button during a rad waste water diversion.

The inspectors noted that the licensee continued to attempt to implement human performance initiatives to prevent personnel errors.

.4 Emergency Preparedness

a. Inspection Scope

The inspectors reviewed scenarios and evaluation reports for drills and exercises conducted between May 1, 2005, and March 31, 2007. The reports were reviewed to identify prior performance weakness and deficiencies in order to evaluate the effectiveness of corrective actions during the biennial exercise.

b. Findings

No findings of significance were identified.

4OA3 Follow-up

a. Inspection Scope

Event Follow-up The inspectors reviewed the one below listed event and degraded condition for plant status and mitigating actions to:

(1) provide input in determining the appropriate agency response in accordance with Management Directive 8.3, "NRC Incident Investigation Program;"

(2) evaluate performance of mitigating systems and licensee actions; and

(3) confirm that the licensee properly classified the event in accordance with emergency action level procedures and made timely notifications to NRC and state/governments, as required.

• June 20, 2007, Units 2 and 3, instrument air header breach The inspectors completed one sample.

Event Report Reviews The inspectors reviewed the below listed LER and related documents to assess:

(1) the accuracy of the LER;

(2) the appropriateness of corrective actions;

(3) violations of requirements; and

(4) generic issues.

.1 (Closed) Licensee Event Report 05000362/2006-006-00, One Train of Component

Cooling Water Considered Inoperable for Longer than Allowed by Technical Specification 3.7.7 due to Gas Entrainment The inspectors determined that a Green noncited violation occurred. This issue is documented in Section 4OA2 of this report as a violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action. The violation of Technical Specification 3.7.7 that is referenced in the licensee event report was caused by the corrective action violation. As a result, the corrective action violation serves to encompass the technical specification violation and is more indicative of licensee performance. This licensee event report is closed.

4OA6 Meetings, Including Exit

On April 16, 2007, the Emergency Preparedness inspector conducted a telephonic exit meeting to present the results of the emergency plan change inspection to Mr. B. Ashbrook, Manager, Onsite Emergency Planning. The inspector confirmed that proprietary information was not provided or examined during the inspection.

On April 20, 2007, the inspectors presented the emergency preparedness inspection results to Dr. R. Waldo, Vice President, Nuclear Generation, and other members of licensee staff who acknowledged the findings.

On June 28, 2007, the resident inspectors presented the quarterly inspection results to Dr. R. Waldo and other members of licensee staff who acknowledged the findings.

The inspectors returned all proprietary information that was examined during the inspections.

4OA7 Licensee-Identified Violations

The following violations of very low safety significance (Green) were identified by the licensee and are violations of NRC requirements which meet the criteria of Section VI of the NRC Enforcement Policy, NUREG-1600, for being dispositioned as NCVs.

• Title 10 CFR 50.54(q) states in part, "A licensee . . . shall follow emergency plans which meet the standards in §50.47(b) and the requirements in appendix E of this part . . ." 10 CFR 50.47(b)(4) states in part, "A standard emergency classification and action level scheme, the bases of which include facility system and effluent parameters, is in use by the nuclear facility licensee . . . " Title 10 CFR 50, Appendix E, IV.B states in part, . . . . "The emergency action levels shall be based on in-plant conditions and instrumentation . . . ." Contrary to the above, between 1985 and January 2007, San Onofre Nuclear Generating Station did not follow an emergency plan meeting the standards of 50.47(b) and Appendix E, because Emergency Action Levels A1-1, A2-1, A2-2, and A3-1 could not be fully implemented, because the upper measurement ranges of Process Radiation Monitors 6753, 6759, and 7808G, did not extend to emergency action level thresholds. This finding is of very low safety significance because either the radiation monitors backed-up ambient radiation measurements and dose assessment, or the affected radiation monitor was one of two installed in the release path, so that the affected monitor was not the sole available indicator of an emergency condition. This issue was identified in the licensees corrective action program as AR 070100476.

• Title 10 CFR 50.54(q) states in part, "A licensee . . . shall follow . . . emergency plans which meet the standards in §50.47(b) . . ." Title 10 CFR 50.47(b)(2) states in part, ".

. . adequate staffing to provide initial facility accident response in key functional areas is maintained at all times . . ." 10 CFR 50.47(b)(15) states, "Radiological emergency response training is provided to those who may be called on to assist in an emergency." Contrary to the above, the licensee identified that on February 6 and 26, 2007, a total of four watch positions requiring self-contained breathing apparatus qualification were staffed by individuals whose self-contained breathing apparatus mask fit test had expired. This finding is of very low safety significance because although the licensees on shift staffing process allowed more than two shifts during a 30-day period to go below emergency plan requirements, the personnel were present and could have performed their required emergency plan functions under most circumstances. This performance deficiency was entered into the licensee's corrective action program as AR 070201084.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

B. Ashbrook, Manager, Emergency Planning

D. Axline, Technical Specialist, Nuclear Regulatory Affairs

D. Breig, Station Manager

B. Corbett, Manager, Health Physics

B. Culverhouse, Manager, Offsite Emergency Planning

J. Hirsch, Manager, Maintenance

K. Johnson, Manager, Design Engineering

M. Johnson, Manager, Environmental Protection

M. Jones, Human Performance Improvement Manager, Nuclear Oversight and Assessment

B. Katz, Vice President, Nuclear Oversight and Regulatory Affairs

W. Marsh, Engineer, Maintenance Engineering

C. McAndrews, Manager, Nuclear Oversight and Assessment

H. Newton, Manager, Site Support Services

N. Quigley, Manager, Mechanical/Nuclear Maintenance Engineering

J. Reilly, Vice President, Engineering and Technical Services

A. Scherer, Manager, Nuclear Regulatory Affairs

A. Shean, Manager, Nuclear Oversight

W. Strom, Manager Electrical, Maintenance Engineering

T. Vogt, Manager, Operations

R. Waldo, Vice President, Nuclear Generation

D. Wilcockson, Manager, Plant Operations

C. Williams, Manager Compliance, Nuclear Regulatory Affairs

T. Yackle, Manager, Maintenance Engineering

NRC personnel

D. Loveless, Senior Reactor Analyst, DRS

C. Speer, Engineering Associate

T. Skaggs-Ryan, Engineering Associate

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

None

Opened and Closed

05000361; NCV Failure to Promptly Identify and Correct Gas Accumulation 2/2007003-01 in the Units 2 and 3 Component Cooling Water Systems (Section 1R15)

Attachment

05000361; NCV Failure to Promptly Correct Deficient Emergency Diesel 2/2007003-02 Generator Potentiometers (Section 4OA2)

Closed

05000362/2006-006-00 LER One Train of Component Cooling Water Considered Inoperable for Longer than Allowed by Technical Specification 3.7.7 due to Gas Entrainment (Section 4OA3)

Discussed

None