IR 05000528/2007003

IR 05000528-07-003, 05000529-07-003, 05000530-07-003; 04/01/07 - 06/30/07; Palo Verde Nuclear Generating Station, Units 1, 2, and 3; Integrated Resident and Regional Report; Maint. Effectiveness, Operability Eval., Follow-up of Events

ML072070484
Person / Time
Site:	Palo Verde
Issue date:	07/20/2007
From:	O'Keefe N NRC/RGN-IV/DRP/RPB-D
To:	Edington R Arizona Public Service Co
References
IR-07-003
Download: ML072070484 (59)

Text

uly 20, 2007

SUBJECT:

PALO VERDE NUCLEAR GENERATING STATION - NRC INTEGRATED INSPECTION REPORT 05000528/2007003, 05000529/2007003, AND 05000530/2007003

Dear Mr. Edington:

On June 30, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at your Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility. The enclosed integrated report documents the inspection findings, which were discussed on June 29, 2007, with you and other members of your staff.

The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your licenses. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

The report documents two NRC identified finding and two self-revealing findings which involved violations of NRC requirements. Three of these findings were evaluated under the risk significance determination process as having very low safety significance (Green). One finding was not suitable for evaluation under the significance determination process; however, it was determined to be of very low safety significance by NRC management review. Because of the very low safety significance of these violations and because they were entered into your corrective action program, the NRC is treating these findings as noncited violations consistent with Section VI.A of the NRC Enforcement Policy. Additionally, the report documents one additional example of an NRC identified violation documented in NRC Inspection Report 05000528; 05000529; 05000530/2007011. Additionally, licensee-identified violations which were determined to be of very low safety significance are listed in this report. If you contest these noncited violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington DC 20555-0001; and the NRC Resident Inspector at the Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility.

Arizona Public Service Company -2-In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS) ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Neil F. O'Keefe, Chief Project Branch D Division of Reactor Projects Dockets: 50-528 50-529 50-530 Licenses: NPF-41 NPF-51 NPF-74

Enclosure:

NRC Inspection Report 05000528/2007003, 05000529/2007003, and 05000530/2007003 w/Attachment: Supplemental Information

REGION IV==

Dockets: 50-528, 50-529, 50-530 Licenses: NPF-41, NPF-51, NPF-74 Report: 05000528/2007003, 05000529/2007003, 05000530/2007003 Licensee: Arizona Public Service Company Facility: Palo Verde Nuclear Generating Station, Units 1, 2, and 3 Location: 5951 S. Wintersburg Road Tonopah, Arizona Dates: April 1 through June 30, 2007 Inspectors: J. Bartleman, Reactor Inspector, Region III D. Bollock, Project Engineer S. Garchow, Operations Engineer J. Groom, Project Engineer B. Henderson, Reactor Inspector T. Jackson, Senior Resident Inspector G. Larkin, Senior Resident Inspector J. Melfi, Resident Inspector J. Nadel, Reactor Inspector B. Tindell , Operations Engineer G. Warnick, Senior Resident Inspector Approved By: Neil F. O'Keefe, Chief, Project Branch D Division of Reactor Projects-1- Enclosure

SUMMARY OF FINDINGS

IR 05000528/2007003, 05000529/2007003, 05000530/2007003; 04/01/07 - 06/30/07; Palo

Verde Nuclear Generating Station, Units 1, 2, and 3; Integrated Resident and Regional Report;

Maint. Effectiveness, Operability Eval., Follow-up of Events.

This report covered a 3-month period of inspection by resident inspectors and regional inspectors. The inspection identified four findings and one additional example of a previous finding. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management's review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.

Cornerstone: Initiating Events

Green.

A self-revealing noncited violation of 10 CFR 50.65(b)(2)(iii) was identified for the failure of engineering personnel to place some components of the condensate demineralizer system into the scope of its program for monitoring the effectiveness of maintenance. Specifically, on October 19, 2006, the Unit 3 reactor was manually tripped when condenser vacuum was degraded due to the failure of condensate demineralizer vessel waste drain Valve 3JSCNUV0232. Prior operating experience at Palo Verde demonstrated that the failure of Valve 3JSCNUV0232 could result in a reactor trip.

However, the licensee did not appropriately scope Valve 3JSCNUV0232 into its program for monitoring the effectiveness of maintenance. This issue was entered into the corrective action program as Condition Report/Disposition Request 3035444.

The finding is greater than minor because it is associated with the initiating events cornerstone attribute of equipment performance and affects the cornerstone objective to limit the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Using Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance since it does not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions will not be available. (Section 1R12)

Green.

A self-revealing noncited violation of 10 CFR 50.65(a)(3) was identified for failure of the licensee to incorporate internal and external industry operating experience into preventative maintenance activities that could have prevented a maintenance rule functional failure of feedwater pump Turbine A, a high risk heat removal system.

Specifically, prior to March 18, 2007, the licensee did not incorporate available operating experience into preventative maintenance instructions to inspect, clean, and verify acceptable equipment condition for the linear variable differential transmitter linkage assembly. Failure to inspect and clean the linear variable differential transmitter linkage assembly resulted in a broken linkage due to binding, causing erratic cycling of the feedwater pump turbine control valves resulting in a manual trip of feedwater Pump A and reactor power cutback to 48 percent power. This issue was entered into the corrective action program as Condition Report/Disposition Request 2984713.

The finding is greater than minor because it is associated with the initiating events cornerstone attribute of equipment performance and affects the cornerstone objective to limit the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown and power operations. Using the Manual Chapter 0609,

"Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance because the finding did not result in exceeding the Technical Specification limit for identified reactor coolant system leakage and did not affect other mitigation systems; the finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions will not be available; and the finding did not increase the likelihood of a fire or internal/external flood. This finding has a crosscutting aspect in the area of problem identification and resolution, associated with operating experience, since engineering personnel failed to account for prior operating experience in determining the maintenance rule scope and appropriate preventive maintenance for Valve 3JSCNUV0232 (P.2(b)). (Section 4OA3.1)

Cornerstone: Mitigating Systems

Green.

The inspectors identified a noncited violation of 10 CFR Part 50, Appendix B,

Criterion XVI, "Corrective Actions," for the failure of inservice inspection personnel to promptly identify and correct a condition adverse to quality. Specifically, since April 19, 2006, floor-welded spray pond pipe Supports 13-SP-030-H-007 and 13-SP-030-H-008 in the essential pipe density tunnel became degraded at the weld due to long term standing water in the tunnel. The licensee thought these supports had been previously identified and placed in the corrective action program, but that was not the case. This issue was entered into the corrective action program as Palo Verde Action Request 2989960.

The finding is greater than minor because if left uncorrected the degradation would have led to a more significant safety concern. The finding is associated with the mitigating systems cornerstone. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance since it only affected the mitigating systems cornerstone and did not represent a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event.

The cause of the finding is also related to the crosscutting aspect of problem identification and resolution with a corrective action program causal factor because the threshold for identifying issues was not sufficiently low and the degraded supports were not identified completely, accurately, and in a timely manner commensurate with their safety significance (P.1.(a)). (Section 1R15)

Cornerstone: Barrier Integrity

Green.

The inspectors identified a noncited violation of 10 CFR Part 50, Appendix B,

Criterion V, "Instructions, Procedures, and Drawings," for the failure of operations and refueling personnel to follow Procedure 40DP-9OP02, "Conduct of Shift Operations," Revision 37, when a load error condition occurred during core reloading. Specifically, on June 18, 2007, operations and refueling personnel failed to recognize that the load error condition was the result of a degraded refueling machine control system and could have resulted in fuel damage, a condition that required an Event Recovery Checklist.

This event, along with another event that occurred in the spent fuel pool on May 3, 2007, that involved human performance errors by refueling personnel, indicate that corrective actions associated with past fuel handling problems may not have been completely effective. (See NCVs05000528/2004003-04 and 05000529/2005003-03).

This issue was entered into the corrective action program as Palo Verde Action Request 3029781.

The finding is greater than minor because it could become a more significant safety concern if left uncorrected in that handling fuel with a degraded refueling machine could result in fuel barrier damage. This finding cannot be evaluated by the significance determination process because Manual Chapter 0609, "Significance Determination Process," Appendix A, "Significance Determination of Reactor Inspection Findings for At-Power Situations," and Appendix G, "Shutdown Operations Significance Determination Process," do not apply to the refueling cavity for the plant conditions that existed during the event. This finding affects the barrier integrity cornerstone and is determined to be of very low safety significance by NRC management review because it was a deficiency that did not result in the actual degradation of fuel. This finding has a crosscutting aspect in the area of human performance associated with decision-making because operations and refueling personnel did not make safety significant decisions using a systematic process, when faced with uncertain or unexpected equipment performance, to ensure safety is maintained (H.1(a)). (Section 4OA3.2)

Licensee-Identified Violations

Violations of very low safety significance which were identified by the licensee have been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensee's corrective action program. These violations and corrective actions are listed in Section 4OA7 of this report.

REPORT DETAILS

Summary of Plant Status

Unit 1 operated at essentially full power until May 19, 2007, when the unit was shutdown for refueling Outage 1R13. At the end of the inspection period, the unit was in Mode 5.

Unit 2 operated at essentially full power until May 12, 2007, when power was reduced to 60 percent to repair a bearing oil leak on main feedwater (MFW) Pump A. Following repairs to the MFW pump, the unit was returned to essentially full power on May 14. On June 16 power was again reduced to 60 percent power to repair the same oil bearing on MFW Pump A. The unit returned to essentially full power following repairs on June 17 and remained there for the duration of the inspection period.

Unit 3 operated at essentially full power until April 12, 2007, when the unit was shutdown to repair the main turbine lube oil booster pump. Following repairs and inspection of the main turbine lube oil system, the unit was returned to essentially full power on May 3. On May 29 the unit reduced power to 40 percent to repair a main condenser tube leak. Following repairs to the main condenser, the unit returned to essentially full power on June 3 and remained there for the duration of the inspection period.

REACTOR SAFETY

Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity

1R01 Adverse Weather Protection

a. Inspection Scope

Readiness For Seasonal Susceptibilities The inspectors completed a review of the licensee's readiness for seasonal susceptibilities involving impending high temperatures. The inspectors:

(1) reviewed plant procedures, the Updated Final Safety Analysis Report (UFSAR), and Technical Specifications to ensure that operator actions defined in adverse weather procedures maintained the readiness of essential systems;

(2) walked down portions of the four systems listed below to ensure that adverse weather protection features (weatherized enclosures, temporary chillers, etc.) were sufficient to support operability, including the ability to perform safety functions;

(3) evaluated operator staffing levels to ensure the licensee could maintain the readiness of essential systems required by plant procedures; and

(4) reviewed the corrective action program (CAP) to determine if the licensee identified and corrected problems related to adverse weather conditions.

C April 18 - 19, 2007, Unit 1, spray pond (SP) system, Trains A and B C April 18 - 19, 2007, Unit 1, turbine building cooling water C April 18 - 19, 2007, Unit 2, main generator/main transformer

C April 18 - 19, 2007, Unit 3, SP system Trains A and B Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

1R04 Equipment Alignment

a. Inspection Scope

Partial Walkdown The inspectors:

(1) walked down portions of the four below listed risk important systems and reviewed plant procedures and documents to verify that critical portions of the selected systems were correctly aligned; and

(2) compared deficiencies identified during the walkdown to the licensee's UFSAR and CAP to ensure problems were being identified and corrected.

C April 5, 2007, Unit 3, emergency diesel generator (EDG) Train B while Train A was out of service for preplanned maintenance

• May 3, 2007, Unit 3, safety injection Train B while Train A was out of service for preplanned maintenance

• May 16, 2007, Unit 3, essential cooling water (EW), and SP system Train A while Train B was out of service for preplanned maintenance

• June 10, 2007, Unit 1, fuel pool cooling Train B while Train A was out of service for preplanned maintenance Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed four samples.

b. Findings

No findings of significance were identified.

1R05 Fire Protection

a. Inspection Scope

Quarterly Inspection The inspectors walked down the eight below listed plant areas to assess the material condition of active and passive fire protection features and their operational lineup and readiness. The inspectors:

(1) verified that transient combustibles and hot work activities were controlled in accordance with plant procedures;

(2) observed the condition of fire detection devices to verify they remained functional;

(3) observed fire suppression systems to verify they remained functional and that access to manual actuators was unobstructed;

(4) verified that fire extinguishers and hose stations were provided at their designated locations and that they were in a satisfactory condition;

(5) verified that passive fire protection features (electrical raceway barriers, fire doors, fire dampers, steel fire proofing, penetration seals, and oil collection systems) were in a satisfactory material condition;

(6) verified that adequate compensatory measures were established for degraded or inoperable fire protection features and that the compensatory measures were commensurate with the significance of the deficiency; and

(7) reviewed the UFSAR to determine if the licensee identified and corrected fire protection problems.

• April 9, 2007, Unit 2, auxiliary building, 40 foot, 52 foot, 70 foot, and 88 foot elevations

• April 10, 2007, Unit 2, SP pump rooms

• April 11, 2007, Unit 1, fuel building, 100 foot, 120 foot, and 140 foot elevations

• April 11, 2007, Unit 2, auxiliary building, 100 foot, 120 foot, and 140 foot elevations

• April 16, 2007, Unit 1, auxiliary building, 40 foot, 52 foot, 70 foot, and 88 foot elevations

• April 19, 2007, Unit 3, control building, 100 foot, 120 foot, and 140 foot elevations

• April 24, 2007, Unit 3, turbine building, 100 and 140 foot elevations following a lube oil spill

• May 3, 2007, Unit 2, diesel generator building, all elevations Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed eight samples.

Annual Inspection On May 25, 2007, the inspectors observed a fire brigade drill to evaluate the readiness of the licensee to fight fires, including the following aspects:

(1) the number of personnel assigned to the fire brigade,

(2) use of protective clothing,

(3) use of breathing apparatuses,

(4) use of fire procedures and declarations of emergency action levels,

(5) command of the fire brigade,

(6) implementation of pre-fire strategies and briefs,

(7) access routes to the fire and the timeliness of the fire brigade response,

(8) establishment of communications,

(9) effectiveness of radio communications,

(10) placement and use of fire hoses,

(11) entry into the fire area,

(12) use of fire fighting equipment,

(13) searches for fire victims and fire propagation,

(14) smoke removal,

(15) use of pre-fire plans,

(16) adherence to the drill scenario,

(17) performance of the post-drill critique, and

(18) restoration from the fire drill.

• May 25, 2007, Unit 3, simulated fire in control building, direct current equipment room Train D Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

1R06 Flood Protection Measures

a. Inspection Scope

Semi-annual Internal Flooding The inspectors:

(1) reviewed the UFSAR, the flooding analysis, and plant procedures to assess seasonal susceptibilities involving internal flooding;

(2) reviewed the UFSAR and CAP to determine if the licensee identified and corrected potential flood protection problems;

(3) inspected underground bunkers/manholes to verify the adequacy of

(a) sump pumps,

(b) level alarm circuits,

(c) cable splices subject to submergence, and

(d) drainage for bunkers/manholes;

(4) verified that operator actions for coping with flooding can reasonably achieve the desired outcomes; and

(5) walked down the four below listed areas to verify the adequacy of:

(a) equipment seals located below the floodline,

(b) floor and wall penetration seals,

(c) watertight door seals,

(d) common drain lines and sumps,

(e) sump pumps, level alarms, and control circuits, and (f)temporary or removable flood barriers.

C June 19, 2007, Unit 3, auxiliary feedwater (AFW) pump rooms and surrounding areas

• June 20, 2007, Unit 2, containment spray, low pressure safety injection, and high pressure safety injection pump rooms and surrounding areas

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

1R07 Heat Sink Performance

a. Inspection Scope

Annual Inspection The inspectors reviewed licensee programs, verified performance against industry standards, and reviewed critical operating parameters and maintenance records for the heat exchangers associated with Unit 1 essential SP Train A. The inspectors verified that:

(1) performance tests were satisfactorily conducted for heat exchangers/heat sinks and reviewed for problems or errors;

(2) the licensee utilized the periodic maintenance method outlined in EPRI NP-7552, "Heat Exchanger Performance Monitoring Guidelines;"

(3) the licensee properly utilized biofouling controls;

(4) the licensees heat exchanger inspections adequately assessed the state of cleanliness of their tubes, and

(5) the heat exchanger was correctly categorized under the maintenance rule.

• June 14, 2007, Unit 1, EDG lube oil and jacket water coolers/heat exchangers Train A

• June 18, 2007, Unit 1, EW water heat exchanger Train A Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

No findings of significance were identified.

1R08 Inservice Inspection Activities

02.01 Inspection Activities Other Than Steam Generator Tube Inspections, PWR Vessel Upper Head Penetration Inspections, Boric Acid Corrosion Control

a. Inspection Scope

The inspection procedure requires review of two or three types of non-destructive examination (NDE) activities and, if performed, one to three welds on the reactor coolant

system pressure boundary. Also the procedure requires review of one or two examinations with recordable indications that have been accepted by the licensee for continued service.

The inspectors directly observed the following non-destructive examinations:

System Component/Weld Identification Examination Method Shutdown Cooling Safe end to piping Dissimilar Metal Visual (VT)

Loop 1 Weld Shutdown Cooling Safe end to piping Dissimilar Metal Visual (VT)

Loop 2 Weld Letdown Line 2B Safe end to piping Dissimilar Metal Visual (VT)

Weld Charging Line Safe end to piping Dissimilar Metal Visual (VT)

Weld Pressurizer Surge Nozzle, Welds 5-34 & 20-1 Ultrasonic (UT)

During the review and observation of each examination, the inspectors verified that activities were performed in accordance with ASME Boiler and Pressure Vessel Code requirements and applicable procedures. The qualifications of all non-destructive examination technicians performing the inspections were verified to be current.

There were no NDE examinations with recordable indications that were accepted for continued service.

Records from seven examples of welding on the reactor coolant system pressure boundary (Class 1) were examined as follows:

System Component/Weld Identification Pressurizer Surge Nozzle, Welds 5-34 & 20-1 Pressurizer Spray Nozzle, Welds 5-33 & 29-1 Pressurizer Safety Nozzle, Weld 5-29 Pressurizer Safety Nozzle, Weld 5-30 Pressurizer Safety Nozzle, Weld 5-31 Pressurizer Safety Nozzle, Weld 5-32 Pressurizer Hot Leg Surge Nozzle, Welds 6-4 & 20-11

Welding procedures and non-destructive examination of the welding conformed to ASME Code requirements and licensee requirements.

The inspector completed one sample under Section 02.01.

b. Findings

No findings of significance were identified.

02.02 Vessel Upper Head Penetration (VUHP) Inspection Activities

a. Inspection Scope

The licensee performed non-destruction examination of 100 percent of reactor vessel upper head penetrations. The inspector directly observed a sample of the examinations as listed below:

System Component/Weld Identification Examination Method Reactor Pressure Control Element Drive Eddy Current Vessel (RPV) Head Mechanism (CEDM) 55 (ET)/UT RPV Head CEDM 62 ET/ UT RPV Head CEDM 87 ET/UT The following sample of examinations were reviewed using stored electronic data and review of printed records:

System Component/Weld Identification Examination Method RPV Head CEDM 15 ET/UT RPV Head CEDM 47 ET/UT/Dye Penetrant (PT)

RPV Head CEDM 48 ET/UT RPV Head CEDM 64 ET/UT/PT The NDE inspections were verified to be in accordance with the requirements of NRCs "First Revised NRC Order (EA-03-009) Establishing Interim Inspection Requirements for Reactor Pressure Vessel Heads at PWRs," issued February 20, 2004. No defects were detected, and no weld repairs were necessary.

The inspectors completed one sample under Section 02.02.

b. Findings

No findings of significance were identified.

02.03 Boric Acid Corrosion Control Inspection (BACC) Activities

a. Inspection Scope

Resident inspectors observed a sample of BACC activities and verified that visual inspections emphasized locations where boric acid leaks can cause degradation of safety significant components.

The inspector reviewed 10 instances where boric acid deposits were found on reactor coolant system piping components:

Component Number Description Action Request SDCHX A Shutdown Cooling Heat Exchanger A 2969301 SIAHV 0684 Valve 3029328 SIAHV 0687 Valve 3028359 SIAV 157 Valve 3029329 SIAV 434 Valve 3023287 SIAV 485 Valve 3029327 SIBHV 0695 Valve 3026562 SIBV 158 Valve 3029589 SIBV 200 Valve 3026794 SIBV 484 Valve 3027719 Because the shutdown cooling heat exchangers are required to be operable during shutdown operations by Technical Specifications 3.6.6 Containment Spray; 3.4.6 RCS Loops Mode 4; 3.4.7 RCS Loops Mode 5, Loops Filled; and 3.4.8 RCS Loops Mode 5, Loops Not Filled; a prompt operability evaluation was conducted for the shutdown cooling heat exchanger. Appropriate ASME code requirements were considered in the evaluation, and it was concluded that the heat exchanger remained operable. The condition of the all the components was appropriately entered into the licensees corrective action program, and corrective actions taken were consistent with ASME code requirements.

The inspectors completed one sample under Section 02.03.

b. Findings

No findings of significance were identified.

02.04 Steam Generator Tube Inspection Activities

a. Inspection Scope

This was the first cycle of operation for the new steam generators installed at Palo Verde Unit 1 during the previous outage. During this refueling outage, no tubes were identified for plugging, and no tubes were identified that met the requirements for in-situ pressure testing, thus no in-situ pressure testing was performed.

The inspector compared the recommended test scope to the actual test scope and found that the licensee had accounted for all known areas of previous wear and had established a test scope that met Technical Specification requirements, EPRI guidelines, and commitments made to the NRC. The scope of the licensees eddy current examinations of tubes in both steam generators included:

• A full length bobbin examination of 100 percent of inservice tubes

• Plus point, rotating coil exams for the U-Bend area of tubes in Rows 1-4

• Plus point, rotating coil exams of special interest locations No new degradation mechanisms were identified during the inspection activities, and all areas of potential degradation, as indicated by plant specific experience, were inspected.

No steam generator tube leakage in excess of three gallons per day was identified prior to entering the refueling outage or during post-shutdown visual inspections.

No loose parts or foreign materials were identified prior to the outage. Several indications during the inspection suggested loose parts, but these were examined with plus point, rotating coil and identified as chemical deposits.

The steam generator tube inspection contractor used eddy current probes that were appropriate to find the type of degradation expected. Extensive use of the plus point, rotating probe was employed.

The inspectors reviewed a sample of steam generator tube inspection data for tubes in which indications were present as listed below:

Steam Generator Tube Row/Line Indication 165/76 Possible Loose Part (PLP)132/85 Distorted Support Indication

48/103 13% throughwall 157/82 PLP 163/82 PLP 165/82 PLP 48/107 13% throughwall The inspectors completed one sample under Section 02.04.

b. Findings

No findings of significance were identified.

02.05 Identification and Resolution of Problems

a. Inspection scope

The inspection procedure requires review of a sample of problems associated with inservice inspections documented by the licensee in the corrective action program for appropriateness of the corrective actions.

The inspector reviewed nine corrective action reports which dealt with inservice inspection activities. Action requests reviewed are listed in the documents reviewed section. From this review, the inspector concluded that the licensee had an appropriate threshold for entering issues into the corrective action program and had procedures that direct a root cause evaluation when necessary. The licensee also had an effective program for applying industry operating experience.

The timeliness of corrective actions taken for CRDR 2827845 was reviewed by the inspector. This issue concerned a number of bolted connections on borated systems that were not properly examined for leakage by removing insulation in accordance with Article IWA-5242(a) of ASME Boiler and Pressure Vessel Code, 1992 edition. Palo Verde Relief Requests 11, 15 and 16, approved by NRC, implemented Code Cases N-533, N-533-1, and N-616, which modified the requirements of Article IWA-5242(a).

In addition, the inspector conducted a review of inspection results of bolted connections conducted for Units 2 and 3. An operability evaluation performed for Unit 1 was based in part on conditions found previously in the other units, which identified a number of instances of significant build up of boric acid deposits beneath the insulation. The licensee correctly followed ASME Code procedures by removing and inspecting individual bolts in the areas where boric acid deposits were found. The bolts examined did not exhibit corrosion in excess of ASME Code standards.

As a result of the inspectors questioning regarding timeliness of the inspection process, the licensee reexamined the operability evaluation as applied in Unit 1 in light of the extent of the boric acid deposits found in the other units, and decided to inspect Unit 1 bolting during the current outage.

These inspections, which were also reviewed by the inspector, identified 10 instances of significant boric acid deposits beneath insulation, but no corrosion of bolts in excess of ASME Code standards.

Corrective actions taken in regard to bolting found with evidence of leakage as a result of these inspections was adequate, based on a review of records.

b. Findings

No findings of significance were identified.

1R11 Licensed Operator Requalification Program

a. Inspection Scope

Quarterly Requalification Activities Review The inspectors observed testing and training of senior reactor operators and reactor operators to identify deficiencies and discrepancies in the training, to assess operator performance, and to assess the evaluator's critique. The training scenario involved a loss of condenser vacuum and a loss of coolant accident.

• May 1, 2007, Simulator Scenario SES009U01, "Loss of Condenser Vacuum, Steam Space LOCA, MVAC," Revision 1 Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

1R12 Maintenance Effectiveness

Quarterly Inspection

a. Inspection Scope

The inspectors reviewed the two below listed maintenance activities to:

(1) verify the appropriate handling of structure, system, and component (SSC) performance or condition problems;

(2) verify the appropriate handling of degraded SSC functional performance;

(3) evaluate the role of work practices and common cause problems; and

(4) evaluate the handling of SSC issues reviewed under the requirements of the maintenance rule, 10 CFR Part 50 Appendix B, and the Technical Specifications.

• April 25, 2007, Unit 3, failure of condensate demineralizer vessel waste drain Valve 3JSCNUV0232 resulting in a reactor trip

• May 10, 2007, Unit 3, failure of fast bus transfer Breaker 3ENANS03B charging springs to charge Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed two samples.

b. Findings

Introduction.

A self-revealing Green noncited violation (NCV) of 10 CFR 50.65(b)(2)(iii)was identified for the failure of engineering personnel to place some components of the condensate demineralizer system into the scope of its program for monitoring the effectiveness of maintenance. Specifically, on October 19, 2006, the Unit 3 reactor was manually tripped when condenser vacuum was degraded due to the failure of condensate demineralizer vessel waste drain Valve 3JSCNUV0232. Prior operating experience at Palo Verde demonstrated that the failure of Valve 3JSCNUV0232 could result in a reactor trip. However, the licensee did not appropriately scope Valve 3JSCNUV0232 into its program for monitoring the effectiveness of maintenance.

Additionally, planned design changes or administrative controls to remove the potential for a reactor trip were not implemented.

Description.

On October 19, 2006, operators manually tripped the Unit 3 reactor as a result of decreasing condenser vacuum and the imminent loss of the MFW pumps.

Following the reactor trip, troubleshooting efforts revealed that Valve 3JSCNUV0232 failed to close following a demineralizer resin rinse. When a resin transfer operation was initiated, condensate demineralizer drain to turbine building sump Valve 3JSCNUV0233 was opened, providing a path from the main condenser to atmosphere. With a pathway from the main condenser to atmosphere, condenser vacuum was degraded and hotwell level began to lower resulting in the loss of two out of three condensate pumps. The loss of the condensate pumps caused pre-trip alarms on both MFW pumps. Recognizing an imminent automatic reactor trip on low steam generator levels if the MFW pumps tripped, the operators manually tripped the Unit 3 reactor.

A root cause investigation was conducted and documented in corrective report/disposition request (CRDR) 2934020. The root cause investigation identified previous operating experience from February 1999, in which the air line on Valve 3JSCNUV0232 failed due to excessive vibration and fatigue cracking. This caused the valve to be partially open during condensate demineralizer vent and drain operations, resulting in a degraded condenser vacuum transient. Additionally, the licensee recognized the potential for a reactor trip if Valve 3JSCNUV0232 failed open during demineralizer vent and drain operations and initiated design modification Work Order (WO) 258848 in February 2003. The purpose of the design modification was to

install an interlocking feature to prevent both Valves 3JSCNUV0232 and 3JSCNUV0233 from being open at the same time. The modification has been on hold since mid 2004.

The cause of the valves failure was determined to be wear in the solenoid portion of the valve actuator to the extent that wear particles prevented movement of the slug operated pilot which shuttles the air position slide valve. This condition permitted the valve to remain open when controls removed power to the solenoid coil. No preventive maintenance had been performed on the valve as the licensee had incorrectly determined that failure of the valve would not result in an adverse impact.

The inspectors questioned engineering personnel with regards to the maintenance rule scoping of the condensate demineralizer system and preventive maintenance for Valve 3JSCNUV0232. The inspectors found that the condensate demineralizer system is part of the secondary chemistry system. However, only the steam generator blowdown system was considered to be scoped into the maintenance rule from the overall secondary chemistry system. The inspectors determined that the condensate demineralizer system and in particular Valve 3JSCNUV0232, should have been scoped into the maintenance rule since its failure could cause a reactor trip, as evidenced by previous operating history.

Analysis.

The performance deficiency associated with this finding was the failure of engineering personnel to properly scope the function associated with Valve 3JSCNUV0232 into the maintenance rule program. The finding is greater than minor because it is associated with the initiating events cornerstone attribute of equipment performance and affects the cornerstone objective to limit the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Using Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance since it does not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions will not be available.

Enforcement.

10 CFR 50.65(b)(2)(iii) requires, in part, that the scope of the monitoring program specified in paragraph (a)(1) of this section shall include nonsafety related SSCs whose failure could cause a reactor scram. Contrary to this, since February 1999, engineering personnel failed to properly scope the necessary SSCs associated with Valve 3JSCNUV0232 into the Palo Verde maintenance monitoring program when prior operating experience demonstrated the potential for a failure of the valve to cause a reactor scram. On October 19, 2006, failure of the valve to close resulted in a reactor scram. The licensee initiated CRDR 3035444 to address the failure to properly scope the SSCs associated with Valve 3JSCNUV0232 into the maintenance monitoring program.

As an interim action, the licensee modified Operating Procedure 40OP-9SC06, "Demineralizer Resin Transfers," Revision 16, to require Manual Valve VS-35 to be closed when performing resin transfers in order to remove the risk of a failure of Valve 3JSCNUV0232 causing a reactor trip. Because the finding is of very low risk significance and has been entered into the CAP as CRDR 3035444, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy:

NCV 05000528; 05000529;05000530/2007003-01, "Failure to Scope Condensate Demineralizer Valve Into Maintenance Rule."

1R13 Maintenance Risk Assessments and Emergent Work Control

a. Inspection Scope

Risk Assessment and Management of Risk The inspectors reviewed the below listed assessment activity to verify:

(1) performance of risk assessments when required by 10 CFR 50.65 (a)(4) and licensee procedures prior to changes in plant configuration for maintenance activities and plant operations;

(2) the accuracy, adequacy, and completeness of the information considered in the risk assessment;

(3) that the licensee recognizes, and/or enters as applicable, the appropriate licensee-established risk category according to the risk assessment results and licensee procedures; and

(4) the licensee identified and corrected problems related to maintenance risk assessments.

• May 16, 2007, Unit 3, risk assessment and management during scheduled essential SP, EW, and essential chilled water Train B outage Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

Emergent Work Control The inspectors:

(1) verified that the licensee performed actions to minimize the probability of initiating events and maintained the functional capability of mitigating systems and barrier integrity systems;

(2) verified that emergent work-related activities such as troubleshooting, work planning/scheduling, establishing plant conditions, aligning equipment, tagging, temporary modifications, and equipment restoration did not place the plant in an unacceptable configuration; and

(3) reviewed the UFSAR to determine if the licensee identified and corrected risk assessment and emergent work control problems.

• April 30, 2007, Unit 3, troubleshooting efforts associated with the control element drive mechanism control system motor generator Set B as described in corrective maintenance WO 3007125

• May 8, 2007, Unit 3, risk assessment and management after fast bus transfer Breaker 3ENANS03B (S01-S03 tie breaker) closing spring failed to charge

• May 10 - 12, 2007, Unit 2, troubleshooting and repair efforts associated with plant protection system Channel A bypass indicator lights as described in corrective maintenance WO 2772114

• May 29, 2007, Unit 3, inoperable SP Pump A due to not meeting acceptable flow criteria per Procedure 73ST-9SP01, "Essential Spray Pond Pumps - Inservice Test," Revision 23

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed four samples.

b. Findings

No findings of significance were identified.

1R15 Operability Evaluations

a. Inspection Scope

The inspectors:

(1) reviewed plant status documents such as operator shift logs, emergent work documentation, deferred modifications, and night orders to determine if an operability evaluation was warranted for degraded components;

(2) referred to the UFSAR and design basis documents to review the technical adequacy of licensee operability evaluations;

(3) evaluated compensatory measures associated with operability evaluations;

(4) determined degraded component impact on any Technical Specifications;

(5) used the Significance Determination Process, to evaluate the risk significance of degraded or inoperable equipment; and

(6) verified that the licensee has identified and implemented appropriate corrective actions associated with degraded components.

• April 1, 2007, Unit 2, drops of oil identified from the telltale drain Valve 2PDGNV602 associated with EDG Train B turbocharger intercooler as described in Palo Verde Action Request (PVAR) 2988724

• April 3, 2007, Unit 3, SP piping Supports 3SP-030-H-0007 and 3SP-030-H-0008 as describe in PVAR 2989960

• April 27, 2007, Unit 3, diesel generator compressor oil seal leak with potential to affect intercooler performance as described in PVAR 3006463

• May 3, 2007, Unit 1, atmospheric dump valve steam leakage past seat as described in PVAR 2982229

• May 11, 2007, Units 1, 2, and 3, Class 1E battery 18 month surveillance test did not satisfy Technical Specification Surveillance Requirement 3.8.4.7 as described in PVAR 3011973

• May 12, 2007, Unit 1, Class 1E Train C battery jars with crack indications as described in PVAR 3012802

• May 29 - 31, 2007, Unit 3, operability assessment associated with SP Train A low flow indication as described in PVAR 3019401

• May 31, 2007, Units 1, 2, and 3, Unit 1 EDG Train B turbocharger lube oil filter 3-way valve incorrect orientation and extent of condition review for other EDGs as described in PVAR 3018721

• June 18, 2007, Unit 1, nitrogen supply to steam Generator 1 economizer feedwater isolation valve installed backwards as described in PVAR 3027193 and CRDR 3028343 Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed nine samples.

b. Findings

Introduction.

The inspectors identified a Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Actions," for the failure of inservice inspection personnel to properly identify and correct a condition adverse to quality. Specifically, two floor-welded pipe supports in the essential pipe density tunnel became degraded at the weld due to long term standing water in the tunnel. The licensee thought these supports had been previously identified and placed in the corrective action program, but that was not the case.

Description.

For over 10 years, there have been water intrusion issues in underground equipment rooms. The essential pipe density tunnel is a below grade pipe chase that contains safety related piping from several systems, including the SP system. The piping enters and exits the tunnel through penetrations in the walls. Water leaks through several piping penetration seals and collects on the floor of the tunnel. Similar water intrusion in other underground equipment areas in the plant have led to NRC violations and significant corrective action plans. The licensee has identified many of the degraded conditions caused by the water and entered them into their corrective action program.

In April 2007, inspectors identified floor-welded pipe Supports 13-SP-030-H-007 and 13-SP-030-H-008 in the Unit 3 essential pipe density tunnel that were significantly rusted and corroded at the weld interface due to standing water on the floor. The supports in question were supporting a safety related SP system pipe. Inservice inspections for these pipe supports on April 29, 2006, satisfied acceptance criteria. When questioned about the degraded supports the licensee initially indicated that there was a WO planned to clean and re-coat the supports. The WO was part of the corrective actions for a previous CRDR which had identified the degraded supports. Upon review of that CRDR and WO, the inspectors challenged the licensee on their initial response. In fact, the CRDR dealt with degraded supports and components in a separate equipment room.

Corrosion products were cleaned from the welds and the water was removed. A prompt operability determination (POD) concluded that a reasonable expectation of operability existed. PVAR 2989960 resulted in corrective actions to generate WOs to clean and re-coat the affected supports.

Analysis.

Inspectors determined that the failure to identify and correct the degraded piping supports was a performance deficiency. The degraded supports represented a condition adverse to quality and should have been identified and placed in the licensees CAP. The finding is greater than minor because if left uncorrected the degradation would have led to a more significant safety concern. The finding is associated with the mitigating systems cornerstone. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance since it only affected the mitigating systems cornerstone and did not represent a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event.

The cause of the finding is also related to the crosscutting aspect of problem identification and resolution with a corrective action program causal factor because the threshold for identifying issues was not sufficiently low and the degraded supports were not identified completely, accurately, and in a timely manner commensurate with their safety significance (P.1.(a)).

Enforcement.

10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Actions," requires, in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and nonconformances are promptly identified and corrected. Contrary to the above, since April 19, 2006, the licensee failed to identify and correct a condition adverse to quality. Specifically, inservice inspection personnel failed to identify that SP pipe Supports 13-SP-030-H-007 and 13-SP-030-H-008 were degraded in the essential pipe density tunnel and place them in the CAP. The licensee has initiated PVAR 2989960 to address the failure to promptly identify and correct the degraded supports. Because the finding is of very low safety significance and has been entered into the CAP as PVAR 2989960, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy: NCV 05000530/2007003-02, "Failure to Promptly Identify Degraded Structural Supports."

1R19 Postmaintenance Testing

a. Inspection Scope

The inspectors selected the six below listed postmaintenance test activities of risk significant systems or components. For each item, the inspectors:

(1) reviewed the applicable licensing basis and/or design-basis documents to determine the safety functions;

(2) evaluated the safety functions that may have been affected by the maintenance activity; and

(3) reviewed the test procedure to ensure it adequately tested the safety function that may have been affected. The inspectors either witnessed or reviewed test data to verify that acceptance criteria were met, plant impacts were evaluated, test equipment was calibrated, procedures were followed, jumpers were properly controlled, the test data results were complete and accurate, the test equipment was removed, the system was properly re-aligned, and deficiencies during testing were documented. The inspectors also reviewed the CAP to determine if the licensee identified and corrected problems related to postmaintenance testing.

• April 11 - 21, 2007, Unit 2, supplemental nitrogen for post station blackout operation of atmospheric dump valves per WO 2952660

• May 1, 2007, Unit 1, spent fuel handling machine upgrade per design modification WO 2778582

• May 2, 2007, Unit 2, furmanite repair to steam generator hot leg blowdown isolation valve per WO 3005217

• May 3, 2007, Unit 3, lube, inspect, and stroke safety injection Valves SIA-HV-684 and SIA-HV-687 per WOs 2892943 and 2892934

• May 3, 2007, Unit 3, containment spray surveillance following preventative maintenance oil change per WO 2956878

• May 7 - 8 2007, Unit 3, agastat relay testing and Valve AFC-HV-033 lube and stroke per WOs 2876221 and 2991657 Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed six samples.

b. Findings

No findings of significance were identified.

1R20 Refueling and Other Outage Activities

a. Inspection Scope

Unit 3 Short Notice Outage for Turbine Oil Booster Pump Failure The inspectors reviewed the following risk significant outage activities to verify defense in depth commensurate with the outage risk control plan, compliance with the Technical Specifications, and adherence to commitments in response to Generic Letter 88-17, "Loss of Decay Heat Removal:"

(1) the risk control plan;

(2) electrical power;

(3) decay heat removal;

(4) reactivity control;

(5) containment closure;

(6) licensee identification and implementation of appropriate corrective actions associated with outage activities.

The inspectors observed the reactor startup after the short notice outage.

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

1R22 Surveillance Testing

a. Inspection Scope

The inspectors reviewed the UFSAR, procedure requirements, and Technical Specifications to ensure that the seven below listed surveillance activities demonstrated that the SSCs tested were capable of performing their intended safety functions. The inspectors either witnessed or reviewed test data to verify that the following significant surveillance test attributes were adequate:

(1) preconditioning;

(2) evaluation of testing impact on the plant;

(3) acceptance criteria;

(4) test equipment;

(5) procedures;

(6) jumper/lifted lead controls;

(7) test data;

(8) testing frequency and method to demonstrate Technical Specification operability;

(9) test equipment removal;

(10) restoration of plant systems;

(11) fulfillment of ASME Code requirements;

(12) updating of performance indicator data;

(13) engineering evaluations, root causes, and bases for SSCs not meeting the test acceptance criteria were correct;

(14) reference setting data; and

(15) annunciators and alarms setpoints. The inspectors also verified that the licensee identified and implemented corrective actions associated with the surveillance testing.

• April 3, 2007, Unit 1, Procedure 73ST-9ZZ18, "Main Steam and Pressurizer Safety Valve Set Pressure Verification," Revision 19

• April 11, 2007, Unit 1, inservice test of low pressure safety injection pump Train A per Procedure 73ST-9SI11, "Low Pressure Safety Injection Miniflow - Inservice Test," Revision 19

• April 12, 2007, Unit 1, inservice test of turbine driven AFW pump per Procedure 73ST-9AF02, "AFA-P01 Inservice Test," Revision 37

• April 19, 2007, Unit 1, inservice test of diesel fuel oil transfer pump Train B per Procedure 73ST-9DF01, "Diesel Fuel Oil Transfer Pump - Inservice Test,"

Revision 15

• April 26, 2007, Unit 3, Procedure 73ST-9SI05, "Leak Test of HPSI/LPSI Containment Isolation Check Valves," Revision 16

• June 11, 2007, Unit 1, local leak rate testing of containment Penetration 40 per Section 8.18 of Procedure 73ST-9CL01, "Containment Leakage Type 'B' and 'C' Testing," Revision 30

• June 14, 2007, Unit 1, local leak rate test per Procedure 73ST-9CL06, "Containment Purge Supply Leak Test (42") Penetration 56," Revision 17 Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed seven samples.

b. Findings

No findings of significance were identified.

1R23 Temporary Plant Modifications

a. Inspection Scope

The inspectors reviewed the UFSAR, plant drawings, procedure requirements, and Technical Specifications to ensure that the below listed temporary modification was properly implemented. The inspectors:

(1) verified that the modifications did not have an effect on system operability/availability;

(2) verified that the installation was consistent with modification documents;

(3) ensured that the post-installation test results were satisfactory and that the impact of the temporary modifications on permanently installed SSCs were supported by the test;

(4) verified that the modifications were identified on control room drawings and that appropriate identification tags were placed on the affected drawings; and

(5) verified that appropriate safety evaluations were completed.

The inspectors verified that the licensee identified and implemented corrective actions associated with temporary modifications.

• June 26-28, 2007, Unit 1, Temporary Modification 3024963 to install vibration monitors on main steam lines Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed one sample.

b. Findings

No findings of significance were identified.

4OA1 Performance Indicator (PI) Verification

Cornerstone: Mitigating Systems

The inspectors sampled licensee data for the Mitigating System Performance Indicators (MSPIs) listed below for the period from April 1, 2006 through March 31, 2007. The definitions and guidance of Nuclear Energy Institute 99-02, "Regulatory Assessment Indicator Guideline," Revision 4, were used to verify the licensees basis for reporting unavailability and unreliability in order to verify the accuracy of PI data. The inspectors reviewed operating logs, Limiting Condition of Operation logs, CRDRs, and the maintenance rule database to verify that the licensee properly accounted for planned and unplanned unavailability as part of the assessment. The inspectors sampled data to

verify that the licensee:

(1) accurately documented the actual unavailability hours for the MSPI systems; and

(2) accurately documented the actual unreliability information for each MSPI monitored component. In addition, the inspectors interviewed licensee personnel associated with PI data collection and evaluation.

• Mitigating System Performance Index - Safety System Functional Failures

• Mitigating System Performance Index - Emergency AC Power Systems

• Mitigating System Performance Index - High Pressure Injection Systems The inspectors completed three samples.

Documents reviewed by the inspectors are listed in the attachment.

b. Findings

No findings of significance were identified.

4OA2 Identification and Resolution of Problems

.1 Routine Review of Identification and Resolution of Problems

The inspectors performed a daily screening of items entered into the licensee's CAP.

This assessment was accomplished by reviewing daily summary reports for PVARs and CRDRs, and attending corrective action review and work control meetings. The inspectors:

(1) verified that equipment, human performance, and program issues were being identified by the licensee at an appropriate threshold and that the issues were entered into the CAP;

(2) verified that corrective actions were commensurate with the significance of the issue; and

(3) identified conditions that might warrant additional follow-up through other baseline inspection procedures.

.2 Selected Issue Follow-up Inspection

a. Inspection Scope

In addition to the routine review, the inspectors selected the four below listed issues for a more in-depth review. The inspectors considered the following during the review of the licensee's actions:

(1) complete and accurate identification of the problem in a timely manner;

(2) evaluation and disposition of operability/reportability issues;

(3) consideration of extent of condition, generic implications, common cause, and previous occurrences;

(4) classification and prioritization of the resolution of the problem;

(5) identification of root and contributing causes of the problem;

(6) identification of corrective actions; and

(7) completion of corrective actions in a timely manner.

C April 24, 2007, Unit 2, aborted reactor startup as described in Significant CRDR 2976449 C April 9 - 20, 2007, Units 1, 2, and 3, main steam and feedwater isolation valve accumulator design and licensing basis as noted in CRDRs 2992315, 2992380 and 2994734

C May 14 - 25, 2007, Units 1, 2, and 3, Palo Verde Setpoint and Out of Tolerance Program including review of identified deficiencies as described in CRDRs 2831585, 2833612, 2908360, and 2916795 C May 28, 2007, Unit 2, failure of closed light indication on steam generator bypass valve as noted in PVAR 3019266 Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed four samples.

b. Findings

Introduction.

The inspectors identified an additional example of the Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Actions," described in NRC Component Design Basis Inspection Report 05000528; 05000529; 05000530/2007011, for the failure to promptly identify and correct significant conditions adverse to quality for failures of Target Rock solenoid-operated valves.

Description.

On May 28, 2007, steam admission bypass solenoid-operated Valve 2JSGAUV0138A failed to give a green closed indicating light when demanded to close.

This same valve also experienced failures on April 3, 2006, March 5, 2007, March 12, 2007, April 11, 2007, and April 15, 2007. Engineering personnel indicated that the failure modes were all symptoms of the same cause, namely valve leakage due to bad design.

In response to the April 11, 2007 failure, the valve was replaced with a single pilot design. Reed switch settings for valve position indication were the focus of the failures on April 15, 2007, and May 28, 2007. After the May 28, 2007 failure, the vendor recommended different reed switch settings, so the valve settings were changed accordingly.

The licensee planned to replace all dual pilot steam admission valves with single pilot valves to reduce the possible failure modes. Fewer failures were expected with single pilot valves and the reed switch setting adjustment per the latest vendor recommendations. In the long term, the licensee planned to replace the AFW pump turbine governors with digital governors. This design modification will eliminate the need for the steam admission bypass valves.

Analysis.

Failure to identify and correct significant conditions adverse to quality involving the Target Rock solenoid-operated valves was a performance deficiency. The finding is greater than minor because it is associated with the equipment performance cornerstone attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the reliability and availability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance because there was no actual loss of safety function to the pump. This finding had crosscutting aspects associated with corrective action of the problem identification and resolution area to ensure that issues potentially impacting nuclear safety are promptly identified, fully evaluated, and that actions are taken to address safety issues in a timely manner (P.1(c)).

Enforcement.

10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Actions," states, in part, that measures shall be established to assure that conditions adverse to quality are promptly identified and corrected. In the case of significant conditions adverse to quality, the measures shall assure that the cause of the conditions is determined and corrective action taken to preclude repetition. Contrary to the above, since April 3, 2006, the licensee did not correct a condition described in corrective action documents as being a significant condition adverse to quality associated with the failures of Target Rock solenoid-operated valves sufficiently to preclude repetition. Because the finding is of very low safety significance and has been entered into the CAP as PVAR 3019266, this violation is being treated as an NCV. This violation represents an additional example of NCV 05000528; 05000529;05000530/2007011-04, "Inadequate Corrective Actions for Target Rock Solenoid-Operated Valves," documented in NRC Inspection Report 05000528; 05000529; 05000530/2007011. Since the corrective actions for the earlier NCV is expected to correct this violation, this example will not be cited separately.

.3 Semiannual Trend Review of Operator Overtime

a. Inspection Scope

The inspectors completed a semi-annual trend review of repetitive or closely related issues that were documented in corrective action documents and monthly trend reports, to identify trends that might indicate the existence of more safety significant issues.

Documents reviewed by the inspectors are listed in the attachment.

• A review of operator overtime was conducted to ensure compliance with Technical Specifications and Procedure 01DP-9EM01, "Overtime Limits,"

Revision 4

b. Findings

No findings of significance were identified.

.4 Annual Sample: Review of Operator Workaround Program

a. Inspection Scope

The inspectors conducted a cumulative review of operator workarounds for Units 1, 2, and 3 and assessed the effectiveness of the operator workaround program to verify that the licensee is:

(1) identifying operator workaround problems at an appropriate threshold;

(2) and entering them into the CAP; and

(3) identifying and implementing appropriate corrective actions. The review included walkdowns of the control room panels, interviews with licensed operators and reviews of the control room discrepancies list, the lit annunciators list, the operator workaround list, the operator burdens list, and the operator challenges tracking system.

b. Findings

No finding of significance were identified.

.5 Multiple/Repetitive Degraded Cornerstone Column and Crosscutting Issues Follow-up

Activities In the NRCs Annual Assessment Letter of Palo Verde dated March 2, 2007, the NRC indicated that improvement efforts in addressing the substantive crosscutting issues through baseline inspections would be monitored, including a detailed assessment following the licensees notification of readiness for closure verification. In a Confirmatory Action Letter dated June 21, 2007, the NRC revised this to indicate the intent to address the substantive crosscutting issues within the Inspection Procedure 95003 supplemental inspection and followup process, since the issues are integral to the performance deficiencies being addressed by your staff.

The inspectors and Region IV personnel conducted weekly teleconferences and conducted periodic discussions with licensee management to monitor their progress in addressing their performance deficiencies and substantive crosscutting issues.

Two public meetings were conducted during this inspection period. On June 6, 2007, a public meeting was held with PVNGS to discuss the status of their assessment and improvement efforts to address plant performance issues that contributed to entering Column 4 of the NRC Action Matrix. On June 7, 2007, a town hall meeting was held with members of the public to answer questions and hear comments. The June 6 and June 7 meeting summaries can be found in ADAMS under ML071740010 and ML071740009, respectively.

During the week of June 18, 2007, five inspectors from the Inspection Procedure 95003 supplemental inspection team were onsite to continue to evaluate the licensees third party assessment of its safety culture. The objective of this inspection was to gain an understanding of how the assessment was conducted and to determine whether it provided independent, comprehensive, valid, and reliable insights into safety culture at the site. The team reviewed the licensees survey tool and assessment methodology. In addition, the inspectors reviewed the "Palo Verde Nuclear Generating Station Independent Safety Culture Assessment Preliminary Phase 1 Results Report," dated June 18, 2007, supporting statistical analysis, and survey write-in comments. Results from the licensees 2005 Safety Culture Assessment, and preliminary results from the Independent Safety Culture Performance Evaluation were also reviewed. The team conducted interviews with licensee management and independent safety culture assessment personnel. The inspection is still ongoing, and the results of the inspection will be documented in NRC Inspection Report 05000528; 05000529; 05000530/2007012.

No findings of significance were identified during this preliminary review.

.6 Cross-References to Problem Identification and Resolution Findings Documented

Elsewhere Section 1R15 describes a finding where inservice inspection personnel had an inappropriately high threshold for recognizing degraded and nonconforming conditions.

Section 4OA2.2 describes an additional example of a previous violation where the licensee failed to promptly identify, fully evaluate, and take corrective actions to address safety issues in a timely manner.

Section 40A3.1 under the Personal Performance Review describes a finding where engineering personnel failed to incorporate operating experience to evaluate preventive maintenance activities for equipment in the maintenance rule.

4OA3 Follow-up of Events and Notices of Enforcement Discretion

Event Report Reviews

a. Inspection Scope

The inspectors reviewed the below listed Licensee Event Reports (LERs) and related documents to assess:

(1) the accuracy of the LER;

(2) the appropriateness of corrective actions;

(3) violations of requirements; and

(4) generic issues.

.1 (Closed) LER 05000528/2006005-00, "Technical Specification Prohibited Condition Due

to Check Valve Not Seated" The failure of safety injection check Valve 1PSIEV134 to fully seat was previously discussed and dispositioned in Section 4OA2.2 of NRC Inspection Report 05000528; 05000529; 05000530/2006005. NCV 05000529/2006005-05 was issued for the failure to adequately evaluate the same condition on a similar valve that had occurred in 2000.

Failure to perform an adequate evaluation of the previous valve failure precluded appropriate corrective actions to prevent the failure of Valve 1PSIEV134. The inspectors reviewed the LER and identified no additional concerns. This LER is closed.

.2 (Closed) LER 05000528/2006006-00, "Reactor Trip due to Core Protection Calculator

Generated Low Departure From Nucleate Boiling Ratio (DNBR) Trip Signal" On October 21, 2006, Unit 1 automatically tripped due to a core protection calculator generated low DNBR ratio trip. The cause of the reactor trip was fluctuation in the control element Assembly (CEA) 29 position indication signal sent from reed switch position Transmitter A to control element assembly Calculator (CEAC) 1, CEA position indication is transmitted by the CEACs to the core protection calculator as a penalty factor in order to prevent departure from nucleate boiling or exceeding a local power density limit due to CEAs being out of position. Recognizing that the CEA 29 position fluctuations were not actual position changes of the CEA, operators were attempting to remove CEAC 1 from service when the automatic reactor trip occurred. Following the reactor trip, maintenance personnel determined the cause of the CEA 29 position fluctuations was due to excessive circuit resistence at a cable connector. The inspectors reviewed this LER and no findings of significance were identified and no violations of NRC requirements occurred. This LER is closed.

.3 (Closed) LER 05000528/2006007-00, "Emergency Diesel Generator Actuation on Loss of

Power to B Train 4.16 kV Buses in Units 1 and 3" On October 26, 2006, a valid actuation of both the Unit 1 EDG Train B and Unit 3 EDG Train B occurred as a result of undervoltage on their respective safety buses. The loss of power to the two safety buses was the result of an apparent protective relay actuation of startup Transformer X01 output breakers to Unit 1 Bus NAN-S06 and Unit 3 Bus NAN-S06, which supply their respective unit Train B safety buses. The output breakers opened as a result of auxiliary Relay 552X actuating due to vibration when closing the cubicle door following maintenance activities. The licensee planned to redesign the circuitry associated with the auxiliary relay to eliminate the vulnerability. In the interim, the licensee has posted a warning sign on the susceptible breaker cubicle doors containing this relay to alert personnel of the potential to de-energize the safety buses when closing the doors. The inspectors reviewed this LER and no findings of significance were identified and no violations of NRC requirements occurred. This LER is closed.

.4 (Closed) LER 05000530/2006007-00, "Manual Reactor Trip Due to Degrading

Condenser Vacuum and Condensate Flow" On October 19, 2006, operators initiated a manual reactor trip in response to lowering condenser hotwell level which had caused two of the three operating condensate pumps to trip coincident with degrading condenser vacuum. The cause of the hotwell level and condenser vacuum reduction was determined to be a failed-open condition of air-operated Valve 3JSCNUV0232 on the condensate demineralizer drain header. The failure of this valve created an opening from the main condenser to atmosphere. The licensee instituted procedural controls to require an in-line manual valve to be closed except during specific demineralizer rinsing operations in order to prevent this event from reoccurring. The inspectors reviewed this LER and identified a self-revealing NCV of 10 CFR 50.65(b)(2)(iii). A finding associated with this event is discussed in Section 1R12 of this report. This LER is closed.

.5 (Closed) LER 05000528/2005009-00, "Technical Specification Violation For Operation

Without Both Reactor Coolant System Loops Operable" Following refueling Outage 1R12 in which the steam generators were replaced, the licensee observed that the measured vibration on shutdown cooling (SDC) suction isolation Valve 1JSIAUV0651 increased significantly from previous cycles. This condition caused the licensee to limit power to approximately 25 percent to maintain vibrations within analyzed limits. The vibration increase was believed to be due to a hydraulic coupling between the reactor coolant system (RCS) hot leg flow and the fundamental acoustic mode on the isolated SDC suction line, aggravated by the increased RCS flow rate following steam generator replacement. On March 18, 2006, Unit 1 was in Mode 3 (Hot Standby) at normal operating pressure and temperature to support data collection for evaluation of this vibration condition. To assess the impact of higher RCS Loop 1 hot leg flow, the licensee secured reactor coolant pump (RCP) 2A in Loop 2. After securing RCP 2A, flow increased in Loop 1 as expected, due to reverse flow in the cold leg of the secured RCP. Observed vibration levels increased to a maximum of 3.05 inches per second (ips) on Valve 1JSIAUV0651, which was beyond the 3.0 ips analyzed limit for

long-term operation. Operations personnel restarted RCP 2A after about one minute, and vibration levels returned to previous levels, about 1.3 ips. This LER was written to address the potential concern that a loss of either RCP 2A or 2B would have increased vibration levels beyond analyzed limits following the twelfth refueling outage. The licensee's analysis concluded that since the plant experienced these vibration levels for only 1 minute, it did not overstress plant components. The licensee permanently relocated Valve 1JSIAUV0651, which changed the fundamental acoustic mode of the SDC line, and vibration levels decreased to less than 0.3 ips.

The licensee concluded that exceeding the analyzed vibration limit caused both RCS loops to be inoperable. The failure to maintain both RCS loops operable was a performance deficiency. The finding is greater than minor because it is associated with the RCS equipment performance attribute of the barrier integrity cornerstone and affects the cornerstone objective to provide reasonable assurance that physical design barriers protect the public from radionuclide releases caused by accidents or events. Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet required a Phase 3 analysis since the finding impacts the RCS barrier integrity cornerstone. Manual Chapter 0609, Appendix M, "Significance Determination Process Using Qualitative Criteria," was used since the significance determination process methods and tools were not adequate to determine the significance of the finding within the established timeliness goals. The finding is determined to have very low safety significance through management review for the following reasons:

(1) Dedicated vibration monitoring was available and updated every 10 minutes, therefore the high vibration condition would have been promptly detected. The plant staff was acutely aware of the vibration issue and therefore it would have received focused attention.

(2) Actions to mitigate the vibration levels would have been simple and straightforward, consisting of a cooldown following the resulting plant trip, or the tripping of an additional RCP.

(3) The licensee's analysis concluded that vibration levels would not have challenged pipe integrity for at least 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br />. Recognizing conservatisms in the calculation, it is likely that the piping could have sustained the resulting stresses for a much longer period of time. And finally,

(4) in the unlikely event that the vibration levels were allowed to persist for a long period of time, an ensuing small-break loss of coolant accident would have a conditional core damage probability of approximately 1.0E-3, given long-term 100 percent power history. Because the plant was being operated at only 25 percent power, the actual incremental conditional core damage probability would have been considerably lower.

This licensee identified finding involved a violation of Technical Specification 3.4.4. The enforcement aspects of the violation are discussed in Section 4OA7. This LER is closed.

.6 (Closed) LER 05000530/2006005-01, "Manual Reactor Trip Due To Loss Of Main

Feedwater" This LER is a supplement to LER 05000530/2006005-00, which was closed in NRC Inspection Report 05000528; 05000529; 05000530/2006004. This supplement provided the root cause of the demineralizer sight glass rupture, which caused the operators to manually trip the reactor. The inspectors reviewed this LER and no additional findings were identified. This LER is closed.

Personnel Performance

a. Inspection Scope

The inspectors:

(1) reviewed operator logs, plant computer data, and/or strip charts for the below listed evolutions to evaluate operator performance in coping with non-routine events and transients;

(2) verified that operator actions were in accordance with the response required by plant procedures and training; and

(3) verified that the licensee identified and implemented appropriate corrective actions associated with personnel performance problems that occurred during the non-routine evolutions sampled.

• On March 18, 2007, Unit 2, alarms were received associated with a loss of feedwater pump Turbine (FWPT) A speed control while operating at full power.

Operators observed local high pressure control valve oscillations. Based on the observations, operations personnel tripped the FWPT which resulted in a reactor power cutback. Operations personnel entered Procedure 40AO-9ZZ09, "Reactor Power Cutback (Loss of Feedpump)," Revision 19, and stabilized the plant at approximately 48 percent power. This event was documented in CRDR 2984713.

• On May 3, 2007, refueling and radiation protection personnel were performing an evolution to remove a new rod storage basket (RSB) that was empty of fuel pins from the Unit 2 SFP. While lifting the RSB lid out of the water to attach rigging equipment, radiation protection personnel measured contact radiation readings of 5 mR per hour. These readings were unexpected since the RSB was supposed to be new. The spent fuel handling machine (SFHM) operator and the task leader then recognized they were in the wrong location. Refueling personnel replaced the lid onto the RSB and proceeded to the correct location to continue the evolution. The licensee's investigation of the event determined that refueling and radiation protection personnel's failure to immediately notify the control room of the human performance error was not in accordance with the explicit direction given by the shift manager at the sensitive issues brief and the expectations of the Palo Verde Standards and Expectations Prevent Events Handbook, Revision 2. This event was documented in CRDR 3011825.

• On June 19, 2007, during the Unit 1 core reload, the shift manager was informed that there was a possibility that the refueling machine hoist underload and overload protections were not functioning properly. This was based upon the receipt of a load error message on June 18, 2007, when a fuel assembly lower end fitting came in contact with the top of an adjacent assembly in the core. The load error message alerts the refueling machine operator when the load on the hoist is less than 100 lbs. The refueling machine was declared inoperable and core reload was suspended per Technical Requirements Manual Limiting Condition for Operability (TLCO) 3.9.102. This event was documented in PVAR 3029781.

Documents reviewed by the inspectors are listed in the attachment.

The inspectors completed three samples.

.1 Loss of MFW Pump Due to Equipment Failure

Introduction.

A self-revealing Green NCV of 10 CFR 50.65(a)(3) was identified for failure of the licensee to incorporate internal and external industry operating experience into preventative maintenance activities that could have prevented a maintenance rule functional failure of FWPT A, a high risk heat removal system. Specifically, the licensee did not incorporate available operating experience into preventative maintenance instructions to inspect, clean and verify acceptable equipment condition for the linear variable differential transmitter (LVDT) linkage assembly. Failure to inspect and clean the LVDT linkage assembly resulted in a broken linkage due to binding, causing erratic cycling of the FWPT control valves resulting in a manual trip of feedwater Pump A and reactor power cutback to 48 percent power.

Description.

On March 18, 2007, the Unit 2 FWPT A operating cylinder LVDT threaded rod end bearing failed. The LVDT provided position indication of the FWPT low pressure and high pressure steam control valves to the FWPT control circuit. The cause of the shearing was attributed to significant binding of the LVDT armature within the bore of the LVDT coil housing due to the build up of a gummy black magnetic deposit. The gummy black deposit substance was identified as the accumulation of grease, oil, and debris.

The licensee concluded that the LVDTs were mistakenly greased four years earlier when the slide block for the limit switches was greased per a preventive maintenance task.

When identified, the visible grease was removed, but no internal cleaning occurred. No corrective action documents were generated, since the condition was not considered significant.

The inspectors conducted an industry operating experience search which indicate there was a considerable history of nearly identical failures as early as 1985. The predominant corrective action was to conduct periodic inspection for ease of armature rod movement within the LVDT windings. The licensee had identified that inspection, cleaning, and replacement tasks were necessary to prevent failure of the LVDT. However, the preventative maintenance tasks were not developed. Review of the Reliability Centered Maintenance (RCM) evaluation indicates that there was sufficient information to identify and manage the single point vulnerabilities inherent in the FWPT control system. The RCM evaluation for the FWPT first performed in February 2002 provided recommendations to replace the LVDT every 20 years and inspect the LVDT linkage and to check for binding. Additionally, the LVDT failure in Unit 2 on July 26, 2006, resulted in a narrowly focused action to inspect only the main turbine control valve LVDT linkages in all three units. These inspections were not completed by March 18, 2007. The failure of the LVDT to move freely, ultimately resulting in its linkage failing due to overload on March 18, 2007, was preventable.

Analysis.

The inspectors determined that the failure to incorporate internal and external industry operating experience into preventative maintenance activities that could have prevented a maintenance rule functional failure of the feedwater Pump A and the resulting plant transient, a high risk heat removal system, was a performance deficiency.

The finding is greater than minor because it is associated with the initiating events

cornerstone attribute of equipment performance and affects the cornerstone objective to limit the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown and power operations. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheet, the finding is determined to have very low safety significance because the finding did not result in exceeding the Technical Specification limit for identified RCS leakage and did not affect other mitigation systems; the finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions will not be available; and the finding did not increase the likelihood of a fire or internal/external flood. This finding has a crosscutting aspect in the area of problem identification and resolution, associated with operating experience, since engineering personnel failed to account for prior operating experience in determining the maintenance rule scope and appropriate preventive maintenance for Valve 3JSCNUV0232 (P.2(b))

Enforcement.

10 CFR 50.65(a)(3) states, in part, that preventative maintenance activities shall be evaluated to take into account, where practical, industry-wide operating experience. Contrary to this requirement, between February 2002 and March 18, 2007, internal and external industry operating experience pertaining to the inspection and cleaning of the LVDT was determined to be applicable but had not been incorporated into preventative maintenance activities for the inspection of the FWPT A operating cylinder LVDT. As a result, a failure of the pump occurred, causing a reactor power cutback to 48 percent power. Because the finding is of very low safety significance and has been entered into the CAP as CRDR 2984713, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy: NCV 05000528; 05000529;05000530/2007003-03, "Failure to Apply Industry Operating Experience to Maintenance Activities Results in a Plant Transient."

.2 Failure of the Underload Interlock Resulting in Load Error Conditions

Introduction.

The inspectors identified a Green NCV of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure of operations and refueling personnel to follow Procedure 40DP-9OP02, "Conduct of Shift Operations," Revision 37, when a load error condition occurred during core reloading. Specifically, operations and refueling personnel failed to recognize that the load error condition was a result of a degraded refueling machine control system and could have resulted in fuel damage, a condition that required an Event Recovery Checklist (ERC). This event, along with another event that occurred in the SFP on May 3, 2007, that involved human performance errors by refueling personnel, corrective actions associated with past fuel handling problems may not have been completely affective.

Description.

On June 18, 2007, core reloading was in progress during refueling Outage 1R13. The refuelng machine had been extensively modified following core offloading. Refueling personnel observed that the refueling machine was not operating as expected, in that, the off-index hoisting capability was functioning intermittently. The intermittent loss of off-index hoisting capability was discussed, and the licensee determined that refueling machine operability was not impacted and core reload could continue. The basis for this decision was that the off-index hoisting feature was an enhancement and on-index hoisting capability was still available. The off-index hoisting feature allows the fuel assembly to be lowered as the refueling machine approaches the core location from adjacent areas. This reduces the probability that a fuel assembly will

make contact with an adjacent assembly causing interference when lowering. On-index moves only allow the fuel assembly to be lowered when it is directly above the designated core location. The licensee initially failed to recognize the intermittent problem with the off-index feature as a potentially degraded condition, and enter it into the CAP per Procedure 90DP-0IP10, "Condition Reporting," Revision 32.

On June 18, 2007, at approximately 1900 hours0.022 days <br />0.528 hours <br />0.00314 weeks <br />7.2295e-4 months <br />, while refueling personnel were lowering a fuel assembly on-index, the hoist stopped at an unexpected elevation indicating that an assembly lower end fitting may have come in contact with the top of an adjacent assembly in the core. The limited senior reactor operator (LSRO) recognized that this was not an unusual condition when lowering burned fuel assemblies on-index. The LSRO attempted to correct the condition by raising the fuel assembly, engaging the spreader, then relowering the assembly, however, the fuel assembly could not be raised.

The Program and Remote Nuclear Inc. (PAR) representative on the refueling machine was consulted regarding why the fuel assembly could not be raised. The PAR representative informed the LSRO that the load bypass, associated with lowering the hoist box to the downstop, was still active allowing the fuel assembly to lower beyond the underload limits resulting in a load error condition. Further, the PAR representative informed the LSRO that the interlock bypass key would need to be used to raise the assembly since a load error message had been received. The load error message alerts the refueling machine operator when the load on the hoist is less than 100 lbs. The LSRO questioned the use of the interlock bypass key since the underload protection should have stopped hoist movement when load on the hoist was less than 1350 lbs, prior to receiving the load error message. However, the PAR representative stated that the refueling machine was operating normally. Due to knowledge deficiencies, a lack of questioning attitude, and incorrect information provided from the PAR representative, the LSRO incorrectly concluded that the load bypass allowed the hoist to be lowered beyond the underload limits. The LSRO incorrectly assumed the new refueling machine must have been designed with a larger range of the downstop bypass than the old machine.

The LSRO also stated to the PAR representative that he believed that the refueling machine should not have been designed to allow the condition observed. The LSRO contacted the shift manager to get permission to use the interlock bypass key to raise the assembly and engage the spreader, despite believing that there may be a design error with the refueling machine. Due to a lack of understanding of the issue, the shift manager gave permission to continue core reload.

Core reload continued for approximately 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and an additional 28 fuel assemblies were loaded, with the machine not operating as expected. On June 19, at 0113 hours0.00131 days <br />0.0314 hours <br />1.868386e-4 weeks <br />4.29965e-5 months <br />, a fuel services section leader became aware of the event that occurred with the load error condition, and informed the shift manager that there was a possibility that the refueling machine hoist underload and overload protection was not functioning properly. The refueling machine was declared inoperable and core reload was suspended per TLCO 3.9.102. Troubleshooting determined that a downlatch limit switch was operating intermittently. The limit switch is associated with input to the software logic that installs the underload and overload protection, and the permissive to enable the off-index feature. The equipment was repaired and retested. Inspections were also performed on all potentially impacted fuel assemblies. No structural damage was identified.

The inspectors determined that the failure of the licensee to identify the relationship between the loss of off-index capability and the loss of underload and overload protection was a missed opportunity. Identification of the degraded condition at the time that refueling personnel observed that the refueling machine was not operating as expected, and performing a functional assessment, could have resolved the degraded refueling machine condition prior to the subsequent event that occurred on June 18. Following the fuel handling event, Procedure 40DP-9OP02, "Conduct of Shift Operations," Revision 37, required performing an ERC to ensure that all potential impacts of the identified deficiency on the safety of the fuel were identified and resolved prior to continuing with movement of the affected fuel. The licensee recognized on June 19 during the following shift that an ERC should have been performed. However, only portions of the ERC were completed at that time, which resulted in more missed opportunities to fully understand the significance of the issues associated with the event. Only after questioning from the inspectors on June 25 for details regarding the event did the licensee complete the actions required by the ERC and initiate actions to identify and evaluate issues associated with the human performance and problem identification and resolution crosscutting aspects of the event.

Similar decision making problems occurred on May 3, 2007, when refueling personnel prepared to remove a new RSB that was empty from the Unit 2 SFP. A sensitive issues brief was held prior to the commencement of the evolution where the operations shift manager expressed his expectations regarding notification of the control room if an off-normal event occurred. The refueling task leader controlled the evolution and had possession of the work documents, which included material balance area (MBA) Form 2-14-2 to identify the SFP coordinates for the RSB location. The MBA form identified grid Location GG-05 as the location of the new RSB.

Without either an MBA short form to follow or further communications with the task leader who had the MBA sheet, the SFHM operator located himself over an incorrect RSB at Location CC-05. A refueling trainee verified the location by confirming that there was a RSB in Location CC-05. Refueling personnel proceeded to remove the RSB lid from the SFP to attach the necessary rigging to lift the RSB. The lid was manually lifted out of the water and radiation protection personnel measured contact radiation readings of 5 mR/hour. These readings were unexpected since the RSB was supposed to be new. The SFHM operator and the task leader then discussed the grid location of the new RSB and recognized they were in the wrong location. Refueling personnel replaced the lid onto the RSB at Location CC-05. Neither the control room nor refueling supervision were notified of the event prior to continuing with the evolution. Failure to stop and notify the control room about the human performance event was not in accordance with the explicit direction given by the shift manager at the sensitive issues brief and the "stop expectation" of the Palo Verde Standards and Expectations Prevent Events Handbook, Revision 2.

Similar performance deficiencies have also been identified by the NRC.

NCV 05000528/2004003-04 described an occurrence where operators failed to recognize the need to perform the ERC. NCV 05000529/2005003-03 described three examples associated with the following performance deficiencies:

(1) failure to complete a functional retest following maintenance on the SFHM;

(2) failure to ensure that spent fuel was in a safe condition, stop fuel handling operations, or contact the shift manager to

determine the need to complete an ERC when a deficiency was identified with fuel handling equipment as required by Procedure 40DP-9OP02, "Conduct of Shift Operations;" and

(3) failure to ensure the MBA short form was present on the SFHM to perform proper independent verification or verify that the bridge and trolley were over the correct fuel assembly as required by Procedure 78OP-9FX03, "Spent Fuel Handling Machine." Crosscutting aspects were also identified in the areas of human performance and problem identification and resolution that were similar to the aspects associated with this finding.

Analysis.

The performance deficiency associated with this finding was the failure to perform an ERC when fuel was potentially damaged due to a degraded refueling machine. The finding is greater than minor because it would become a more significant safety concern if left uncorrected in that handling fuel with a degraded refueling machine could result in fuel barrier damage. This finding cannot be evaluated by the significance determination process because Manual Chapter 0609, "Significance Determination Process," Appendix A, "Significance Determination of Reactor Inspection Findings for At-Power Situations," and Appendix G, "Shutdown Operations Significance Determination Process," do not apply to the refueling cavity for the plant conditions that existed during the event. This finding affects the barrier integrity cornerstone and is determined to be of very low safety significance by NRC management review because it was a deficiency that did not result in the actual degradation of spent fuel. The dominant crosscutting aspect for this finding was in the area of human performance associated with decision-making because operations and refueling personnel did not make safety significant decisions using a systematic process, when faced with uncertain or unexpected equipment performance, to ensure safety is maintained (H.1(a)).

Enforcement.

10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," requires, in part, that activities affecting quality shall be prescribed by procedures, and that the activities shall be accomplished in accordance with these procedures. Procedure 40DP-9OP02, "Conduct of Shift Operations," Revision 37, required the performance of an ERC following a fuel handling event to ensure that all potential impacts of the identified deficiency on the safety of the fuel were identified and resolved prior to continuing with movement of the affected fuel. Contrary to the above, on June 18, 2007, operations and refueling personnel failed to perform an ERC as required by Procedure 40DP-9OP02, when a fuel handling event occurred during core reload. Specifically, a fuel assembly came in contact with the top of an adjacent fuel assembly in the core. Operations and refueling personnel failed to recognize that the indications were the result of a degraded refueling machine and could have resulted in fuel damage. Consequently, core reload continued with the refueling machine in a degraded condition to load an additional 28 fuel assemblies. Fuel movement stopped once the degraded condition was recognized, however, an ERC was still not performed until the following shift. Inspections were performed for the potentially affected fuel assemblies and no structural damage was identified. Because the finding is of very low safety significance and has been entered into the CAP as PVAR 3029781, this violation is being treated as an NCV consistent with Section VI.A of the Enforcement Policy:

NCV 05000528/2007003-04, "Failure to Perform Event Recovery Checklist."

4OA5 Other Activities

(Open) TI 2515/166, Pressurized Water Reactor Containment Sump Blockage, Unit 1

a. Inspection Scope

The inspector reviewed the licensees implementation of Unit 1 plant modifications and procedure changes committed to in the licensees response to Generic Letter 2004-02, Potential Impact of Debris Blockage on Emergency Recirculation During Design Basis Accidents at Pressurized Water Reactors. In addition, the inspector verified that the changes were reviewed and documented in accordance with 10 CFR 50.59 processes.

As directed by Temporary Instruction 2515/166, the inspector observed the physical installation of the sump strainers as committed to in the licensees response to Generic Letter 2004-02. No concerns with the physical modifications were identified.

The inspector also reviewed the licensees procedures and programs for accounting for and controlling equipment tags, latent debris, unqualified coatings, and chemicals inside containment. Programs to identify the scope of equipment tags, coatings, debris, and chemicals that have the potential to cause screen blockage were adequate, and the licensee has made needed changes to relevant procedures to control introduction of these items in the future.

The modifications to the sump strainers provides a large increase in strainer area, and tests were performed at the manufacturers facilities to validate that this area is sufficient to maintain sump function with predicted debris loading. At the time of this inspection, the final test reports had not been received and reviewed by the licensee or this inspector. However, preliminary review of the test data by licensee personnel indicated that the strainer area was adequate with a large margin. Final closeout of this Temporary Instruction will require review of the final test reports to validate that head loss and flow through the strainers with assumed debris loading will meet safety design requirements.

b. Findings

No findings of significance were identified.

4OA6 Meetings, Including Exit

The inspector presented the results of this inservice inspection to Mr. Michael Perito, Plant Manager, and other members of licensee management on June 7, 2007. Licensee management acknowledged the inspection findings.

The inspectors presented the resident inspection results to Mr. R. Edington, Senior Vice President, Nuclear, and other members of the licensee's management staff at the conclusion of the inspection on June 29, 2007. The licensee acknowledged the findings presented.

The inspectors noted that while proprietary information was reviewed, none would be included in this report.

4OA7 Licensee-Identified Violations

The following violations of very low significance (Green) were identified by the licensee and are violations of NRC requirements which meet the criteria of Section VI.A of the NRC Enforcement Policy, NUREG-1600, for being dispositioned as an NCVs.

• Technical Specification 3.4.4 requires that two RCS loops be operable and in operation in Modes 1 and 2. Contrary to the above, during periods of operation with all four reactor coolant pumps in service, from December 20, 2005, to March 18, 2006, the RCS did not comply with General Design Criterion 15, "Reactor Coolant System Design," in that there was not always sufficient margin to assure that the design conditions of the reactor coolant pressure boundary were not exceeded during any condition of normal operation, including anticipated operational occurrences. Specifically, previous vibration limits associated with SDC suction isolation Valve 1JSIAUV0651 following the replacement steam generator outage could have been exceeded if either RCP in Loop 2 failed during Modes 1 or 2. Valve 1JSIAUV0651 was permanently relocated to resolve this problem. The finding is determined to be of very low safety significance by management review since this condition would be rapidly recognized, the actions to mitigate this condition were simple and straightforward, and the vibrations levels would not have challenged pipe integrity for at least 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br />. This finding was documented in CRDR 2877313 and LER 05000529/2005009-00 (Section 4OA3.5).

• Technical Specification 5.4.1.a requires written procedures be established, implemented, and maintained covering the activities specified in Regulatory Guide 1.33, Appendix A, February 1978. Regulatory Guide 1.33, Appendix A, Item 9.a, requires, in part, procedures for "performing maintenance that can affect the performance of safety related equipment." Contrary to this requirement, on November 30, 2006, during the performance of planned maintenance, maintenance personnel failed to properly implement the work instructions, and improperly installed the thermal overloads on the Unit 1, Train A, fuel building essential filtration unit supply breaker, causing that unit, and the associated train of pump room exhaust air cleanup system (PREACS), to be inoperable under certain conditions. This condition was discovered on April 12, 2007, and corrected. The PREACS train was restored to operable status on April 14, 2007.

The licensee entered this item into the CAP as PVAR 2992623. This finding is determined to be of very low safety significance because it does not represent a loss of system safety function and the finding does not screen as risk significant due to a seismic, flooding, or severe weather initiating event.

• 10 CFR Part 50, Appendix B, Criterion III, "Design Control," requires, in part, that design control measures shall provide for verifying or checking the adequacy of design, and can be accomplished by the performance of a suitable testing program. Contrary to the above, during original construction and prior to placing EDG 1B in-service to support power operations, the testing program failed to

identify that the EDG 1B turbocharger filter three way changeover valve was not installed per design. The valve was installed 180 degrees out of position, such that no lube oil flow would be available to the turbocharger in the mid-position while shifting between filters. The valve was installed incorrectly until identified during EDG maintenance on May 25, 2007. This issue was entered into the CAP as PVAR 3018721. The finding is of very low safety significance because there was no actual loss of safety function to the EDG since damage to the turbocharger would not occur during the short time the valve was in the mid-position while swapping to the clean filter.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

G. Andrews, Director, Performance Improvement

S. Bauer, Department Leader, Regulatory Affairs

J. Bayless, Senior Engineer

R. Bement, Vice President, Nuclear Operations

P. Borchert, Director, Operations

P. Brandjes, Department Leader, Maintenance

R. Buzard, Senior Consultant, Regulatory Affairs

D. Carnes, Director, Nuclear Assurance

P. Carpenter, Department Leader, Operations

R. Cavalieri, Director, Outages

K. Chavet, Senior Consultant, Regulatory Affairs

D. Coxon, Unit Department Leader, Operations

R. Eddington, Senior Vice President, Nuclear

D. Elkington, Consultant, Regulatory Affairs

T. Engbring, Senior Engineer

J. Gaffney, Director, Radiation Protection

T. Gray, Department Leader, Radiation Protection

K. Graham, Department Leader, Fuel Services

M. Grigsby, Unit Department Leader, Operations

D. Hansen, Senior Consulting Engineer

R. Henry, Site Rep., SRP

J. Hesser, Vice President, Engineering

R. Indap, Senior Engineer

M. Karbasian, Director, Engineering

W. Lehman, Senior Engineer

D. Marks, Section Leader, Regulatory Affairs

S. McKinney, Department Leader, Operations Support

J. Mellody, Department Leader, PV Communications

E. O<Neil, Department leader, Emergency Preparedness

M. Perito, Plant Manager, Nuclear Operations

F. Poteet, Senior ISI Engineer

M. Radspinner, Section Leader, Systems Engineering

T. Radtke, General Manager, Emergency Services and Support

H. Ridenour, Director, Maintenance

F. Riedel, Director, Nuclear Training Department

J. Scott, Section Leader, Nuclear Assurance

M. Shea, Director, 95003

E. Shouse, Representative, EPE

M. Sontag, Department Leader, Performance Improvement

D. Straka, Senior Consultant, Regulatory Affairs

K. Sweeney, Department Leader, Systems Engineering

J. Taylor, Nuclear Project Manager, PNM

J. Taylor, Unit Department Leader, Operations

Attachment

D Vogt, Section Leader, OPS STA

T. Weber, Section Leader, Regulatory Affairs

J. Wood, Department Leader, Nuclear Training Department

NRC Personnel

M. Runyan, Senior Reactor Analyst

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000528;

05000529; NCV Failure to Scope Condensate Demineralizer Valve Into

05000530/2007003-01 Maintenance Rule (Section 1R12)

05000530/2007003-02 NCV Failure to Identify Degraded Structural Supports (Section 1R15)

05000528;

05000529; NCV Failure to Apply Industry Operating Experience to

05000530/2007003-03 Maintenance Activities Results in a Plant Transient (Section 4OA3.1)

05000528/2007003-04 NCV Failure to Perform Event Recovery Checklist (Section 4OA3.2)

Closed

05000528/2006005-00 LER Technical Specification Prohibited Condition Due to Check Valve Not Seated (Section 4OA3.1)

05000528/2006006-00 LER Reactor Trip due to Core Protection Calculator Generated Low DNBR Trip Signal (Section 4OA3.2)

05000528/2006007-00 LER Emergency Diesel Generator Actuation on Loss of Power to B Train 4.16 kV Buses in Units 1 and 3 (Section 4OA3.3)

05000530/2006007-00 LER Manual Reactor Trip Due to Degrading Condenser Vacuum and Condensate Flow (Section 4OA3.4)

05000528/2005009-00 LER Technical Specification Violation For Operation Without Both Reactor Coolant System Loops Operable (Section 4OA3.5)

05000530/2006005-01 LER Manual Reactor Trip Due to Loss of Main Feedwater (Section 4OA3.6)

Attachment

Discussed

05000528;

05000529; NCV Inadequate Corrective Actions for Target Rock

05000530/2007011-04 Solenoid-Operated Valves (Section 4OA2.2)