IR 05000528/2009002

IR 05000528-09-002, 05000529-09-002 & 05000530-09-002 on 01/01/09 - 03/31/09 for Palo Verde

ML091250061
Person / Time
Site:	Palo Verde
Issue date:	05/05/2009
From:	Hay M C NRC/RGN-IV/DRP/RPB-D
To:	Edington R K Arizona Public Service Co
References
IR-09-002
Download: ML091250061 (52)

Text

May 5, 2009

Randall K. Edington, Executive Vice President, Nuclear and Chief Nuclear Officer Mail Station 7602 Arizona Public Service Company P.O. Box 52034 Phoenix, AZ 85072-2034

SUBJECT: PALO VERDE NUCLEAR GENERATING STATION - NRC INTEGRATED INSPECTION REPORT 05000528/2009002, 05000529/2009002, AND 05000530/2009002

Dear Mr. Edington:

On March 31, 2009, the U.S. Nuclear Regulatory Commission (NRC) completed an integrated inspection at your Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility. The enclosed integrated report documents the inspection findings, which were discussed on April 7, 2009, with Mr. Bement and other members of your staff.

The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your licenses. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

This report documents three NRC identified findings of very low safety significance (Green). Two of these findings were determined to involve violations of NRC requirements. Additionally, four licensee-identified violations, which were determined to be of very low safety significance, are listed in this report. However, because of the very low safety significance and because they were entered into your corrective action program, the NRC is treating these findings as non-

cited violations consistent with Section VI.A.1 of the NRC Enforcement Policy. If you contest these non-cited violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission Region IV, 612 E. Lamar Blvd, Suite 400, Arlington, Texas 76011-4125; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington DC 20555-0001; and the NRC Resident Inspector at the Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility. In addition, if you disagree with the characterization of any finding in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the Regional Administrator, Region IV, and the NRC Resident Inspector at Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility. The information you provide will be considered in accordance with Inspection Manual Chapter 0305. Arizona Public Service Company -2-

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, and its enclosure, will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,/RA/

Michael C. Hay, Chief Projects, Branch D Division of Reactor Projects Docket Nos. 50-528 50-529 50-530 License Nos

. NPF-41 NPF-51 NPF-74

Enclosure:

NRC Inspection Report 05000528/2009002, 05000529/2009002, and 05000530/2009002

w/Attachment:

Supplemental Information

cc w/enclosure: Steve Olea Arizona Corporation Commission 1200 W. Washington Street Phoenix, AZ 85007

Douglas K. Porter, Senior Counsel Southern California Edison Company Law Department, Generation Resources P.O. Box 800 Rosemead, CA 91770 Chairman Maricopa County Board of Supervisors 301 W. Jefferson, 10th Floor Phoenix, AZ 85003 Aubrey V. Godwin, Director Arizona Radiation Regulatory Agency 4814 South 40 Street Phoenix, AZ 85040 Arizona Public Service Company -3-

Scott Bauer Director Regulatory Affairs Palo Verde Nuclear Generating Station Mail Station 7636 P.O. Box 52034 Phoenix, AZ 85072-2034 Dwight Mims, Vice President Regulatory Affairs and Performance Improvement Palo Verde Nuclear Generating Station Mail Station 7636 P.O. Box 52034 Phoenix, AZ 85072-2034 Jeffrey T. Weikert Assistant General Counsel El Paso Electric Company Mail Location 167 123 W. Mills El Paso, TX 79901 Eric J. Tharp Los Angeles Department of Water & Power Southern California Public Power Authority P.O. Box 51111, Room 1255-C Los Angeles, CA 90051-5700 James Ray Public Service Company of New Mexico 2401 Aztec NE, MS Z110 Albuquerque, NM 87107-4224 Geoffrey M. Cook Southern California Edison Company 5000 Pacific Coast Hwy, Bldg. N50 San Clemente, CA 92672 Robert Henry Salt River Project 6504 East Thomas Road Scottsdale, AZ 85251 Brian Almon Public Utility Commission William B. Travis Building P.O. Box 13326 1701 North Congress Avenue Austin, TX 78701-3326 Arizona Public Service Company -4-

Karen O'Regan Environmental Program Manager City of Phoenix Office of Environmental Programs 200 West Washington Street Phoenix, AZ 85003 John C. Taylor Director, Nuclear Generation El Paso Electric Company 340 East Palm Lane, Suite 310 Phoenix, AZ 85004 Chief, Radiological Emergency Preparedness Section National Preparedness Directorate Technological Hazards Division Department of Homeland Security 1111 Broadway, Suite 1200 Oakland, CA 94607-4052 Chairperson, Radiological Assistance Committee Region IX Federal Emergency Management Agency Department of Homeland Security 1111 Broadway, Suite 1200 Oakland, CA 94607-4052 Arizona Public Service Company -5-

Electronic distribution by RIV: Regional Administrator (Elmo.Collins@nrc.gov) Deputy Regional Administrator (Chuck.Casto@nrc.gov) DRP Director (Dwight.Chamberlain@nrc.gov) DRP Deputy Director (Anton.Vegel@nrc.gov) DRS Director (Roy.Caniano@nrc.gov)

DRS Deputy Director (Troy.Pruett@nrc.gov) Senior Resident Inspector (Ryan.Treadway@nrc.gov) Resident Inspector (Michelle.Catts@nrc.gov) Resident Inspector (Joseph.Bashore@nrc.gov) Resident Inspector (Jim.Melfi@nrc.gov) Branch Chief, DRP/D (Michael.Hay@nrc.gov) PV Site Secretary (acting) (Ryan.Treadway@nrc.gov) Senior Project Engineer, DRP/D (Don.Allen@nrc.gov) Public Affairs Officer (Victor.Dricks@nrc.gov) Team Leader, DRP/TSS (Chuck.Paulk@nrc.gov) RITS Coordinator (Marisa.Herrera@nrc.gov) Regional State Liaison Officer (Bill.Maier@nrc.gov) NSIR/DRP/ER (Steve.LaVie@nrc.gov)

Only inspection reports to the following: DRS STA (Dale.Powers@nrc.gov) OEDO RIV Coordinator (John.Adams@nrc.gov)

ROPreports

SUNSI Review Completed: __mch____ADAMS: Yes No Initials: __mch Publicly Available Non-Publicly Available Sensitive Non-Sensitive

R:\_REACTORS\_PV\2009\PV2009-002RP-RITadams.doc ML#091250061 RIV:RI:DRP/D RI:DRP/D RI:DRP/D SRI:DRP/D C:DRS/EB1 C:DRS/OB JBashore MCatts JFMelfi RITreadwayTFarnholtz RLantz

/RA/-email /RA/-email /RA/-email /RA/-email /RA/ /RA/ 04/27/2009 04/27/2009 04/27/2009 04/27/2009 04/27/2009 04/28/2009 C:DRS/EB2 C:DRS/PSB1 C:DRS/PSB2 C:DRP/D NO'Keefe MShannon GWerner MHay /RA/ /RA/ /RA/ /RA/

04/28/2009 04/29/2009 04/29/2009 04/29/2009 OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax

-1- Enclosure U.S. NUCLEAR REGULATORY COMMISSION REGION IV Dockets: 50-528, 50-529, 50-530 Licenses:

NPF-41, NPF-51, NPF-74 Report: 05000528/2009002, 05000529/2009002, 05000530/2009002 Licensee:

Arizona Public Service Company

Facility:

Palo Verde Nuclear Generating Station, Units 1, 2, and 3 Location:

5951 S. Wintersburg Road Tonopah, Arizona Dates: January 1 through March 31, 2009

Inspectors:

J. Bashore, Resident Inspector M. Catts, Resident Inspector J. Melfi, Resident Inspector R. Treadway, Senior Resident Inspector E. Uribe, Reactor Inspector P. Elkmann, Senior Emergency Preparedness Inspector G. Guerra, CHP, Emergency Preparedness Inspector Approved By:

Michael C. Hay, Chief, Project Branch D Division of Reactor Projects

-2- Enclosure

SUMMARY OF FINDINGS

IR 05000528/2009002, 05000529/2009002, 05000530/2009002; 01/01/09 - 03/31/09; Palo Verde Nuclear Generating Station, Units 1, 2, and 3; Integrated Resident and Regional Report; Maintenance Effectiveness, Operability Evaluations, and Other Activities.

This report covered a 3-month period of inspection by resident inspectors and regional inspectors. Three Green findings, two of which were non-cited violations, were identified. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management's review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 4, dated December 2006.

Cornerstone: Mitigating Systems

Green.

The inspectors identified a finding for the failure of engineering and maintenance personnel to adequately implement timely corrective actions for deficiencies associated with the essential spray pond sodium hypochlorite chemical addition system. Specifically, between May 2006 and March 2009, corrective actions to replace degraded sodium hypochlorite valves with a more reliable chemical addition system were not taken resulting in the Unit 2 spray pond Train A chemistry pH level being out of specification high on two occasions. This issue was entered into the licensee's corrective action program as Palo Verde Action Request 3277070.

The finding is more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of human performance associated with decision making because the licensee did not communicate bases for decisions to personnel with a need to know such that work is performed safely in a timely manner H.1(c) (Section 1R12).

Green.

The inspectors identified a non-cited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure of operations personnel to follow procedures to declare the essential spray pond inoperable. Specifically, on November 13, 2008, operations personnel failed to follow procedures to declare Unit 2 essential spray Pond A inoperable and perform a 10 CFR 50.59 screening when a compensatory measure, such as acid addition, was required to restore operability of the spray pond. This resulted in the performance of a calculation and an evaporative test to verify operability of essential spray Pond A for the mission time without taking credit for compensatory measures. This issue was entered into the licensee's corrective action program as Palo Verde Action Request 3258988.

The finding is more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its technical specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of human performance associated with decision-making because safety-significant decisions were not verified to validate underlying assumptions and identify unintended consequences [H.1 (b)] (Section 1R15).

Green.

The inspectors identified 5 examples of a non-cited violation of License Condition 2.C.(7), 2.C.(6), and 2.F for Unit 1, Unit 2, and Unit 3, respectively, for the failure of engineering and maintenance personnel to follow procedures to adequately inspect and repair fire penetration seals. Specifically, between 2004 and August 2008, engineering and maintenance personnel failed to inspect and repair fire penetration seals, which provide protection to safety-related equipment during fire events, resulting in the licensee declaring 4 fire penetration seals degraded and 1 non-functional. This issue was entered into the licensee's corrective action program as Palo Verde Action Request 3295124.

The finding is more than minor because it was associated with the external factors attribute (i.e. fire) of the mitigating systems cornerstone and affected the cornerstone objective of ensuring the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to require additional evaluation under Inspection Manual Chapter 0609, Appendix F, "Fire Protection Significance Determination Process." Based on the analysis performed, the inspector concluded that the degradation of the fire barrier penetration seals represented a low degradation of the fire confinement element of the fire protection program, the degraded fire barrier penetration seals had no credible fire damage state, and that the fire ignition sources present could not damage the post-fire safe shutdown equipment, and therefore determined the finding to have very low safety significance. This finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program because the licensee failed to implement the corrective action program with a low threshold for identifying issues P.1(a) (Section 4AO5).

B. Licensee-Identified Violations

Violations of very low safety significance that were identified by the licensee have been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensee's corrective action program. These violations and corrective action tracking numbers are listed in Section 4OA7 of this report.

REPORT DETAILS

Summary of Plant Status

Unit 1 operated at full power until March 26, 2009, when power was reduced to approximately 12 percent to facilitate maintenance on the stator cooling water system. The unit returned to full power on March 29, 2009, and remained at full power for the duration of the inspection period.

Unit 2 operated at full power until January 15, 2009, when power was reduced to approximately 90 percent due to elevated gasses in the 'C' main transformer. The trend for elevated gasses stabilized and the unit began power ascension on January 16, 2009.

The unit returned to full power on January 17, 2009, and remained at full power for the duration of the inspection period.

Unit 3 began the inspection period at full power and remained at full power for the duration of the inspection period.

REACTOR SAFETY

Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity, and Emergency Preparedness

1R01 Adverse Weather Protection

.1 Readiness for Seasonal Extreme Weather Conditions

a. Inspection Scope

The inspectors performed a review of the licensee's adverse weather procedures for seasonal extremes (e.g., extreme high temperatures, extreme low temperatures, or hurricane season preparations). The inspectors: verified that weather-related equipment deficiencies identified during the previous year were corrected prior to the onset of seasonal extremes; and evaluated the implementation of the adverse weather preparation procedures and compensatory measures for the affected conditions before the onset of, and during, the adverse weather conditions.

During the inspection, the inspectors focused on plant-specific design features and the licensee's procedures used to mitigate or respond to adverse weather conditions. Additionally, the inspectors reviewed the Updated Final Safety Analysis Report and performance requirements for systems selected for inspection, and verified that operator actions were appropriate as specified by plant-specific procedures. Specific documents reviewed during this inspection are listed in the attachment. The inspectors also reviewed corrective action program items to verify that the licensee was identifying adverse weather issues at an appropriate threshold and entering them into their corrective action program in accordance with station corrective action procedures. The inspectors' reviews focused specifically on the following plant systems:

• January 20, 2009, Unit 2, fuel pool cooling system, both trains

• January 20, 2009, Unit 3, fuel pool cooling system, both trains

• January 20, 2009, Unit 2, essential chillers and essential chilled water system, both trains

• January 20, 2009, Unit 3, essential chillers and essential chilled water system, both trains These activities constitute completion of one readiness for seasonal adverse weather sample as defined in Inspection Procedure 71111.01-05.

b. Findings

No findings of significance were identified.

.2 Readiness for Impending Adverse Weather Conditions

a. Inspection Scope

Since extreme cold conditions were forecast in the vicinity of the facility for January 13, 2009, the inspectors reviewed the licensee's overall preparations/protection for the expected weather conditions. On January 13, 2009, the inspectors walked down the Unit 2 and Unit 3 control room ventilation and emergency diesel generator systems because their safety-related functions could be affected or required as a result of the extreme cold conditions forecast for the facility. The inspectors observed insulation, heat trace circuits, and weatherized enclosures to ensure operability of affected systems. The inspectors reviewed licensee procedures and discussed potential compensatory measures with control room personnel. The inspectors focused on plant management's actions for implementing the station's procedures for ensuring adequate personnel for safe plant operation and emergency response would be available. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of one readiness for impending adverse weather condition sample as defined in Inspection Procedure 71111.01-05.

b. Findings

No findings of significance were identified.

1R04 Equipment Alignment

a. Inspection Scope

The inspectors performed partial system walkdowns of the following risk-significant systems:

• January 21, 2009, Unit 3, 13.8 kilovolt alternating current power distribution system while startup transformer NAN-X01 is out of service

• February 5, 2009, Unit 3 emergency diesel generator Train A support systems

• March 11, 2009, Unit 3, containment spray and low pressure safety injection system Train A injection, recirculation, and shutdown cooling flow paths

• March 25, 2009, Unit 1, stator cooling water system during degraded system flow conditions The inspectors selected these systems based on their risk significance relative to the reactor safety cornerstones at the time they were inspected. The inspectors attempted to identify any discrepancies that could affect the function of the system, and, therefore, potentially increase risk. The inspectors reviewed applicable operating procedures, system diagrams, Updated Final Safety Analysis Report, technical specification requirements, administrative technical specifications, outstanding work orders, condition reports, and the impact of ongoing work activities on redundant trains of equipment in order to identify conditions that could have rendered the systems incapable of performing their intended functions. The inspectors also walked down accessible portions of the systems to verify system components and support equipment were aligned correctly and operable. The inspectors examined the material condition of the components and observed operating parameters of equipment to verify that there were no obvious deficiencies. The inspectors also verified that the licensee had properly identified and resolved equipment alignment problems that could cause initiating events or impact the capability of mitigating systems or barriers and entered them into the corrective action program with the appropriate significance characterization. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of four partial system walkdown samples as defined in Inspection Procedure 71111.04-05.

b. Findings

No findings of significance were identified.

1R05 Fire Protection

Quarterly Fire Inspection Tours

a. Inspection Scope

The inspectors conducted fire protection walkdowns that were focused on availability, accessibility, and the condition of firefighting equipment in the following risk-significant plant areas:

• January 13, 2009, Unit 2, emergency diesel generator building, all accessible elevations

• January 14, 2009, Unit 1, control building, all accessible elevations

• January 15, 2009, Unit 1, condensate storage tank pump house and tunnel

• March 25, 2009, Unit 3, emergency diesel generator building, all accessible elevations

• March 31, 2009, Unit 3, auxiliary building , 40 foot, 52 foot, 70 foot, and 88 foot elevations The inspectors reviewed areas to assess if licensee personnel had implemented a fire protection program that adequately controlled combustibles and ignition sources within the plant; effectively maintained fire detection and suppression capability; maintained passive fire protection features in good material condition; and had implemented adequate compensatory measures for out of service, degraded or inoperable fire protection equipment, systems, or features, in accordance with the licensee's fire plan. The inspectors selected fire areas based on their overall contribution to internal fire risk as documented in the plant's Individual Plant Examination of External Events with later additional insights, their potential to affect equipment that could initiate or mitigate a plant transient, or their impact on the plant's ability to respond to a security event. Using the documents listed in the attachment, the inspectors verified that fire hoses and extinguishers were in their designated locations and available for immediate use; that fire detectors and sprinklers were unobstructed, that transient material loading was within the analyzed limits; and fire doors, dampers, and penetration seals appeared to be in satisfactory condition. The inspectors also verified that minor issues identified during the inspection were entered into the licensee's corrective action program. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of five quarterly fire protection inspection samples as defined in Inspection Procedure 71111.05-05.

b. Findings

No findings of significance were identified.

.2 Annual Fire Protection Drill Observation

a. Inspection Scope

On February 4, 2009, the inspectors observed a fire brigade activation to evaluate the readiness of the licensee to fight fires. The observation evaluated the readiness of the plant fire brigade to fight fires. The inspectors verified that the licensee staff identified deficiencies; openly discussed them in a self-critical manner at the drill debrief, and took appropriate corrective actions. Specific attributes evaluated were: (1) proper wearing of turnout gear and self-contained breathing apparatus; (2) proper use and layout of fire hoses; (3) employment of appropriate fire fighting techniques; (4) sufficient firefighting equipment brought to the scene; (5) effectiveness of fire brigade leader communications, command, and control; (6) search for victims and propagation of the fire into other plant areas; (7) smoke removal operations; (8) utilization of pre planned strategies; (9) adherence to the preplanned drill scenario; and (10) drill objectives.

These activities constitute completion of one annual fire-protection inspection sample as defined in Inspection Procedure 71111.05-05.

b. Findings

No findings of significance were identified.

1R07 Heat Sink Performance

a. Inspection Scope

The inspectors reviewed design documents (e.g., calculations and performance specifications), program documents, test and maintenance procedures, and corrective action documents for the inspection samples selected. The inspectors interviewed chemistry and engineering personnel and reviewed the corrective action documents related to Special Inspection Report 05000528; 05000529; 05000530/2006011.

The inspector selected heat exchangers that ranked high in the plant-specific risk assessment and were directly connected to the safety-related spray pond system. The inspector selected the following heat exchangers:

• Ultimate Heat Sink (Essential Spray Pond System)

• Diesel Generator Intercoolers

• Essential Cooling Water Heat Exchanger For heat exchangers directly connected to the safety-related spray pond system, the inspectors verified whether testing, inspection, maintenance, and monitoring of biotic fouling and microfouling programs are singularly or in combination adequate to ensure proper heat transfer. Specifically, the inspectors reviewed: (1) heat exchanger test methods and test results from performance testing; (2) chemical treatments for microfouling and controls for macrofouling; and (3) whether test results appropriately considered differences between testing conditions and design conditions. Additionally, the inspectors verified that the licensee: (1) performed condition monitoring and operation consistent with design assumptions in the heat transfer calculations; (2) evaluated the potential for water hammer, as applicable; (3) instituted appropriate chemistry controls for the heat exchangers; (4) reviewed periodic flow testing at or near maximum design flow for redundant and infrequently used heat exchanger; (5) verified that the number of plugged tubes were within pre-established limits based on heat transfer capacity; and (6) reviewed visual inspection records, to determine the structural integrity of the heat exchanger.

For the ultimate heat sink and its subcomponents, the inspectors verified that the licensee established appropriate controls for macrofouling and biological fouling. Since the licensee has an ultimate heat sink made up of two spray ponds per unit, a system walk-down was performed to verify the licensee had: (1) sufficient reservoir capacity; (2) performed periodic monitoring and trending of sediment build-up; (3) periodic performance monitoring of heat transfer capability; (4) periodic performance monitoring of the ultimate heat sink structural integrity; (5) instrumentation is available and functional; (6) review licensee controls to prevent clogging due to macrofouling; and (7) biocide treatments were conducted as scheduled, controlled, and the results monitored, trended, and evaluated.

Additionally, the inspectors examined the internal condition of the Unit 2 essential spray ponds Train A and B, and associated cooling water piping, to determine the effectiveness of the chemistry control program. The inspector concluded that adequate controls are being implemented to ensure the operability of the selected heat exchangers. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of three triennial heat sink inspection samples as defined in Inspection Procedure 71111.07-05.

b. Findings

No findings of significance were identified.

1R11 Licensed Operator Requalification Program

a. Inspection Scope

Quarterly Inspection On January 27, 2009, the inspectors observed a crew of licensed operators in the plant's simulator to verify that operator performance was adequate, evaluators were identifying and documenting crew performance problems and training was being conducted in accordance with licensee procedures. The inspectors evaluated the following areas:

• Licensed operator performance

• Crew's clarity and formality of communications

• Crew's ability to take timely actions in the conservative direction

• Crew's prioritization, interpretation, and verification of annunciator alarms

• Crew's correct use and implementation of abnormal and emergency procedures

• Control board manipulations

• Oversight and direction from supervisors

• Crew's ability to identify and implement appropriate technical specification actions and emergency plan actions and notifications The inspectors compared the crew's performance in these areas to pre-established operator action expectations and successful critical task completion requirements. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of one licensed-operator requalification program inspection sample as defined in Inspection Procedure 71111.11.

b. Findings

No findings of significance were identified.

1R12 Maintenance Effectiveness

a. Inspection Scope

The inspectors evaluated degraded performance issues involving the following risk significant systems:

• January 6 - 12, 2009, Unit 1, emergency diesel generator Train A control room equipment air handling unit motor and coupling replacement as documented in Palo Verde Action Request 3268162

• February 25, 2009, Unit 2, sodium hypochlorite chemical addition system for the essential spray ponds as documented in Palo Verde Action Request 3206115

• March 21, 2009, Unit 2, inverter power supply replacement affecting control room annunciators on control boards one, four, and five as documented in Palo Verde Action Request 3292456 The inspectors reviewed events such as where ineffective equipment maintenance has resulted in valid or invalid automatic actuations of engineered safeguards systems and independently verified the licensee's actions to address system performance or condition problems in terms of the following:

• Implementing appropriate work practices

• Identifying and addressing common cause failures

• Scoping of systems in accordance with 10 CFR 50.65(b)

• Characterizing system reliability issues for performance

• Charging unavailability for performance

• Trending key parameters for condition monitoring

• Ensuring proper classification in accordance with 10 CFR 50.65(a)(1) or (a)(2)

• Verifying appropriate performance criteria for structures, systems, and components classified as having an adequate demonstration of performance through preventive maintenance, as described in 10 CFR 50.65(a)(2), or as requiring the establishment of appropriate and adequate goals and corrective actions for systems classified as not having adequate performance, as described in 10 CFR 50.65(a)(1) The inspectors assessed performance issues with respect to the reliability, availability, and condition monitoring of the system. In addition, the inspectors verified maintenance effectiveness issues were entered into the corrective action program with the appropriate significance characterization. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of three quarterly maintenance effectiveness samples as defined in Inspection Procedure 71111.12-05.

b. Findings

Introduction.

The inspectors identified a Green finding for the failure of engineering and maintenance personnel to adequately implement timely corrective actions for deficiencies associated with the essential spray pond sodium hypochlorite chemical addition system.

Specifically, between May 2006 and March 2009, corrective actions to replace degraded sodium hypochlorite valves with a more reliable chemical addition system were not taken resulting in the Unit 2 spray pond Train A chemistry pH level being out of specification high on two occasions.

Description.

On January 28, 2009, an area operator discovered the ten gallon per minute sodium hypochlorite addition Valve 2P-SPN-V491 leaking by its seat following completion of a chemical addition to the Unit 2 essential spray pond Train B. This valve was opened for a sodium hypochlorite addition on the previous shift, but was closed and verified closed by two auxiliary operators. The impact of the valve leaking by its seat resulted in an uncontrolled addition of sodium hypochlorite to the Unit 2 essential spray pond Train B. An over-addition of sodium hypochlorite causes elevated levels of pH that adversely affect the operation of the essential spray ponds.

The essential spray ponds are required to have a pH between 7.4 and 8.0 to prevent a critical pH from occurring, which causes short-term calcium phosphate precipitants, or scaling, over the spray pond mission time of 26 days. In 2006, calcium phosphate precipitants caused clogging of the safety-related emergency diesel generator heat exchangers as described in NRC Inspection Report 05000528; 05000529; 05000530/2006011. Chlorine does not have a required range; however, the elevated chlorine causes elevated pH levels in the essential spray ponds. Uncontrolled additions of hypochlorite raise pH levels in the essential spray pond to a value that could affect its operability.

There are six essential spray ponds and associated sodium hypochlorite chemical addition systems at the Palo Verde Nuclear Generating Station. The failure on January 28, 2009, was the third failure of an essential spray pond chemical addition system valve since July of 2008. During their review, the inspectors noted that there have been ninety-eight corrective maintenance work orders performed on essential spray pond chemical addition system valves since 2003. In March 2001, design modification work Order 2374458 was initiated to replace the sodium hypochlorite chemical addition system, but implementation had not begun until 2005. Additionally, in May 2004, condition report disposition Request 2708901 documented an identified trend of high failure rates for the hypochlorite valves. The inspectors noted that condition report disposition Request 2708901 was closed as "review" without any corrective actions or formal evaluation performed. On May 20, 2006, significant condition report disposition Request 2897810 was issued in response to a loss of thermal performance of the essential cooling water and emergency diesel generator intercooler heat exchangers. One of the root causes identified was that Palo Verde had not adequately addressed the significance or impacts of changes made to the essential spray pond chemistry control program.

A contributing cause identified was that Palo Verde was unable to effectively prioritize or establish a system to ensure spray pond problems were effectively resolved. The inspectors noted one of the corrective actions for both the contributing cause and root cause was to have a permanent modification for the chemical addition system in place and operational by September 2008. As of March 30, 2009, the chemical addition system permanent modification was neither in place nor operational.

The chemical addition system valves are diaphragm operated rising stem valves. This type of valve is used across the industry and at Palo Verde for applications associated with hypochlorite and other highly corrosive chemicals. A review of industry and Palo Verde operating experience identified a trend of issues associated with failure of the diaphragm valves, particularly if they are installed and used beyond their nominal five year shelf life.

The Palo Verde Water Reclamation Facility identified this type of valve as having high failure rates and instituted preventive maintenance tasks to minimize the failure rate. The preventive maintenance tasks included retorquing bonnet bolts, stem lubrication, and setting the stem stop. At Palo Verde Nuclear Generating Station, these valves were classified as non-key equipment and therefore considered as "run to failure" components. In addition, vendor technical Document VTD-R026-0001, "Resitoflex Installation and Maintenance Data for Type VD40 Diaphragm Valve," defined torque values for body to bonnet bolts as well as instructions for setting the stem travel stop. During their review, the inspectors noted that no preventive maintenance was prescribed for these valves, the work orders did not contain instructions for torque values and travel stop settings, and did not reference the vendor technical document within the work order.

Analysis.

The performance deficiency associated with this finding was the failure of engineering and maintenance personnel to adequately implement timely corrective actions for deficiencies associated with the essential spray pond chemical addition system. The finding is more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of human performance associated with decision making because the licensee did not communicate bases for decisions to personnel with a need to know such that work is performed safely in a timely manner H.1(c).

Enforcement.

Enforcement action does not apply because the performance deficiency did not involve a violation of regulatory requirements. The finding is of very low safety significance and the issue was entered into the corrective action program as Palo Verde Action Request 3277070. FIN 05000528;05000529;05000530/2009002-01, "Failure to Correct Deficient Condition for the Essential Spray Pond Chemical Addition System Valves High Failure Rate."

1R13 Maintenance Risk Assessments and Emergent Work Control

a. Inspection Scope

Risk Assessment and Management of Risk The inspectors reviewed licensee personnel's evaluation and management of plant risk for the maintenance and emergent work activities affecting risk-significant and safety-related equipment listed below to verify that the appropriate risk assessments were performed prior to removing equipment for work:

• January 6, 2009, Unit 1, emergency diesel generator Train A out of service due to unplanned maintenance for replacement of the control room equipment air handling unit motor and coupling

• January 7, 2009, Unit 1, emergency diesel generator, essential chiller, containment spray, essential cooling water, and control room emergency air temperature control system, Train A, out of service for planned maintenance

• January 9, 2009, Unit 1, emergent work to replace Main Steam Isolation System AB matrix relay that exceeded time response

• January 13, 2009, Unit 1 and Unit 3 planned maintenance activities with startup transformer NAN-X02 out of service

• February 3, 2009, Unit 3, emergency diesel generator, emergency spray pond system, essential cooling water system, and high pressure safety injection system, Train B, out of service for planned maintenance

• March 19, 2009, Unit 2, impending inverter power supply replacement affecting control room annunciators on control boards one, four, and five The inspectors selected these activities based on potential risk significance relative to the reactor safety cornerstones. As applicable for each activity, the inspectors verified that licensee personnel performed risk assessments as required by 10 CFR 50.65(a)(4) and that the assessments were accurate and complete. When licensee personnel performed emergent work, the inspectors verified that the licensee personnel promptly assessed and managed plant risk. The inspectors reviewed the scope of maintenance work, discussed the results of the assessment with the licensee's probabilistic risk analyst or shift technical advisor, and verified plant conditions were consistent with the risk assessment. The inspectors also reviewed the technical specification requirements and inspected portions of redundant safety systems, when applicable, to verify risk analysis assumptions were valid and applicable requirements were met. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of six maintenance risk assessment and emergent work control inspection samples as defined in Inspection Procedure 71111.13-05.

c. Findings

No findings of significance were identified.

1R15 Operability Evaluations

a. Inspection Scope

The inspectors reviewed the following issues:

• January 20, 2009, Unit 2, spray pond chemistry out of specification

• January 26 2009, Unit 2, ground on Class 1E 125 Volts DC power Train B when atmospheric dump valve nitrogen supply valves 178 and 185 were deenergized

• February 12, 2009, Units 1, 2, and 3, operability determination for atmospheric dump valves meeting mission time during a main steam line break

• February 23 - 24, 2009, Units 1, 2, and 3, operability determination for all emergency diesel generators following Unit 3 emergency diesel generator Train A fuel injector metering rod failure

• March 18, 2009, Units 1, 2, and 3, operability determination for main steam line constant spring hanger tension rod crack.

• March 13, 2009, Unit 1, 2, and 3, operability determination for core protection calculators and reactor coolant system for inaccurate assumption of available over power margin for analysis of 12 finger control element assembly drop event for power levels below 50 percent The inspectors selected these potential operability issues based on the risk-significance of the associated components and systems. The inspectors evaluated the technical adequacy of the evaluations to ensure that technical specification operability was properly justified and the subject component or system remained available such that no unrecognized increase in risk occurred. The inspectors compared the operability and design criteria in the appropriate sections of the technical specifications and Updated Safety Analysis Report to the licensee's evaluations, to determine whether the components or systems were operable. Where compensatory measures were required to maintain operability, the inspectors determined whether the measures in place would function as intended and were properly controlled. The inspectors determined, where appropriate, compliance with bounding limitations associated with the evaluations. Additionally, the inspectors also reviewed a sampling of corrective action documents to verify that the licensee was identifying and correcting any deficiencies associated with operability evaluations. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of six operability evaluation inspection samples as defined in Inspection Procedure 71111.15-05.

1. Failure to Perform an Adequate Operability Determination for High Chlorine in the Essential Spray Pond

Introduction.

The inspectors identified a Green non-cited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure of operations personnel to follow procedures to declare essential spray Pond 2A inoperable and perform a 10 CFR 50.59 screening when a compensatory measure, such as acid addition, was required to restore operability of the spray pond.

Description.

On November 13, 2008, chemistry personnel found Unit 2 essential spray Pond A chemistry out of specification high with chlorine levels at 30 parts per million and pH at 8.33. Chemistry Procedure 74DP-9CY04, "Systems Chemistry Specification," Revision 61, Section 3.11, stated that when pH is greater than 8.2, an operability determination is required. Operations personnel performed an immediate operability determination and documented that essential spray Pond A was operable based on a proceduralized compensatory measure for acid addition that would bring the pH back into specification. Operations personnel added acid to the spray pond and chemistry returned within normal specifications in approximately eight hours.

The essential spray ponds are required to have a pH between 7.4 and 8.0 to prevent a critical pH from occurring, which causes short-term calcium phosphate precipitants or scaling, over the 26 day spray pond mission time. In 2006, calcium phosphate precipitants caused clogging of the safety-related emergency diesel generator heat exchangers as described in NRC Inspection Report 05000528; 05000529; 05000530/2006011. Chlorine does not have a required range; however, the elevated chlorine causes elevated pH in the essential spray ponds.

Inspectors reviewed this operability determination on December 1, 2008, and determined that the proceduralized compensatory measure to add acid, credited in the operability determination to maintain operability, could not be relied on in a design basis accident since the acid addition system is not safety related and would not work during a loss of offsite power. Inspectors expressed concern that the nonconforming condition of Procedure 74DP-9CY04 was not adequately assessed for essential spray pond operability. The inspectors questioned the shift manager's decision that an adequate operability determination was performed in accordance with Procedure 40DP-9OP26, "Operability Determination and Functional Assessment," Revision 23. Procedure 40DP-9OP26, Section 5.16 stated, in part, that to be operable the structure, system, or component must be capable of performing the safety function specified by its design, within the required range of physical design conditions, initiation times, and mission times. Procedure 40DP-9OP26, Section 5.13 stated that the mission time is the duration of the structure, system, or component's operation that is credited in the design basis for the structure, system or component to perform its specified safety function. Since acid addition could not be credited for operability, the inspectors determined that the operability determination was not adequate to ensure essential spray Pond A could meet the mission time.

Additionally, the inspectors reviewed Procedure 40DP-9OP26, Section 3.5.3, which provided actions when a compensatory measure, such as acid addition, is required to restore operability. The procedure stated, in part, that the structure, system, or component is inoperable until the compensatory measure is addressed by performing a 10 CFR 50.59 review. However, operations personnel did not declare the essential spray ponds inoperable or perform the 10 CFR 50.59 review.

Due to the inspectors' questions, operations personnel reviewed the operability determination and determined it was inadequate to support operability. The operability of the spray pond had already been restored by acid addition, so chemistry personnel performed additional analysis using the French Creek Model to verify that scaling would not occur for the 26 day spray pond mission time without taking credit for acid addition.

However, the French Creek Model was not validated for pH values as high as 8.33, so an evaporative test was needed to validate the results of the French Creek Model. The evaporative test verified the results of the French Creek Model that no scaling would occur during the 26 day mission time with a pH of 8.33.

Due to the inadequate operability determination, the Operability Determination Quality Review Metric changed from Green to Yellow for November 2008, and Adverse CRDR 3262892 was written to: perform an evaporation test to provide more margin for spray pond chemistry limits, provide a review of the event, and consider producing an immediate operability determination template for treatment of spray pond chemistry issues.

Analysis.

The performance deficiency associated with this finding was the failure of operations personnel to follow procedures to declare essential spray Pond 2A inoperable and perform a 10 CFR 50.59 screening when a compensatory measure, such as acid addition, was required to restore operability of the spray pond. The finding is more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of human performance associated with decision making because safety-significant decisions were not verified to validate underlying assumptions and identify possible unintended consequences H.1(b).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings. Procedure 40DP-9OP26, "Operability Determination and Functional Assessment," Revision 23, Section 3.5.3 provided actions when a compensatory measure, such as acid addition, is required to restore operability and stated; "the structure, system, or component is inoperable until the compensatory measure has been addressed via a 10 CFR 50.59." Contrary to the above, on November 13, 2008, operations personnel failed to follow procedures to declare essential spray Pond 2A inoperable and perform a 10 CFR 50.59 screening when a compensatory measure, such as acid addition, was required to restore operability of the spray pond. This resulted in the performance of a calculation and an evaporative test to verify operability of essential spray Pond A for the mission time without taking credit for compensatory measures. Because this finding is of very low safety significance and has been entered into the licensee's corrective action program as CRDR 3262892, this violation is being treated as an NCV, consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000529/2009002-02, "Failure to Perform an Adequate Operability Determination for High Chlorine in the Essential Spray Pond."

1R19 Post-Maintenance Testing

a. Inspection Scope

The inspectors reviewed the following post-maintenance activities to verify that procedures and test activities were adequate to ensure system operability and functional capability:

• January 07, 2009, Unit 1, emergency diesel generator Train A control room equipment air handling unit motor and coupling replacement

• January 31, 2009, Unit 3, emergency diesel generator Train B following preventive maintenance and under frequency relay replacement

• March 20, 2009, Unit 1, emergency diesel generator Train B following scheduled maintenance

• March 21, 2009, Unit 2, inverter power supply replacement affecting control room annunciators on control boards one, four, and five

• March 30, 2009, Unit 1, stator cooling water system strainer and filter replacement due to flow blockage

• March 31, 2009, Unit 1 containment 100 foot level personnel access lock inner door seal replacement The inspectors selected these activities based upon the structure, system, or component's ability to affect risk. The inspectors evaluated these activities for the following (as applicable):

• The effect of testing on the plant had been adequately addressed; testing was adequate for the maintenance performed

• Acceptance criteria were clear and demonstrated operational readiness; test instrumentation was appropriate The inspectors evaluated the activities against the technical specifications, the Updated Final Safety Analysis Report, 10 CFR Part 50 requirements, licensee procedures, and various NRC generic communications to ensure that the test results adequately ensured that the equipment met the licensing basis and design requirements. In addition, the inspectors reviewed corrective action documents associated with post-maintenance tests to determine whether the licensee was identifying problems and entering them in the corrective action program and that the problems were being corrected commensurate with their importance to safety. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of six post-maintenance testing inspection samples as defined in Inspection Procedure 71111.19-05.

b. Findings

No findings of significance were identified.

1R22 Surveillance Testing

a. Inspection Scope

The inspectors reviewed the Updated Final Safety Analysis Report, procedure requirements, and technical specifications to ensure that the eight surveillance activities listed below demonstrated that the systems, structures, and/or components tested were capable of performing their intended safety functions. The inspectors either witnessed or reviewed test data to verify that the significant surveillance test attributes were adequate to address the following:

• Preconditioning

• Evaluation of testing impact on the plant

• Acceptance criteria

• Test equipment

• Procedures

• Jumper/lifted lead controls

• Test data

• Testing frequency and method demonstrated technical specification operability

• Test equipment removal

• Restoration of plant systems

• Fulfillment of ASME Code requirements

• Updating of performance indicator data

• Engineering evaluations, root causes, and bases for returning tested systems, structures, and components not meeting the test acceptance criteria were correct

• Reference setting data

• Annunciators and alarms setpoints The inspectors also verified that licensee personnel identified and implemented any needed corrective actions associated with the surveillance testing.

• January 4, 2009, Unit 2, in-service test of atmospheric dump Valve 178

• January 15, 2009, Unit 1, radiation monitor SQB-RU-1 containment isolation valve stroke time testing

• January 23, 2009, Unit 2, in-service test of Train B high pressure safety injection pump

• January 26, 2009, Unit 1, auxiliary feedwater system, Train B, recirculation flow inservice test

• January 27, 2009, Unit 1, high pressure safety injection and miscellaneous safety injection valves stroke time testing

• February 6, 2009, Unit 1, auxiliary feedwater system, Train A, recirculation flow inservice test

• February 11, 2009, Units 1, 2, and 3, channel check of containment area radiation monitor and primary coolant monitor

• March 11, 2009, Unit 3, containment sump level instrumentation LY-19 and LY-10 inservice test Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of eight surveillance testing inspection samples as defined in Inspection Procedure 71111.22-05.

b. Findings

No findings of significance were identified.

Cornerstone:

Emergency Preparedness

1EP1 Exercise Evaluation

a. Inspection Scope

The inspectors reviewed the objectives and scenario for the 2009 biennial emergency plan exercise to determine if the exercise would acceptably test major elements of the emergency plan. The scenario simulated a sheared reactor coolant pump shaft and subsequent seal failure, fission product barrier failures, core damage and a radiological release to the environment via a failed containment penetration to demonstrate the licensee's capabilities to implement the emergency plan.

The inspectors evaluated exercise performance by focusing on the risk-significant activities of event classification, offsite notification, recognition of offsite dose consequences, and development of protective action recommendations, in the Simulator Control Room and the following dedicated emergency response facilities:

• Technical Support Center

• Operations Support Center

• Emergency Operations Facility The inspectors also assessed recognition of, and response to, abnormal and emergency plant conditions, the transfer of decision making authority and emergency function responsibilities between facilities, onsite and offsite communications, protection of emergency workers, emergency repair evaluation and capability, and the overall implementation of the emergency plan to protect public health and safety and the environment. The inspectors reviewed the current revision of the facility emergency plan, emergency plan implementing procedures associated with operation of the licensee's emergency response facilities, procedures for the performance of associated emergency functions, and other documents as listed in the attachment to this report.

The inspectors compared the observed exercise performance with the requirements in the facility emergency plan, 10 CFR 50.47(b), 10 CFR Part 50, Appendix E, and with the guidance in the emergency plan implementing procedures and other federal guidance.

The inspectors attended the post-exercise critiques in each emergency response facility including the simulator control room to evaluate the initial licensee self-assessment of exercise performance. The inspectors also attended a subsequent formal presentation of critique items to plant management. Specific documents reviewed during this inspection are listed in the attachment.

These activities constitute completion of one sample as defined in Inspection Procedure 71114.01-05.

b. Findings

No findings of significance were identified.

1EP6 Drill Evaluation

a. Inspection Scope

The inspectors evaluated the conduct of a routine licensee emergency drill on February 4, 2009, to identify any weaknesses and deficiencies in classification, notification, and protective action recommendation development activities. The inspectors observed emergency response operations in the control room (simulator) and the technical support center to determine whether the event classification, notifications, and protective action recommendations were performed in accordance with procedures. The inspectors also attended the licensee drill critique to compare any inspector-observed weakness with those identified by the licensee staff in order to evaluate the critique and to verify whether the licensee staff was properly identifying weaknesses and entering them into the corrective action program. As part of the inspection, the inspectors reviewed the drill package and other documents listed in the attachment.

These activities constitute completion of one sample as defined in Inspection Procedure 71114.06-05.

b. Findings

No findings of significance were identified.

4OA1 Performance Indicator Verification

.1 Data Submission Issue

a. Inspection Scope

The inspectors performed a review of the data submitted by the licensee for the fourth Quarter 2008 performance indicators for any obvious inconsistencies prior to its public release in accordance with Inspection Manual Chapter 0608, "Performance Indicator Program." This review was performed as part of the inspectors' normal plant status activities and, as such, did not constitute a separate inspection sample.

b. Findings

No findings of significance were identified.

.2 Unplanned Scrams per 7000 Critical Hours (IE01)

a. Inspection Scope

The inspectors sampled licensee submittals for the Unplanned Scrams per 7000 Critical Hours performance indicator for Palo Verde Units 1, 2 and 3 for the period from the fourth quarter 2007 through the fourth quarter 2008. To determine the accuracy of the performance indicator data reported during those periods, performance indicator definitions and guidance contained in NEI Document 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 5, was used. The inspectors reviewed the licensee's operator narrative logs, issue reports, event reports and NRC integrated inspection reports for the period of January 2008 through December 2008 to validate the accuracy of the submittals.

The inspectors also reviewed the licensee's issue report database to determine if any problems had been identified with the performance indicator data collected or transmitted for this indicator and none were identified. Specific documents reviewed are described in the attachment to this report.

These activities constitute completion of three unplanned scrams per 7000 critical hours samples as defined in Inspection Procedure 71151-05.

b. Findings

No findings of significance were identified.

.3 Unplanned Scrams with Complications (IE02)

a. Inspection Scope

The inspectors sampled licensee submittals for the Unplanned Scrams with Complications performance indicator for Palo Verde Units 1, 2 and 3 for the period from the fourth quarter 2007 through the fourth quarter 2008. To determine the accuracy of the performance indicator data reported during those periods, performance indicator definitions and guidance contained in NEI Document 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 5, was used. The inspectors reviewed the licensee's operator narrative logs, issue reports, event reports and NRC integrated inspection reports for the period of January 2008 through December 2008 to validate the accuracy of the submittals. The inspectors also reviewed the licensee's issue report database to determine if any problems had been identified with the performance indicator data collected or transmitted for this indicator and none were identified. Specific documents reviewed are described in the attachment to this report.

These activities constitute completion of three unplanned scrams with complications samples as defined in Inspection Procedure 71151-05.

b. Findings

No findings of significance were identified.

.4 Unplanned Transients per 7000 Critical Hours (IE03)

a. Inspection Scope

The inspectors sampled licensee submittals for the Unplanned Transients per 7000 Critical Hours performance indicator for Palo Verde Units 1, 2 and 3 for the period from the fourth quarter 2007 through the fourth quarter 2008. To determine the accuracy of the performance indicator data reported during those periods, performance indicator definitions and guidance contained in NEI Document 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 5, was used. The inspectors reviewed the licensee's operator narrative logs, issue reports, maintenance rule records, event reports and NRC integrated inspection reports for the period of January 2008 through December 2008 to validate the accuracy of the submittals. The inspectors also reviewed the licensee's issue report database to determine if any problems had been identified with the performance indicator data collected or transmitted for this indicator and none were identified. Specific documents reviewed are described in the attachment to this report.

These activities constitute completion of three unplanned transients per 7000 critical hours samples as defined in Inspection Procedure 71151-05.

b. Findings

No findings of significance were identified.

.5 Drill/Exercise Performance (EP01)

a. Inspection Scope

The inspectors sampled licensee submittals for the Drill/Exercise Performance performance indicator for the period January through December 2008. To determine the accuracy of the performance indicator data reported during those periods, performance indicator definitions and guidance contained in NEI Document 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 5, was used. The inspectors reviewed the licensee's records associated with the performance indicator to verify that the licensee accurately reported the indicator in accordance with relevant procedures and the NEI guidance. Specifically, the inspectors reviewed licensee records and processes including procedural guidance on assessing opportunities for the performance indicator; assessments of performance indicator opportunities during predesignated control room simulator training sessions, performance during the 2007 biennial exercise, and performance during other drills. Specific documents reviewed are described in the attachment to this report.

These activities constitute completion of the drill/exercise performance sample as defined in Inspection Procedure 71151-05.

b. Findings

No findings of significance were identified.

.6 Emergency Response Organization Drill Participation (EP02)

a. Inspection Scope

The inspectors sampled licensee submittals for the Emergency Response Organization Drill Participation performance indicator for the period January through December 2008. To determine the accuracy of the performance indicator data reported during those periods, performance indicator definitions and guidance contained in NEI Document 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 5, was used. The inspectors reviewed the licensee's records associated with the performance indicator to verify that the licensee accurately reported the indicator in accordance with relevant procedures and the NEI guidance. Specifically, the inspectors reviewed licensee records and processes including procedural guidance on assessing opportunities for the performance indicator; performance during the 2007 biennial exercise and other drills; and revisions of the roster of personnel assigned to key emergency response organization positions. Specific documents reviewed are described in the attachment to this report.

These activities constitute completion of the emergency response organization drill participation sample as defined in Inspection Procedure 71151-05.

b. Findings

No findings of significance were identified.

.7 Alert and Notification System (EP03)

a. Inspection Scope

The inspectors sampled licensee submittals for the Alert and Notification System performance indicator for the period January through December 2008. To determine the accuracy of the performance indicator data reported during those periods, performance indicator definitions and guidance contained in NEI Document 99-02, "Regulatory Assessment Performance Indicator Guideline," Revision 5, was used. The inspectors reviewed the licensee's records associated with the performance indicator to verify that the licensee accurately reported the indicator in accordance with relevant procedures and the NEI guidance. Specifically, the inspectors reviewed licensee records and processes including procedural guidance on assessing opportunities for the performance indicator; performance during the 2007 biennial exercise, and results of periodic alert notification system operability tests. Specific documents reviewed are described in the attachment to this report.

These activities constitute completion of the alert and notification system sample as defined in Inspection Procedure 71151-05.

b. Findings

No findings of significance were identified.

4OA2 Identification and Resolution of Problems

Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity, Emergency Preparedness, Public Radiation Safety, Occupational Radiation Safety, and Physical Protection

.1 Routine Review of Identification and Resolution of Problems

a. Inspection Scope

As part of the various baseline inspection procedures discussed in previous sections of this report, the inspectors routinely reviewed issues during baseline inspection activities and plant status reviews to verify that they were being entered into the licensee's corrective action program at an appropriate threshold, that adequate attention was being given to timely corrective actions, and that adverse trends were identified and addressed. The inspectors reviewed attributes that included: the complete and accurate identification of the problem; the timely correction, commensurate with the safety significance; the evaluation and disposition of performance issues, generic implications, common causes, contributing factors, root causes, extent of condition reviews, and previous occurrences reviews; and the classification, prioritization, focus, and timeliness of corrective actions. Minor issues entered into the licensee's corrective action program because of the inspectors' observations are included in the attached list of documents reviewed.

These routine reviews for the identification and resolution of problems did not constitute any additional inspection samples. Instead, by procedure, they were considered an integral part of the inspections performed during the quarter and documented in Section 1 of this report.

b. Findings

No findings of significance were identified.

.2 Daily Corrective Action Program Reviews

a. Inspection Scope

In order to assist with the identification of repetitive equipment failures and specific human performance issues for follow-up, the inspectors performed a daily screening of items entered into the licensee's corrective action program. The inspectors accomplished this through review of the station's daily corrective action documents.

The inspectors performed these daily reviews as part of their daily plant status monitoring activities and, as such, did not constitute any separate inspection samples.

b. Findings

No findings of significance were identified.

.3 Selected Issue Follow-up Inspection

a. Inspection Scope

During a review of items entered in the licensee's corrective action program, the inspectors recognized a corrective action item documenting a failed surveillance for the Unit 2 atmospheric dump Valve 2JSGBHV0178 that occurred on January 4, 2009. The licensee entered the issue into their corrective action program and performed trouble-shooting activities in accordance with approved procedures. During their troubleshooting activities, the licensee determined the failed surveillance was caused by a degraded linkage between the valve and the valve positioner. The licensee replaced the failed linkage, performed post-maintenance testing, and successfully re-performed the subsequent surveillance test.

The inspectors reviewed the corrective action documents and troubleshooting plan for this issue and noted the licensee's troubleshooting efforts did not evaluate the issue to identify a direct cause for the degraded linkage. The inspectors communicated this concern with the licensee and stated that if the direct cause is not identified, it could result in future failures.

The licensee concurred with the inspectors' concerns and entered them into the corrective action program.

These activities constitute completion of one in-depth problem identification and resolution samples as defined in Inspection Procedure 71152-05.

b. Findings

No findings of significance were identified.

4OA3 Event Follow-up

.1 Event Report Reviews

a. Inspection Scope

The inspectors reviewed the below listed Licensee Event Reports and related documents to assess: (1) the accuracy of the License Event Report; (2) the appropriateness of corrective actions; (3) violations of requirements; and (4) generic issues.

.1 (Closed) Licensee Event Report 05000528; 05000529; 05000530/2007-002-00, Train B Class 1E Pressurizer Backup Heater Control Circuit Would Not be Properly Isolated For a Control Room Fire Event.

On July 5, 2007, the licensee identified that the control room circuit for the Class 1E pressurizer backup heater may not properly isolate for certain postulated control room fires, as required by their Operating License. Specifically, when the Local/Remote switch associated with the Train B Class 1E pressurizer backup heaters is in the local position, a postulated line to ground fault on the conductor in conjunction with a concurrent negative line to ground fault from the same battery system due to fire damage in the control room, could cause a fuse to fail open. This would prevent the backup heater breaker from closing from the remote shutdown panel resulting in the loss of Train B Class 1E pressurizer backup heater local control. This issue had existed since original plant construction and affected all three units. The licensee documented this deficiency in Condition Report Disposition Request 3038288. As a compensatory measure, the licensee revised Procedure 40AO-9ZZ19, "Control Room Fire," Revision 17, to require that operators manually close Breaker E-PGB-L32E prior to evacuating the main control room. The licensee will maintain the compensatory actions in effect until they modify the circuit so that it properly isolates the control room circuit for the Train B Class 1E pressurizer heaters The licensee-identified violation of License Condition 2.C.(7), 2.C.(6), and 2.F for Unit 1, Unit 2, and Unit 3, respectively, is documented in Section

4OA7 of this report.

The inspectors reviewed this License Event Report and no additional findings of significance and no additional violation of NRC requirements occurred. This Licensee Event Report is closed.

.2 (Closed) Licensee Event Report 05000529/2007-005-00, Emergency Diesel Generator Inoperability Resulting From a Degraded Speed Control Circuit

On March 30, 2008, during surveillance testing on the Unit 2 emergency diesel generator train A, the surveillance requirement acceptance criteria for steady state frequency (59.7 to 60.7 Hz) was not met. During the surveillance test, the licensee noted that the output frequency was oscillating outside of the allowed range. The licensee's investigation into the oscillations determined that the failure mechanism that caused the oscillations had existed since the emergency diesel generator was shutdown a month before on December 13, 2007. During their review, the licensee determined the cause of the frequency oscillations to be an age-related failure of the transducer epoxy for the speed control governor pilot valve. The inspector reviewed the licensee's corrective actions and documented their results in NRC inspection report 2008-003, section 1R22. Additionally, a licensee identified violation is described in section

4OA7 of this report.

The inspectors reviewed this License Event Report and no additional findings of significance and no additional violation of NRC requirements occurred. This Licensee Event Report is closed.

4OA5 Other Activities

.1 Quarterly Resident Inspector Observations of Security Personnel and Activities

a. Inspection Scope

During the inspection period the inspectors conducted observations of security force personnel and activities to ensure that the activities were consistent with licensee security procedures and regulatory requirements relating to nuclear plant security. These observations took place during both normal and off-normal plant working hours. These quarterly resident inspector observations of security force personnel and activities did not constitute any additional inspection samples. Rather, they were considered an integral part of the inspectors' normal plant status reviews and inspection activities.

b. Findings

No findings of significance were identified.

.2 (Closed) Unresolved Item 05000528; 529; 530/2008004-02: Potentially Degraded Fire Penetration Seals

a. Inspection Scope

The inspectors performed this inspection by reviewing the documents listed in the Attachment, independently evaluating the described fire barrier penetration seal deficiencies, and interviewing fire protection personnel involved in the inspections and evaluations. For the fire barrier penetration seals that had the most degradation, the inspectors performed a walk down that visually verified that the licensee appropriately performed their evaluations.

b. Findings

Introduction.

The inspectors identified 5 examples of a non-cited violation of License Condition 2.C.(7), 2.C.(6), and 2.F for Unit 1, Unit 2, and Unit 3, respectively, for the failure of engineering and maintenance personnel follow procedures to adequately inspect and repair fire penetration seals. Specifically, between 2004 and August 2008, engineering and maintenance personnel failed to inspect and repair fire penetration seals, which provide protection to safety-related equipment during fire events, resulting in the licensee declaring 4 fire penetration seals degraded and 1 non-functional.

Description.

From July through November, 2008, engineering and maintenance personnel performed inspections of all penetrations and barriers in response to Information Notice 2007-01, "Recent Operating Experience Concerning Hydrostatic Barriers," to verify adequate internal flood protection for safety-related equipment. The inspectors reviewed Information Notice 2007-01 in February 2007 and determined that engineering personnel credited fire protection inspection Procedure 14FT-9FP70, "Appendix R & Former Technical Specification Penetration Seal Surveillance," Revision 7, for flood inspections even though the acceptance criteria for fire protection allows small gaps in seals while flood protection does not. During the flood penetration and barrier inspections, the licensee declared 4 fire penetration seals degraded and 1 non-functional. The inspectors questioned why the licensee was finding degraded fire seals when they currently had a program and Procedure 14FT-9FP70 in place to periodically inspect the fire seals.

One example of a degraded seal found during the flood seal inspections is Seal 1AZYD379*007. This seal is located on the 94 foot elevation of the emergency diesel generator building and provides a barrier between the emergency diesel generator building's seismic gap and the outside trench. This 19 inch deep seal was found on August 1, 2008 with a hole at least six inches wide and ten inches deep, and it was indeterminate if additional low density foam material existed in the rest of the barrier. This seal was declared nonfunctional per Procedure 73TI-9ZZ86, "Visual Assessment of Hydrostatic/Flood, HELB, EDP, and RAD Barriers and Penetrations," Revision 2, since the seal did not meet the depth requirements.

Seal 1AZYD379*007 was last inspected in 2005 and 2007 and was declared satisfactory, but did not meet the acceptance criteria of Procedure 14FT-9FP70. Procedure 14FT-9FP70, Section 1.0 stated, in part, that the objective of this procedure is to perform a visual inspection on a sample of Appendix R and Former Technical Specification penetration seals and identify degradations. Procedure 14FT-9FP70, Section 1.1.2.2, for acceptance criteria, stated, in part, that for low density foam-1, 2, or 3 that seal damage up to 3/8" deep shall constitute a degraded functional deficiency which must be repaired. Since Seal 1AZYD379*007 could not meet this acceptance criteria, the seal should have been declared nonfunctional and an evaluation or repair should have been performed. For all of these degraded seals, the licensee implemented a fire watch until the seal was either repaired or an evaluation showed the seal would still perform its function. The licensee performed a case-by-case evaluation on each of the degraded seals and determined that with the low fire loading, the suppression systems in the area, and the location of the seals relative to the fire, the seals would have performed their functions.

During the inspections of these fire penetration seals, the licensee declared 4 fire penetration seals degraded and 1 non-functional. These seals had all been inspected since 2004 and should have been declared degraded or nonfunctional at that time. The degradation of these seals occurs very slowly over a long period of time, and would not have degraded drastically since 2004. The licensee wrote PVAR 3295124 to address the inadequate inspection and repair of these seals.

This issue was opened as an unresolved issue in NRC Inspection Report 05000528; 529; 530/2008004-02, " Potentially Degraded Fire Penetration Seals," to determine the extent of degraded flood penetration seals at Palo Verde and the potential aggregate effect of more than one seal penetration failure. This unresolved item is closed.

Analysis.

The performance deficiency associated with this finding involved the failure of engineering and maintenance personnel to follow procedures to adequately inspect and repair fire penetration seals. The finding is more than minor because it was associated with the external factors attribute (i.e. fire) of the mitigating systems cornerstone and affected the cornerstone objective of ensuring the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to require additional evaluation under Inspection Manual Chapter 0609, Appendix F, "Fire Protection Significance Determination Process." Based on the analysis performed, the inspector concluded that the degradation of the fire barrier penetration seals represented a low degradation of the fire confinement element of the fire protection program, the degraded fire barrier penetration seals had no credible fire damage state, and that the fire ignition sources present could not damage the post-fire safe shutdown equipment, and therefore determined the finding to have very low safety significance. This finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program because the licensee failed to implement the corrective action program with a low threshold for identifying issues P.1(a).

Enforcement.

Arizona Public Service Company Operating License NPF 41, 51 and 74, License Condition 2.C.(7), 2.C.(6), and 2.F for Unit 1, Unit 2, and Unit 3 respectively, require, in part, that the licensee implement and maintain in effect all provisions of the approved fire protection program as described in the UFSAR, as supplemented and amended, and approved in the Palo Verde Safety Evaluation Report through Supplement 11. The UFSAR, Section 9.5.1.5.4(B) required that the licensee test and inspect, at regular intervals, fire barriers including penetration seals. Procedure 14FT-9FP70, "Appendix R and Former Tech Spec Penetration Seal Surveillance," Revision 7, Section 1.0 stated, in part, that the objective of this procedure is to perform a visual inspection on a sample of Appendix R and former Technical Specification penetration seals and identify degradations. Contrary to the above, between 2004 and August 2008, engineering and maintenance personnel failed to follow procedures to adequately inspect and repair fire penetration seals, which provide protection to safety-related equipment during fire events, resulting in the licensee declaring 4 degraded and 1 non-functional. Because this violation was determined to be of very low safety significance and has been entered into the licensee's CAP as PVAR 3295124, this violation is being treated as a NCV, consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000528;05000529;05000530/2009002-03, "Failure to Periodically Inspect or Test, and Repair Fire Penetration Seals."

4OA6 Meetings, Including Exit

On March 6, 2009, the inspectors presented the onsite emergency preparedness inspection results to Mr. R. Bement, Vice President, Nuclear Operations and other members of his staff, who acknowledged the findings. The inspectors confirmed that proprietary, sensitive, or personal information examined during the inspection had been returned to the identified custodian.

On March 20, 2009, the inspector presented the triennial heat sink performance inspection results to Mr. L. Cortopassi, Plant Manager, and other members of the licensee's staff. The licensee acknowledged the inspection observations.

On April 7, 2009, the inspectors presented the inspection results to Mr. R. Bement, Vice President Nuclear Operations, and other members of the licensee's staff at the conclusion of the inspection. The licensee acknowledged the findings presented.

The inspectors noted that while proprietary information was reviewed, none would be included in this report.

4OA7 Licensee-Identified Violations

The following violations of very low safety significance (Green) were identified by the licensee and are violations of NRC requirements that meet the criteria of the NRC Enforcement Policy, to be dispositioned as Non-Cited Violations.

• Title 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, or drawings. Contrary to the above, on January 5, 2009, the licensee identified that maintenance personnel did not implement the requirements of procedure 30DP-9MP02, "Fastener Tightening / Preload," during maintenance activities on the Unit 1 emergency diesel generator, Train A, equipment room essential air handling unit fan / motor coupling. This issue has been entered into the licensee's corrective action program as Palo Verde Action Request 3268162 and Condition Report Disposition Request 3269649. The finding is of very low safety significance because it did not result in a loss of safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to seismic, flooding, or severe weather initiating event.

• License Conditions 2.C.(7), 2.C.(6), and 2.F for Unit 1, Unit 2, and Unit 3, respectively, specifies that the licensee shall comply with the requirements of the fire protection program as specified in the license. The Final Safety Analysis Report, Section 9.5.1, which describes the fire protection program stated, "Fire protection features shall be capable of limiting fire damage so that one train of systems necessary to achieve and maintain hot shutdown conditions from either the control room or emergency control station(s) are free of fire damage." Contrary to the above, on July 5, 2007, the licensee determined that the existing design failed to ensure that the Train B Class 1E pressurizer backup heaters, which were required to achieve hot shutdown, remained free of fire damage during certain control room fire scenarios. The licensee documented this deficiency in Condition Report Disposition Record 3038288 and immediately implemented compensatory measures. This finding was evaluated in a Phase 3 SDP and found to have very low safety significance (Green). This item is further discussed in Section 4OA3.1.

• Title 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, or drawings. Contrary to the above, the licensee identified that operations personnel did not implement the requirements of 40DP-9OP26, "Operability Determination and Functional Assessment," when evaluating a degraded condition associated with the Unit 2 emergency diesel generator Train A speed control governor on January 15, 2008. This issue has been entered into the licensee's corrective action program as Palo Verde Action Request 3153187 and Condition Report Disposition Request 3153212. The finding is of very low safety significance because it did not result in a loss of safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to seismic, flooding, or severe weather initiating event.

• Appendix E to 10 CFR Part 50, IV F.1, requires that the emergency response program include a description of specialized initial and periodic retraining programs to be provided to emergency personnel, including radiological monitoring teams and repair and damage control teams. Contrary to the above, between March 11 and July 3, 2008 electrical maintenance personnel and radiation protection technicians stood emergency response organization watch without having received periodic retraining appropriate to their duties. Specifically, Condition Report 3196481 documented that no respirator qualified personnel were on emergency response organization duty for 6 shifts for the electrical craft; and Condition Report 3145062 documented that one radiation protection technician assigned as a B-player had lapsed on an annual training requirement. This finding is of very low safety significance because it was a failure to comply with an NRC requirement, the issue was associated with Emergency Preparedness Planning Standard B(2), problem, was not a risk significant planning standard problem as defined by Manual Chapter 0609 Appendix B, and was not a functional failure of the planning standard capability. ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

G. Andrews, Director, Performance Improvement

S. Bauer, Department Leader, Regulatory Affairs

R. Bement, Vice President, Nuclear Operations

P. Borchert, Unit 1 Assistant Plant Manager

F. Burdick, Regulatory Affairs

R. Burge, Sr. Engineer

R. Buzard, Section Leader, Compliance

D. Carnes, Unit 2 Assistant Plant Manager

K. Chavet, Senior Consultant, Regulatory Affairs

L. Cortopossi, Plant Manager, Nuclear Operations

D. Coxon, Unit Department Leader, Operations

A. Dave, Design Engineer

E. Dutton, Acting Director of Nuclear Assurance

R. Edington, Executive Vice President, CNO

D. Elkington, Consultant, Regulatory Affairs

W. Grover Hettel, Director, Operations

D. Hautala, Senior Engineer, Regulatory Affairs

J. Hesser, Vice President, Engineering

G. Hettel, Director, Operations

L. Johnson, Department Leader, Chemistry

M. Karbasian, Director, Design Engineering

A. Krainik, Department Leader, License Renewal

S. Koski, Fire Protection Engineer

F. Lake, Performance Improvement

M. McGhee, Acting Director, Operations

D. Mims, Vice President, Regulatory Affairs and Performance Improvement

J. Molden, Director, Engineering

T. Monk, Shift Manager, Operations

W. Pierce, Section Leader, Emergency Planning

T. Radtke, General Manager, Emergency Services and Support

D. Raught, System Engineer

M. Ray, Director, Emergency Planning Programs

H. Ridenour, Director, Maintenance

J. Ridriguez, Engineer, Regulatory Affairs

S. Sawtschenko, Department Leader, Emergency Preparedness

J. Scott, Department Leader, Nuclear Assuarance

M. Shea, Director, Safety Culture

B. Sullivan, Shift Manager, Operations

J. Summy, Director, Plant Engineering

T. Weber, Section Leader, Regulatory Affairs

Nuclear Regulatory Commission Personnel

M. Runyan, Senior Reactor Analyst, Region IV

Mahdi Hayes, Project Engineer Peter Jayroe, Project Engineer Chris Smith, Reactor Engineer

-A2- Attachment

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000528;529;530/2009002-01 FIN Failure to Correct Deficient Condition for the Essential Spray Pond Chemical Addition System

Valves High Failure Rate (1R12)

05000529/2009002-02 NCV Failure to Perform an Adequate Operability Determination for High Chlorine in the Essential

Spray Pond (Section 1R15)

05000528;529;530/2009002-03 NCV Failure to Periodically Inspect or Test, and Repair Fire Penetration Seals (Section 4OA5)

Closed

05000528;529;530/2008004-02 URI Potentially Degraded Fire Penetration Seals (Section 4OA5)

05000528;529;530/2007-002-00 LER Train B Class 1E Pressurizer Backup Heater Control Circuit Would Not Be Properly Isolated For A Control Room Fire Event (Section 4OA3)

05000529/LER-2007-005-00 LER Emergency Diesel Generator Inoperability Resulting From a Degraded Speed Control Circuit (Section 4OA3)

LIST OF DOCUMENTS REVIEWED

In addition to the documents called out in the inspection report, the following documents were selected and reviewed by the inspectors to accomplish the objectives and scope of the inspection and to support any findings:

Section 1R01: Adverse Weather Protection

Procedures

Number

Title Revision40OP-9ZZ17 Cold Weather Procedures 34 40OP-9WC01 Normal Chilled Water System 27

40OP-9OP06 Operations Department Repetitive Task Program 105

40DP-9OP18 Motor Operated Valve Operating Guidelines 18

40DP9OP10 Components Locked For Operational Convenience 50

Miscellaneous

Unit 1, 2 and 3 operator logs for December 16, 2008 thru January 21, 2009

-A3- Attachment

PVNGS Standing Order 108, UFSAR room cooler capacity for Unit 3 LPSI & CS Train A may be exceeded if insulation is removed

PVNGS Standing Order 141, Broken heat trace circuit for line CHBL149 (RWT to SI pumps)

Section 1R04: Equipment Alignment

Procedures

Number Title Revision40OP-9NA03 13.8 KV Electrical System (NA) 31

40OP-9NB01 4.16 KV Non-Class 1E Power (NB) 23

40OP-9PB01 4.16 KV Class 1E Power (PB) 24

73ST-9XI39 RWT Outlet Check Valve, ECCS Sump Check Valve, and SI Miniflow Leakage Test

40OP-9SI02 Recovery from Shutdown Cooling to Normal Operating Lineup 85

Drawings

Number

Title Revision01-E-NAA-004 Single Line Diagram 13.8 KV Non-Class 1E Power System Switchgear 1E-NAN-S01

02-E-NAA-004 Single Line Diagram 13.8 KV Non-Class 1E Power System Switchgear 2E-NAN-S01

03-E-NAA-004 Single Line Diagram 13.8 KV Non-Class 1E Power System Switchgear 3E-NAN-S01

01-E-PBA-001 Single Line Diagram 4.13 KV Class 1E Power System Switchgear 1E-PBA-S03

01-E-PBA-002 Single Line Diagram 4.13 KV Class 1E Power System Switchgear 1E-PBB-S04

02-E-NAA-002 Single Line Diagram 13.8 KV Non-Class 1E Power System Intermediate Switchgear 2E-NAN-S06 and Startup SFMR AE-NAN-

X02

9 01-M-SIP-001 P&I Diagram Safety Injection and Shutdown Cooling 42

PVARs

3295710

3294499

Miscellaneous

-A4- Attachment Palo Verde Nuclear Generating Station Permit 160498

Section 1R05: Fire Protection

Procedures

Number Title Revision Pre-Fire Strategies Manual

30DP-0WM12 Housekeeping

14DP-0FP33 Control of Transient Combustibles

73TI-9ZZ86 Visual Assessment of Hydrostatic/Flood, HELB, EDP, and RAD Barriers and Penetrations

Calculations

Number Title Revision13-AC-ZZ-0200 Penetration Seal Qualifications 12

PVARs

3203134

3206120

3207581

3207816

3209413

3213332

217048

3218773

3221773

3229969

3235248

3239156

246026

3246532

3247506

3249175

3249580

3252454

CRAIs

22253

3222801

3222806

3222812

3230894

3231226

239328

3247360

3248907

3249892

3251908

3251942

3255660

Penetrations Evaluated

1AZYD312*029 1AZYD379*007

1AZYD312*194 1AZYD312*200

2AZYD312*060 2AZYD312*094

2AZYD379*008 2AZYD379*009

2AZYD379*010 2AZYD379*011

3AZYD312*040 3AZYD312*045

3AZYD312*068 3AZYD312*116

3AZYD312*155 3AZYD312*156

3AZYD312*161 3AZYD312*168

3AZYD312*185 3AZYD312*186

3AZYD379*008 3AZYD379*009

3AZYD379*010 3AZYD379*011

Miscellaneous

Updated Final Safety Analysis Report, Section 9.5, Revision 14

Specification 13-AN-0340, Technical Specification for The Installation of Penetration Seals,

Revision 9

NUREG 1552, Fire Barrier Penetration Seals in Nuclear Power Plants, July 1996

-A5- Attachment

Section 1R07: Heat Sink Performance

Procedures

Number Title Revision73DP-9ZZ11 Heat Exchanger Condition Monitoring 6

74DP-9CY03 Chemistry Control Instruction 5

70TI-9EW03 EW Heat Exchanger Improved Test Performance 0

70TI-9EW01

Thermal Performance Testing of Essential Cooling Water Heat Exchangers

90DP-0IP10 Condition Reporting 43

40DP-9OP26 Operability Determination and Functional Assessment 23

01DP-0AP12 Palo Verde Action Request Processing 10

Calculations

Number Title Revision13-MC-ZY-527 Outside Area - Essential Spray Pond System 5

01-MC-SP-501 Essential Spray Pond System E.C.W.S Heat Exchanger Train A

and B 0

13-MC-DG-411 DG Heat Exchanger Minimum Flow Rate vs. Inlet SP Water Temperature

Drawings

Number Title Revision01-M-SPP-001 P&I Diagram Essential Spray Pond System

SWMS#2870151 Operability Determination #305

M018-351-1 Lube Oil Heat Exchanger Data Sheet

PVARs

3102356

3301495*

3299202*

3300578*

3300843*

3102356

3197571

3197571

CRDRs

2748191

2897810

2980620

2902498

2905161

2748191

2913430

2928230

2945423

3067255

3094374

2913430

-A6- Attachment

3123168

3206115

3257588

3262892

3279750

3123168 9-5-0125

9-5-0125

CRAIs

2905247

3060342

3087384

3090929

3098735

2905247

3123169

3267418

3272467

3123169

Miscellaneous

Essential Spray Pond System Chemistry Control Strategy, Rev. 4

Essential Cooling Water System Training Manual, Volume 7, Rev. 5

Essential Spray Pond System Training Manual, Volume 6, Rev. 3

Essential Spray Pond System Design Basis Manual, Rev. 18

Letter dated January 26, 1990 response to GL 89-13

Section 1R11: Licensed Operator Requalification Program

Procedures

Number Title RevisionSES-0-09-AP-01 Operator Requalification Training Scenario

EPIP-99 Emergency Plan Standard Appendices

40EP-9EO02 Reactor Trip

40EP-9EO09 Functional Recovery 39

Section 1R12: Maintenance Effectiveness

Procedures

Number Title Revision30DP-9MP02 Fastener Tightening / Preload

32MT-9ZZ84 AC Motor Operational Testing

41AL-1RK1C DG RMS HVAC TRBL

30DP-9MP03 System Cleanliness and Foreign Material Exclusion Controls

72ST-9RX11 COLSS Margin Alarms

72ST-9RX03 DNBR/LHR/AZTILT/ASI With COLSS Out of Service

40ST-9ZZ35 PDIL Alarm Circuit Operability Surveillance 12

-A7- Attachment

40AO-9ZZ15 Loss of Annunciators

02DP0ZZ02 PVNGS Site Tagging Standard

01DP-0AP12 Palo Verde Action Request Processing

74DP-9CY04 Chemistry Department Practices 1

PVARs

3268162

3203273

3135202

3134910

3187783

3192705

3188752

3198796

CRDRs

3269649

3206115

3255289

Work Orders

268167

3268182

3269405

2880364

3292478

3270879

2743415

Miscellaneous

Engineering Evaluation

3275138, Evaluation of EDG 1A Essential Fan/Motor 1 MHDAA01 Degraded Coupling, February 18, 2009

Recommendation to Downgrade

CRDR 3269649 from Significant to Apparent Cause, January 16, 2009

Charter and Pre-Evaluation Tailboard Checklist for

CRDR 3269649

DFWO

3268182 Disposition, January 6, 2009

VTD-R233-00003, Vendor Technical Document Rexnord Installation Instructions for Thomas Type DBZ,

DBA-A,

DBZ-B Couplings, Revision 3

Engineering Evaluation

3269405, EDG 1A Availability with Essential AHU 1MHDAA01 Out of Service, January 8, 2009

13-VTM-A220-0002, Vendor Technical Manual for American Air Filter Air Handling Units Volumes 1 and 2, Revision 4

Units 1, 2, and 3 Emergency Diesel Generator Equipment Room Essential Air Handling Units Motor Vibration History Reactor Plant Event Notification Worksheet 44924, Unit 2, dated March 21, 2009

13-MC-SP-0309, "Essential Spray Pond System Chemistry Design Basis Analysis Calculation," Revision 0

Section 1R13: Maintenance Risk Assessments and Emergent Work Control

-A8- Attachment

Procedures

Number Title Revision70DP-0RA05 Assessment and Management of Risk When Performing Maintenance in Modes 1 and 2

40DP-9OP02 Conduct of Shift Operations 43

01DP-0ZZ01 Systematic Troubleshooting 3

30DP-9MP01 Conduct of Maintenance 57

PVARs

3268162

2979877

3158083

3131500

3119261

3094118

3107443

2976497

TSSCRs

3268206

Work Orders

268167

3268182

3269405

2880364

3076677

3096995

3132108

3123779

3094704

3078516

3179579

3094636

3034905

3094671

3094616

3090968

3119489

3118352

3118960

3179721

3035796

3163059

3150929

3169552

3008429

3160083

3032464

3034867

3122112

3185326

3092608

3118949

2967686

3133025

3211738

3135026

3118963

3187521

3183525

3143319

3204076

3210325

3183074

3263186

3273068

3273069

3042679

3198756

3132616

3205721

3217428

3263188

3057335

3068142

3096050

3269642

3011577

Miscellaneous

Schedule Tracker for Week of 01/05/2009 Cycle 65 Week 06 (A) Train Scheduler's Evaluation of PV Unit 1 for week of 01/05/2009

Scheduler's Evaluation of PV Unit 1 for week of 01/12/2009

Scheduler's Evaluation of PV Unit 2 for week of 01/12/2009

Scheduler's Evaluation of PV Unit 3 for week of 01/12/2009

SRP Switchyard Work for Jan 03 thru Jan 10

Section 1R15: Operability Evaluations

Procedures

-A9- Attachment Number Title Revision40AO-9ZZ11 CEA Malfunctions 13

Drawings

Number Title Revision/Date02-J-SGL-008

Control Logic Diagram Nitrogen Supply Valves for Atmospheric Dump Valves

PVARs

3200564

3249004

3258988

3249757

3277070

3297495

3928124

3187026

3257843

CRDRs

3301009

3187316

3260048

CRAIs

3301010

3294351

3197919

3197968

3197970

3197992

3197994

3266067

Work Orders

201186

Miscellaneous

Chemistry Timeline for November 13, 2008 High Chlorine Event and 2008 History Night Order January 28, 2009 for Over-Chlorination of Unit 2 Spray Pond Chemistry Control Instruction,09-009, January 1, 2009

Unit 2 Essential Spray Pond A pH Trend, November 2008

Unit 2 Essential Spray Pond A Chlorine Trend, November 2008

Unit 2 Essential Spray Pond A Dispersant Trend, November 2008

Calculation of Scaling Probability of Unit 2 Essential Spray Pond A, November 13, 2008

Potential Deficiencies in Control Rod Drop Analysis Prompt Operability Determination Summary including Palo Verde Action Requests

3187026,

3257843, and

3298124, Revision 3

Potential Deficiencies in Control Rod Drop Analysis Prompt Operability Determination Summary for Palo Verde Action Requests

3187026, Revision 0

-A10- Attachment Potential Deficiencies in Control Rod Drop Analysis Prompt Operability Determination Summary for Palo Verde Action Requests

3187026, Revision 1

Potential Deficiencies in Control Rod Drop Analysis Prompt Operability Determination Summary for Palo Verde Action Requests

3187026 and

3257843, Revision 2

Nuclear Analysis calculation,

NA-13-C00-2002-005.01

Updated Safety Analysis Report, Section 15.4.3, Single Full Strength Control Element Assembly Drop, Revision 14

Degraded or Non-Conforming Condition Discovery Checklist, Dated June 10, 2008

CEA Drop Immediate Operability Assessment, Dated June 10, 2008

Adverse

CRDR 3187316 Evaluation, Revision 0

Adverse

CRDR 3260048 Evaluation, Revision 0

Section 1R19: Post-Maintenance Testing

Procedures

Number Title Revision30DP-9MP02 Fastener Tightening / Preload 7

2MT-9ZZ84 AC Motor Operational Testing 21

2MT-9ZZ84 AC Motor Operational Testing 25

37MT-9ZZ01 Vibration Survey 13

30DP-9MP11 Field Use of Rigging 27

30DP-9MP03 System Cleanliness and Foreign Material Exclusion Controls 13

73ST-9DG08 Class 1E Diesel Generator Load Rejection, 24 Hour Rated Load and Hot Start Test Train B

01DP-0ZZ01 Systematic Troubleshooting 3

30DP-9MP01 Conduct of Maintenance 57

40AL-9RK6B Panel B06B Alarm Response GEN STATOR CLG WTR SYS TRBL 3

40OP-9CE01 Stator Cooling System 13

73ST-9CL03 Containment Airlock Door Seal Leak Test 22

30DP-9MP03 System Cleanliness and Foreign Material Exclusion Controls 15

Drawings

-A11- Attachment Number Title Revision13-J108-579-1 Schematic Diagram

DC-1K12-AS Inverter

B 13-J108-252 Wiring Diagram Cabinet

A 13-J108-14 Power Distribution Block Diagram

B 13-J108-9 System Arrangement -

Annunciators

A 13-J108-274-1 Voltage Sense Adjustment

A 13-J108-80-13 Terminal Arrangement

O 13-J108-31-22 Terminal Arrangement

Q 13-J108-30-20 Terminal Arrangement

O 13-J108-32-24 Terminal Arrangement - Equipment Bay 6

Q 13-J108-33-18 Terminal Arrangement

O 13-J108-34-15 Terminal Arrangement

13-J108-35-22 Terminal Arrangement - Equipment Bay 7

Q 13-J108-39-10 Bays 5-8 Equipment Cubicle Mechanical Ass-Annun

Q 13-J108-40-20 Terminal Arrangement

Q 13-J108-41-15 Terminal Arrangement

O 13-J108-42 Terminal Arrangement Bay A Front, Annunciator

01-M-CEP-001 P&I Diagram Stator Cooling 14

Work Orders

268167

3269405

3268182

2880364

3248452

2835485

257945

3275395

3293309

3272459

3290246

3121777

292478

3270789

2743415

3303356

3303622

3303624

3304150

3305634

3305455

3304150

3277584

3102833

3094380

3303624

3303622

3304150

PVARs

3268162

3303292

33032924

CRDRs

3269649

3240757

-A12- Attachment

Miscellaneous

Engineering EWR's

3269405,

3303562 and

3304150

Unit 1 Stator Cooling Degraded Flow Troubleshooting Game Plan Palo Verde Nuclear Generating Station Permit

161042

Engineering Document Change 2005-00569

Trouble-shooting game Plan for Unit 1 Stator Cooling Water Degraded Flow

Section 1R22: Surveillance Testing

Procedures

Number Title Revision73ST-9SI10 HPSI Pumps Miniflow - Inservice Test 41

36ST-9RC02 Containment Building Sump Instrumentation Calibration 22

74ST-9SQ08 RMS Monthly Channel Check 2

73ST-9AF02-1

AFA-P01 Recirc Flow - Inservice Test

73ST-9XI14-1 Train B HPSI Injection and Miscellaneous SI Valves - Inservice Test 25

73ST-9AF03-1

AFB-P01 Recirc Flow - Inservice Test 20

73ST-9XI40 HC Valves - Inservice Test 4

73ST-9SG05 ADV Nitrogen Accumulator Drop Test 28

73ST-9XI20 ADV's In-Service Test 23

Drawings

Number Title Revision13-C-CTS-701 Condensate Storage Tank Liner Plate Plans Sections, and Details 8

03-M-CTP-001 P & I Diagram Condensate Storage and Transfer System 11

13-C-CTS-703 Condensate Storage Tank Liner Plate Elevation, Plans Sections, and Details

13-J-03K-065 Condensate Storage Tank Level Setting Diagram 7

13-J-03D-135 Outside Area Isometric

CTA-LT-35 &

CTB-LT-36 Sensing Line 2

-A13- Attachment Number Title Revision13-J-ZYF-005 Instrument Location Plan Condensate Water Storage Tank Area 3

PVARs

3268007

3259901

3200564

3188019

3268007

Work Orders

275110

3142183

3296229

3021591

3089701

3089663

268008

3113314

3113539

3113422

3072058

3094861

3094695

3094612

3154351

3268008

Miscellaneous

Unit 2 Control Room Logs - January 4, 2009

Unit 3 Control Room Logs - January 13, 2008 - March 13, 2009

Design Basis Manual, SQ System, Revision 12

Design Basis Manual, AF System, Revision 17

Updated Safety Analysis Report, Section 10.4.9, Revision 14

Trouble-Shooting Game Plan for Unit 2

ADV-0178 failing open during 73ST-9XI20, dated 1/4/09

Section 1EP1: Exercise Evaluation

Procedures

Number Title Revision16DP-0EP19 Performance Indicator Emergency Preparedness Cornerstone

16DP-0EP20 Emergency Planning Conduct of Operations

EPIP - 02 Operations Support Center Actions

EPIP - 04 Emergency Operations Facility Actions

EPIP - 59 Emergency Planning Training Program Description 6

CRDRs

2981601

3123374

3144307

3144370

3145062

2981601

3145063 3196481

Section 1EP6: Drill Evaluation

Procedures

-A14- Attachment Number Title RevisionEPIP-02 Operation Support Center Actions

EPIP-03 Technical Support Center Actions

EPIP-04 Emergency Operations Facility Actions

EPIP-14 Dose Assessment

EPIP-99 Emergency Plan Implementing Procedure Standard Appendices, Appendix B, Protective Action Recommendations

EPIP-99 Emergency Plan Implementing Procedure Standard Appendices, Appendix D, Notifications

Section 4OA1: Performance Indicator Verification

Procedures

Number Title Revision71DP-0PI01 Performance Indicator Data Mitigating System Cornerstone

93DP-0LC09 Data Collection and Submittal Using INPO's Consolidated Data Entry System

73DP-9ZZ08 Thermal Performance Monitoring and Reporting 3

73DP-9PP01 Thermal Performance Monitoring and Evaluation Process 5

MISCELLANEOUS

NUMBER

TITLE REVISION / DATE

Palo Verde Nuclear Generating Station, Siren Operating Manual

November 1, 2008Work Order

216191

Palo Verde EPZ Siren Test October 8, 2008

Notification Siren System

ERO Team Full Scale Drill Report 1/23/08 9/25/08 11/19/08 12/10/08

-A15- Attachment MISCELLANEOUS

NUMBER

TITLE REVISION / DATE

Emergency Preparedness Call-in Drill Report 1/24/08 2/28/08 3/19/08 4/23/08

Emergency Preparedness Evaluated Scenario Report 3/5/08

Emergency Preparedness Drill Report 5/21/08

Emergency Preparedness Augmentation Drill Report 6/26/08 7/18/08

8/12/08 9/28/08 10/16/08 11/21/08 12/28/08

Section 4OA2: Identification and Resolution of Problems

Procedures

Number Title Revision01DP-0AP12 Palo Verde Action Request Processing 4 01DP-0AP12 Palo Verde Action Request Processing 5 01PR-0AP04 Corrective Action Program 0

01DP-0AP16 PVNGS Self-Assessment and Benchmarking 0 60DP-0QQ02 Trend Analysis and Coding 18

Miscellaneous

System Health Reports, January and February, 2009 PVNGS Monthly Trend Report, January and February 2009 PVNGS SIIP Performance Indicators, January and February 2009

Section 4OA3: Follow-up of Events and Notices of Enforcement Discretion

Procedures

Number Title Revision40AO-9ZZ19 Control Room Fire

40AO-9ZZ19 Control Room Fire 16

-A16- Attachment Drawing

Title Revision01-E-RCB-010 Elementary Diagram Reactor Coolant System Pressurizer Backup Heaters

02-E-RCB-010 Elementary Diagram Reactor Coolant System Pressurizer Backup Heaters

03-E-RCB-010 Elementary Diagram Reactor Coolant System Pressurizer Backup Heaters

03-E-PGB-021 Elementary Diagram 480V Class 1E Power System 0

PVARs

3153387

2979274

CRDRs

3038288

3153212

3094341

2906584

2892720

2937248

2947667

2959958

2969930

2980261

2981674

Miscellaneous

TA-13-C00-2000-004, CENTS Computer Code Analysis of the 10

CFR 50, Appendix R, Fire Protection Scenarios, Revision 03

2-03407-WLS/AKK/GAM, Individual Plant Examination of External Events (IPEEE) for Severe Accident Vulnerabilities (Generic Letter No. 88-20, Supplement 4)

Component data sheet for EDG Speed Control Governor

ESI Daily Field Service Report dated March 7, 2007 for EDG frequency oscillations Woodward Governor 2301 Speed Control Governor Vendor Technical Manual

Section 4OA5: Other Activities

Procedures

Number Title Revision14FT-9FP70 Appendix R & Former Tech Spec Penetration Seal Surveillance 7 73DP-0FP01 Fire Protection Test Program Requirement 6

14DP-0Fp02 Fire System Impairments and Notifications 14

31MT-9ZZ12 Replacement/Rework of Penetration and Internal Conduit Seals 7

PVARs

-A17- Attachment

3206120

3295124

CRDRs

3206882

3100593

2970134

CRAIs

2970135

Work Orders

23120

772558

Miscellaneous

Engineering Calculation 13-AC-ZZ-0200 for Fire Penetration Seal Qualifications