IR 05000528/2006008

IR 05000528-06-008 IR 05000529-06-008 and IR 05000530-06-008 on 1/17-3/28/2006 for Palo Verde Nuclear Station Units 1, 2, and 3; Biennial Identification and Resolution of Problems

ML061310033
Person / Time
Site:	Palo Verde
Issue date:	05/10/2006
From:	Gody A Operations Branch IV
To:	James M. Levine Arizona Public Service Co
References
IR-06-008
Download: ML061310033 (53)

Text

SUBJECT:

PALO VERDE NUCLEAR STATION - NRC PROBLEM IDENTIFICATION AND RESOLUTION INSPECTION REPORT 05000528,529,530/2006008

Dear Mr. Levine:

From January 17 through February 3, 2006, the U. S. Nuclear Regulatory Commission (NRC)

conducted the onsite portion of a team inspection at your Palo Verde Nuclear Station. From February 6 through March 28, 2006, the team inspection was completed in the Region IV offices. The enclosed report documents the inspection findings, which were discussed with your staff as described in Section 4OA6 of this report.

This inspection examined activities conducted under your license as they relate to the identification and resolution of problems, and compliance with the Commission's rules and regulations and the conditions of your operating license. The team reviewed approximately 240 condition reports and work orders, associated root and apparent cause evaluations, and other supporting documents. The team reviewed cross-cutting aspects of NRC and licensee-identified findings and interviewed personnel regarding the condition of a safety conscious work environment at the Palo Verde Nuclear Station. The team also reviewed the corrective actions associated with the substantive cross-cutting issues in human performance and problem identification and resolution, identified by the NRC in the 2004 Annual Assessment letter dated March 2, 2005 (ML050610294), and in the 2005 Mid-Cycle Assessment letter dated August 30, 2005 (ML052430131). In addition, the team also reviewed your corrective actions in the emergency preparedness program associated with a Severity Level III violation cited in NRC Inspection Report 05000528,529,530/2005011.

Overall performance had declined since the last problem identification and resolution inspection. The team identified notable issues in both the processes and procedures of your corrective action program as described below. The team found that established thresholds for identifying and classifying issues were appropriately low, although several instances were identified where adverse conditions were not entered into the corrective action program for evaluation. Programmatic goals for completion of problem evaluations, consistent with industry standards, were routinely not met because of process problems and lack of management enforcement of timeliness goals. Ineffective and incomplete corrective actions led to a number of repeat problems that could have been prevented. Untimely problem evaluations and

Arizona Public Service Company -2-corrective actions continued to result in a significant number of self-disclosing and NRC-identified findings and violations. The team concluded that while a safety-conscious work environment exists at your facility, isolated concerns were raised by your staff during the interviews. These concerns were associated with not having sufficient personnel to accomplish long-term improvements, a loss of trust that management would not subject the staff to negative consequences for raising issues, some confusion about when to place an adverse condition into your corrective action program, and a decrease in confidence that the corrective action program will adequately address problems.

The Performance Improvement Plan that you developed to address the substantive cross-cutting issues in human performance and problem identification and resolution appears to be extensive, however, the inspection results indicate that it is too early in the implementation of corrective actions in the plan to make an assessment of its effectiveness. Also, the corrective actions taken have not significantly reduced observed error rates. As stated in our 2005 Annual Assessment letter to you dated March 2, 2006, the NRC will continue to evaluate your corrective actions to address these issues. A problem identification and resolution inspection has been planned to be conducted in early 2007 to perform this evaluation.

The report documents five findings that were evaluated under the risk significance determination process as having very low safety significance (Green). The NRC has also determined that violations were associated with all five of these findings. In addition, two licensee-identified violations determined to be of very low safety significance are also listed in this report. The violations are being treated as noncited violations because they were of very low safety significance and because they were entered in your corrective action program consistent with Section VI.A of the Enforcement Policy. If you contest the violations or the significance of the violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U. S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C. 20555-0001, with copies to the Regional Administrator, U. S. Nuclear Regulatory Commission, Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas, 76011; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555-0001; and the NRC Resident Inspector at the Palo Verde Nuclear Station.

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records component of NRC's document system (ADAMS). ADAMS is accessible from the NRC web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Anthony T. Gody, Chief Operations Branch Division of Reactor Safety

Arizona Public Service Company -3-Docket: 50-528, 529, 530 License: NPF-41, NPF-51, NPF-74

Enclosure:

NRC Inspection Report 05000528; 529 ; 530/2006008 w/Attachment: Supplemental Information

REGION IV==

Docket.: 50-528, 529, 530 License: NPF-41, NPF-51, NPF-74 Report No.: 05000528,529,530/2006008 Licensee: Arizona Public Service Facility: Palo Verde Nuclear Generating Station Location: 5951 S. Wintersburg Tonopah, Arizona Dates: January 17 through March 28, 2006 Team Leader: R. Lantz, Senior Operations/Emergency Preparedness Inspector Operations Branch Inspectors: S. Garchow, Operations Engineer, Operations Branch M. Haire, Operations Engineer, Operations Branch R. Azua, Reactor Inspector, Project Branch C J. Larsen, Security Inspector, Plant Support Branch G. Werner, Senior Reactor Inspector, Project Branch D D. Dumbacher, Resident Inspector, Project Branch B P. Benvenuto, Resident Inspector, Project Branch D Approved By: Anthony T. Gody, Chief Operations Branch Division of Reactor Safety-1- Enclosure

SUMMARY OF FINDINGS IR 05000528,529,530/2006008; 1/17-3/28/2006; Palo Verde Nuclear Station Units 1, 2, and 3; Biennial Identification and Resolution of Problems.

The inspection was conducted by two senior reactor inspectors, three reactor inspectors, two resident inspectors, and a security inspector. Five Green noncited violations were identified during this inspection. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.

Identification and Resolution of Problems The inspectors reviewed approximately 175 condition reports, 65 work orders, associated root and apparent cause evaluations, and other supporting documentation to assess problem identification and resolution activities. Overall, performance declined when compared to the previous problem identification and resolution assessment. Significant delays in evaluation of the significance of an identified problem, as well as identification of appropriate corrective actions, resulted in large corrective action backlogs, some repeat events, and examples of continued noncompliance. The delays in completion of corrective actions continued to result in a significant number of self-disclosing and NRC-identified violations and findings. While the licensee initiated actions to address the substantive cross-cutting issues in human performance and problem identification and resolution, the majority of the corrective actions were not complete and some of the initial completed actions were not effective. Also, competing priorities between resources and the backlog of corrective actions created a condition where many corrective actions were significantly delayed in their completion, contributing to failures to adequately implement the corrective action process.

The team concluded that while a safety-conscious work environment exists at your facility, isolated concerns were raised by your staff during the interviews. These concerns were associated with not having sufficient personnel to accomplish long-term improvements, a loss of trust that management would not subject the staff to negative consequences for raising issues, some confusion about when to place an adverse condition into your corrective action program, and a decrease in confidence that the corrective action program will adequately address problems. (Section 4OA2).

A. Inspector-Identified and Self-Revealing Findings Cornerstone: Mitigating Systems

• Green. A self-revealing, noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," was identified for the failure to correct, and preclude repetition of, a significant condition adverse to quality involving multiple failures of the turbine driven auxiliary feedwater pump. Specifically, the licensee-2- Enclosure

failed to perform a timely evaluation to determine the cause of the Units 2 and 3 turbine driven auxiliary feedwater pump governor power supply resistor failures.

Approximately 7 months following the Unit 2 and 3 failures, the Unit 2 turbine driven auxiliary feedwater pump governor failed again because of the same resistor failure. The licensee entered the deficiency into their corrective action program as Condition Report Disposition Request 2871541 for resolution.

The finding is more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the availability of systems that respond to initiating events. The failure of the Unit 2 turbine driven auxiliary feedwater pump governor power supply resistor affected the availability of the auxiliary feedwater system. Using the Phase 1 worksheet in Manual Chapter 0609, "Significance Determination Process," the finding is determined to have very low safety significance because it only affected the mitigating systems cornerstone and did not result in an actual loss of safety function. The cause of the finding is related to the cross-cutting element of problem identification and resolution, in that, delays in the evaluation of the resistors failures allowed a subsequent failure prior to completion of the corrective actions. (Section 4OA2e(2)(i))

• Green. A noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI,

"Corrective Action," was identified for the failure to correct an adverse trend of contaminated oil samples in a timely manner. Specifically, on April 1, 2005, the licensee identified an increasing trend of incorrect lubricant oil additions and contaminated oil samples and entered the deficiency in their corrective action program. As of January 2006, the frequency of oil control problems documented in the corrective action program had not decreased. The inspectors concluded that the corrective actions taken to correct the identified oil control deficiencies were not adequate. The licensee entered the deficiency into their corrective action program as Condition Report Disposition Request 2785915 for resolution.

The finding is more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the reliability and availability of systems that respond to initiating events. Using Manual Chapter 0609,

"Significance Determination Process," Phase 1 worksheet, the finding was determined to have very low safety significance because it only affected the mitigating systems cornerstone and did not result in the loss-of-safety function of a single train or system. The cause of the finding is related to the cross-cutting elements of human performance and problem identification and resolution, in that, poor work practices resulted in multiple oil contamination events and the corrective actions taken were ineffective in promptly correcting the condition.

(Section 4OA2e(2)(ii))

• Green. A noncited violation of 10 CFR Part 50, Appendix B, Criterion XI, "Test Control," was identified for the failure to perform required testing of the Unit 3 essential cooling water system Pump EWP01 breaker in accordance with requirements and acceptance limits. Pump EWP01 breaker test procedure established tolerances and acceptance criteria for the breaker sub-component-3- Enclosure

clearances that were documented as not being met. The licensee entered the deficiency into their corrective action program as Condition Report Disposition Request 2865792 for resolution.

This finding was more than minor since it affected the equipment performance attribute of the Mitigating Systems cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609,

"Significance Determination Process," Phase 1 worksheet, the finding was determined to have very low safety significance because the condition was a qualification deficiency confirmed not to result in loss of function. The cause of the finding is related to the cross-cutting elements of human performance and problem identification and resolution, in that, maintenance personnel failed to properly implement maintenance procedures, and the deficient conditions were not identified by supervisory review of the completed procedures.

(Section 4OA2e(2)(iii))

• Green. A noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI,

"Corrective Action," was identified for the failure to identify and correct a condition adverse to quality involving auto-testing of safety-related relays.

Specifically, for approximately 100 days, between May 5 and July 14, 2004, and again between May 2 and June 3, 2005, the Unit 1 "A" Balance of Plant Engineered Safety Feature Actuation System sequencer was placed in continuous auto-test as a corrective action to assist in diagnosing reliability issues. Approximately 3 months later, on October 10, 2005, during testing, the sequencer failed to shed one of the loads as required. The long-term use of the continuous auto-test feature was determined to be the most likely cause of the relay failure. The licensee entered the deficiency into their corrective action program as Condition Report Disposition Request 2796883 for resolution.

The finding is greater than minor because it affects the equipment performance attribute of the Mitigating Systems cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609,

"Significance Determination Process," Phase 1 worksheet, the finding is determined to have very low safety significance because the finding did not result in the loss of safety function of any component, train, or system. The cause of the finding is related to the cross-cutting element of problem identification and resolution in that the licensee failed to adequately evaluate and correct a condition adverse to quality. (Section 4OA2e(2)(iv))

• Green. A noncited violation of 10 CFR 50.65(a)(2) was identified for the failure to demonstrate that the performance or condition of the low pressure safety injection/shutdown cooling Pump 2A was adequate. Specifically, in May 2005, the licensee failed to accurately account for 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> of unavailability time for the low pressure safety injection/shutdown cooling Pump 2A, which when re-evaluated, exceeded the performance trigger to enter (a)(1) monitoring. The licensee entered this deficiency into their corrective action program as Condition Report Disposition Request 2865315 for resolution.

-4- Enclosure

The finding is more than minor because it affects the equipment performance attribute of the mitigating systems cornerstone objective to maintain availability and reliability of structures systems and components needed to respond to initiating events and had a credible impact on safety. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 worksheet, the finding is determined to have very low safety significance because there was no design deficiency and the low pressure safety injection/shutdown cooling Pump 2A failure did not exceed the allowed technical specification outage time.

The cause of the finding is related to the cross-cutting element of human performance in that the initial evaluation and subsequent supervisory reviews failed to identify the need for additional monitoring of the low pressure safety injection/shutdown cooling Pump 2A. (Section 4OA2e(2)(v))

Cornerstone: Emergency Preparedness

• N/A. The inspectors assessed the licensees evaluation associated with the change to radiological emergency action levels, which decreased the effectiveness of the emergency plan. This performance deficiency was previously characterized as a Severity Level III violation of 10 CFR 50.54(q) in NRC Inspection Report 05000528,529,530/2005011. The inspectors determined that the licensee satisfactorily evaluated the Severity Level-III violation. The licensees evaluation identified two root causes of the performance deficiency:

(1) failure to ensure adequate radiation protection expertise review of the emergency action levels changes that were made to Procedure EPIP-99,

"Standard Appendices," Revision 2, because of inadequate radiation protection expertise within the emergency planning organization and failure to conduct a required cross-organizational review, and (2) failure of management to address knowledge and ability challenges within the emergency planning organization resulting from attrition of health physics/radiation protection experienced personnel, inadequate training on procedure change requirements, and inadequate management of workload. The inspectors concluded that the licensees evaluation and implemented corrective actions were appropriate to reasonably prevent recurrence of the 10 CFR 50.54(q) violation.

Given the licensees acceptable performance in addressing the performance deficiency, the Severity Level III violation is closed. (Section 4OA5)

B. Licensee-Identified Violations.

Two violations of very low safety significance, which were identified by the licensee, have been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensees corrective action program. The violations and corrective actions are listed in Section 4OA7 of this report.

-5- Enclosure

REPORT DETAILS 4 OTHER ACTIVITIES (OA)

4OA2 Identification and Resolution of Problems The inspectors based the following conclusions, in part, on all issues that were identified in the assessment period, which ranged from June 1, 2004, (the last biennial problem identification and resolution inspection) to the end of the inspection on March 28, 2006.

The issues are divided into two groups. The first group (Current Issues) included problems identified during the assessment period where at least one performance deficiency occurred during the assessment period. The second group (Historical Issues) included issues that were identified during the assessment period where all the performance deficiencies occurred outside the assessment period.

a. Effectiveness of Problem Identification (1) Inspection Scope The inspectors reviewed items selected across the seven cornerstones to determine if problems were being properly identified, characterized, and entered into the corrective action program (CAP) for evaluation and resolution. The inspectors performed field walkdowns of selected systems and equipment to inspect for deficiencies that should have been entered in the CAP. The inspectors also observed control room operations and reviewed operator logs, plant tracking logs, and station work orders to ensure conditions adverse to quality were being entered into the CAP. Additionally, the inspectors reviewed a sample of self assessments, trending reports, system health reports, and various other documents related to the CAP.

The inspectors interviewed station personnel, attended condition review committee and corrective action review board meetings, and evaluated corrective action documentation to determine the licensees threshold for entering problems in their CAP. In addition, the inspectors reviewed the licensees evaluation of selected industry operating experience information, including operator event reports, NRC generic letters and information notices, and generic vendor notifications to ensure that issues applicable to Palo Verde Nuclear Station were appropriately addressed.

(2) Assessment The inspectors determined that the licensee's problem identification aspect of the CAP had issues in several general areas. These issues included narrowly focused root cause evaluations, informal and inconsistent use of internal and external operating experience, ineffective use of equipment and human performance trending tools, and ineffective supervisory reviews. The CAP database, "Site Wide Management System (SWMS)," although capable, was not fully utilized because of difficulties in using the system. Problems were generally identified and placed into the corrective action program at an appropriate threshold, however, the evaluation of the appropriate-6- Enclosure

threshold was not timely in a majority of instances reviewed, partially because of a lack of strong management commitment to enforce timely evaluations, as well as a programmatic attribute that encouraged delayed evaluations. Interviews of plant staff revealed some confusion regarding when a condition should be placed into the corrective maintenance system rather than the condition reporting system.

On March 23, 2006, the inspectors received the results of a performance improvement self-assessment report, and the associated planned corrective actions in response to the corrective action timeliness concerns that the inspectors debriefed on February 3, 2006. The licensee indicated that improvements in performance had been observed since programmatic changes were made to the corrective action process in February 2006. A review of the effectiveness of those actions was not conducted during this inspection.

As illustrated in the examples and observations listed below, a notable number of NRC-identified and self-disclosing issues were documented during the period. This reflected a decline in performance when compared to the previous problem identification and resolution assessment (NRC Inspection Report 05000528; 529; 530/2004006).

Some problems were not identified and entered into the CAP at the first opportunity and in other instances NRC involvement was required to ensure adverse conditions were appropriately addressed under the CAP. The trend of NRC-identified findings evaluated as having problem identification and resolution (PI&R) aspect in identification of problems has been fairly steady since 2004, with four findings in 2004 and three in 2005. The inspectors concluded that the evaluation area of effectiveness of problem identification continued to challenge the organization.

Current Issues Example 1: Licensed and non-licensed operations personnel failure to follow an emergency operating procedure, which resulted in a loss of the positive displacement charging pump. The licensee failed to identify procedural adherence issues of the control room supervisor until the NRC questioned the operator actions. (NRC Inspection Report 05000528,529,530 (IR) 2004013).

Example 2: During the loss-of-offsite power event, a risk assessment was not performed to determine which units turbine driven auxiliary feedwater (TDAFW) pump steam traps should be drained first. Consequently, the licensee inappropriately drained Unit 1 TDAFW pump before Unit 2 TDAFW pump, which had a higher risk profile because of an inoperable emergency diesel generator (EDG). This issue was not identified in the CAP program until prompted by the NRC. (IR 2004013)

Example 3: The licensee failed to identify that maintenance technicians used inadequate procedures during core protection calculator software installation and testing. (IR 2005004)

Example 4: The licensee failed to identify, and properly account for, three detector functional failures occurring from May 2004 to June 2005. Consequently, the licensee did not establish appropriate goal setting and monitoring for the detectors. (IR 2005005)

-7- Enclosure

Example 5: The licensee failed to identify that maintenance personnel were continuing to use guidelines for safety-related activities that had been previously evaluated as obsolete. (IR 2005005)

Observation 1: The SWMS software platform appears adequate to perform the elements required of a CAP, however, based on interviews conducted during the inspection, the software was difficult to use and was therefore not being utilized to support all the functions needed by the organization. For example, tracking and trending limitations in the SWMS software have resulted in development of a separate trend tracking database that requires a manual entry of information from SWMS. The inspectors observed both condition review committee and the corrective action review board meetings, and observed that the licensee relied primarily on the collective memory of the participants rather than the SWMS database to determine if there were past events or history related to the condition report disposition request (CRDR) being evaluated. The inspectors noted that the Operations Department had developed their own internal data base for historical problem identification.

Historical Issues Example 1: Fuel handling personnel failed to recognize that a fuel assembly had contacted a solid object during fuel, and that the fuel damage procedure provided inadequate guidance to consistently evaluate fuel damage. (IR 2004011)

Example 2: Engineering and operations personnel failed to identify the degradation of polyethylene insulating channels on Class 1E station batteries. (IR 2004002)

b. Prioritization and Evaluation of Issues (1) Inspection Scope The inspectors reviewed CRDRs, work orders, and operability evaluations to assess the licensees ability to evaluate the importance of adverse conditions. The inspectors reviewed a sample of CRDRs, apparent and root cause analyses to ascertain whether the licensee properly considered the full extent of causes and conditions, generic implications, common causes, and previous occurrences. The inspectors also attended various meetings to assess the threshold of prioritization and evaluation of issues identified.

In addition, the inspectors reviewed licensee evaluations of selected industry operating experience reports, including licensee event reports, NRC generic letters, bulletins and information notices, and generic vendor notifications to assess whether issues applicable to Palo Verde Nuclear Station were appropriately addressed.

The inspectors performed a historical review of CRDRs and notifications written over the last 5 years that addressed the emergency diesel generators and the reactor coolant system.

-8- Enclosure

(2) Assessment The inspectors concluded that problems were generally prioritized and evaluated in accordance with the licensees CAP guidance and NRC requirements, with the significant exception of meeting timeliness goals stated in the licensee's program guidance and consistent with industry standards. The inspectors found that for the sample of root cause reports reviewed, the licensee was generally self-critical and thorough in evaluating the causes of significant conditions adverse to quality, but the evaluations were not completed in a timely manner when compared to industry standards, and did not meet the goals of the licensee's program.

The inspectors assessed the overall timeliness history for completion of root and apparent cause evaluations. The process for conducting an evaluation involved three main steps: (1) determination of CRDR significance, (2) assignment of an evaluation leader and team, and (3) completion of the evaluation, including identification of the root and contributing causes and associated corrective actions. The inspectors observed that the time utilized for determination of significance varied greatly, but on average, required more than 30 days for all 2005 significant and apparent cause CRDRs. One contributor to this extended time frame was routine use of an interim classification of potentially significant, the use of which has been discontinued since the completion of this inspection. Selection and assignment of the evaluation team required another 2 weeks on average. Finally, completion of the root or apparent cause evaluations required an additional 105 and 68 days, respectively. The inspectors noted that one root cause evaluation required 306 days from significance determination to evaluation completion, and one apparent cause evaluation took 253 days to complete.

The inspectors concluded that the licensee had significant challenges associated with timely completion of significance determinations and root cause evaluations, which subsequently delay corrective action implementation. The licensee had not established a timeliness goal for completion of the significance determination, but did establish a timeliness goal for completion of root cause evaluations of 30 days from completion of the significance determination; even so, of the 198 apparent cause evaluations for 2005, less than 25 percent were completed within that 30-day goal, and of the 43 root cause evaluations for 2005, 5 (12 percent) were completed within 30 days.

The inspectors evaluated the licensee audits and self-assessments as typically critical and thorough, however, the follow-through with corrective actions to address those findings were typically lacking and not timely. The licensee completed a thorough and critical audit of the corrective action program in November 2005. One area reviewed in the audit was response to NRC violations. Condition report disposition requests had been written to enter the problems noted in the audit into the CAP, however, in several instances, the inspectors noted that the only response for some CRDRs as of the time of the inspection was initiation of additional CRDRs, even when the original CRDR effectively captured the problem noted in the new CRDR. This action was not in accordance with the licensee CAP and effectively delayed taking actions on the audit findings. The inspectors concluded that a lack of management focus and accountability on timeliness of the CAP contributed to these excessive times, and allowed significant delays in the completion of corrective actions associated with these issues.

-9- Enclosure

The operability determination process was inconsistently applied, as indicated by several NRC findings related to failure to promptly evaluate conditions for operability. In some instances, the inspectors noted that questions concerning operability would be answered outside the formally described process. The inspectors noted that this practice could remove opportunities for formal evaluation of potentially significant issues.

The trend of NRC identified findings with PI&R aspects in evaluation of problems has been improving since 2004, with seven findings in 2004 and two in 2005, however, the inspectors identified two additional findings during this inspection in the evaluation area.

The problems with the CAP that results in delays in completion of the evaluations did not directly result in regulatory findings in this area, but have contributed significantly to findings in the third PI&R evaluation category (effectiveness of corrective actions).

The inspectors concluded that the evaluation area of prioritization and evaluation of issues is a significant challenge to the organization.

Current Issues Example 1: Engineering personnel identified a nonconforming condition associated with a pressurizer heater modification and initiated a CRDR, but did not inform the shift manager or shift technical advisor until the next day. A prompt operability determination was not completed when the nonconforming condition was initially identified.

(IR 2004005)

Example 2: Engineering and operations personnel failed to consider water intrusion into the electrical conduit for EDG fuel oil transfer pump Train A as a condition that could affect the ability of the EDG to perform its specified function, and consequently, declared EDG Train A operable without performing an operability determination.

(IR 2004005)

Example 3: Engineering and operations personnel failed to implement requirements in the stations condition reporting and operability determination procedures following identification of a potentially degraded condition. Engineering personnel did not forward documents regarding the assessment of a voided condition of the emergency core cooling system in a timely manner and once operations received the information, they did not perform a prompt operability determination. (IR 2004014)

Example 4: The licensee failed to place a degraded main generator excitation limiter circuit into the work control process and, therefore, the degraded condition did not receive an operability determination. (IR 2004013)

Example 5: Engineering personnel failed to do an adequate extent of condition involving failed resistors for TDAFW pump governor power supply. The applicability of the condition to Unit 1 was not evaluated. (IR 2004004)

Example 6: Engineering personnel did not perform an adequate extent of condition review for the improper nozzle dam locking rings. (IR 2004004)

-10- Enclosure

Example 7: The licensee failed to submit two licensee event reports to report shutdowns required by technical specifications. (IR 2005005)

Example 8: The licensee failed to implement corrective actions to preclude repetition of failures of gasket retaining bolts in various safety-related valves. (IR 2005003)

Historical Issues Example 1: During a draindown to midloop in October 2004, there was a level transient caused by an inadequate draindown procedure. A similar event occurred in April 2004 and the licensee failed to specify the correct drain rates in the procedure as part of their corrective actions. Engineering documents were available that had the correct drain rates; however, the licensee failed to identify these documents prior to changing the procedure as a result of the April 2004 event. (IR 2004005)

Observations 1) Inability to Meet Timeliness Goals The inspectors noted multiple examples of failures to meet timeliness goals. A sample of those observations is listed below:

• CRDR 2819047, "Equipment reliability shortfalls," was identified on July 28, 2005, as part of an external evaluation and classified as apparent cause, the second highest significance rating. The inspectors identified during the onsite inspection in January 2006 that the apparent cause evaluation had not yet been completed.

• Significant CRDR 2720228, "Failure of AFW Pump Governor," identified failure of the TDAFW pump on several occasions because of a failed resistor in the governor. The root cause involved evaluation of two events, in May and July 2004. The evaluation due date was August 14, 2004, however, the evaluation was not completed until February 4, 2005.

• CRDR 2624427 was initiated in response to the pressurizer spray valve failure that resulted in a manual reactor trip on July 29, 2003. The NRC issued Noncited Violation 05000529/2004006-03 for procedure deficiencies revealed during the event. The licensee determined that one of the root causes was that the spray valve control design was not tolerant to single failure. The corrective action was to design and install a solenoid valve that would allow the operators to bleed off control air, shutting the valve. The inspectors noted that even though this corrective action had been designated as priority 2 (high priority) with a due date of August 11, 2004, no design work had been started as of January 2006.

• The inspectors reviewed 50 corrective action items associated with significant CRDRs and found 8 had been completed by the due date. In addition, the inspectors reviewed the 56 priority 2 corrective actions that were open and found-11- Enclosure

30 of the 56 actions were overdue. Many of these overdue corrective actions had exceeded their due date extensions. The inspectors also noted the backlog of open restraining corrective actions had been consistently increasing over the last 4 years, from 515 in 2002 to 1156 at year end 2005. A restraining corrective action typically required equipment modification or more than administrative procedure changes.

• CRDR Program Report (October 2005), noted that the number of CRDRs that have been open greater than 180 days has exceeded the station goal for the entire year. The "Timeliness of Corrective Action" window in the Business Plan Metrics for 2005 was red. A red window was defined as a significant weakness.

2) Cause Evaluation Process Problems The inspectors determined there were several contributors to the failure to meet timeliness goals for completion of root cause evaluations.

• A review of six significant CRDRs determined it took an average of 48 days to make a final determination of the significance of the CRDR.

• Interviews with root cause evaluators indicated that it often takes several weeks before they are assigned as the lead evaluator for root cause evaluation.

Interviews with licensee management indicated that identifying resources to perform the evaluation required negotiation with other managers, which contributed to the delay in starting the evaluation. A review of 15 significant CRDRs indicated that the evaluations typically began 6-8 weeks after the initiating event.

• The inspectors concluded that the CRDR evaluation process burdens the lead evaluator with time consuming administrative activities. For example, the root cause leader was required to obtain approval for each individual corrective action from each proposed corrective action owner. If a proposed corrective action included a plant modification, the team leader was required to obtain approval for the modification. These activities could take several weeks depending on the number of corrective actions identified, and all must be completed before the root cause investigation is complete.

• All of the individuals interviewed stated that the 30-day management expectation could not be met given the current CRDR process. The inspectors also noted that the licensee's 30-day evaluation goal was measured from the day the CRDR was classified significant, and not from the time of the event as was the industry standard.

3) Potentially Ineffective Use of Program Resources

• The inspectors reviewed actions taken to address CAP weaknesses identified in licensee audits, self assessments, and NRC assessments. Several changes were made to the CAP procedure, as well as the site organization in December 2005 and January 2006. The inspectors reviewed the changes but could not-12- Enclosure

assess the potential success of those changes since they had been made recently and many were still being implemented.

• The inspectors were concerned with two corrective actions made to address problems observed in initial CRDR significance screening and completion of corrective actions. Recently, the corrective action review board was assigned the responsibility of reviewing all new CRDRs on a daily basis in addition to and as a backup for the condition review committee. Also, four independent reviews (CRDR Responsible Manager, Performance Improvement Group, Quality Assurance, and Corrective Action Review Board) are now required prior to closure of a significant CRDR, however, since these reviews had not solved observed performance deficiencies, a fifth closeout review was being considered. Both of these actions appeared to focus resources on identifying problems with corrective action implementation, instead of preventing problems and improving the timeliness of corrective action completion.

4) Emergency Diesel Generators and Reactor Coolant System Review

• The inspectors reviewed the performance, past issues and corrective actions on two safety significant systems - EDG and the reactor coolant system (RCS). The inspectors identified one noncited violation associated with the use of the auto-test feature of the BOP-ESFAS sequencer, documented in Section 4OA2e(2)(iv) of this report.

• The licensee's most significant concern for the EDG system was electrical parts obsolescence. For example, the automatic voltage regulators have experienced several failures, and vendor support and replacement parts are difficult to obtain.

The system engineer mentioned that funds have been approved for a general upgrade of the electrical components in the EDG system.

• The RCS list of top significant issues includes more than 80 items, 25 of which are classified high priority, and 32 that are not currently being worked. Reactor coolant pump lube oil seal leaks have been a longstanding reliability issue, and several modification attempts have been unsuccessful to correct the problems.

• The inspectors concluded that resource limitations and corrective action backlogs are contributing to delays in completion of identified corrective actions in the EDG and RCS. The inspectors concluded that both of these systems represent significant challenges in the corrective action system.

5) High Pressure Safety Injection Bearing Degradation Evaluation Problems i) Oil Sampling and Bearing Replacement History

• Historically at Palo Verde Nuclear Station, high pressure safety injection (HPSI)

pump oil iron concentrations ([FE+]) typically increase following bearing replacement maintenance from a normal of 5 - 15 ppm to a reading of 20 -

30 ppm, then return to normal. In January 2005, the routine HPSI Pump 3B oil sample results indicated bearing wear particles in the oil with a [FE+] of 80 ppm.

-13- Enclosure

Delays were documented in the CAP associated with the recognition, sample results and the operability call. Interviews with the oil engineer indicated that a

[FE+] of 80 ppm would have likely made the sample appear dark because of immersed particles. The sample had been marked as satisfactory in appearance. The bearing was subsequently replaced in February 2005 because of the elevated [FE+] reading.

• In May 2005 the HPSI Pump 2B's mechanical seal was replaced, which required bearing replacements. The bearings had been in place for a full service cycle of approximately 5 years, during which oil samples/changes had been conducted routinely on a 12-month frequency. Maintenance records did not document any problems associated with these bearings/oil samples, however, inspection of the removed bearings showed abrasive scratches on the bearings.

• In July 2005, the routine oil sample of the HPSI Pump 2B indicated a [FE+] of 171 ppm. The oil sample, which would have been very dark, was documented as a satisfactory sample. As in the January 2005 HPSI 3B Pump example above, delays were documented in the CAP associated with problems obtaining the sample results for use in an operability determination. The shift manager ultimately declared the HPSI Pump 2B inoperable 14 days after the sample was taken, based on the oil engineer's assessment that the pump could not meet its 180-day mission time. A significant CRDR was initiated and the bearings were replaced again, while the evaluation of the source of the elevated [FE+]

continued.

• The bearing vendor supplied an analysis that implied insufficient oil supply and inadequate oil viscous properties, which the licensee disagreed with based on the lack of evidence of adhesive wear, polishing wear and thermal distress.

Approximately 1 month after bearing replacement, the [FE+] was slightly higher than normal, indicating potential abnormal wear was occurring on the bearings.

The licensee then obtained the services of a different bearing and oil consultant, who identified (in October 2005) that aluminum oxide wear particles were cutting the bearings, but the source of the contaminants was unknown. The consultant stated that the HPSI Pump 2B bearings they inspected all had the same type of hard particle damage, which was characterized as unacceptable, but not indicative of a precursor to imminent failure. The licensee had not previously associated the HPSI Pump 3B bearing problems with the HPSI Pump 2B bearings in the corrective action system, but based on the consultant's analysis results, added additional team members to the root cause efforts in January 2006. The root cause evaluation was submitted to nuclear assurance for review on April 19, 2006, but has not been approved as completed as of the issuance of this report. The inspectors did not review the results of the root cause evaluation during this inspection.

ii) Corrective Action Program Observations:

• The licensee had similar oil results on three sets of HPSI Pump 2B bearings and one set of HPSI Pump 3B bearings back to the start of 2005. The licensee-14- Enclosure

missed early opportunities to identify the source of the contaminants, and this resulted in two additional out-of-service windows for HPSI Pump 2B.

• The oil sampling program was weak, in that, it did not identify abnormal visual sample indications, had no threshold for operability evaluations, and only required samples on a 12-month frequency until mid 2005 when the frequency was changed to 6 months.

• In January 2006, even though the root cause was not complete and the source of oil contaminants was not yet identified, routine oil sample frequency was restored to 12 months. As of February 3, 2006, there was no requirement to perform oil samples following HPSI bearing replacements.

• A significant CRDR was not initiated on the HPSI Pump 3B despite the shift managers decision to declare the pump inoperable because of oil sample results.

• As of February 3, 2006, there was still no defined threshold for declaring an HPSI pump inoperable based on high [FE+].

• The root cause evaluation for HPSI bearing degradation, which began in July 2005, was still not approved as completed as of the issuance of this report, over eight months since the evaluation began.

c. Effectiveness of Corrective Actions (1) Inspection Scope The inspectors reviewed plant records, primarily CRDRs and work orders, to verify that corrective actions related to identified problems were developed and implemented, including corrective actions to address common cause or generic concerns. The inspectors sampled specific technical issues to evaluate the adequacy of the licensees operability determinations.

Additionally, the inspectors reviewed a sample of CRDRs that addressed past NRC identified violations, for each affected cornerstone, to ensure that the corrective actions adequately addressed the issues as described in the inspection reports. The inspectors also reviewed a sample of corrective actions closed to other CRDRs, work orders, or tracking programs to ensure that corrective actions were still appropriate and timely.

In both the 2004 Annual Assessment letter dated March 2, 2005 (ML050610294), and the 2005 Mid-cycle Assessment letter dated August 30, 2005 (ML052430131), the NRC identified substantive cross-cutting issues in the areas of human performance and problem identification and resolution. The inspectors evaluated the licensees actions to address the substantive cross-cutting issues.

-15- Enclosure

(2) Assessment The timeliness issues in problem evaluation described above significantly challenge the success of the CAP in the effectiveness area. These challenges inhibit the effectiveness, as well as the timeliness of corrective actions. The inspectors noted instances where corrective actions were closed without completion, where repeat events occurred because of slow or ineffective corrective actions, and other instances where corrective action implementation was delayed with no documented or apparent reason.

The backlog of maintenance issues and other corrective actions continue to present a significant challenge to equipment reliability and corrective action effectiveness. The inspectors noted that while the licensee seems to possess a very good equipment trending tool, none of the approximately 20 engineers interviewed were able to use the tool. Equipment issues trending, when conducted, were routinely kept in personal logs and memory.

The inspectors observed that the licensee had developed an extensive performance improvement plan to address the substantive cross-cutting issues in human performance and PI&R, which included corrective actions and completion due dates.

The evaluation of the issue required a substantial part of the remainder of 2005 to complete, and only a small percentage of the corrective actions as defined in the performance improvement plan have been accomplished. The inspectors identified that many of the planned corrective actions were vague, and would require additional evaluation to identify specific corrective actions. The inspectors also noted that of the corrective actions that had been completed, several were not completed by the projected due dates, or were not fully effective. The inspectors also noted the trend of human performance and problem identification and resolution related problems had remained essentially steady since identification of the cross-cutting issues. The inspectors could not evaluate the potential effectiveness of the actions taken in the performance improvement plan.

The trend of NRC identified findings with PI&R aspects in effectiveness of corrective actions has been fairly steady since 2004, with seven findings in 2004, six in 2005, and one additional finding identified during this inspection in the effectiveness of corrective action area. The inspectors concluded that the area of effectiveness of corrective actions continued to significantly challenge the organization.

Current Issues Example 1: Engineering personnel failed to enter a nonconformity report from the steam generator fabricator into the CAP. Subsequently, an actual steam generator tube leak occurred which would have likely been prevented. (IR 2004009)

Example 2: Operations staff failed to identify and correct spent fuel pool inventory losses. (IR 2004003)

Example 3: The licensee failed to correct a condition associated with an emergency diesel generator excitation circuit failure. This condition prevented the diesel generator from achieving rated voltage within the required time. (IRV 2004013)

-16- Enclosure

Example 4: The licensee failed to perform a licensing document change request and 10 CFR 50.59 screening for the abandonment of the boronometer. (IR 2005004)

Example 5: The licensee failed to identify and correct a deficiency in the method of testing the auxiliary feedwater pump discharge check valves. (IR 2005003)

Example 6: The licensee failed to correct a condition adverse to quality associated with operating charging pumps and the boric acid makeup pump with suction from the refueling water tank. The use of this lineup resulted in a loss of charging flow in July 2004 and February 2005. (IR 2005002)

Example 7: The licensee failed to properly implement procedures for refueling equipment on three separate instances. (IR 2005003)

Example 8: Failure to monitor telltale drains resulted in spent fuel pool leakage to the environment. (IR 2005004)

Example 9: The licensee failed to promptly correct an adverse condition with the refueling water tank instrument pit. (IR 2005005)

Historical Issues Example 1: Engineering personnel failed to promptly correct a programmatic lack of routine inspection and maintenance for essential spray pond system piping and components. The CRDR process was not utilized to evaluate this condition.

(IR 2004004)

Example 2: The licensee failed to correct a condition of fuel handling personnel conducting repeated violations of procedures, as well as long-standing degraded material conditions of fuel handling equipment. (IR 2004011)

Example 3: The licensee failed to promptly correct a degraded condition (lack of lubrication preventive maintenance for remote operators) associated with reach rods on safety-related manual valves. (IR 2004004)

Example 4: An emergency operating procedure failed to provide guidance to ensure that TDAFW pumps were maintained operable following a main steam isolation signal.

The licensee modified surveillance and normal operating procedures in 1990 as a result of TDAFW pump trips, but did not modify emergency operating procedures accordingly.

(IR 2004013)

Observation 1 - Performance Improvement Plan

• Item CRAI 2787233 (completed) required designation of all procedures for "level of use" in response to several instances of failure to follow procedures. The actions were noted as completed in the corrective action system and in Section 4.3.2.2 of Part-2 of the Performance Improvement Plan. The inspectors observed that for designated "combined use" procedures, neither Operations'

-17- Enclosure

nor Reactor Engineering's procedures were marked for level of use in each procedure section as required on the majority of those procedures sampled. The inspectors noted that Operations did have a formal "combined use" policy in the

"Conduct of Operations" procedure, however, the acceptability of that policy to meet the corrective action items action had not been documented in the associated CRDR. Condition Report Disposition Request 2865483 was written to address this observation.

• Item CRAI 2792305 (completed) directed development of leadership expectations for accountability and positive culture change in procedure adherence; the corrective action included a requirement for leaders to document two observations per day to engage leaders in coaching and performance improvement. The inspectors reviewed a sample of documented observations from 15 different leaders, approximately 150 total observations. The majority of the observations did not involve plant work activities, but included meetings, document reviews, and conference calls. The few in-plant observations were typically less than 5 minutes in duration. The inspectors concluded that the observation program, as it was being implemented, did not meet the intent of the original corrective action. During the second onsite inspection week, the licensee instituted a revised observation program that required 10 short observations per day. The inspectors could not assess the potential success of the revised program at this time.

• The inspectors observed that the rate of human performance errors remained essentially unchanged since it was identified as a cross-cutting issue by the NRC in March 2004. According to an internal licensee trend report, the number of human performance events over the last 2 years has remained relatively stable at approximately 10 events per month. Procedure use and adherence have been a specific management focus area over the last year, yet little improvement has been achieved. Some managers interviewed by the inspectors felt procedure use and adherence has actually declined over the last year.

Procedure use and adherence continue to be one of the top three problem areas according to the December 2005 Leadership Effectiveness Program Review.

• A review of the actions taken to date to improve human performance was performed by the team. The inspectors noted the corrective actions for human performance to date rely heavily on generic training and administrative programs. For example, much of the licensee staff was sent to a mandatory procedure use and adherence training class over the last year. In addition, once a month a document called "HP Talking Points" often focuses on procedure use and was read to the site staff by the work group supervisor. However, little evidence of followup and on-the-job reinforcement in specific work groups was found outside the Operations Department.

• There were 40 corrective actions identified in the root cause analysis performed in response to the human performance cross-cutting issue. Of these, approximately one-half have not been completed, and only 6 of those accomplished were completed on or before the scheduled due date. One corrective action established a new human performance organization to improve-18- Enclosure

station performance in December 2005. This group was still getting organized and has not had time to affect any change in job-site behaviors. Because of the timing of the inspection, the inspectors were unable to assess how effective this group will be in improving performance.

• A booklet on management expectations and standards was recently distributed to the licensee staff, however, the inspectors did not observe the booklet being used in the station. This booklet is now undergoing revision. Another corrective action was to establish expectations for procedure change requests and this action was closed after issuing a memorandum. Another corrective action required the use of a human performance simulator to improve job-site behaviors in areas such as self-checking and procedure use. The first step in the associated action plan was to evaluate industry best practices with an assigned due date of November 2005. This action was closed in December 2005 to another correction action, which was still open.

• The inspectors observed that many of the identified corrective actions for human performance issues were generally vague and did not address a specific problem, but required a corrective action based on a future evaluation. For example, Item CRAI 2837280; 4.1.5.1 states, "Use human performance metrics to identify behaviors that are in need of change. Develop Action Plans for improvements for any negative trend or less than top quartile or other industry leading performance." Corrective Action Item 2837298; 4.3.2.4 states, "Program and process changes, resulting from Item CRAI 2830460 efforts, will be developed and initiated." Corrective Action Item 2837286; 4.2.3.2 "Develop a communication strategy to inform staff of the process that will be used to assess the appropriateness of employment-related decisions." The inspectors could not evaluate the potential success of these corrective actions.

Observation 2 - Effect and Status of Corrective Action and Maintenance Backlogs

• The inspectors reviewed the October 2005 Corrective Action Trend Report. The work order backlog was identified as 852 work orders, with an average age of 154 days. The licensees goal was 800 work orders with an average age of 91 days. The inspectors noted that although the number of work orders in the backlog is within 7 percent of the goal, the timeliness goal was exceeded by 70 percent. This is consistent with the observation of timeliness challenges in the corrective action process. According to the October trend report, there were 87 open evaluations older than 30 days, 166 adverse or significant CRDRs open and older than 180 days, and the average closure time was 130 days.

-19- Enclosure

d. Assessment of Safety Conscious Work Environment (1) Inspection Scope The inspectors interviewed approximately 40 individuals from different departments representing a cross section of functional organizations and supervisory and non-supervisory personnel. These interviews assessed whether conditions existed that would challenge the establishment of a safety conscience work environment. The inspectors also reviewed the results of the "2005 Nuclear Safety Culture Assessment" conducted by Synergy Consulting Services.

(2) Assessment The inspectors concluded that a safety conscious work environment exists at the Palo Verde Nuclear Station. Employees felt free to enter issues into the CAP, as well as raise safety concerns to their supervision, the employee concerns program, and the NRC.

A concern was raised with regard to the loss of personnel resources to Performance Improvement Plan staffing. Approximately eight of the interviewees pointed to one of the items identified in the "2005 Nuclear Safety Culture Assessment," i.e., that plant personnel were concerned with having to do "more with less." The interviewees were concerned about the impact the loss of these personnel to the new Performance Improvement Plan organization would have on the day-to-day work accomplishment, which did not seem to resolve the issue identified in the safety culture assessment.

The inspectors received two isolated comments regarding trust of site management and the perception that negative consequences could occur as a result of raising safety issues. All of the interviewees believed that potential safety issues were being addressed and there were no instances identified where these individuals had experienced negative consequences for bringing safety issues to the NRC.

The majority of the interviewees expressed a concern with the timeliness of corrective actions. For safety significant issues, there was confidence that the issue would be addressed, although not as quickly as warranted. For less safety significant issues, there was less confidence that those issues would be ultimately resolved because of past experiences where corrective actions were identified but not completed, then removed from the corrective action process with the original issue left uncorrected.

The inspectors also received approximately fifteen comments regarding confusion surrounding the use of a dual entry system for condition reporting. Specifically, line workers were tasked with determining if an identified concern should be recorded as a CRDR or as a corrective maintenance item. When line workers were asked to define the determining threshold for documentation as a corrective maintenance action vice a CRDR, the responses were varied and lacked specificity. Although no examples were identified during this inspection where a significant safety concern had been incorrectly handled through the corrective maintenance process, the lack of clear knowledge as to the defining boundary was a concern. The inspectors determined that licensee management was aware of this concern and was taking actions to address it.

-20- Enclosure

e. Specific Issues Identified During This Inspection (1) Inspection Scope During the reviews described in Sections 4OA2 a.(1), 4OA2 b.(1), and 4OA2 c.(1),

above, the inspectors identified the following findings.

(2) Findings and Observations (i) 10 CFR Part 50, Appendix B, Criterion XVI Violation Introduction. A Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," was identified for failure to correct, and preclude repetition of a significant condition adverse to quality involving the failure of the Unit 2 turbine driven auxiliary feedwater pump.

Description. In May 2004 on Unit 2, and in July 2004 on Unit 3, the licensee experienced a failure of the governor control circuits on the respective Unit's turbine drive auxiliary feedwater pump. These failures were indicated by an alarm annunciator in the respective Unit's main control room. In both instances, the governor control circuits were repaired within the Technical Specification allowed outage times by replacing the governor power supply resistor with an in-stock spare resistor. The licensee initiated CRDR 2720228 on July 5, 2004, then classified it as significant, which required a root cause evaluation.

The root cause charter was issued to the investigation team leader on July 23, 2004. The assigned due dates for the interim and final evaluation reports were August 14 and September 30, 2004, respectively. The evaluations were not completed until November 10 and February 4, 2005, respectively. The governor power supply resistors were sent to an outside testing facility in late October 2004 and the failure report was received on December 13, 2004. During interviews, one of the individuals involved with the root cause evaluation acknowledged that the root cause was not timely and was delayed mainly because of the unavailability of the investigation leader. The leader was involved in numerous events/activities in 2004 including the three unit trip, a Unit 2 trip because of a lighting strike, and the Fall 2004 refueling outage.

The root cause evaluation determined that the loss of control power to the governor control systems was caused by an open circuit in the governor power supply control voltage dropping resistor. Degradation of small cracks in the ceramic body of the resistors, due primarily to heating and cooling cyclic stress, caused the open circuit in the resistors. The licensee initiated two corrective actions (one short-term and one long-term) to prevent recurrence. The short-term corrective action involved X-ray examination of in-stock spare resistors to ensure the resistors were free of cracks, and subsequent installation of the "crack free" resistors. This action had a due date of April 1, 2005. Prior to the completion of the corrective action, the Unit 2 turbine driven AFW pump governor failed on March 19, 2005, because of a cracked power supply resistor.

-21- Enclosure

As in the prior instance, the resistor was replaced and the turbine driven AFW pump was restored to service within the Technical Specification allowed outage time.

The long-term corrective action was based on the licensee's identification of numerous industry events (starting in 1977) associated with this particular resistor. The licensee developed a modification to change out the power supply resistor with a higher wattage and more reliable resistor. This larger resistor has been installed in all three turbine driven AFW pump governor control circuits and no additional governor failures have occurred.

Analysis. The performance deficiency associated with this finding involved the failure to promptly correct and prevent recurrence of a significant condition adverse to quality. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the availability of systems that respond to initiating events. The failure of the Unit 2 turbine driven auxiliary feedwater pump governor power supply resistor affected the availability of the AFW system. Using the Phase 1 worksheet in Manual Chapter 0609,

"Significance Determination Process," the finding is determined to have very low safety significance because it only affected the mitigating systems cornerstone and did not result in an actual loss of safety function. The cause of the finding is related to the cross-cutting element of problem identification and resolution in that delays in the evaluation of the resistors failures allowed a subsequent failure prior to completion of the corrective actions.

Enforcement. 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action,"

requires in part, that conditions adverse to quality be promptly corrected, and for significant conditions adverse to quality that corrective actions are taken to prevent recurrence. Contrary to the above, the licensee did not correct a significant condition adverse to quality in a timely manner and did not prevent recurrence. Specifically, the licensee failed to perform a timely evaluation to determine the cause of the turbine driven AFW pump governor failures in May and July 2004, and a significant delay in implementation of corrective actions failed to prevent the subsequent Unit 2 turbine driven AFW pump governor failure in March 2005. Because the finding is of very low safety significance and has been entered into the CAP as CRDR 2871541, this violation is being treated as an noncited violation, consistent with Section VI.A of the NRC Enforcement Policy: NCV 05000529/2006008-01, "Untimely Corrective Actions for the Turbine Driven Auxiliary Feedwater Pump Governor Power Supply Resistor Failures."

(ii) Lubricating Oil Control Program Introduction. A Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," was identified for the failure to promptly correct a condition adverse to quality, specifically an adverse trend of contaminated oil samples.

-22- Enclosure

Description. While reviewing corrective action documents related to contaminated oil samples, the inspectors noted that CRDR 2785915, dated April 1, 2005, documented an increasing trend of oil sample results that indicated contamination in the sampled system. The licensee cited three CRDRs written in a short time previous to April 1, 2005, and determined during the evaluation that several potential causes resulted in the contamination found in the oil samples, including contaminated sampling equipment, contaminated secondary containers, small additions of the incorrect oil, weaknesses in the oil control and accountability process, and informal management of bulk storage locations. The licensee also documented a concern that nondedicated oils could be used in quality applications because of process control vulnerabilities. As a consequence of the review, the licensee identified several corrective actions, which included the establishment of tolerances for lubricant viscosity, the dedication of all oils onsite, and the augmentation of controls for the bulk and secondary lubrication oil storage locations.

The inspectors conducted an independent review of the licensee CRDR database to assess the history of oil-related CRDRs. The inspectors identified 12 CRDRs that documented 31 instances of contaminated oil samples between July 1, 2004, and April 1, 2005, when CRDR 2785915 was written. Despite the identification of the adverse trend and development of corrective actions, the inspectors identified another 19 CRDRs written between April 1, 2005, and January 2006, documenting another 31 instances of contaminated oil samples.

Four of these instances occurred in December 2005, and three more in January 2006. The samples were taken from safety and non-safety equipment, with at least 6 from the safety injection pumps. The majority of the instances of contamination were traces of the incorrect oil found in the samples. There was only one instance where a complete oil change had been done with the incorrect type of oil and, in this instance, the licensee determined that the oil was similar enough to the correct oil, such that, the component affected was not declared inoperable nor degraded. The inspectors agreed with this determination.

The inspectors determined that CRDRs identifying contaminated oil samples were being written at the same or higher frequency as before the identification of the adverse trend in CRDR 2785915. The inspectors determined that the corrective actions identified were ineffective, in that, they were not implemented until January 2006. Those actions included the removal of the lockers containing oil from inside the radiological control area, and the management directive that restricted the conduct of oil changes to mechanical maintenance personnel only.

The inspectors could not evaluate the effectiveness of those corrective actions.

Analysis. The performance deficiency associated with this finding involved the failure to promptly correct a condition adverse to quality. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the reliability and availability of systems that respond to initiating events. Using Manual Chapter 0609, "Significance Determination Process," Phase 1 worksheet, the finding is determined to have very low safety significance because it only affected the mitigating systems cornerstone and did-23- Enclosure

not result in the loss of safety function of a single train or system. The cause of the finding is related to the cross-cutting elements of human performance and problem identification and resolution, in that, poor work practices resulted in multiple oil contamination events and the corrective actions taken were ineffective in promptly correcting the condition.

Enforcement. 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action,"

requires, in part, that measures shall be established to assure that conditions adverse to quality are promptly corrected. Contrary to this, after identification of the adverse condition in April 1, 2005, of contamination in multiple oil samples in safety-related equipment because of oil control program weaknesses, the licensee failed to implement programmatic corrective actions for a period of approximately 9 months, resulting in a continuation of the adverse condition.

Because the finding is of very low safety significance and has been entered into the licensees CAP as CRDR 2785915, this violation is being treated as a noncited violation consistent with Section VI.A of the Enforcement Policy:

NCV 05000528,529,530/2006008-02, "Failure to Promptly Correct an Adverse Trend of Contaminated Oil Samples."

(iii) Failure to Meet Maintenance Test Requirements Introduction. A Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XI, "Test Control," was identified for failure to perform testing in accordance with the maintenance test procedure, in that, out of tolerance conditions were accepted without adjustment or explanation.

Description. The licensee completed a routine Essential Cooling Water System Pump EWP01 circuit breaker "Inspect and Adjust" maintenance test in Unit 3 on August 10, 2005. The test was completed, reviewed by a maintenance supervisor, and the breaker returned to service based on satisfactory test results. The inspectors reviewed the completed Breaker EWP01 test results and determined that several recorded breaker measurements did not meet certain tolerances and acceptance criteria identified in the test for the breaker sub-component clearances. For example, many steps required the maintenance technician to "Verify" a breaker measurement read between two given values and required the technician to record the as-found measurement. "Verification" is defined in Procedure 01TD-0AP01, "Technical Dictionary," Revision 5, as "An act of confirming, substantiating, or assuring that an activity or condition has been implemented in conformance with specified requirements." The inspectors noted several instances where the recorded measurement was not within the values given, and yet no note or explanation was given as to how the "Verify" step was accomplished. The inspectors then sampled five additional similar breaker test results and noted several other similar examples in each of the five additional tests. The inspectors noted that the breaker maintenance tests were reviewed by maintenance supervisors without comment on why the failure to meet required tolerances and acceptance criteria was acceptable.

At the inspector's request, the licensee investigated the breaker tests, interviewed maintenance technicians and supervisors, and determined that in-24- Enclosure

each case where an acceptance criterion was not met, an adjustment had been made to restore the breaker to within the acceptance criteria. Also, in some instances, the tolerances stated in the test procedure were not considered as required to pass the maintenance test, so out-of-tolerance conditions were accepted as long as the final acceptance criteria were met. The inspectors determined that the breakers were adequately tested and returned to service, however, the test procedures were not followed in accordance with procedure usage requirements.

Analysis. The finding affected the mitigation systems cornerstone, and was more than minor since it affected the Equipment Performance attribute of the Reactor Safety Mitigating Systems cornerstone, specifically the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609, "Significance Determination Process," Phase 1 worksheet, the finding was determined to have very low safety significance because the condition was a qualification deficiency confirmed not to result in loss of function in accordance with Part 9900, Technical Assessment, "Operability Determination Process for Operability and Functional Assessment." This finding is similar to more than minor example 2.c.

in NRC Inspection Manual Chapter 0612, Appendix E, in that, the issue was repetitive and affected multiple breakers tested. The cause of the finding is related to the cross-cutting elements of human performance and problem identification and resolution in that maintenance personnel failed to properly implement maintenance procedures, and the deficient conditions were not identified by supervisor review of the completed procedures.

Enforcement. 10 CFR Part 50, Appendix B, Criterion XI, "Test Control,"

requires, in part, that the licensee establish a test program to assure that structures, systems, and components will perform satisfactorily in service, that the tests are performed in accordance with written test procedures, which incorporate the requirements and acceptance limits contained in applicable design documents, and that results shall be documented and evaluated to assure that test requirements have been satisfied. Contrary to the above, the breaker maintenance test performed on August 10, 2005, was not performed in accordance with the procedure, nor reviewed adequately so as to provide assurance of the EWP01 pump's operability. Specifically, the test tolerances and acceptance limits were documented as outside tolerance limits, and the supervisory review failed to justify acceptability of the results. Because of the very low safety significance and the licensees action to place this issue in their CAP as CRDR 2865792, this violation is being treated as a noncited violation in accordance with Section VI.A.1 of the Enforcement Policy:

NCV 05000530/2006008-03, "Failure to Meet Maintenance Test Requirements."

(iv) Failure to Identify BOP-ESFAS Sequencer Degradation Introduction. A Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," was identified for the failure to identify and correct a condition adverse to quality, involving the failure of the Unit 1 "A"

-25- Enclosure

balance of plant engineered safety feature actuation system (BOP ESFAS)

sequencer.

Description. On April 25, 2004, after conducting an EDG over speed trip test, the control room received a "BOP ESFAS STALL" annunciator, indicative of a potential problem with the "A" BOP ESFAS sequencer. After troubleshooting and repairs, the sequencer was placed in the auto-test mode for enhanced monitoring. The sequencer remained in auto-test mode from May 5 until July 14, 2004. The auto-test mode provided real-time diagnostic indications by sending small electrical pulses to verify proper operation of the sequencer, including the relays that actuate the engineered safety functions. The auto-test pulses do not fully cycle the relays, but caused them to partially move. According to operators interviewed, this relay movement also created an audible sound in the control room. Some operations and engineering staff informally raised questions concerning the potential impact of continuous auto-testing on the relays, but did not initiate a CRDR to obtain a formal evaluation. Engineering staff did conduct a 10 CFR 50.59 screen and concluded that there was no adverse impact on the relays of continuous auto-testing based on the design of the sequencer, and that continuous auto-testing would increase the reliability of the system.

Approximately a year later, on May 1, 2005, because of a normally lit sequencer light that was out, operators conducted a test of the "A" BOP ESFAS sequencer.

During the test, the sequencer stalled again. Without immediately being able to identify a root cause, the licensee replaced the sequencer and placed it in the auto-test mode again for enhanced monitoring. The sequencer remained in the auto-test mode from May 2 to June 3, 2005. Approximately 3 months later, on October 10, 2005, during Refueling Outage 1R12 and while performing the

"Class 1E Diesel Generator and Integrated Safeguards Test, Train A," Revision 9, the sequencer failed to shed a service water pump as expected in Step 8.4 of the procedure. The sequencer is relied on by design to remove all loads from the Class 1E 4160V Safety Bus PBA-SO3 to support safe EDG starting and loading onto the safety bus.

After investigating, the licensee determined that the failure to shed the essential water pump from the bus was because of a failed Potter & Brumfield (P&B)

relay. The licensee found wear marks on the bearing surfaces of the relay armature and on the contact surfaces. Some of the contact surfaces were also contaminated with wear products that prohibited contact closure. After questioning by the inspectors, the licensee continued their investigation and determined that the operation of the sequencer for long periods in the auto-test mode was the most likely cause of the relay failure. After further research, the licensee also determined that in May 2003, an instrumentation and control engineer had recommended not using the auto-test mode in response to CRDR 2598652, "CRDR Recommends that the BOP ESFAS Auto-tester be Run Continuously in Both Channels and in All Three Units" due to concerns about damage to the relays. His concerns were not formally addressed in the CRDR process, but were captured in an electronic mail message to individuals in the engineering department that were addressing CRDR 2598652.

-26- Enclosure

The inspectors determined that the licensee incorrectly concluded in May 2004, and again in May 2005, that operation of the sequencer long-term in the auto-test mode would have no adverse effect on the sequencer function. The licensee also failed to identify the long-term pulsing of the relays as a condition adverse to quality, and failed to correct it. The inspectors also noted that Information Notice 92-04, "Potter & Brumfield Model MDR Rotary Relay Failures," although focused on a different failure mechanism than was experienced in this finding, described several failures of P&B relays in the industry and discussed contributors to those failure mechanisms, which included among others, testing frequency and operational cycling. The inspectors concluded that although the exact failure mechanism experienced in this finding (relay fouling) was not described in Information Notice 92-04, the insights could have prompted a more detailed review of the potential degradation of the P&B relays from auto-testing at Palo Verde.

Analysis. The performance deficiency associated with the finding was the failure of the licensee to identify and correct a condition adverse to quality. Specifically the licensee incorrectly concluded that long-term operation of the BOP ESFAS sequencer in the auto-test mode and the resultant pulsing of the P&B relays would have no adverse impact on the equipment. The finding is greater than minor because it affects the equipment performance and human performance attributes of the Mitigating Systems cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609,

"Significance Determination Process," Phase 1 worksheet, the finding is determined to have very low safety significance because the finding did not result in the loss of safety function of any component, train, or system. The cause of the finding is related to the cross-cutting element of problem identification and resolution in that the licensee evaluation failed to adequately evaluate and correct a condition adverse to quality.

Enforcement. 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action,"

requires, in part, "that measures be established to assure that conditions adverse to quality are promptly identified and corrected." Contrary to this, between July 14, 2004 and May 1, 2005, the licensee failed to identify and correct a condition adverse to quality. Specifically, the licensee did not identify the adverse impact of long-term auto testing of the sequencer, and did not correct the situation resulting in a relay failure. Because the finding is of very low safety significance and has been entered into the licensees CAP as CRDR 2796883, this violation is being treated as a noncited violation consistent with Section VI.A of the Enforcement Policy: NCV 05000528/2006008-04, "Failure to Identify and Correct an Adverse Condition Associated with the BOP-ESFAS Sequencer."

(v) Failure to Promptly Monitor LPSI 2A in Maintenance Rule Introduction: A Green noncited violation of 10 CFR 50.65 (a)(2) was identified associated with the Unit 2 low pressure safety injection (LPSI)/ shutdown cooling-27- Enclosure

(SDC) 2A Pump. The licensee failed to identify the need to set goals and monitor the condition of LPSI/SDC Pump 2A as required by 10 CFR 50.65(a)(1).

Description: The inspectors reviewed the accounting of the unavailability time associated with the LPSI/SDC Pump 2A degraded mechanical seal of May 16, 2005. Responding to inspectors questioning, the licensee determined that 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> of unavailability time had not been accounted for after the May 2005 event.

The additional 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> resulted in a revised unavailability of 1.57 percent.

Procedure 70DP-OMR01, "Maintenance Rule," Section 3.5.2.1.3, requires that an a(1) review is required when an unavailability first trigger is exceeded. The unavailability first trigger for the LPSI/SDC Pump 2A was established at 1.5 percent unavailability. The inspectors concluded that on May 16, 2005, the licensee failed to accurately account for unavailability time for the LPSI/SDC Pump 2A and therefore failed to perform the required (a)(1) evaluation.

Analysis: The inspectors determined that this finding was greater than minor because it affected the reactor safety mitigating systems cornerstone objective to maintain availability and reliability of safety-related components needed to respond to initiating events and was similar to the greater than minor Example 7.b. in Appendix E of Manual Chapter 0612. Using the Manual Chapter 0609,

"Significance Determination Process," Phase 1 worksheet, the finding was determined to have very low safety significance because there was no design deficiency and the LPSI/SDC Pump 2A failure did not exceed the allowed technical specification outage time. The cause of the finding is related to the cross-cutting element of human performance in that initial evaluation and subsequent supervisory reviews failed to identify the need for additional monitoring of the LPSI/SDC Pump 2A.

Enforcement: 10 CFR 50.65, "Requirements for Monitoring the Effectiveness of Maintenance at Nuclear Power Plants," paragraph (a)(2) states, in part, that

"Monitoring as specified in paragraph (a)(1) . . . is not required where it has been demonstrated that the performance of a . . . component is being effectively controlled through the performance of appropriate preventative maintenance. . . ." Contrary to the above, because of the failure to accurately determine unavailability time for the LPSI/SDC Pump 2A in May 2005, the licensee failed to establish goals and monitor the LPSI/SDC Pump 2A under paragraph a(1). Because this failure to enter the LPSI/SDC pump 2A into 10 CFR 50.65 (a)(1) monitoring is of very low safety significance and has been entered into the licensees CAP as CRDR 2865315, this violation is being treated as a noncited violation, consistent with Section VI.A.1 of the NRC Enforcement Policy. NCV 05000529/2006008-05, "Failure to Promptly Monitor LPSI 2A in Maintenance Rule."

-28- Enclosure

4OA3 Event Follow-up (71153)

(Closed) Licensee Event Report (LER) 05000529/2005-004-00, Technical Specification Required Shutdown Due to Core Protection Calculators Inoperable On August 22, 2005 at approximately 1750 Mountain Standard Time (MST), Unit 2 completed a reactor shutdown required by technical specifications. The shutdown was required due to all four channels of the core protection calculators (CPC) being declared inoperable on August 22 at 1326 MST based on information from the CPC vendor that software changes that had previously been implemented in Unit 2 CPCs changed the way the CPCs would operate for a failed sensor. The investigation determined a CPC system requirement specification was not properly translated into the CPC software by the vendor. The software has been corrected.

The inspectors determined that there was no performance deficiency associated with the installation of the flawed CPC software, nor the resultant technical specification required shutdown, therefore no new findings were identified in the inspectors review.

This LER is closed.

(Closed) LER 05000529/2005-002-00, Technical Specification 3.0.4 Violation; Mode Change Made With One of Two Required LPSI Pumps Inoperable On May 16, 2005, Unit 2 entered Mode 3 with pressurizer pressure greater than or equal to 1837 psia while LPSI Pump A had a degraded mechanical seal. Technical Specification Limiting Condition of Operation 3.5.3 required two emergency core cooling systems to be operable in Modes 1, 2, and in Mode 3 when pressurizer pressure is greater than or equal to 1837 psia. On June 10, 2005, mechanical maintenance engineering determined the cause of the seal degradation was inadequate venting of the seal prior to one or more pump starts while the Unit was in MODE 5. Therefore, the operability status of LPSI Pump 2A was indeterminate when Limiting Condition of Operation 3.5.3 (Mode 3, greater than or equal to 1837 psia) conditions were entered on May 16, 2005. This was identified by the licensee as a violation of Technical Specification 3.0.4. and is documented in Section 4OA7 of this report. The condition would not have prevented the fulfillment of the safety function and did not result in a safety system functional failure. This LER is closed.

(Closed) LER 05000528/2005-002-00, Technical Specification 3.0.4 Violation: Mode Change With Safety Injection Valve Not in Its Required Position On February 17, 2005, at approximately 1752 hours0.0203 days <br />0.487 hours <br />0.0029 weeks <br />6.66636e-4 months <br /> MST, Unit 1 was in Mode 3, Hot Standby, and increasing reactor coolant system temperature and pressure to return the unit to power operation. The licensee identified that a violation of Technical Specification 3.0.4 occurred when a mode change occurred with a safety injection valve not in its required position. The condition was discovered on February 21, 2005, when an auxiliary operator noted dual light position indication at the valve breaker. The finding is documented in Section 4OA7 of this report as a second example of a Technical Specification 3.0.4 violation. This LER is closed.

-29- Enclosure

(Closed) LER 05000529/2005-003-00, Two Independent Trains of Auxiliary Feedwater Inoperable On June 23, 2005, Door C-A06, a watertight fire door that functions as the train separation barrier between auxiliary feedwater pump Room "A" and "B," was found opened by maintenance personnel with no compensatory measures established. An investigation concluded that a security officer had failed to close C-A06 after leaving the auxiliary feedwater pump room area. Because there was no loss of fire detection and suppression capability, no excessive fire loading in the two rooms, and no actual loss of normal feedwater on June 23, 2005, there was no actual safety consequence associated with the two essential trains of auxiliary feedwater being rendered inoperable for 43 minutes.

No new findings were identified in the inspectors review. This LER is closed.

(Closed) LER 05000528/2005-006-00, Technical Specification Required Shutdown on Emergency Diesel Generator "A" Failure to Start During Post Maintenance Testing On March 18, 2005, Palo Verde Unit 1 Control Room personnel commenced a reactor shutdown required by Technical Specification 3.8.1. Unit 1 Diesel Generator "A" had failed to start during its post-maintenance retest. It had been determined that the cause of the diesel generator failure was because of a governor failure. The condition would not have prevented the fulfillment of any safety function and did not result in a safety system functional failure. This report also discusses the failure to complete the required LER within 60 days of the event, which occurred on March 18, 2005. This failure to report the LER on time was documented as Noncited Violation 05000528; 05000529;05000530/2005005-04, "Failure to Submit LER to Report Shutdown Required by Technical Specifications."

No new findings were identified in the inspectors review. This LER is closed.

4OA5 Root Cause Evaluation Review The inspectors assessed the licensees root cause evaluation associated with a change to radiological emergency action levels, which decreased the effectiveness of the emergency plan. This performance issue was related to the emergency preparedness cornerstone in the reactor safety strategic performance area, and was previously characterized as a Severity Level III violation of 10 CFR 50.54(q) in NRC Inspection Report 05000528,529,530/2005011. The licensees evaluation consisted of the Significant Root Cause Investigation Report for CRDR 2774185 and its associated attachments. The inspectors used the guidance in NRC Inspection Procedure 95001,

"Inspection for One Or Two White Inputs in A Strategic Performance Area," to assess the licensee's evaluation.

(1) Problem Identification (95001)

a. Determine that the evaluation identifies who (i.e., licensee, self-revealing, or NRC), and under what conditions the issue was identified.

-30- Enclosure

This issue was identified by an NRC inspector during in-office review of Revision 29 to the Palo Verde Nuclear Generating Station Emergency Plan, and Revision 2 to Emergency Plan Implementing Procedure 99, "EPIP Standard Appendices," both submitted September 1, 2004. These revisions were compared to their previous revisions, to the criteria of Nuclear Energy Institute 99-01, "Methodology for Development of Emergency Action Levels," Revision 2, to the NRC Safety Evaluation transmitted to the licensee March 19, 2004, and to the requirements of 10 CFR 50.54(q), 10 CFR 50.47(b), and 10 CFR Part 50, Appendix E, to determine if the revisions represented a decrease in the effectiveness of the emergency plan. The inspectors concluded that the changes submitted were a decrease in the effectiveness and communicated this conclusion to the licensee. On February 3, 2005, the licensee documented the issue in CRDR 2774185 for evaluation and corrective actions.

b. Determine that the evaluation documents how long the issue existed, and prior opportunities for identification.

Short-term The changes to the emergency plan that resulted in the decrease in the effectiveness were generated in July 2004 and implemented on August 6, 2004. After the issue was identified by the NRC inspector, the emergency plan was revised to eliminate the decrease in the effectiveness on February 4, 2005.

The licensees evaluation of this issue noted three opportunities during the development and review of the proposed plan changes to identify the issue and prevent the inappropriate implementation of the decrease in the effectiveness. Neither the change preparer, nor two subsequent reviewers identified the changes as a decrease in the effectiveness. Additionally, the evaluation noted that emergency planning personnel missed an additional opportunity for identification by failing to submit the proposed changes to radiation protection for required cross-organizational review.

Long-term The licensees evaluation identified a twofold root cause of the performance issue:

(1) failures to ensure adequate radiation protection review of the Emergency Action Level changes that were made to Procedure EPIP-99 (Revision 2) through adequate radiation protection expertise within emergency planning and cross-organizational review; (2) failure of management to address knowledge and ability issues within emergency planning (resulting from attrition of radiation protection-experienced emergency planning personnel, inadequate training on procedure change requirements, and inadequate management of workload demands). The evaluation identified that plant management had several precursor indicators, from results of internal audits and self-assessments, of the need to address knowledge and ability issues within emergency planning over the 5 years leading up to the decrease in the effectiveness.

The inspectors concluded that the licensees evaluation adequately identified the duration of the issue and prior opportunities for identification.

-31- Enclosure

c. Determine that the evaluation documents the plant-specific risk consequences (as applicable) and compliance concerns associated with the issue.

The licensees evaluation acknowledged that the emergency plan changes implemented on August 6, 2004, constituted a compliance issue, in that, it decreased the effectiveness of the emergency plan without prior approval by the NRC, which is contrary to the 10 CFR 50.54(q) process. The evaluation also acknowledged the safety significance of the change in that it might have delayed declaration of the appropriate Emergency Action Level under certain conditions since one of the trigger points for implementing the Emergency Action Level (site boundary dose rate as measured by portable instrumentation) was removed by the change. However, the evaluation also indicated that the implementing procedure (EPIP-01) used by the radiological field assessment team, still directed the radiological field assessment team members to notify the radiological assessment coordinator if any Emergency Action Level thresholds are exceeded and further indicates the correct site boundary dose rate trigger points for Emergency Action Levels (even though that trigger point had been removed from Procedure EPIP-99). The evaluation reported an assessment that concluded that, of 14 radiological field assessment team team members tested, all 14 would have recommended the correct Emergency Action Level declaration at the appropriate site boundary dose rate threshold. The evaluation concluded from the results of this assessment that, even during the 6 months in which the Procedure EPIP-99, Revision 2, change was in effect, the correct Emergency Action Levels would have been declared based on site boundary dose rate indications. The inspectors concluded that the licensee effectively evaluated risk consequences and compliance concerns associated with the issue.

(3) Root Cause, Extent of Condition, and Extent of Cause Evaluation a. Determine that the problem was evaluated using a systematic method(s) to identify root cause(s) and contributing cause(s).

The licensee used its own "PVNGS Root Cause Investigation Manual" to evaluate this issue, which incorporated event and causal factors analysis, "Why Staircase," change analysis, hazard-barrier-target analysis, common cause analysis, fault tree analysis, and the "Prevent Events" model. The inspectors determined that the licensee followed its procedures for performing a root-cause investigation.

b. Determine that the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem.

Overall, the inspectors concluded that the root-cause evaluation identified and assessed the potential contributors to the decrease in the effectiveness in sufficient detail to identify appropriate corrective actions.

The licensee identified two root causes for the decrease in the effectiveness:

• Failure of emergency planning personnel to send Emergency Action Level changes in Procedure EPIP-99, Revision 2, to radiation protection personnel for a required cross-organizational review, coupled with emergency planning-32- Enclosure

personnel lacking radiation protection expertise allowed Emergency Action Levels 3-16 and 3-19 to be changed inadequately in Procedure EPIP-99, Revision 2, which caused a decrease in the effectiveness of the emergency plan without NRC approval. (Root-Cause-1)

• Palo Verde Nuclear Generating Station management had not successfully addressed knowledge and ability issues within emergency planning that resulted from: 1) radiation protection expertise lost from emergency planning as a result of personnel changes and not replaced; 2) 10 CFR 50.54(q) training not provided to emergency planning personnel after the responsibility for making procedure changes to the emergency plan was moved from the Emergency Services Program Department to emergency planning ; and 3) emergency planning not successful in managing its workload (primarily procedure changes). (Root-Cause-2)

The inspectors evaluated the root-cause evaluation report against the requirements of the licensees "PVNGS Root Cause Investigation Manual," and determined that the root-cause evaluation followed the administrative procedure requirements.

c. Determine that the root cause evaluation included consideration of prior occurrences of the problem and knowledge of prior operating experience.

The inspectors concluded that the evaluation effectively included consideration of prior occurrences of the problem and knowledge of prior operating experience. The evaluation effectively assessed the licensees failure in recent years to address symptoms and evidences of organizational weakness (as outlined in Root-Cause-2)

apparent in the licensees own operating experience and its failure to adequately address related industry operating experience issues.

d. Determine that the root cause evaluation addresses the extent of condition and the extent of cause of the problem.

The NRC defines Extent of Condition as "the extent to which the actual condition exists within other plant processes, equipment, or human performance." Included in the licensees evaluation was a thorough review of all Emergency Action Level changes made since the last formal NRC emergency plan approval in 1994.

The NRC defines Extent of Cause as "the extent to which the root causes of an identified problem has impacted other plant processes, equipment, or human performance." The licensees evaluation considered the potential for the root-causes for this issue to have impacted other procedure change processes and identified similar error precursors.

The inspectors concluded that the extent of condition and extent of cause reviews were adequate.

-33- Enclosure

(4) Corrective Actions a. Determine that appropriate corrective action(s) are specified for each root/contributing cause or that there is an evaluation that no actions are necessary.

The root-cause investigation clearly indicated which corrective actions were identified to address each root cause. The inspectors determined that the corrective actions associated with the root-cause evaluation were appropriate for the root causes identified; however, the inspectors did note that one corrective action seemed to fall short of establishing programmatic barriers to prevent recurrence of one of the conditions that contributed to this issue. The licensees evaluation identified the lack of radiation protection expertise in the emergency planning organization as one of the root causes for this issue. The corrective action was to assign an individual with radiation protection expertise to the emergency planning organization; however, no program requirements for radiation protection expertise were implemented, so no barriers other than personnel memory are in place to ensure that future attrition doesnt create the same lack of radiation protection expertise that contributed to this issue.

b. Determine that the corrective actions have been prioritized with consideration of the risk significance and regulatory compliance.

The inspectors concluded that the corrective actions were reasonably prioritized. The decrease in the effectiveness was immediately corrected and subsequent corrective actions are being addressed appropriately.

c. Determine that a schedule has been established for implementing and completing the corrective actions.

Several corrective actions for the root-cause investigation were not yet completed; however, the inspectors reviewed the completed corrective actions and concluded that they had been generally implemented in a timely and effective manner, although one example of failure to follow the corrective action procedure was identified by the inspectors. One of the corrective action items had been closed as complete when, in actuality, it was only partially complete; however, the remaining actions were progressing independent of the corrective action items and would have been completed anyway. The licensee wrote CRDR 2865444 to address the administrative errors that led to the inappropriate closeout of the corrective action items. The inspectors determined that no violation of NRC regulations occurred since the actual corrective actions were being carried out in spite of the administrative tracking error, and the failure to follow an administrative procedure was entered into their corrective action system.

d. Determine that quantitative or qualitative measures of success have been developed for determining the effectiveness of the corrective actions to prevent recurrence.

The inspectors determined that the root-cause investigation established general effectiveness review criteria for determining the future effectiveness of corrective actions through a self-assessment. The licensee stated that the criteria in the effectiveness review will include a review of procedure changes to ensure they meet 10 CFR 50.54(q)

requirements.

-34- Enclosure

4OA6 Exit Meeting On February 3, 2006, at the end of the onsite portion of the inspection, the inspection findings were discussed with Mr. and other members of your staff. The inspectors continued in-office document reviews and conducted an exit meeting with Mr. and other members of your staff on March 28, 2006.

4OA7 Licensee Identified Violations The following violations of very low safety significance (Green) were identified by the licensee and were violations of NRC requirements which met the criteria of Section IV of the NRC Enforcement Policy for being dispositioned as noncited violations.

• 10 CFR 50.54(q) states, in part, that "a licensee . . . shall follow and maintain in effect emergency plans which meet the standards in §50.47(b) and the requirements in Appendix E of ths part." The Palo Verde Emergency Plan, Section 7.2, "Communications Systems," describes the onsite and offsite communication systems, which include the use of a microwave transmission system capability for the private branch exchange, as well as a ringdown facsimile and public information circuit from the emergency operating facility to offsite authorities. Contrary to the emergency plan, in March 2005, the microwave transmission system was replaced with a fibre-optic system without performing a 10 CFR 50.54(q) evaluation and without coordination with the emergency planning staff. In January 2006, the change to the facility configuration was discovered, and was documented in CRDR 2861534. The finding was determined to be of very low safety significance because it was a violation of regulatory requirements, but was not a planning standard problem since communication capabilities were not decreased as a result of the change to the facility. The cause of the finding had human performance cross-cutting aspects in the area of personnel. The licensee has planned action to strengthen the work control process in the Information Services Telecom group, and correct the communication system description in the Emergency Plan.

• Technical Specification 3.0.4 establishes limitations on changes in modes or other specified conditions in the Limiting Condition for Operation Applicability when an Limiting Condition for Operation is not met. In LERs 05000529/2005-002-00 and 05000528/2005-002-00, the licensee reported that Mode changes occurred with 1) a Safety Injection valve not in its required position, and 2) a LPSI pump inoperable because of a degraded seal. The limitations of Technical Specification 3.0.4 were not met in these instances, since the applicable Limiting Condition for Operation actions would not permit continued operation in the higher Mode for an unlimited time. This finding (two examples) is only of very low safety significance because there was not an actual loss of safety function. The cause of the finding had human performance cross-cutting aspects in the area of personnel.

Attachment: Supplemental Information-35- Enclosure

Supplemental Information Partial List of Persons Contacted Licensee S. Bauer, Department Leader, Regulatory Affairs D. Carnes, Director, Nuclear Assurance B. Chapin, Director, Performance Improvement C. Churchman, Director, Engineering C. Eubanks, Vice President, Operations J. Gaffney, Director, Radiation Protection T. Gray, Department Leader, Radiation Protection D. Hautala, Senior Compliance Engineer J. Hesser, Director, Emergency Services D. Leech, Department Leader, Performance Improvement J. Levine, Executive Vice President, Generation D. Marks, Performance Advocate, Regulatory Affairs D. Mauldin, Vice President, Engineering M. McGhee, Department Leader, Operations M. Muhs, Department Leader, Maintenance E. ONeil, Department Leader, Emergency Preparedness T. Radtke, General Manager, Services and Support F. Riedel, Director, Nuclear Training Department J. Scott, Department Leader, Nuclear Assurance C. Seaman, General Manager, Regulatory Affairs, Performance Improvement V. Setergren, Department Leader, Performance Improvement M. Shea, Director, Maintenance M. Sontag, Department Leader, Performance Improvement D. Straka, Senior Consultant, Regulatory Affairs C. Zell, Director, Work Management LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED Opened and Closed 05000529/2006008-01 NCV "Untimely Corrective Actions for Feedwater Resistor Failures" 05000528,529,530/2006008-02 NCV "Failure to Promptly Correct an Adverse Trend of Contaminated Oil Samples"05000530/2006008-03 NCV "Failure to meet Maintenance Test Requirements"05000528/2006008-04 NCV "Failure to Identify and Correct an Adverse Condition Associated with the BOP-ESFAS Sequencer"

-1- Attachment

05000529/2006008-05 NCV "Failure to Promptly Monitor LPSI 2A in Maintenance Rule" Closed 05000529/2005-004-00 LER Technical Specification Required Shutdown Due to Core Protection Calculators Inoperable 05000529/2005-002-00 LER Technical Specification 3.0.4 Violation; Mode Change Made With One of Two Required LPSI Pumps Inoperable 05000528/2005-002-00 LER Technical Specification Violation: Mode Change With Safety Injection Valve Not In Its Required Position 05000529/2005-003-00 LER Two Independent Trains of Auxiliary Feedwater Inoperable 05000528/2005-006-00 LER Technical Specification Required Shutdown on EDG "A" Failure to Start During Post Maintenance Testing 05000528,529,530/2005011-01 VIO Change to radiological emergency action levels which decreased the effectiveness of the emergency plan.

Documents Reviewed In addition to the documents called out in the inspection report, the following documents were selected and reviewed by the inspectors to accomplish the objectives and scope of the inspection and to support any findings:

Procedures Administrative Procedures 90DP-OIP10, "Condition Reporting," Revisions 23, 24 93DP-OLC07, "10 CFR 50.59 and 72.48 Screenings and Evaluations," Revision 12 51DP-9OM03, "Site Scheduling," Revision 12 51DP-9OM09, "Outage Planning and Implementation," Revision 2 PVNGS Policy No. 104; Section 100, "Nuclear Safety Principles," Revision 0 01DP-OEM09, "Employee Concerns Program," Revision 0-2- Attachment

90DP-0IP09, "Differing Professional Opinions (DPO)," Revision 9 Maintenance Procedures 30DP-0AP01, "Maintenance Instruction Writers Guide," Revision 27 39DP-9ZZ04, "Valve Services Maintenance - Motor Operated Valves," Revision 10 39MT-9ZZ02, "PM or EQ Inspection of the GL 89-10 Limitorque SMB/SB Motor Operated Valve Actuators," Revision 16 06DP-7AC01, "Conduct of PVIS Field Services," Revision 0 31DP-9ZZ01, "Lubricant Sampling," Revision 7 37DP-9MP04, "Lubricant Evaluations," Revision10 73DP-9ZZ05, "Lubrication of Plant Equipment," Revision 19 Surveillance Procedures Procedure 73DP-9XI01, "Pump and Valve Inservice Testing Program - Component Tables,"

Revision 18 Procedure 73ST-1XI11, "Safety Injection Train A ECCS Throttle Valves - Inservice Test,"

Revision 17 Operating Procedures Abnormal Operating Procedure 40AO-9ZZ13, "Loss of Class Instrument or Control Power,"

Revision 8 Abnormal Operating Procedure 40AO-9ZZ17, "Loss of Class Instrument or Control Power,"

Revision 0 Operations Procedures 40DP-9WP01, "Operations Processing of Work Orders," Revision 1 40DP-9OP26, "Operability Determination", Revision 15 40OP-9SA01, "BOP-ESFAS OPERATION", Revision 21, 22, and 23-3- Attachment

Engineering Procedures System Training Manuals:

Breakers System Descriptions, non-class Volume 29A 13.8 kV Non-class 1E Power System Breakers System Descriptions,-class Volume 29B 4.16 kV Non-class 1E Power System (NB)

Breakers System Descriptions,-class Volume 29C 480V Non-class LC/MCC Power System (NG/NH)

Class IE 4.16 kV Power System (PB)

Volume 28A Class IE 4.16 kV Power System (PB)

480 VAC Class IE LC / MCC Power 480 VAC Class IE LC / MCC Power System (PG)

Volume 28C 125 VDC IE Power System (PK)

Volume 28D 125 VDC IE Power System (PK)

Work Orders 2719184 2722157 2773782 2836165 2858391 2449966 2732276 2774514 2838278 2858455 2552294 2746466 2776032 2842080 2859921 2552295 2749050 2783964 2845020 2860315 2552296 2749051 2796592 2845297 2861270 2647198 2750251 2809441 2847738 2861958 2702095 2752313 2809442 2849003 2861963 2712466 2760523 2818530 2853084 2861978 2712661 2760526 2825030 2854575 2716751 2716181 2760528 2829382 2857509 2709779 2716751 2760529 2830391 2858080 2760530 2719184 2760530 2831514 2858310 2716751 2722156 2766545 2834324 2858390 Significant CRDRs 2624427 2775921 2800733 2829384 2844023 2669474 2780273 2819034 2833010 2844115 2687292 2780286 2819043 2833743 2845317 2687507 2788450 2821210 2835132 2859071 2703945 2796883 2823509 2835976 2720228 2714809 2800534 2825485 2841586 2783251-4- Attachment

CRDRs 2399780 2731336 2791716 2821546 2847516 2858439 2596182 2735474 2791838 2822997 2849362 2858978 2596985 2735992 2792051 2823704 2849881 2859220 2624427 2740302 2795636 2825004 2849883 2859272 2660103 2741036 2799382 2825486 2850565 2859373 2669474 2746160 2800779 2826154 2851540 2859401 2682409 2746954 2800972 2827862 2852080 2859409 2686201 2753414 2807956 2828477 2852098 2859444 2686271 2760295 2808100 2828558 2852099 2859450 2687507 2762043 2812433 2828622 2852133 2859635 2693582 2762063 2812449 2828919 2852145 2859659 2693619 2762065 2812536 2830632 2853745 2859726 2694880 2762114 2814209 2831091 2854017 2860698 2695262 2771440 2815959 2831151 2854697 2861534 2709451 2775015 2817016 2832944 2854954 2862310 2711719 2777637 2818524 2833225 2856233 2862482 2715731 2781949 2818612 2834180 2856380 2862512 2716019 2784263 2819031 2835326 2857133 2862653 2719183 2785643 2819047 2842053 2857479 2863686 2719463 2785819 2819772 2844853 2857524 2864821 2721947 2786615 2820753 2845804 2857810 2865382 2722006 2786620 2820810 2847145 2858333 2865792 2726509 2787338 CRAIs 2412506 2806475 2820402 2857996 2859434 2760585 2820400 2856432 2859428 2688541 2806471 Licensee Event Reports (LER)

50-529/2003-001-00, "Reactor Trip with Loss of Forced Circulation due to Failed Pressurizer Main Spray Valve" 50-528/2004-005-01, "Missed ST on Shutdown Cooling Valve RCS Pressure Interlocks" 50-528/2005-006-00, "TS Required Reactor Shutdown on EDG "A" Failure to Start During Post Maintenance Testing" 50-529/2005-003-00, "Two Independent Trains of Auxiliary Feedwater Inoperable" 50-528/2005-002-00, "Technical Specification violation: mode change with safety injection valve not in its required position" 50-529/2005-002-00, "T/S 3.0.4 Violation; mode change made with one of two required LPSI pumps inoperable"

-5- Attachment

50-530/2004001-00, "RCS Pressure Boundary Leakage Caused by Degraded Alloy 600 Component" Vendor Manuals Limitorque Technical Update 93-01, "SMB-3 Gear Efficiencies" Assessments and Audits Engineering Report: "Failure Analysis of SKF 7411 bearings," 9/27/05 SOER 83-05 Closeout Recommendations, File 91-029-404 and 89-029-404 PI&R Readiness Review, November 7-23, 2005 Quality Assurance Surveillances Nuclear Assurance Department (NAD) Audit Planning for 2005 NAD Design Control Audit Number 2005-005 Miscellaneous 2004 and 2005 System Health Reports for Auxiliary Feedwater System RIS 2005-05, "Grid Reliability and the Impact on Plant Risk and the Operability of Offsite Power" RIS 2005-20, "Revision to Guidance Formerly Contained in NRC Generic Letter 91-18, Information to Licensees Regarding Two NRC Inspection Manual Sections on Resolution of Degraded and Nonconforming Conditions and on Operability" Code Case OMN-1, "Alternative Rules for Preservice and Inservice Testing of Certain Electric Motor-Operated Valve Assemblies in Light-Water Reactor Power Plants OM Code-1995, Subsection ISTC" Plant Change Request 92-13-AF-00, "Replacement of Voltage Dropping Resistor with DC to DC Converter" Licensing Document Change Request 04-F020 87DP-0MC09 Item Procurement Specification (IPS) Requirements 33 IEEE Recommended Practice for Maintenance, Testing, and Replacement of Vented Lead-Acid Batteries for Stationary Applications. IEEE Std 450-1995-6- Attachment

IEEE Recommended Practice for Maintenance, Testing, and Replacement of Large Lead Storage Batteries for Generating Stations and Substations. IEEE Std 450-1980 Work History for 1EPKCF13 , Unit 1C Battery , 1/1/2004 to 1/17/2006 HPSI pump Oil trend plots showing Fe content Reviewed last 4 seal replacements 1MSIBP02 (replaced 10-13-02), 2MSIAP02 (replaced 3-20-02), 2MSIBP02 (replaced 4-24-05) and 3MSIBP02 (replaced 10-31-04) HPSI pumps. 01/30/06 10 CFR 50.59 Evaluation E-05-0028 Root Cause Investigation Manual For Significant CRDRs, November 2005, Revision 4 CRDR Program Report July - August and September - November Industry Key Performance Indicator Basis Document DRAFT, Human Performance Program Significant Human Performance - Event Criteria 4th Quarter 2005 Leader Effectiveness Assessment Performance Improvement Plan (PIP)

Control Room Logs for Operability Determination of CRDR 2844023 Palo Verde Nuclear Generating Station Business Plan Metrics Palo Verde Site Work Order Backlog Human Performance Talking Points Level 1 Projects by Leader Functional Responsibilities, Organizational Structure and Charter for PVNGS Performance Improvement Team, Rev 0 Palo Verde Expectations and Standards [for] Preventing Events Leadership Effectiveness Program 2005 Business Plan Procedure Use Stand Down, November 16, 2005 Root Cause Team Charter for Significant CRDR 2841586 Benchmark Trip Report From Fitzpatrick Nuclear Power Plant (Entergy) & Callaway Nuclear Power Plant (Ameren)

-7- Attachment

Information Request 1 - November 2005 Palo Verde PIR Inspection (IP 71152; Inspection Report 50-528,529,530/2006-08)

The inspection will cover the period of June 1, 2004 to January 26, 2006. All requested information should be limited to this period unless otherwise specified. The information may be provided in either electronic or paper media or a combination of these, although electronic on a CD is preferred.

Please provide the following information to Ryan Lantz in the Region IV Arlington office by November 21, 2005:

5. Summary list of all CRDRs of significant conditions adverse to quality opened or closed during the period 6. Summary list of all open CRDRs which were generated during the period 7. Summary list of all open CRDRs which were generated prior to the latest refueling outage 8. Summary list of all CRDRs closed during the specified period 9. A list of all corrective action documents that subsume or "roll-up" one or more smaller issues for the period 10. List of all root cause analyses completed during the period 11. List of root cause analyses planned, but not complete at end of the period 12. List of plant safety issues raised or addressed by the employee concerns program during the period 13. List of action items generated or addressed by the plant safety review committees during the period 14. All quality assurance audits and surveillances of corrective action activities completed during the period 15. A list of all quality assurance audits and surveillances scheduled for completion during the period, but which were not completed-8- Attachment

16. All corrective action activity reports, functional area self-assessments, and non-NRC third party assessments/audits completed during the period 17. Corrective action performance trending/tracking information generated during the period and broken down by functional organization 18. Copy of the governing procedures/policies/guidelines for:

1. Condition Reporting/ CRDR generation 2. Corrective Action Program 3. Root Cause Evaluation/Determination 4. Procedure Change Process/ Control 5. Operability Determinations (in Problem Identification Process)

6. The Palo Verde Performance Improvement Plan, updated with current status 19. A listing of all external events (OE) evaluated for applicability at Palo Verde during the period 20. CRDRs or other actions generated for each of the items below:

(1) Part 21 Reports (2) NRC Information Notices (3) NRC Bulletins (4) NRC Generic Letters (5) All LERs issued by Palo Verde during the period (6) Cited and NCVs issued to Palo Verde during the period (21) Safeguards event logs for the period (22) Radiation protection event logs (23) Current system health reports or similar information (24) Current predictive performance summary reports or similar information (25) Corrective action effectiveness review reports generated during the period (26) Current Organization Chart/ with phone/contact numbers (27) Description of Plant Performance Indicators (not NEI 99-02 Pis) and status/trends-9- Attachment

Information Request 2 - January 2006 Palo Verde PIR Inspection (IP 71152; Inspection Report 50-528,529,530/2006-08)

The inspection will cover the period of June 1, 2004 to February 3, 2006. All requested information should be limited to this period unless otherwise specified. The information may be provided in either electronic or paper media or a combination of these, although electronic on a CD is preferred.

Please assemble the following information (electronic if not indicated otherwise) segregated by Inspector. The information should be made available as soon as possible beginning January 17, 2006, at the PIR inspection workspace. If you have questions, please contact me or the inspector directly (emails given)

Mark Haire: Email MSH2@nrc.gov 1. All INTERNAL and EXTERNAL audits in the EP area (including EP Audit #0501)

2. CRDRs 2829230 and 2774206 details 3. Evaluation packages/ Actions taken for the following Generic Communications:

a. RIS 04-13, 04-15, 05-02, and 05-08 b. IN 2004-19, 2005-06, 2005-19 Greg Werner: Email GEW@nrc.gov Note: Please print all CRDRs/LER packages requested.

Doc Reference System/ Description Item(s) Requested N/A System Health Reports 2005 2nd and 3rd qtr system health reports N/A AFW open and closed MWOs List of 2005 MWOs with a brief description of what was done. If possible highlight those that were reworked and/or exceeded scheduled work hrs N/A AFW procedure and drawing List of Annunciator, AOP, EOP, backlog and P&IDs outstanding procedure change requests with a brief description N/A AFW - PM program List of AFW PM basis backlog and deferred PMs CRDR 2720228 Significant CRDR for failure of CRDR details AFA-P01 governor-10- Attachment

CRDR 2783251 Significant CRDR for seismic CRDR details alarm on AFW pump CRDR 28822997 IN 05-23 Vibration Induced CRDR details Degradation of Butterfly Valves CRDR 2753414 Containment penetration vent and CRDRs 2753414 and 340341 drain valves CRDR 818612 Nitrogen accumulator drop testing CRDR details does not take into account all pressure drops CRDR 2800972 Single active failure vulnerability in CRDR details AFW system CRDR 2746954 Dropping resistor failure for CRDRs 2746954, 2720228, TDAFW pmp governor 2709451, & WM 2732276 CRDR 2791716 Broken bolts on SI valves CRDR details CRDR 2815959 SFP leakage detection system CRDRs 2815959 and 2814209 &

blocked - look at WM to see if a WM 2773782 CRDR was initiated LER 2-2005-003 Two Independent Trains of LER and associated documents Auxiliary Feedwater Inoperable Dave Dumbacher:Email DED@nrc.gov 1. Engineering Audit Program procedure/ guidance.

2. Interview with Engineering Audit lead/owner to discuss program details.

3. Interviews (week 1) with system engineer(s) for SI system, DC system, and large voltage breakers.

4. System description training lesson plans for the SI system and DC voltage (battery,chargers and inverters) system.

5. Root Cause / Extent of Condition reviews for the 95-002 gas intrusion; SI pump mechanical seal problem; vibration of the Shutdown cooling lines off the RCS hotleg; and any breaker failures.

James Larsen: Email jrl1@nrc.gov Security Event Log entries, 04-07-069, 05-08-124, 05-09-138, and 05-11-181 thru 194 Security related CRDRs: 2798253, 2822270, 2822931, 2782187, 2774296, 287302, 2701132, 2716846, 2769720, 2819915, 2839186, 2774115, 2774387, 2788522, 2798646, 2768115, 2778202, 2790517.

-11- Attachment

Steve Garchow: Email smg@nrc.gov 1. A list of all significant CRDRs that have not been fully evaluated (i.e. the root cause is not complete). The list should included the CRDR numbers, a summary statement for each, and when the CRDR was initially written.

2. The charters for the 10 oldest CRDRs listed in 1. above.

3. If available, I would like trend graphs spanning the last two years showing the:

a. CRDR backlog for all CRDRs b. CRDR backlog for significant CRDRs c. CRAI backlog d. CRDR average age e. CRAI average age 4. If available, I would also like a trend graph showing the percent of CRDRs written over the last two years that were binned as human performance.

5. Access to the work order data base.

6. A list of all CRDRs over the last 3 years that include EDG or diesel generator in the title.

7. The first two pages of all CRDRs written in response to a reactor trip in the last 4 years.

8. Full package of the following CRDRs: 2734665, 2632229, 2605291, and 2780286.

9. CRAI 2759062.

10. Any trend showing error rates over the last 2 years.

-12- Attachment

Information Request 3 - January 2006 Palo Verde PIR Inspection (IP 71152; Inspection Report 50-528,529,530/2006-08)

The inspection will cover the period of June 1, 2004 to February 3, 2006. All requested information should be limited to this period unless otherwise specified. The information may be provided in either electronic or paper media or a combination of these, although electronic on a CD is preferred.

Please assemble the following information (electronic if not indicated otherwise) segregated by Inspector. The information should be made available as soon as possible beginning January 30, 2006, at the PIR inspection workspace. If you have questions, please contact me or the inspector directly (emails given)

Mark Haire: Email MSH2@nrc.gov 1. Copy of the NAD assessment that resulted in the January 2004 "Top Ten" list, a list of all CRDRs generated as a result of the assessment, and full details of the EP-related CRDRs.

Greg Werner: Email GEW@nrc.gov 1. General data on all Open SIG CRDRs, and all Closed SIG CRDRs that were closed in the last 2 years, showing:

A) Date the CRDR was originally opened, and date classified as SIG if it was not originally classified as SIG.

B) Date of completion of the associated root cause evaluation. This date should reflect the final date when all revisions/ reworks/ modifications of the root cause were completed.

C) Date of completion of corrective actions. This may be the CRDR closure date... but may not be if the CRAIs were closed to other CRAI/ CRDRs.

Dave Dumbacher:Email DED@nrc.gov 1) Copy of last 5 completed STs for 4KV breaker maintenance.

Steve Garchow: Email smg@nrc.gov-13- Attachment