IR 05000530/2009006

IR 05000528; 05000529; 05000530-09-006; 02/02/2009 - 02/27/2009; Palo Verde, Nuclear Generating Station Units 1, 2, and 3; Biennial Baseline Inspection of the Identification and Resolution of Problems, and Other Activities

ML090790431
Person / Time
Site:	Palo Verde
Issue date:	03/20/2009
From:	Anton Vegel NRC/RGN-IV/DRP
To:	Edington R NRC Region 4
References
CAL-4-07-004, EA-04-221, EA-09-057 IR-09-006
Download: ML090790431 (89)

Text

UNITED STATES NUC LE AR RE G UL AT O RY C O M M I S S I O N R E GI ON I V 612 EAST LAMAR BLVD , SU I TE 400 AR LI N GTON , TEXAS 76011-4125 March 20, 2009 EA-04-221 CAL-4-07-004 EA-09-057 Randall K. Edington, Executive Vice President, Nuclear/CNO Mail Station 7602 Arizona Public Service Company P.O. Box 52034 Phoenix, AZ 85072-2034 SUBJECT: PALO VERDE NUCLEAR GENERATING STATION UNITS 1, 2 AND 3 - NRC PROBLEM IDENTIFICATION AND RESOLUTION, AND CONFIRMATORY ACTION LETTER (CAL-4-07-004) FOLLOWUP INSPECTION REPORTS 05000528/2009006, 05000529/2009006, 05000530/2009006, AND NOTICE OF VIOLATION

Dear Mr. Edington:

On February 27, 2009, the U.S. Nuclear Regulatory Commission (NRC) completed a team inspection at your Palo Verde Nuclear Generating Station, Units 1, 2, and 3. The inspection examined activities related to problem identification and resolution, as well as activities related to the NRC Revised Confirmatory Action Letter, dated February 15, 2008, and the Site Integrated Improvement Plan, dated December 31, 2007. The enclosed inspection report documents the inspection findings which were discussed on February 27, 2009, and March 13, 2009, with Mr. Bement and other members of your staff during exit meetings.

This inspection reviewed activities conducted under your license as they relate to the identification and resolution of problems, compliance with the Commission's rules and regulations, and the conditions of your operating license. Within these areas, the inspection involved examination of selected procedures and representative records, observations of activities, and interviews with personnel. The inspection team reviewed approximately 350 action requests, work orders, associated root and apparent cause evaluations, and other supporting documentation to assess problem identification and resolution activities.

On the basis of the sample selected for review, the inspection team concluded that Palo Verde Nuclear Generating Stations implementation of the corrective action program has improved since the last problem identification and resolution team inspection. The team determined that Palo Verde Nuclear Generating Station staff had a low threshold for identifying problems, and issues were prioritized and evaluated commensurate with their safety significance. Corrective actions were typically implemented in a timely manner and addressed the identified causes of problems. Lessons learned from industry operating experience were usually reviewed and

Arizona Public Service Company -2-applied when appropriate, and, in most cases, audits and self-assessments were critical with appropriate actions taken to address identified issues.

The team determined that once problems were identified, your staff usually entered the issues into the corrective action program, but your staff was inconsistent in ensuring that identified problems were thoroughly evaluated in a timely manner. The team identified some issues with the quality of evaluations and the linking of corrective action documents to corrective actions.

The team identified that some operability assessments and reportability reviews were not being implemented consistent with procedural guidance and many of these assessments did not demonstrate the appropriate level of technical rigor to support conclusions made for operability.

Your ability to effectively evaluate problems has improved, but is requiring additional layers of review to ensure the correct conclusions. The NRC will continue to focus our inspections in this area to verify sustained improvement is demonstrated.

The team conducted interviews with 34 individuals. On the basis of the interviews conducted during this inspection, observations of plant activities, and reviews of the corrective action and employee concerns programs, the team determined that site personnel were willing to raise safety issues and document them in the corrective action program. The team observed that workers at the site felt free to report problems to their management and were willing to use the employee concerns program.

During this inspection, the NRC completed assessing your actions associated with a Yellow finding (Violation 05000528,529,530/2004014-01) opened in the fourth quarter of 2004 involving voided containment sump suction piping for all three units. Detailed observations, assessments, and conclusions of the inspection are presented in the enclosed inspection report.

These inspections concluded that the root causes of the finding were adequately defined and understood and the corrective actions resulting from the evaluations appropriately addressed the identified causes and extent of condition. Based on our inspection results the NRC considers the Yellow finding closed.

During this inspection, the NRC completed reviewing all associated tasks for seven of the twelve Confirmatory Action Letter key performance areas documented in the February 15, 2008,

"Revised Palo Verde Nuclear Generating Station Confirmatory Action Letter,"

(ADAMS ML080460653). Specifically, reviews were completed for Key Performance Areas 1, 3, 4, 5, 7, 8, and 10. Key Performance Area 1 involved root and contributing causes identified in Palo Verde Nuclear Generating Station evaluations in response to the Yellow finding associated with voided containment sump suction piping for all three units. Key Performance Area 3 involved problem identification and resolution performance issues. Key Performance Area 4 involved human performance issues. Key Performance Area 5 involved problems with the implementation of engineering programs. Key Performance Area 7 involved issues identified during the 2007 independent safety culture assessment. Key Performance Area 8 involved problems associated with standards and expectations for performance and holding individuals accountable for nuclear safety. Key Performance Area 10 involved problems associated with the emergency preparedness program. Based on our inspection results we consider these seven key performance areas closed. The closure of these seven Confirmatory Action Letter key performance areas is discussed in more detail in Section 4OA5 of this report.

One violation is cited in the enclosed Notice of Violation and the circumstances surrounding it are described in detail in the subject inspection report. The violation involved failure to adequately translate design basis maximum condensate storage tank temperature requirements into procedures to ensure the plant is operated within its design basis (EA-09-057). Although

Arizona Public Service Company -3-determined to be of very low safety significance (Green), this violation is being cited in the Notice because not all of the criteria specified in Section VI.A.1 of the NRC Enforcement Policy for a noncited violation were satisfied. Specifically, Palo Verde Nuclear Generating Station failed to restore compliance within a reasonable time after the violation was first identified in NRC Inspection Report 05000528, 05000529, 05000530/2007012. You are required to respond to this letter and should follow the instructions specified in the enclosed Notice when preparing your response. The NRC will use your response, in part, to determine whether further enforcement action is necessary to ensure compliance with regulatory requirements.

This report documents seven additional NRC-identified and/or self-revealing findings of very low safety significance (Green). All of these findings were determined to involve violations of NRC requirements. However, because these violations were of very low safety significance and the issues were entered into your corrective action program, the NRC is treating these findings as noncited violations, consistent with Section VI.A.1 of the NRCs Enforcement Policy. The noncited violations are described in the subject inspection report. If you contest the violations or the significance of the violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U. S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C. 20555-0001, with copies to the Regional Administrator, U. S. Nuclear Regulatory Commission, Region IV, 612 East Lamar Blvd, Suite 400, Arlington, Texas 76011; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555-0001; and the NRC resident inspector at the Palo Verde Nuclear Generating Station facility.

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Anton Vegel, Deputy Director Division of Reactor Projects Docket Nos: 50-528 50-529 50-530 License Nos: NPF-41 NPF-51 NPF-74

Enclosures:

Notice of Violation and NRC Inspection Report 05000528/2009006, 05000529/2009006, 05000530/2009006 w/Attachments:

1. Supplemental Information 2. Information Request

Arizona Public Service Company -4-

REGION IV==

Docket: 50-528, 50-529, 50-530 License: NPF-41, NPF-51, NPF-74 Report: 05000528/2009006; 0500529/2009006; 0500530/2009006 Licensee: Arizona Public Service Company Facility: Palo Verde Nuclear Generating Station Units 1, 2,and 3 Location: 5951 S. Wintersburg Road Tonopah, Arizona Dates: February 2, 2009 through February 27, 2009 Inspectors: N. Taylor, Senior Resident Inspector (Team Leader)

D. Bollock, Project Engineer M. Catts, Resident Inspector D. Dumbacher, Senior Resident Inspector H. Freeman, Senior Reactor Inspector E. Knutson, Senior Resident Inspector D. Proulx, Senior Project Engineer R. Rodriguez, Senior Reactor Inspector E. Ruesch, Reactor Inspector R. Walton, Senior Reactor Inspector Approved By: Anton Vegel, Deputy Director Division of Reactor Projects-1- Enclosure 2

SUMMARY OF FINDINGS

IR 05000528; 05000529; 05000530/2009006; 02/02/2009 - 02/27/2009; Palo Verde, Nuclear

Generating Station Units 1, 2, and 3; Biennial baseline inspection of the Identification and Resolution of Problems, and Other Activities.

The report covered a 4-week period of inspection by three senior resident inspectors, a resident inspector, and six region-based inspectors. Eight Green findings of very low safety significance were identified during the inspection. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 4, dated December 2006.

Identification and Resolution of Problems The team concluded that the implementation of the corrective action program at the Palo Verde Nuclear Generating Station was generally effective. Once entered into the system, items were screened and prioritized in a timely manner using established criteria. The station properly evaluated items entered into the corrective action program commensurate with their safety significance. Corrective actions addressed the identified causes. The team selected and reviewed approximately 350 risk-informed action requests, work orders, associated root and apparent cause evaluations, and other supporting documentation to assess problem identification and resolution activities. The inspectors verified that the licensee had taken actions to address previous NRC findings. The team performed a five year review of the diesel generator performance and a focused review of inverter systems to determine whether problems were being effectively addressed and that the corrective action program was effective in identifying problems. As a result of these reviews, the team concluded that when site personnel identified problems, they entered them into the corrective action program at a low threshold; however, the team identified several issues with the quality of evaluations and linking of corrective action documents. Corrective actions were generally implemented in a timely manner, although the team identified several corrective actions associated with conditions adverse to quality that were not completed in a timely manner. The team also identified that operability assessments and reportability reviews were not being implemented consistent with procedural guidance and, although the equipment remained operable, many of these assessments did not demonstrate the appropriate level of technical rigor to support conclusions made for operability.

The team determined that in most cases the licensee identified, reviewed, and applied industry operating experience relevant to the facility, and had entered applicable items into the corrective action program. The team noted that the licensee was evaluating industry operating experience when performing root cause and apparent cause evaluations. The team also noted that Quality Assurance audits and other self-assessment activities were generally effective.

Based on 34 interviews conducted during this inspection, observations of plant activities, and reviews of the corrective action and nuclear safety concerns programs, the team determined that site personnel were willing to raise safety issues and document them in the corrective action program. The team observed that workers at the site felt free to report problems to their management, and were willing to use the Employee Concerns Program.

Cornerstone: Mitigating Systems

Green.

The inspectors identified a noncited violation of 10 CFR Part 50,

Appendix B, Criterion V, "Instructions, Procedures, and Drawings," when, on November 8, 2008, Palo Verde Nuclear Generating Station did not adequately test the emergency diesel generator to verify that a newly identified emergency diesel generator governor issue, would not cause the emergency diesel generators start time to exceed the Technical Specification allowable limit of 10 seconds. Palo Verde Nuclear Generating Station did not specify testing requirements and acceptance criteria to ensure continued operability of the affected emergency diesel generators. As an immediate corrective action, Palo Verde Nuclear Generating Station reevaluated the issue and specified additional testing requirements with specific acceptance criteria for the affected emergency diesel generators pending completion of a hardware modification that would eliminate the issue. The licensee documented this performance deficiency in Palo Verde Action Request 3280971.

The finding was more than minor because, if left uncorrected, it had the potential to lead to a more significant safety concern; specifically, that emergency diesel generator start time in excess of the Technical Specification allowable maximum may not have been promptly identified. The finding is associated with the mitigating systems cornerstone. The finding was evaluated in accordance with Inspection Manual Chapter 0609.04, and determined to be of very low safety significance because the finding was confirmed not to result in loss of operability or functionality. The finding had a crosscutting aspect in the problem identification and resolution component of the corrective action program because Palo Verde Nuclear Generating Station did not thoroughly evaluate operability of the emergency diesel generators that remained susceptible to governor-related start time degradation P.1.c] (Section 4OA2).

Green.

The inspectors identified a Green noncited violation of 10 CFR Part 50,

Appendix B, Criterion V, Instructions, Procedures and Drawings, for the licensees failure to follow procedures for identifying the significance of a significant condition adverse to quality. Specifically, the Action Request Review Committee screened Palo Verde Action Request 3221258 as an adverse Condition Report Disposition Request, despite the fact that the Procedure 01DP-OAP12 required it to be screened as significant. This error resulted in the failure to understand the failure mode associated with a safety related essential cooling water pump such that corrective actions would prevent recurrence. The licensee documented the failure to properly screen this issue for significance in Palo Verde Action Request 3288713.

The finding is more than minor because the finding is associated with the equipment performance attribute of the mitigating systems cornerstone, and affects the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The inspectors utilized Manual Chapter 0609.04, "Phase 1 -

Initial Screening and Characterization of Findings," to determine that the finding was of very low safety significance because it did not represent a design or qualification deficiency, did not result in a loss of safety function, or screen as a risk-significant external event. The cause of this finding is related to the problem identification and resolution crosscutting component of corrective action program, in that licensee failed to properly classify and evaluate a significant condition adverse to quality P.1(c) (Section 4OA2).

Green.

The inspectors identified a noncited violation of 10 CFR Part 50,

Appendix B, Criterion V, "Instructions, Procedures, and Drawings," when, on February 10, 2009, it was determined that 62 scaffolds that did not comply with the engineering installation specification had been in place in the three units in excess of 90 days, and that these scaffold installations had not been screened in accordance with 10 CFR 50.59, nor had these nonconforming conditions been evaluated for their potential impact on equipment operability. As immediate corrective action, Palo Verde Nuclear Generating Station informed the applicable control room operators of the 62 nonconforming conditions and operability assessments were performed under Palo Verde Action Requests 3283371, 3283489, and 3281680. Additionally, Palo Verde Nuclear Generating Station initiated Palo Verde Action Request 3283865 to perform 10 CFR 50.59 screenings on the 62 non-compliant scaffolds.

The finding was more than minor because it is associated with the mitigating systems cornerstone attribute for protection against external factors and affected the cornerstone objective of ensuring the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. The finding, associated with the mitigating systems cornerstone, was evaluated in accordance with Inspection Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," and determined to be of very low safety significance per the Significance Determination Process because the finding was not a design or qualification deficiency, did not represent a loss of a system/train safety function, and did not screen as potentially risk significant due to external events. The finding had a crosscutting aspect in the human performance component of resources because Palo Verde Nuclear Generating Station did not ensure that adequate personnel were assigned to ensure that long term scaffold installations remained compliant with applicable procedural requirements H.2.a] (Section 4OA2).

Green.

The inspectors identified a cited violation of 10 CFR Part 50, Appendix B,

Criterion III, "Design Control," for the failure of engineering personnel to translate the design basis maximum condensate storage tank temperature requirements into procedures to ensure the plant is operated within its design basis. This issue was entered into the licensees corrective action program as Palo Verde Action Requests 3289578 and 3289530.

This finding is greater than minor because it is associated with the mitigating systems cornerstone attribute of equipment performance and affected the cornerstone objective of ensuring the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. Using the Inspection Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding is determined to have very low safety significance since it only affected the mitigating systems cornerstone and did not represent a loss of system safety function. The cause of this finding had crosscutting aspects associated with corrective action component of the problem identification and resolution area in that engineering personnel failed to thoroughly evaluate problems such that resolutions ensured that the problems were resolved P.1(c) (Section 4OA2).

Green.

The inspectors identified a noncited violation of 10 CFR Part 50,

Appendix B, Criterion XVI, Corrective Action, for the licensees failure to promptly identify and correct a condition adverse to quality. Specifically, the licensee failed to incorporate industry and vendor recommended preventative maintenance requirements to prevent the age related degradation of safety-related inverter components. This finding was entered into the licensees corrective action program as Palo Verde Action Request 3291971.

The inspectors determined that the failure to identify the necessary maintenance practices and take corrective actions prior to the 2008 inverter failures was a performance deficiency. This finding is more than minor because it affects the equipment performance attribute of the mitigating systems cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1 worksheets, the team determined that a Phase 2 analysis was required because the finding represented a loss of system safety function. A Phase 2/Phase 3 significance determination was performed by an NRC senior reactor analyst. Based on a bounding analysis, the analyst determined that the delta core damage frequency result was less than 1.0E-7/yr. This noncited violation was therefore determined to be of very low safety significance. This finding has a crosscutting aspect in the problem identification and resolution component of operating experience, in that the licensee failed to implement operating experience through changes to station procedures P.2(b) (Section 4OA2).

Green.

The inspectors identified a noncited violation of 10 CFR Part 50,

Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure of operations personnel to follow the corrective action program to ensure that degraded and nonconforming conditions associated with safety related systems and systems important to safety were properly reviewed for operability.

Specifically, between December 21, 2006, and January 30, 2009, operations personnel failed to perform adequate operability determinations of Palo Verde Action Requests associated with the component design basis review project and other site projects, resulting in 97 Palo Verde Action Requests that either needed an immediate operability determination or a functional assessment, or needed more information to provide reasonable assurance of operability. Of the 97 examples 20 occurred following implementation of corrective actions associated with the Confirmatory Action Letter to improve this process and therefore are reflective of current performance. This issue was entered into the licensee's corrective action program as Palo Verde Action Request 3281099.

The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings,"

the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program because 9 of the 20 examples, reflective of current performance, were not thoroughly evaluated such that the resolutions address causes and extent of conditions, as necessary, including properly evaluating for operability conditions adverse to quality P.1(c) (Section 4OA5).

Green.

The inspectors identified a noncited violation of 10 CFR Part 50,

Appendix B, Criterion V, Instructions, Procedures, and Drawings, for the failure of operations personnel to follow the corrective action program to ensure that degraded and nonconforming conditions associated with safety related systems and systems important to safety were reviewed for operability. Specifically, between December 21, 2006 and January 30, 2009, operations personnel failed to perform adequate operability determinations of Palo Verde Action Requests associated with the component design basis review project and other site projects, resulting in 97 Palo Verde Action Requests that either needed an immediate operability determination or a functional assessment, or needed more information to provide reasonable assurance of operability. Of the 97 examples 20 occurred following implementation of corrective actions to improve this process and therefore are reflective of current performance. This issue was entered into the licensee's corrective action program as Palo Verde Action Request 3281099.

The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, Phase 1 - Initial Screening and Characterization of Findings, the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of human performance associated with resources because 11 of the 20 examples, reflective of current performance, were the result of inadequate procedural guidance governing the conduct of operability determinations to ensure that conditions adverse to quality are properly evaluated for their potential operability impacts H.2(c) (Section 4OA5).

Green.

A self-revealing noncited violation of 10 CFR Part 50, Appendix B,

Criterion XVI, "Corrective Action," was identified for the failure of the licensee to promptly identify and correct a condition adverse to quality associated with the high pressure safety injection system piping. Specifically, between January 18, 1989, and October 12, 2006, the licensee failed to ensure that select sections of Unit 1 high pressure safety injection Train B piping were inspected to prevent erosion due to cavitation. This resulted in a through-wall leak in the high pressure safety injection Train B recirculation line. This issue was entered into the licensees corrective action program as Condition Report/Disposition Request 2932507.

The performance deficiency associated with this finding involved the licensees failure to promptly identify and correct a condition adverse to quality associated with the high pressure safety injection system piping. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the reliability and availability of systems that respond to initiating events. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. This finding was evaluated as not having a crosscutting aspect because the performance deficiency is not indicative of current performance (Section 4OA5).

Licensee-Identified Violations

A violation of very low safety significance, which was identified by the licensee has been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensees corrective action program. This violation and corrective actions are listed in Section 4OA7 of this report.

REPORT DETAILS

4OA2 Problem Identification and Resolution

The team based the following conclusions, in part, on a review of issues that were identified in the assessment period, which ranged from March 20, 2007, (the last biennial Problem Identification and Resolution inspection) to the end of the on-site portion of the inspection on February 27, 2009.

.1 Assessment of Corrective Action Program

a. Inspection Scope

The inspection team reviewed the procedures describing Palo Verde Nuclear Generating Stations corrective action program. The licensee identified problems for evaluation and resolution by initiating condition reports in their condition reporting system. The team evaluated the methods for assigning and tracking issues to ensure that issues were screened for operability and reportability, prioritized for evaluation and resolution in a timely manner commensurate with their safety significance, and tracked to identify adverse trends and repetitive issues. In addition, the team interviewed plant staff and management to determine their understanding of and involvement with the corrective action program. The condition reports and other documents reviewed, as well as key personnel contacted, are listed in Attachment 1 of this report.

The team reviewed condition reports to determine if site personnel properly identified, characterized, and entered problems into the corrective action program for evaluation and resolution. The team selected items from the maintenance, operations, engineering, emergency preparedness, physical security, radiation safety, and oversight programs to ensure that the licensee appropriately addressed problems identified in each functional area. The team selected a risk informed sample of condition reports that had been issued since the last NRC Problem Identification and Resolution inspection conducted in March 2007. The team considered risk insights from the NRC's and Palo Verde Nuclear Generating Stations risk analyses to focus the sample selection and plant tours on risk significant systems and components. The corrective action review was expanded to five years for evaluation of the emergency diesel generator system.

The team selected items from various processes used at Palo Verde Nuclear Generating Station to verify that they were appropriately considered for entry into the corrective action program. Specifically, the team reviewed a sample of operability determinations, engineering system health reports, and completed surveillance tests. The team also reviewed work orders for selected components to determine if station personnel entered issues identified during the performance of preventive maintenance into the corrective action program.

The team reviewed condition reports to assess whether the licensee adequately evaluated and prioritized identified problems. The issues reviewed encompassed the full range of evaluations, including root cause analyses, apparent cause evaluations, and common cause analyses. The review included the appropriateness of the assigned significance, the scope and depth of the causal analysis, and the timeliness of

resolution. For significant conditions adverse to quality, the team reviewed the licensee's corrective actions to preclude recurrence. The team observed meetings of the Action Request Review Committee and Condition Review Group, in which station management reviewed new condition reports for prioritization and assignment, and evaluated root cause evaluations and selected apparent cause evaluations including associated corrective action assignments. The team also reviewed equipment operability determinations, reportability assessments, and extent of condition reviews for selected problems.

The team reviewed the corrective actions associated with selected condition reports to determine whether the actions addressed the identified causes of the problems. The team reviewed condition reports for repetitive problems to determine whether previous corrective actions were effective. The team also reviewed licensee timeliness in implementing corrective actions and their effectiveness in precluding recurrence for significant conditions adverse to quality. The team reviewed corrective actions associated with selected noncited violations and findings to determine whether the station properly evaluated and resolved these issues.

b. Assessment

(1) Identification of Issues The team concluded that problems were generally identified and documented in accordance with the licensees corrective action program guidance and NRC requirements. The licensee had written approximately 26,000 action requests during the two year period of review, which demonstrated that the licensee was identifying problems and entering them into the corrective action program. The team concluded the licensee was identifying problems at an appropriately low threshold, however three themes related to identification of issues needed additional focus: Palo Verde Action Request (PVAR) initiation timeliness; sensitivity to degraded or unacceptable conditions; and, identification of newly discovered degraded, nonconforming, or unanalyzed conditions during audits and other review processes. The team noted the following examples of these issues:

• PVAR 3221258 was initiated at 4 pm on September 10, 2008, to record the excessive mechanical seal leakage from mechanical Seal 3MEAP01. The pump was subsequently declared inoperable and a 72-hour Technical Specification Action Statement was entered to repair the pump. The inspectors discovered that operations personnel were aware of the leakage for at least two shifts prior to a PVAR being written. This does not meet the timeliness requirements of Procedure 01DP-0OP12, which requires PVARs for degraded conditions to be written by the end of the current shift.

• On July 6, 2007, PVAR 3037396 was written by operations stating that the outage would need to be extended due to check Valve 1PSIEV123 leaking past its seat. On July 10, 2007, the corrective action group wrote PVAR 3038324 to document;

(1) that 1PSIEV123 failed a surveillance test and that a maintenance rule evaluation was required,( 2) failure to initiate a PVAR/Condition Report/Disposition Request (CRDR) to document the valves test failure, and

(3) a potential Technical Specification violation had occurred requiring a review for reportability. Actions taken to document the failed surveillance test and

missed reportability and maintenance rule evaluation, were examples of sensitivity to degraded conditions which resulted in failing to document an adverse condition.

• Procedure 01DP-0AP12, Palo Verde Action Request Processing, Revision 9, requires in Paragraph 3.2.14 that equipment related PVARs shall be initiated and processed out of the initiation step no later than the end of the current shift.

Contrary to this procedural requirement, a nonconforming condition associated with assumed maximum condensate storage tank temperature was identified by Palo Verde Nuclear Generating Station staff on October 4, 2007, but PVAR 3073243 was not initiated for this condition until October 8, 2007. After prompting by the inspectors, the licensee documented this condition in PVAR 3285240. This was an example of untimely initiation of a PVAR.

• The control room review performed for PVAR 3073243, associated with condensate storage tank temperature control, was incorrect and the need for an immediate operability determination was missed, as determined by the apparent cause evaluation performed under the PVAR. The incorrect control room review was identified by the NRC on October 23, 2007 and corrected, but no PVAR was written to capture why the senior reactor operator had missed the need for an immediate operability determination. After prompting by the inspectors, this error was subsequently recorded in PVAR 3283326 and again in PVAR 3284779.

This was an example of failure to initiate a PVAR for a procedural error discovered during other review processes.

• In September 2007, during apparent cause evaluation for CRDR 3053386, associated with reactor coolant system piping break analysis, the evaluator determined that the 10 CFR 50.59 evaluation that had been performed for an engineering calculation was discrepant in that it did not identify that a change in calculation methodology had been made. No PVAR was initiated to reflect that the 50.59 evaluation had not recognized the change of methodology. After discussions with the inspectors, the licensee documented this condition in PVAR 3283590. This was another review that did not document a new adverse condition.

• During plant tours, as part of the scope of the Problem Identification and Resolution inspection, the inspectors identified eleven examples of plant conditions which met the threshold for the licensee to initiate a PVAR. These examples covered items such as broken pipe support brackets, vent port orientation allowing potential foreign material exclusion entry, loose electrical connectors, missing valve position indicators, multiple lube oil and fuel oil leaks on the emergency diesel generator skids, leaking service water relief valves, damaged floor drain covers, drained emergency diesel generator crankcase manometer indicators, and soot buildup on one of the emergency diesel generator turbocharger inlet pipes.

• The inspectors observed personnel performing cleaning on a protected train emergency diesel generator and questioned whether this was consistent with licensee expectations for limiting work on protected equipment. Palo Verde Nuclear Generating Station indicated that procedures were in place concerning limitations of maintenance activities to be performed on protected equipment.

The licensee initiated PVAR 3283346 to evaluate whether procedural guidance concerning limitations on activities, such as equipment walkdowns and cleaning, was warranted. This was an example of not identifying or recognizing uncontrolled plant activities.

• A fix-it-now senior reactor operator indicated that PVAR screening to determine which PVARs should be sent for control room review was primarily done by the fix-it-now senior reactor operators. This was contrary to Procedure 01DP-0AP12, Palo Verde Action Request Processing, which states that this evaluation shall normally be performed by the work control senior reactor operator. This was an example of failure to initiate a PVAR for a known procedure noncompliance.

• On September 16, 2008, recovery efforts following failure and transfer of the Unit 2 Class 1E Vital Inverter PNC-N13, and downpower of associated 120 Vac bus necessitated preparations to shut down the unit per Technical Specification Limiting Conditions for Operation 3.0.3 requirements. The control room supervisor took actions per Procedure 40OP-9ZZ05, Power Operations, which included a review of the maneuvering box tools, a plan for shutdown using a combination of boration and rod movement. However, in the existing plant configuration, control element assemblies were not available. Therefore, the shutdown plan reviewed and approved by the control room supervisor was not a valid or usable method of reducing plant power for shutdown. This unacceptable plan was not identified by the licensee during their reviews of this event. The licensee has now documented this performance deficiency in PVAR 3283886.

This is an example of a significant CRDR review not capturing the need for additional guidance to shut down the plant when control element assembly indications may not be available.

(2) Prioritization and Evaluation of Issues The team reviewed PVARs to assess the licensees ability to properly prioritize and evaluate issues. The team noted that evaluations appeared to be adequate, however some examples of programmatic breakdowns and inadequate evaluations were noted.

Programmatic Problems with Evaluations

• The licensee failed to perform 16 apparent cause evaluations for maintenance rule functional failures, as required by procedures, due to programmatic CRDR closure deficiencies. The corrective action program did not allow for indeterminate causes of functional failures to provide closure for the associated CRDRs. As a process workaround, the CRDRs were downgraded to an adverse closed classification prior to implementing the apparent cause evaluation to support a maintenance rule functional failure assessment.

CRDR 3272899 and CRAI 3283927 were written to address the process issues.

Each missed apparent cause evaluation was reevaluated and all failures were counted toward the Palo Verde Nuclear Generating Stations maintenance rule 10 CFR 50.65 (a)(1) monitoring criteria. No systems were moved to an (a)(1)status as a result of these failures.

• The licensee failed to evaluate the acceptability of long term scaffolding installations. Approximately 62 scaffolds were in existence past the procedural

requirement to initiate a PVAR and perform a 10 CFR 50.59 review. The lack of review for these scaffolds was not recognized by the licensee until discussions with the inspectors.

Inadequate Evaluations

• Low pressure safety injection and containment spray pumps full flow recirculation during testing, as discussed in NRC Bulletin 88-04, was not evaluated as requested in PVAR 3085457 to ensure sufficient flow to cool the pumps when all safety injection pumps are started. PVAR 3284044 was written to address the concern and implement immediate corrective actions.

• Regarding the September 11, 2008, mechanical seal failure of an essential cooling water pump, the licensee failed to perform a causal evaluation and propose actions to preclude repetition due to a screening error by the Action Request Review Committee.

(3) Operability Determinations Problems with the quality of operability determinations have been raised during prior NRC inspections. The team reviewed the licensees actions to improve in this area, and concluded that these actions have improved performance but have not been fully effective. The team identified numerous examples of inadequate operability determinations:

• Numerous PVARs initiated from the component design basis review project did not receive the necessary immediate operability determination screening to evaluate degraded, nonconforming, or unanalyzed conditions; or did not fully evaluate the issue for operability. This resulted in approximately 97 PVARs needing an immediate operability determination or an immediate operability determination reevaluation. To address this issue, PVAR 3281099 was written.

• An NRC-identified Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," was identified when, on November 8, 2008, Palo Verde Nuclear Generating Station did not adequately assess how a newly identified emergency diesel generator governor issue, that could cause the emergency diesel generator's start time to exceed the Technical Specification allowable limit of 10 seconds, could affect three other susceptible emergency diesel generators. Palo Verde Nuclear Generating Station did not specify testing requirements and acceptance criteria to ensure continued operability of the affected emergency diesel generators pending completion of a hardware modification that would eliminate the issue.

• A required immediate operability determination was missed for PVAR 3283489 associated with scaffold program controls. No immediate operability determination was performed until prompted by inspectors (62 of these scaffolds were not compliant with stand-off distance specifications, which was a nonconforming condition). An immediate operability determination was performed satisfactorily on February 19, 2009.

• The control room review performed for the failure to control the condensate storage tank maximum design temperature discussed in PVAR 3073243 was incorrect and the need for an immediate operability determination was missed, as determined by the apparent cause evaluation performed under the PVAR. The incorrect control room review was identified by the NRC on October 23, 2007, and corrected. The component design basis review team initiated a PVAR in August 2008 for failure to complete corrective actions for this issue. That PVAR also received an inadequate operability determination.

(4) Root Cause Evaluations The team reviewed the root cause evaluation, apparent cause evaluation, and direct cause evaluation procedures. The team noted that during the inspection period for 2007 and 2008, the 3-month rolling average root and apparent cause evaluation age had trended down from a 47 day average to a 25 day average.

(5) Effectiveness of Corrective Actions Overall, the team concluded that the licensee generally developed appropriate corrective actions to address problems, although the team identified several examples of ineffective corrective actions. These examples included:

• The failure to adequately translate the design basis condensate storage tank maximum temperature requirements into applicable procedures.

• The failure to identify and correct deficiencies in preventative maintenance practices associated with the safety-related inverters.

• The failure to ensure that the select sections of emergency core cooling system piping were inspected to prevent erosion due to cavitation which resulted in a through-wall leak in the Unit 1 high pressure safety injection Train B recirculation line. To address this issue, PVAR 3285128 was written.

• Four findings from the 2008 resident inspector reports demonstrated ineffective corrective actions:

Noncited Violation 05000528; 05000530/2008004-06: Failure to correct a condition adverse to quality with the refueling water tank instruments in a timely manner. On June 16, 2006, engineering personnel inadvertently closed a CRDR that assigned work orders to be completed to correct deficiencies associated with flooding of the refueling water tank instrument pit, which left these deficiencies uncorrected for Units 1 and 3 until June 2008.

Noncited Violation 05000528/2008003-07: Involving the failure to take timely corrective action for a condition adverse to quality resulting in safety injection Tank 1A becoming inoperable. Between August 2007 and June 2008, operations and maintenance personnel failed to identify and correct the source of a nitrogen leak on safety injection Tank 1A.

Noncited Violation 05000529/2008003-04: Involving the failure to prevent recurrence of a significant condition adverse to quality for the feedwater isolation valves. Between June 28, 1998, and July 17, 2006, engineering personnel failed

to implement adequate corrective actions to preclude recurrence of a significant condition adverse to quality. Specifically, for three times in eight years, the four-way 'N' valve for an economizer main feedwater isolation valve became lodged in the center blocked position, preventing fast closure of the main feedwater isolation valve upon receipt of a main steam isolation signal.

Noncited Violation 05000528/2008004-04: Involving the failure to provide an adequate procedure to control essential spray pond missile hazards. Since January 15, 1997, engineering personnel failed to establish adequate procedures to ensure evaluation and approval of transient missile structure hazards that have an effect on the operability of the essential spray ponds. Appropriate corrective actions were not taken to address safety issues and adverse trends in a timely manner, commensurate with their safety significance.

c. Findings

(1) Inadequate Operability Evaluation for Potential Emergency Diesel Generator Slow Start Issue

Introduction.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," when, on November 8, 2008, Palo Verde Nuclear Generating Station did not adequately test the emergency diesel generator to verify that a newly identified emergency diesel generator governor issue, that could cause the emergency diesel generator's start time to exceed the Technical Specification allowable limit of 10 seconds, could affect three other susceptible emergency diesel generators. Palo Verde Nuclear Generating Station did not specify testing requirements and acceptance criteria to ensure continued operability of the affected emergency diesel generators pending completion of a hardware modification that would eliminate the issue.

Description.

Following installation of design modification work Order 2835485, "Upgrade Emergency Diesel Generator [EDG] Governing System to a Woodward 2301-A Electronic Governor and an EDG 50-PLS Governor Actuator," in October 2008, Emergency Diesel Generator 1A demonstrated slower than expected emergency mode start times. On some occasions, the start time was in excess of the Technical Specification requirement of 10 seconds.

One feature of the newly installed governor was to perform slow engine starts while in the test mode of operation. This feature, called the start fuel limit, was desirable because allowing the engine to start in 12 to 18 seconds, when it was not required for emergency use, reduced stress and wear on the engine. Following discussion with the vendor, Palo Verde Nuclear Generating Station determined that the slow emergency mode start times were an unanticipated consequence of the start fuel limit feature.

During an engine slow start, the start fuel limit feature operates by causing the governor to reposition from the full fuel position during a portion of the start sequence. This sends less fuel to the engine than it would receive in the emergency start mode and results in a slower start time. The fuel start limit is disengaged at a specific engine speed during the start, based on output from the governor's speed sensor; however, it is then reengaged during engine coast down, at such time as the signal from the speed sensor is lost.

Depending on the governor hydraulic pressure at the time that this occurs, the governor

actuator may reposition to provide reduced fuel. During a subsequent engine start in the emergency mode, the time required for governor hydraulic pressure to increase and reposition the actuator to the full fuel position accounts for the increased start time.

This issue was addressed by modifying the start fuel limit feature circuitry so that it could only be applied during test mode starts from the initial engine start signal until the specified dropout speed was reached during engine acceleration. This modification was accomplished through a revision to design modification work Order 2835485.

New governors had previously been installed per design modification work Order 2835485 on Emergency Diesel Generator 1B and both Unit 2 emergency diesel generators. Pending modification of the start fuel limit feature on those emergency diesel generators, an immediate operability determination was performed under PVAR 3248239. This immediate operability determination concluded that the potentially affected emergency diesel generators remained operable based on their previously demonstrated ability to meet surveillance testing acceptance criteria. The inspectors did not consider this to be adequate justification, given that the condition had been previously unknown and required a hardware modification to correct. After discussion of this issue with the inspectors, Palo Verde Nuclear Generating Station generated a prompt operability determination under PVAR 3280971. The prompt operability determination stated that numerous variable conditions could mask a degrading trend in emergency diesel generator start time, and therefore required that increased frequency testing be performed on the remaining emergency diesel generators that did not have the modification to the start fuel limit feature installed (at that time, Emergency Diesel Generators 1B and 2B).

As part of PVAR 3280971, Palo Verde Nuclear Generating Station reevaluated the issue and specified additional testing requirements with specific acceptance criteria for the affected emergency diesel generators pending completion of a hardware modification that would eliminate the issue.

Analysis.

The performance deficiency associated with this event was that Palo Verde Nuclear Generating Station did not adequately assess how an identified emergency diesel generator governor issue, that could cause the emergency diesel generator's start time to exceed the Technical Specification allowable limit of 10 seconds, could affect two susceptible emergency diesel generators. The finding is associated with the mitigating systems cornerstone. The finding is more than minor because, if left uncorrected, it had the potential to lead to a more significant safety concern; specifically, that emergency diesel generator start time in excess of the Technical Specification allowable maximum may not have been promptly identified. The finding was evaluated in accordance with Inspection Manual Chapter 0609.04, and determined to be of very low safety significance because the finding was confirmed not to result in loss of operability or functionality. The finding had a crosscutting aspect in the problem identification and resolution component of the corrective action program because Palo Verde Nuclear Generating Station did not thoroughly evaluate operability of the emergency diesel generators that remained susceptible to governor-related start time degradation P.1(c).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," states, in part, "Instructions, procedures, or drawings shall include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily accomplished."

Contrary to the above, from November 8, 2008, until February 13, 2009, Palo Verde Nuclear Generating Station did not establish appropriate quantitative or qualitative acceptance criteria for determining that the start times for Emergency Diesel Generators 1B and 2B were not degraded while these engines were awaiting installation of a hardware modification that would eliminate their susceptibility to an identified emergency diesel generator governor-related start time degradation phenomenon.

Because this noncompliance is of very low safety significance and was entered into the corrective action program as PVAR 3280971, this violation is being treated as a noncited violation, consistent with Section VI.A.1 of the NRC Enforcement Policy:

NCV 05000530/2009006-01, "Inadequate Operability Evaluation for Potential Emergency Diesel Generator Slow Start Issue.

(2) Failure to Follow Procedure for Screening Significant Condition Adverse to Quality

Introduction.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings, for the licensees failure to follow procedures for identifying a significant condition adverse to quality.

Specifically, the Action Request Review Committee screened PVAR 3221258 as an adverse CRDR, despite the fact that the Procedure 01DP-OAP12 required it to be screened as significant. As a result, no causal determination was performed to understand the mode of failure of the mechanical seal and no actions were identified to prevent the recurrence of mechanical seal failures on essential cooling water Pump 3MEWAP01.

Description.

Essential cooling water Pump 3MEWAP01 developed mechanical seal leakage on September 10, 2008. The licensee recorded this condition in PVAR 3221258 and went on to perform an immediate determination of operability, which supported the continuing operability of the pump. A subsequent prompt operability determination on September 11, 2008, determined that the operability of the pump could not be supported based upon feedback from the vendor. As a result, the licensee declared Pump 3MEWAP01 inoperable and entered the applicable action statement of Technical Specification 3.7.7 on September 11, 2008.

The licensee initiated PVAR 3221708 on September 11, 2008, to record the inoperability of Pump 3MEWAP01. This condition was further documented in the related CRDR 224261. As a result of entering the Technical Specification action statement, the licensee initiated a repair activity for the pump, requiring removal of essential cooling water Train A from service for approximately 50 hours5.787037e-4 days <br />0.0139 hours <br />8.267196e-5 weeks <br />1.9025e-5 months <br />. On September 13, 2008, repairs to the pump were completed, the pump was declared operable and the applicable Technical Specification action statement was exited.

The Action Request Review Committee reviewed CRDR 3224261 on September 17, 2008. The procedure directing this activity, Appendix E of Nuclear Administrative Technical Manual Procedure 01DP-0AP12, Palo Verde Action Request Processing, Revision 9, provides CRDR classification level examples. This appendix provides the following as an example of a significant condition:

An equipment failure in systems AF, DG, PB, PK, SI, SP or EW that does not result in the loss of a safety function of a train, but where the equipment failure indirectly introduces risk by requiring the immediate removal of a train from service to conduct corrective maintenance.

Contrary to this procedure, on September 17, 2008, the Action Request Review Committee screened this failure as an adverse CRDR, despite the fact that the Procedure 01DP-OAP12 required it to be screened as significant. As a result, no causal determination was performed to understand the mode of failure of the mechanical seal and no actions were identified to prevent the recurrence of mechanical seal failures on essential cooling water Pump 3MEWAP01.

The inspectors questioned licensee personnel regarding the cause of this screening error. The licensee interviewed members of the Action Request Review Committee, who reported that they had screened the issue as adverse based upon an assumption all required actions to preclude repetition had already been completed. By making this assumption, the Action Request Review Committee bypassed the normal root cause investigation process.

The licensee documented the failure to properly screen this issue for significance in PVAR 3288713.

Analysis.

The performance deficiency associated with this finding involved the licensees failure to follow procedures for identifying the significance of conditions adverse to quality. Specifically, the Action Request Review Committee screened PVAR 3221258 as an adverse CRDR, despite the fact that the Procedure 01DP-OAP12 required it to be screened as significant. The finding is more than minor because the finding is associated with the equipment performance attribute of the mitigating systems cornerstone, and affects the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The inspectors utilized Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," to determine that the finding was of very low safety significance because it did not represent a design or qualification deficiency, did not result in a loss of safety function, or screen as a risk-significant external event.

The cause of this finding is related to the problem identification and resolution crosscutting component of corrective action program, in that licensee failed to properly classify and evaluate a significant condition adverse to quality P.1(c).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings, requires, in part, that activities affecting quality shall be prescribed by procedures and shall be accomplished in accordance with these procedures. Contrary to this requirement, on September 17, 2008, the licensee failed to follow procedures for identifying a significant condition adverse to quality. Specifically, the Action Request Review Committee screened PVAR 3221258 as an adverse CRDR, despite the fact that the Procedure 01DP-OAP12 required it to be screened as significant. This error resulted in the failure to understand the failure mode associated with a safety related essential cooling water pump such that corrective actions would prevent recurrence. Because the finding is of very low safety significance and has been entered into the licensees corrective action program as PVAR 3288713, this violation is being treated as a noncited violation consistent with Section VI.A of the Enforcement Policy: NCV 05000530/2009006-02, "Failure to Follow Procedure for Screening Significant Condition Adverse to Quality.

(3) Failure to Perform 10 CFR 50.59 Screenings on Scaffolds Installed for Greater Than 90 Days

Introduction.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," when, on February 10, 2009, it was determined that 62 scaffolds did not comply with the engineering installation specifications had not been screened to ensure compliance with 10 CFR 50.59 and had not been evaluated for their potential impact on equipment operability.

Description.

On February 10, 2009, the inspectors attended a meeting of the Action Request Review Committee. During this meeting, the committee reviewed PVAR 3281680, concerning long term scaffolding. It stated that, as of November 26, 2008, there were 529 scaffolds in various areas of the plant that had been installed for greater than 90 days. Of these, 62 did not meet the minimum 2-inch clearance between the scaffold and safety related structures, systems, or components, and risk important non-safety related structures, systems, or components, as specified by Specification 13-CN-0380, Installation Specification for Seismic Category IX &

Non-Seismic Scaffolding, Revision 10. The PVAR further stated that these 62 scaffolds had been evaluated by engineering and were found to be acceptable. The inspectors questioned whether screening in accordance with 10 CFR 50.59 had been performed for these scaffolds and whether the applicable control room operators had been made aware of the nonconforming conditions so that an operability determination could be performed.

Specification 13-CN-0380, Revision 10, which was implemented in August 2008, expanded the minimum scaffold clearance specification from one inch to two inches, from safety related piping to safety related structures, systems, or components and risk important non-safety related structures, systems, or components. The revision indicated that inspection of all installed scaffolds had identified that some did not meet the new clearance specification, and that these scaffolds would be evaluated. Although initial evaluation calculations were completed in January 2009, and all noncompliant scaffolds had been found to be acceptable, screening in accordance with 10 CFR 50.59 had not been initiated. The team noted that 62 scaffolds did not comply with Specification 13-CN-0380, and each was required to be screened for 10 CFR 50.59 applicability in accordance with Procedure 81DP-0DC13, Deficiency (DF) Work Order, Revision 25, however, the team noted that this had not been performed.

As immediate corrective action, Palo Verde Nuclear Generating Station informed the applicable control room operators of the 62 nonconforming conditions and operability assessments were performed under PVARs 3283371, 3283489, and 3281680.

Additionally, Palo Verde Nuclear Generating Station initiated PVAR 3283865 to perform 10 CFR 50.59 screenings on the 62 noncompliant scaffolds.

Analysis.

The inspectors concluded that failure to perform 10 CFR 50.59 screenings on 62 scaffolds that had been installed for greater than 90 days and were noncompliant with Specification 13-CN-0380 was a performance deficiency. Additionally, the inspectors determined that failure to inform the control room operators of the noncompliant scaffolds, in lieu of having completed the required 10 CFR 50.59 screenings, so that an operability evaluation could be performed, was a performance deficiency. The finding was more than minor because it is associated with the mitigating systems cornerstone

attribute for protection against external factors and affected the cornerstone objective of ensuring the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. The finding was evaluated in accordance with Inspection Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," and determined to be of very low safety significance (Green) per the Significance Determination Process because the finding was not a design or qualification deficiency, did not represent a loss of a system/train safety function, and did not screen as potentially risk significant due to external events; specifically, it was not risk significant due to a seismic initiating event because all of the noncompliant scaffolds were evaluated by engineering to be satisfactory. The finding had a crosscutting aspect in the human performance component of resources because Palo Verde Nuclear Generating Station did not ensure that adequate personnel were assigned to ensure that long term scaffold installations remained compliant with applicable procedural requirements

H.2(a).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," states, in part, "Activities affecting quality shall be prescribed by documented instructions, procedures, or drawings . . . and shall be accomplished in accordance with these instructions, procedures, or drawings.

Contrary to the above, from November 26, 2008, until February 10, 2009, Palo Verde Nuclear Generating Station did not accomplish the requirement of Procedure 81DP-0DC13, Deficiency (DF) Work Order, Revision 25, to ensure that 10 CFR 50.59 screening had been addressed in the case of 62 scaffolds that had been installed for greater than 90 days and were noncompliant with Specification 13-CN-0380, Installation Specification for Seismic Category IX & Non-Seismic Scaffolding, Revision 10; nor did they accomplish the requirement of Procedure 30DP-9WP11, Scaffolding Instructions, to generate PVARs prior to the scaffolds 75th day of existence, and thereby inform the control room operators of the need to evaluate operability with respect to the nonconforming conditions. Because this noncompliance is of very low safety significance and was entered into the corrective action program as PVAR 3283865, this violation is being treated as a noncited violation, consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000530/2009006-03, " Failure to Perform 10 CFR 50.59 Screenings on Scaffolds Installed for Greater Than 90 Days.

(4) Failure to Implement Adequate Design Controls

Introduction.

The inspectors identified a Green cited violation of 10 CFR Part 50, Appendix B, Criterion III, "Design Control," for the failure of engineering personnel to translate the design basis maximum condensate storage tank temperature requirements into procedures to ensure the plant is operated within its design basis.

Description.

During a review of the licensees corrective actions following a 2007 noncited violation, the inspectors developed a concern regarding the adequacy of the licensees corrective actions. In NRC Integrated Inspection Report 2007012, inspectors identified a noncited violation regarding the licensees failure to translate the design basis maximum condensate storage tank temperature requirements into procedures to ensure the plant was operated within its design basis. The inspectors documented that a condensate storage tank maximum temperature of 120°F was used in Calculation 13-MC-CT-0205, "Condensate Storage Tank," Revision 4, Calculation 13-MC-CT-0307, "CST Minimum Level Setpoint," Revision 4, and Calculation 13-MC-AF-0309, "AF Hydraulic Calculation for Q-Trains," Revision 7, to

ensure sufficient condensate storage tank volume and net positive suction head for the auxiliary feedwater pumps during a design basis accident. Additionally, during this inspection period the inspectors identified that this temperature value is an input assumption in the containment peak pressure analysis, described in Table 6.2.1-6A of the Updated Final Safety Analysis Report. For these reasons, the inspectors determined that this variable was an important design basis assumption that needed to be protected.

The inspectors noted that no routine monitoring of this temperature had been performed prior to the issuance of the noncited violation in 2007, nor was the variable monitored through control room alarms, Technical Specification surveillance requirements or otherwise. In response to the 2007 noncited violation, the licensee initiated PVAR 3073243 on October 4, 2007. Corrective action was initiated under CRDR 3076781 on the same date, which generated two CRAIs. CRAI 3076782 was written on October 15, 2007 to determine appropriate procedure changes needed to direct monitoring CST temperature and responding to high temperature conditions, implement additional CRAI(s) as needed to implement changes. Actions per this CRAI were completed on November 8, 2007, and identified that two procedures needed to be revised to restore compliance (Procedure 40DP-9OPA2, Area 2 Operator Logs, Modes 1-4, and Procedure 40OP-9ZZ14, Feedwater and Condensate). Condition Report Action Item 3090209 was initiated on November 8, 2007, to implement procedure changes needed to direct monitoring CST temperature and responding to high temperature conditions. The team noted that when this CRAI was completed on March 20, 2008, however, only Procedure 40DP-9OPA2, Area 2 Operator Logs, Modes 1-4, was revised by adding a shiftly check of the locally indicated tank temperature and creating an operator action to contact the control room supervisor if temperature limits were approached and to contact engineering if the temperature limits were exceeded. No rationale was provided in CRAI 3090209 for not revising the other affected procedure (Procedure 40OP-9ZZ14, Feedwater and Condensate).

On August 5, 2008, PVAR 3208445 and CRDR 3211046 were written after licensee personnel performing the component design basis review discovered that the resolution of CRDR 3076781 was incomplete and that no basis had been provided for the temperature limits that had been added to the Procedure 40DP-9OPA2 log sheets.

PVAR 3208445 went on to state that:

no changes were initiated to direct monitoring of CST temperature during operations which reject water from the hotwell to the CST. Such operations can cause the CST to exceed its design temperature. Sections 7.3 and 28.3 of procedure 40OP-9ZZ14, at a minimum, should be revised accordingly.

Several actions were assigned from CRDR 3211046. Condition Report Action Item 3211047 was initiated on August 15, 2008 with the purpose of addressing necessary changes to Procedure 40OP-9ZZ14 during operations which reject water from the hotwell to the condensate storage tank. An evaluation was performed under this CRAI, and the CRAI was closed with the following comment: No actions will be completed at this time pending outcome of CRAI 3178187 which covers an expanded scope. In the evaluation attached to the CRAI, the following justification was provided for not making any changes to Procedure 40OP-9ZZ14:

Discussions at the time with Engineering and Operations determined that the appropriate changes to make would be to set a temperature limit on the CST and

monitor it by the logs. The range picked was60-100 to allow time for the engineer to be notified and an action plan put into place based on the evolutions in progress in the unit.

The action actually taken, however, was to revise Procedure 40DP-9OPA2 and add an upper temperature limit of 110°F, after which the control room supervisor would be notified. The revised procedure did not require notification of engineering until the condensate storage tank temperature exceeded 120°F, in excess of its analytical limit in the Updated Final Safety Analysis Report accident analysis. Additionally, the evaluation performed under CRAI 3211047 documented that there was no technical basis for the temperature limits chosen, nor was there any basis for using the one available temperature indication as a measure of condensate storage tank bulk temperature.

Lastly, CRAI 3211047 was closed to CRAI 3178187 to develop a basis for the temperature limits assigned and address the inability to measure condensate storage tank bulk temperature. This indication question, the subject of CRAI 3272479 initiated on January 14, 2009, was still in a working status at the time of this inspection.

The inspectors reviewed CRAI 3178187, which was written on May 22, 2008, to expand the scope of CRDR 2951254 to perform an analysis of all factors that could add heat to the condensate storage tank. The scope of CRDR 2951254 was subsequently expanded on December 29, 2008, and CRAI 3178187 was closed. The inspectors reviewed CRDR 2951254 and determined that at the time of the inspection no action had been completed to provide a basis for the assigned temperature limits for the condensate storage tank.

Through this complex chain of events, the inspectors determined that not only was the original opportunity in PVAR 3073243 missed, but the subsequent discovery of the missed corrective action in CRDR 3211046 also failed to restore compliance as the issue shuffled through the corrective action program. After seventeen months of activity, two PVARs, three CRDRs, and five CRAIs, no action had yet been taken to provide a basis for the temperature limits added to the operating logs, provide a bulk temperature indication for the condensate storage tank, or to monitor tank temperature during evolutions known to add hot water to the tank.

After reviewing these documents and discussing the issue with members of the licensees staff, the inspectors toured the facility and learned that the temperature indication used to monitor the condensate storage tank temperature is located at the inside wall of the tank, approximately 4 feet off the bottom. The normal water level in the tank is approximately 45 feet. Based on these physical considerations, the inspectors determined that the installed temperature indicator provides a non-conservative indication of bulk fluid temperature in the tank.

The licensee provided an analysis to attempt to bound the maximum possible temperature that the tank could see in a normal unit shutdown (the most severe normal transient that the tank would experience due to hotwell rejection). This analysis demonstrated that if the entire volume of the hotwell (approximately 116,000 gallons) at the worst case temperature (approximately 149°F) was transferred to the condensate storage tank at its minimum normal level (approximately 35 feet) and maximum allowable temperature (110°F), the resulting bulk temperature in the tank would reach 119°F. The inspectors determined that this analysis was refuted by actual plant data, which demonstrated that the condensate storage tank temperature rose by at least 7°F

during a routine plant shutdown on March 28, 2008. In the actual event, the licensee estimates that approximately 12,000 gallons of condensate at approximately 115°F caused an increase in condensate storage tank temperature of at least seven degrees.

Given the shiftly nature of the logged readings, the actual temperature rise may have actually been greater than 7°F. This actual plant data demonstrates that normal plant activities such as a unit shutdown have the potential to threaten the design maximum temperature of the condensate storage tank.

Based on a review of available documents, interviews with plant employees, walkdowns of the facility, and a review of actual plant data, the inspectors determined that the licensee had not yet restored compliance in that the design basis condensate storage tank temperature had not been adequately translated into procedures. Based on this observation by the inspectors, the licensee initiated PVARs 3289578 and 3289530, which proposed a number of procedural changes. Prior to the inspection team leaving the site, the licensee implemented several procedural changes, including an increased frequency of log readings, identifying more restrictive operating limits on tank temperature, and the addition of a cautionary note in the system operating procedure.

The inspectors noted that at the end of the inspection, the licensee had still not identified a technical basis for the temperature limits in the operating logs, nor had an appropriate method of monitoring condensate storage tank bulk temperature been identified.

Analysis.

The performance deficiency associated with this finding was the licensees failure to adequately translate the design basis condensate storage tank maximum temperature requirements into applicable procedures. This finding is greater than minor because it is associated with the mitigating systems cornerstone attribute of equipment performance and affected the cornerstone objective of ensuring the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. Using the Inspection Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding is determined to have very low safety significance since it only affected the mitigating systems cornerstone and did not represent a loss of system safety function. The cause of this finding had crosscutting aspects associated with corrective action component of the problem identification and resolution area in that engineering personnel failed to thoroughly evaluate problems such that resolutions ensured that the problems were resolved P.1(c).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion III, Design Control, requires, in part, that the design basis for structures, systems, and components be translated into specifications, drawings, procedures, and instructions. Contrary to the above, from 1985 through February 27, 2009, the licensee failed to adequately translate design basis information into specifications, drawings, procedures, and instructions. Specifically, the licensee failed to adequately translate design basis maximum condensate storage tank temperature requirements into procedures to ensure the plant is operated within its design basis. This example was of very low safety significance and was entered into the corrective action program as PVAR 289530. Due to the licensees failure to restore compliance from the previous NCV 05000528,529,530/2007012-02 within a reasonable time after the violation was identified, this violation is being cited in a Notice of Violation consistent with Section VI.A of the Enforcement Policy: VIO 05000528,529,530/2009006-04, Failure to Implement Adequate Design Controls.

.2 Assessment of the Use of Operating Experience

a. Inspection Scope

The team examined the licensee's program for reviewing industry operating experience, including reviewing the governing procedure and self-assessments. A sample of operating experience CRDRs that had been issued during the assessment period was reviewed to assess whether the licensee had appropriately evaluated the CRDRs for relevance to the facility. The team then examined whether the licensee had entered those items into its corrective action program and assigned actions to address the issues. The team reviewed a sample of root cause evaluations and action requests to verify that the licensee had appropriately included industry operating experience.

b. Assessment Overall, the team determined that the licensee was adequately evaluating industry operating experience for relevance to the facility, and had entered applicable items in the corrective action program in accordance with stations procedures. The team concluded that the licensee was evaluating industry operating experience when performing root cause and apparent cause evaluations. Both internal and external operating experience were being incorporated into lessons learned for training and pre-job briefs. There were some examples found where the review of operating experience could have been more effective.

• Following the modification to install new emergency diesel generator governors, Emergency Diesel Generators 1B and 2B exhibited slower than expected frequency recovery from load sequencing during testing, requiring engineering evaluation for continued operability. This condition could have been foreseen and addressed as a part of the modification process, based on similar results experienced by others using the new governors. Also, this should have been anticipated because the new governors do not have the same anticipatory response to frequency changes that the old governors did.

• One issue listed in NRC Information Notice 2007-27 concerned failure of a compression fitting in an emergency diesel generator jacket water cooling system. Palo Verde Nuclear Generating Station's response stated that there had been no compression fitting failures in their emergency diesel generator jacket water cooling systems. This did not address the point of the operating experience, which was compression fitting failures, rather than failures involving jacket water cooling systems.

• In response to NRC Information Notice 2008-05, fires involving emergency diesel generator manifolds, Palo Verde Nuclear Generating Station noted that Cooper Bessemer emergency diesel generators have the exhaust manifolds on top of the engine, and are therefore not susceptible to fuel oil leaking onto them. Palo Verde Nuclear Generating Station could have considered the issue more generically, for example, to examine whether other Cooper Bessemer specific configurations (such as the exhaust inlet to the turbocharger, which is prone to leakage until the engine is up to temperature) might present a fire hazard.

• Noncited Violation 05000528,529,530/2008002-01: Failure to establish preventative maintenance procedures for emergency diesel generator fuel oil injection pump O-rings as discussed in vendor information.

• Noncited Violation 05000528,529,530/2008005-01: Failure to promptly identify and correct degraded hydrostatic flood penetration seals.

• Low pressure safety injection and containment spray pumps full flow recirculation, as discussed in NRC Bulletin 88-04, was not thoroughly evaluated as requested in PVAR 3085457 to ensure sufficient flow to cool the pumps when all safety injection pumps are started. To address this issue, PVAR 3284044 was written.

• Operating experience searches for the four 2008 inverter failures focused specifically on Elgar inverters and on Limiting Condition for Operation 3.0.3 entries resulting from inverter failures. There were no searches for evaluations of more generic operating experience associated with inverters and other uninterruptible power supplies.

c. Findings

Failure to Identify and Correct Age-Related Degradation of Safety-Related Inverters

Introduction.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the licensees failure to promptly identify and correct a condition adverse to quality. Specifically, the licensee failed to incorporate industry and vendor recommended preventative maintenance requirements to prevent the age-related degradation of safety-related inverter components.

Description.

Between April and September 2008, the licensee experienced four failures of safety-related Class-1E vital inverters. The failures occurred on four different inverters installed in two different units.

• On April 24, 2008, Palo Verde Unit 1 experienced a failure of Class-1E Vital Inverter 1E-PNC-N13 resulting in a loss of power to one of four 120 VAC vital distribution panels (PNC-D27) and a subsequent entry into Technical Specification Limiting Condition for Operation 3.0.3. The licensee entered this event into the corrective action program as PVAR 3164931 and identified it as a significant condition under CRDR 3165478. The immediate corrective action was to replace the inverters DC-DC converter card.

• On July 25, 2008, Palo Verde Unit 2 experienced a failure of Class-1E Vital Inverter 2E-PNB-N12. This failure was entered into the corrective action program as PVAR 3202233, which led to an equipment root cause of failure analysis in CRDR 3202468 and identified a faulty silicon-controlled rectifier as the apparent cause of the failure. The licensee replaced the defective silicon-controlled rectifiers and returned the inverter to service.

• On September 15, 2008, Palo Verde Unit 2 experienced a spurious static switch transfer of Inverter 2E-PNC-N13. As in the April event, troubleshooting required entry into Limiting Condition for Operation 3.0.3. The licensee entered this failure

into the corrective action program as PVAR 3222864 and identified it as a significant condition under CRDR 3225731. The licensee has not determined the cause of this inverter failure.

• On September 18, 2008, Palo Verde Unit 2 experienced a spurious static switch transfer of Inverter 2E-PNA-N11. This failure was entered into the corrective action program as PVAR 3225531. The licensee has not determined the cause of this inverter failure.

The inspectors reviewed the licensees treatment of the events in the corrective action program and performed a search for relevant operating experience. The inspectors determined that the licensee had missed several opportunities to identify substantial industry data concerning age-based degradation of inverter electronic components.

The inspectors reviewed the Electric Power Research Institute (EPRI) Technical Report TR-100491, UPS Maintenance and Application Guide, dated August 1994.

This report provided statistical evidence that the failure rate of inverters and chargers increase with age . . . with rapid rate of increase after 15 years. Additionally, EPRI reported that the likely failure modes included high resistance and/or intermittent connections at solder joints cause by long term oxidation. Based upon the industry trend data, EPRI defined a recommended preventative maintenance strategy, as well as guidance for conducting periodic major overhauls of safety-related inverters. The inspectors reviewed the EPRI recommendations, and determined that several important maintenance practices were recommended but not implemented by the licensee, including:

(1) detailed visual inspections and chemical cleaning of inverter subcomponent connections for oxidation or corrosion buildup,

(2) twelve-year replacement interval for vital circuit boards, and

(3) twelve-year replacement interval for silicon-controlled rectifiers. The inspectors learned that the inverter system engineers were aware of the existence of the EPRI guidance, but had failed to incorporate this industry standard into maintenance practices for the inverter system. On February 27, 2009, the licensee completed a review of EPRI TR-100491 and determined that they would incorporate some of the recommended maintenance practices therein, including a ten-year replacement interval for the DC-DC converter circuit board.

Additionally, the inspectors reviewed CRAI 3212930, which was written on August 21, 2008, to evaluate the recommendations made by an inverter system expert. On August 19, 2008, the licensee had contacted a technical representative from the company that provides service support for the inverter system at Palo Verde Nuclear Generating Station to solicit his recommendations regarding the adequacy of the preventative maintenance program for the inverters. In his response on August 20, 2008, the expert provided Palo Verde Nuclear Generating Station several specific recommendations, including the following:

(1) that silicon-controlled rectifier testing should be done under operating conditions as opposed to performing a general static test,

(2) removing silicon-controlled rectifier drive boards and inspecting them under a bright light at magnification to detect degrading solder connections, and

(3) conducting chemical cleaning of circuit board connectors to remove oxidation from the conductors.

The expert went on to state that he had provided these same recommendations to the licensees staff in the previous years. Regarding the potential degradation of the silicon-controlled rectifier drive boards, the expert stated that:

there is no fix for this in Elgar equipment . . . other than to conduct very detailed preventative maintenance procedures specifically designed to detect problems caused by age. Vibration, over enough time, will cause cracks in solder joints.

Oxidation will always form on electrical contacts over a period of time.

In response to the information provided by the technical expert, Palo Verde Nuclear Generating Station reviewed the maintenance practices in place at the time and determined that while the specific maintenance practices recommended by the expert were not included in the existing maintenance program, the existing program was adequate to maintain the reliability of the inverters.

The inspectors noted that at least two of the inverter failures in 2008 were related to recommended preventive maintenance practices that had not been implemented at Palo Verde Nuclear Generating Station. The April 24, 2008, failure of Inverter 1E-PNC-N13 was caused by a failed DC-DC converter circuit board, which EPRI had recommended replacing every 10-12 years in the 1994 technical report. Additionally, the July 25, 2008, failure of Inverter 2E-PNB-N12 was caused by a failed silicon-controlled rectifier, which EPRI had also recommended replacing on a 10-12 year interval. In contrast, the manufacturers technical expert offered that the silicon-controlled rectifiers did not require periodic replacement as long as thorough visual inspections that the manufacturer recommended were performed. The inspectors reviewed Maintenance Procedure 32MT-9ZZ58, Preventive Maintenance of Elgar Inverters, Revision 29, and determined that it did not contain any of these recommended maintenance practices.

The licensee has since completed an action item in CRAI 3245187 to evaluate the preventive maintenance program for the inverters. This action was completed on February 27, 2009, and recommends future incorporation of some of the EPRI and manufacturer-recommended maintenance practices.

Analysis.

The inspectors determined that the failure to identify the necessary maintenance practices and take corrective actions prior to the 2008 inverter failures was a performance deficiency. This finding is more than minor because it affects the equipment performance attribute of the mitigating systems cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using Inspection Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings, the team determined that a Phase 2 analysis was required because the finding represented a loss of system safety function. Phase 2 and Phase 3 significance determinations were performed by an NRC Senior Reactor Analyst. Using bounding assumptions, the analyst estimated a delta-CDF of 2.9E-7/yr, and further concluded that by removing conservatisms the result would decrease at least an order of magnitude (<2.9E-8/yr).

Therefore, this finding was determined to be of very low safety significance (Green).

This finding has a crosscutting aspect in the problem identification and resolution component of operating experience, in that the licensee failed to implement operating experience through changes to station procedures P.2(b).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion XVI, Corrective Action, requires, in part, that measures shall be established to assure that conditions adverse to quality are promptly identified and corrected. Contrary to this requirement, between August 1994 and February 27, 2009, conditions adverse to quality were not promptly identified and corrected, specifically the licensee failed to

utilize relevant industry operating experience to understand that an age-related degradation mechanism was occurring on their safety-related inverters, and failed to implement industry and vendor recommended maintenance activities to prevent at least two failures of the safety-related inverters. Because this finding was determined to be of very low safety significance and was entered into the licensees corrective action program as PVAR 3291971, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy:

NCV 05000528,529,530/2009006-05, Failure to Identify and Correct-Related Degradation of Safety-Related Inverters.

.3 Assessment of Self-Assessments and Audits

a. Inspection Scope

The team reviewed a sample of licensee self-assessments and audits to assess whether the licensee was regularly identifying performance trends and effectively addressing them. The team reviewed audit reports to assess the effectiveness of assessments in specific areas. The specific self-assessment documents reviewed are listed in

1. b. Assessment

The team concluded that the licensee had a generally effective self-assessment process. The team observed that Palo Verde Nuclear Generating Stations management was involved in developing the topics and objectives of self-assessments.

The team observed that the assignment of the assessment team included members with the proper skills and experience to ensure an effective self-assessment was conducted and the team members included individuals from outside organizations.

c. Findings

No findings of significance were identified.

.4 Assessment of Safety Conscious Work Environment

a. Inspection Scope

The inspection team conducted individual interviews with 34 individuals. The interviewees represented various functional organizations and ranged across contractors, staff, and supervisor levels. The team conducted these interviews to assess whether conditions existed that would challenge the establishment of a safety conscious work environment at Palo Verde Nuclear Generating Station.

b. Assessment General comments from most interviewed was that the working environment and safety culture had improved significantly. This was attributed largely to the new management team and their commitment to improved site performance. The working staff and first line supervisors are convinced that the leaders are trying to do the right thing.

Management insistence on accountability and standards was seen as a positive comment in interviews.

Workers consistently indicated a willingness to raise safety concerns. Interviews revealed that everyone understands the multiple avenues to raise safety concerns. The most popular preference was to use direct supervision first as an avenue for safety concerns. Many of the staff were not familiar with the concept of the Employee Concerns Program team. Those with knowledge of the Employee Concerns Program believed it is a viable option today but perhaps not previously. Most individuals provided clear responses that a chilled environment does not exist. While two interviewees singled out certain supervisors as difficult to work with, the interviewees indicated that they were not concerned the supervisors might take adverse action against them for raising concerns.

Interview comments received associated with the corrective action program reflect that it has significantly changed the focus of plant personnel. Many view the corrective action program as the driver for recent improvements. However, many comments were received indicating all aspects of the process have room for improvement. For example, comments were received that more feedback to initiators is needed; deadlines sometimes negatively affect CRDR/CRAI quality; corrective action program priorities are not understood; the tracking and retrieval of CRDR/CRAIs is very cumbersome; timeliness of CRAIs is a concern; closeout of CRDRs is occurring without adequate action, etc. Some responses indicated the corrective action program focus on equipment reliability was positive and headed in the right direction. The additional levels of review quality were seen by some as likely to uncover and correct adverse work products.

c. Findings

No findings of significance were identified.

4OA3 Followup of Events and Notices of Enforcement Discretion

.1 (Closed) Licensee Event Report (LER) 05000528,529,530/2004-009-00, Emergency

Core Cooling System Piping Voids May Have Prevented Fulfillment of Safety Function On July 30, 2004, control room personnel were informed that a voided section of emergency core cooling system suction piping might prevent the fulfillment of the safety function to remove residual heat and mitigate the consequences of a loss of coolant accident. The licensee took initial compensatory measures for control room operators to open the inboard containment sump isolation valves allowing 90 percent of the voided piping to fill during accident conditions. Measures were taken to return the containment sumps to their design configurations by filling the piping with borated water. These actions were completed on all three units on August 4, 2004. The failure to maintain design control of containment sump recirculation piping was previously discussed and dispositioned as an Apparent Violation in Inspection Report 05000528,529,530/2004014-01. This condition was documented in CRDR 2726509. This LER is closed.

.2 (Closed) LER 05000528,529,530/2004-009-01, Emergency Core Cooling System Piping

Voids May Have Prevented Fulfillment of Safety Function On July 30, 2004, control room personnel were informed that a voided section of emergency core cooling system suction piping might prevent the fulfillment of the safety

function to remove residual heat and mitigate the consequences of a loss of coolant accident. The licensee took initial compensatory measures for control room operators to open the inboard containment sump isolation valves allowing 90 percent of the voided piping to fill during accident conditions. Measures were taken to return the containment sumps to their design configurations by filling the piping with borated water. These actions were completed on all three units on August 4, 2004. The failure to maintain design control of containment sump recirculation piping was previously discussed and dispositioned as an Apparent Violation in Inspection Report 05000528,529,530/2004014-01. This condition was documented in CRDR 2726509. This LER is closed.

.3 (Closed) LER 05000530/2007-001-00, Condition Prohibited by Technical Specification

Resulting from Containment Spray Nozzle Blockage On December 5, 2007, the licensee identified that two lower containment spray nozzles on Unit 3 were plugged from boron buildup. The cause of the event was a seat leak in a containment isolation valve. The licensee declared containment spray Train A inoperable, cleaned the nozzles, and repaired the leaking isolation valve. Documents reviewed as part of this inspection are listed in Attachment 1. The enforcement aspects of this issue are discussed in Section 4OA7 of this report. This LER is closed.

.4 (Closed) LER 05000528/2008-002-00, Technical Specification - Limiting Condition for

Operation 3.0.3 for Greater than 1 Hour On April 24, 2008, Palo Verde Unit 1 experienced a failure of Class-1E Vital Inverter 1E-PNC-N13 resulting in a loss of power to one of four 120 volt AC vital distribution panels (PNC-D27). The loss of the distribution panel resulted in the loss of the pulse counter indication on 67 control element assemblies causing two of the three required control element assembly position indications to be lost for 67 of the 89 control element assemblies. This required entry into Technical Specification Limiting Condition for Operation 3.0.3. Limiting Condition for Operation 3.0.3 requires action to be initiated within one hour to place the unit in Mode 3 within seven hours. The licensee exited Limiting Condition for Operation 3.0.3 after 59 minutes following restoration of power to Distribution Panel PNC-D27. The licensee later determined that Limiting Condition for Operation 3.0.3 had been exited prematurely; the limiting condition for operation should have been exited following functional verification of control element assembly pulse counters, which occurred 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 25 minutes after entry into the limiting condition for operations. The required one-hour actions of Limiting Condition for Operation 3.0.3 had been initiated prior to having exceeded one hour in the limiting condition for operation.

On April 25, the licensee again entered Limiting Condition for Operation 3.0.3 when Panel PNC-D27 was powered down in order to transfer it back to its normal power supply. The licensee initiated the required actions of Limiting Condition for Operation 3.0.3 and exited the limiting condition for operation after 68 minutes; no power reduction was initiated. This LER was issued on June 23, 2008, for exceeding one hour in Limiting Condition for Operation 3.0.3 on two occasions. This event is further discussed in Section 4OA2.2.c of this report as part of NCV 05000528;529;530/2009006-05, for the failure to promptly identify and correct a condition adverse to quality associated with the failure of the inverter. The inspectors reviewed this LER and identified no additional findings of significance and no additional violation of NRC requirements. This LER is closed.

.5 (Closed) LER 05000529/2008-003-00, Technical Specification - Limiting Condition for

Operation 3.0.3 for Greater than 1 Hour On September 15, 2008, Palo Verde Unit 2 experienced a spurious static switch transfer of Inverter 2E-PNC-N13. On September 16, 2008, distribution Panel PNC-D27 was powered down to support troubleshooting, requiring entry into Limiting Condition for Operation 3.0.3 due to the loss of redundant position indications for 67 of 89 control element assemblies. Prior to exceeding one hour in Limiting Condition for Operation 3.0.3, the licensee initiated action to place the unit in Mode 3 within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />.

The Limiting Condition for Operation was exited after 88 minutes after power was restored to distribution Panel PNC-D27 from the voltage regulator; no power reduction was initiated. The licensee submitted this LER on November 17, 2008, for exceeding one hour in Limiting Condition for Operation 3.0.3. This event is further discussed in Section 4OA2.2.c of this report as part of NCV 5000528;529;530/2009006-05, for the failure to promptly identify and correct a condition adverse to quality associated with the failure of the inverter. The inspectors reviewed this Licensee Event Report and identified no additional findings of significance and no additional violation of NRC requirements.

This LER is closed.

4OA5 Other Activities

.1 Multiple/Repetitive Degraded Cornerstone Column and Crosscutting Issues Followup

Activities - Quarterly Confirmatory Action Letter (CAL) Inspection This inspection was the fifth in a series of inspections to be performed by the NRC to assess the progress that Palo Verde Nuclear Generating Station made with respect to the implementation of their Site Integrated Improvement Plan (SIIP) and to verify their progress in addressing the specific actions in the NRC CAL dated February 15, 2008.

The revised CAL contains a subset of actions delineated in the SIIP that the NRC determined were necessary to address the performance insights identified by Palo Verde Nuclear Generating Station assessment activities and the Inspection Procedure 95003 Supplemental Inspection. The key performance areas that Palo Verde Nuclear Generating Station has committed to address are as follows: Yellow and White findings as documented in NRC Inspection Reports 05000528,529,530/2004014 and 2006012, problem identification and resolution issues, human performance issues, engineering programs, review of current equipment evaluations, safety culture, accountability, change management, emergency preparedness, longstanding equipment deficiencies, and backlog.

The areas to be inspected are identified in the revised CAL. The licensee submitted a list of the specific tasks, including due dates, associated with the action plans and strategies for each of the CAL items on March 31, 2008. The items selected for this quarterly CAL inspection were based on the completion due dates provided by the licensee from their submittal dated, December 31, 2007.

a. Inspection Scope

The inspectors selected the SIIP tasks listed below for an in-depth review. The inspectors considered the following during the review of the licensees actions:

(1) SIIP task matches the CRAI description;

(2) corrective actions address and correct the SIIP

task;

(3) corrective actions address the action plan problem statement and primary causes;

(4) verification of SIIP task completion;

(5) timely completion of corrective actions in accordance with the SIIP schedule;

(6) review of metrics and measures for improved performance;

(7) independent verification of improved performance; and

(8) closure of SIIP task in accordance with procedures. The inspectors also:

(1) walked down portions of the associated risk important systems;

(2) attended closure review board panels;

(3) interviewed CAL task personnel; and

(4) reviewed root and apparent causes to verify effectiveness of task closures.

• Task 2.1.D.5.d (CAL Item 8 and SIIP Action Plan 6, Strategy 5)

(CRAI 3075733) - Provide training developed under Task 2.1.D.5.b to Department Leaders and Managers.

• Task 2.1.D.5.e (CAL Item 8 and SIIP Action Plan 6, Strategy 5)

(CRAI 3075737) - Provide training developed under Task 2.1.D.5.b to Section Leaders and Team Leaders.

• Task 2.1.D.5.h (CAL Item 8 and SIIP Action Plan 6, Strategy 5) (CRAI 3075743) -

Incorporate expected behaviors from Task 2.1.D.5.b into the Observation Program/Workplace Observation tool used by the site to observe work behaviors.

• Task 2.2.B.8 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3108392) -

Develop targeted staffing strategy for Operations/ Engineering/Maintenance/

Radiation Protection/Chemistry and other groups.

• Task 2.3.C.1.a (CAL Item 7 and SIIP Action Plan 12, Strategy 7)

(CRAI 3076254) - Establish attributes/competencies for key positions to include Nuclear Safety, Safety culture, and Safety Conscience Work Environment behaviors.

• Task 2.4.B.4 (CAL Item 7 and SIIP Action Plan 12, Strategy 7) (CRAI 3065020) -

Develop/implement a formal Management Succession Plan and associated policy.

• Task 3.2.5.b (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 6)

(CRAI 3047258) - Develop means to resolve oversight conclusion differences.

• Task 3.2.5.d (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 6)

(CRAI 3032400) - Nuclear Assurance Department to implement use of the Significant CRDR Root Cause evaluation grade sheet and provide to corrective action program for tracking/trending.

• Task 3.2.5.e (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 6)

(CRAI 3032402) - Develop/implement Root Cause review and quality grading training for the Corrective Action Review Board and Nuclear Assurance Department.

• Task 3.3.2.b (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 6)

(CRAI 3047269) - Establish core group of apparent-cause evaluators to perform causal evaluations.

• Task 3.3.2.c (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 6)

(CRAI 3032399) - Implement job qualification for Root Cause investigations, reviews, and approvals.

• Task 3.4.10.a (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3038047) - Revise the Corrective Action Review Board Charter and corrective action program Procedures to require Corrective Action Review Board review of closeout actions and documentation for Priority 2 corrective actions.

• Task 3.4.7.a (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047290) - Review existing trend capabilities and implement interim actions to provide immediate capabilities to perform simple trending.

• Task 3.4.7.b (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047291) - Conduct meeting with Performance Improvement Department Management Team to determine desire trend capabilities and develop a plan to implement these capabilities.

• Task 3.4.7.c (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047292) - Establish priority and schedule for implementing electronic business revisions in support of monitoring of the Corrective Action Program.

• Task 3.4.7.d (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047293) - Implement trending process wherein the advocates review departmental data and identify potential trends in a quarterly report for each department.

• Task 3.4.7.e (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047294) - Benchmark industry for trending programs and present to senior management.

• Task 3.4.7.f (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047295) - Implement a monthly departmental trend day and quarterly trend day process where departments review trends on a monthly basis and senior management reviews roll-ups on a quarterly basis.

• Task 3.4.7.g (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3023691) - Modify existing Palo Verde trend program to a basic level to track gross subjects and numbers associated with program process and equipment failures.

• Task 3.4.7.h (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047298) - Incorporate changes into the trend program to be forward looking to provide insights on why trends are occurring.

• Task 3.5.3.f (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 4)

(CRAI 3038019) - Incorporate performance objectives for corrective action program evaluation and closure timeliness and quality into Performance Management Processes.

• Task 3.7.3.q, 3.7.8.k, 3.7.11.b (CAL Item 1 and SIIP Action Plan 15, Focus Area 2, Strategy 2; and CAL Item 1 and SIIP Action Plan 15, Focus Area 10, Strategy 1) (CRAI 2825641) - Engineering to communicate to all potential design basis manual users, on the possible limitations of the design basis manual and measures to effectively use the design basis manuals. Accuracy of the design basis manuals may not be 100 percent; there may be errors of omission (primarily unincorporated engineering design changes, and other possible omissions) and possibly inaccurate content. Users should validate and quality verify the information with other resources when possible. Users should also understand the context of set point information; the differences between safety limits, operational bands, instrument inaccuracies, etc.

• Task 3.7.11.e (CAL Item 1 and SIIP Action Plan 15, Focus Area 10, Strategy 3)

(CRAI 2842003) - Conduct additional UFSAR reviews using the identified scope from the CESSAR to UFSAR conversion project.

• Task 3.7.5.hh and 11.4.15 (CAL Item 1 and SIIP Action Plan 15, Focus Area 4, Strategy 1) (CRAI 3066464) - Establish a process to formally provide technical information by the engineering staff.

• Task 3.7.8.c (CAL Item 1 and SIIP Action Plan 15, Focus Area 7, Strategy 5)

(CRAI 3033594) - Implement and complete remediation plan for identified Design Engineers following completion of incumbent analysis conducted on CRAI 3021273.

• Task 3.7.8.d (CAL Item 1 and SIIP Action Plan 15, Focus Area 7, Strategy 5)

(CRAI 3033595) - Implement and complete remediation plan for identified Design Engineers following completion of incumbent analysis conducted on CRAI 3021285.

• Task 3.7.8.v (CAL Item 1 and SIIP Action Plan 15, Focus Area 7, Strategy 5)

(CRAI 3111714) - Develop a remediation plan and complete remediation of component engineers where analysis performed under CRAI 3033591 identified knowledge gaps.

• Task 3.7.9.a (CAL Item 1 and SIIP Action Plan 15, Focus Area 8) (CRAI 2856973) - Monitor Performance Indicators applicable to the backlog of undispositioned deficiencies, Engineering Design Changes, and CRDR evaluations assigned to Nuclear Engineering.

• Task 4.1.F.24 (CAL Item 8 and SIIP Action Plan 1, Strategy 12)

(CRAI 3105766) - During daily operability determination review, corrective action program-operability determination Senior Reactor Operator will identify operability determinations with loss of CLB design margin &/or use of compensatory measure and add to the list of significant operability determinations.

• Task 4.1.F.25 (CAL Item 8 and SIIP Action Plan 1, Strategy 12)

(CRAI 3105767) - Revise operability determination procedure to require the Operations Unit Department Leader to periodically review operability

determinations corrective action due dates and change due dates as necessary based on safety significance or aggregate impacts.

• Task 4.1.F.26 (CAL Item 8 and SIIP Action Plan 1, Strategy 12)

(CRAI 3112466) - Revise Shift Manager Turnover to link the list of significant operability determinations to Shift Manager Turnover.

• Task 4.1.G.16 (CAL Item 3 and SIIP Action Plan 3, Strategy 3) (CRAI 2928885) -

Implement an Engineering Operations Support team.

• Task 4.1.G.5 (CAL Item 8 and SIIP Action Plan 1, Strategy 8) (CRAI 3064339) -

Incorporate key operations department attributes and behaviors of an operationally focused organization identified in Task 4.1.G.4.

• Task 4.4.1 (CAL Item 1 and SIIP Action Plan 15, Focus Area 3 Part 2, Strategy 6) (CRAI 3062971) - Integrate Safety discussions in the context of Plant Status during meetings (Nuclear, Industrial, Radiological, and Safety Culture).

• Task 4.4.17 (CAL Item 7 and SIIP Action Plan 12, Strategy 7) (CRAI 3082455) -

Verify that the competencies in Task 2.3.C.1.a address Nuclear Safety, Safety Culture, and SCWE behaviors.

• Task 4.4.8.b (CAL Item 7 and SIIP Action Plan 12) (CRAI 3062994) -

Effectiveness Review - Safety Culture 2008 assessment.

• Task 5.1.E.4 (CAL Item 3 and SIIP Action Plan 3, Strategy 4) (CRAI 3062969) -

Incorporate initial training on Operability Determinations into the Engineering Training Program.

• Task 6.1.1.b (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 1)

(CRAI 3032685) - Incorporate leadership fundamentals into HR Performance Management Process.

• Task 6.1.6 (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 1)

(CRAI 3007272) - Review human performance program to include appendix on risk assessment process.

• Task 6.11.1.b (CAL Item 8 and SIIP Action Plan 1, Strategy 11)

(CRAI 3110609) - Incorporate into initial licensed and non-licensed operator training programs the stations expectations and industry standards regarding operations ownership of equipment deficiencies.

• Task 6.11.1.c (CAL Item 8 and SIIP Action Plan 1, Strategy 11)

(CRAI 3105770) - All available Control Room Supervisors will attend the INPO Operations supervisor Professional Development seminar in 2008.

• Task 6.11.2.a (CAL Item 8 and SIIP Action Plan 1, Strategy 11)

(CRAI 3115433) - Perform a focused self-assessment to identify specific weaknesses in operator fundamentals.

• Task 6.11.2.g (CAL Item 8 and SIIP Action Plan 1, Strategy 11)

(CRAI 3112302) - Perform an assessment on the operations teams execution of these performance standards in both the simulator and on-shift.

• Task 6.2.4.b (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 6)

(CRAI 3032697) - Develop/implement training for coach-the-coach.

• Task 6.4.4.b (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 6)

(CRAI 3065864) - Implement training with training instructors to reinforce human performance and industrial safety behaviors.

• Task 6.5.2.a (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 302269) - Second quarter 2007, review and determine if additional analysis is required for declining human performance.

• Task 6.5.2.b (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 302271) - Third quarter 2007, review and determine if additional analysis is required for declining human performance.

• Task 6.5.2.e (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022274) - Second quarter 2008, review and determine if additional analysis is required for declining human performance.

• Task 6.5.2.f (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022275) - Third quarter 2008, review and determine if additional analysis is required for declining human performance.

• Task 6.5.2.g (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022276) - Fourth quarter 2008, review and determine if additional analysis is required for declining human performance.

• Task 6.6.1.a (CAL Item 1 and SIIP Action Plan 15, Focus Area 3 Part 1)

(CRAI 2981667) - Evaluate what programs or processes will be included in a rollup program to determine current status of human performance. Completion date February 29, 2009.

• Task 6.7.10 (CAL Item 1 and SIIP Action Plan 15, Focus Area 6, Strategy 2)

(CRAI 2988523) - Develop a database for retrieval and knowledge management of operating experience. Identify target population and train on how to use database efficiently. Include a shortcut on Kiosk menu for retrievability.

• Task 6.7.5 (CAL Item 1 and SIIP Action Plan 15, Focus Area 6, Strategy 2)

(CRAI 2988519) - Train and identify operating experience point of contacts in departments and pertinent Performance Improvement Team staff members on the use of external INPO website.

• Task 6.7.6 (CAL Item 1 and SIIP Action Plan 15, Focus Area 6, Strategy 4; and CAL Item 3 and SIIP Action Plan 6 Part 2, Strategy 6) (CRAI 2988507) -

Develop/implement method and controls to ensure operating experience, particularly high-tiered operating experience cannot be eliminated from other procedures, processes and training.

• Task 9.1.A.4 (CAL Item 10 and SIIP Action Plan 8, Strategy 7) (CRAI 3063198) -

Emergency Operations Directors Performance Management Plans are to include an expectation that they are responsible for their teams performance, commencing in 2008.

• Task 9.1.A.6 (CAL Item 10 and SIIP Action Plan 8, Strategy 6) (CRAI 3063200) -

Develop strategies (posters, lanyard cards, etc.) to communicate Emergency Planning Program elements to the line organization.

• Task 9.1.A.21 (CAL Item 10 and SIIP Action Plan 8, Strategy 6) (CRAI 3076194)

- Implement the communications strategies developed in CRAI 3063200.

• Task 9.1.A.33 (CAL Item 10 and SIIP Action Plan 8, Strategy 5) (CRAI 3082331)

- Create an Emergency Preparedness Training Review Group as well as the appropriate number of Emergency Preparedness Training Advisory Committees.

• Task 9.2.A.15 (CAL Item 10 and SIIP Action Plan 8, Strategy 4) (CRAI 3065520)

- Conduct training on Emergency Action Levels with Emergency Coordinator and Emergency Operations Director-qualified individuals.

• Task 9.2.A.16 (CAL Item 10 and SIIP Action Plan 8, Strategy 4) (CRAI 3065521)

- Ensure the initial training for Emergency Coordinators contains training on Emergency Action Levels and their bases. This includes the Emergency Coordinator in the Control Room and the Emergency Coordinator in the Technical Support Center.

• Task 9.2.A.22 (CAL Item 10 and SIIP Action Plan 8, Strategy 4) (CRAI 3065522)

- Ensure the continuing training programs for Emergency Coordinator contains biennial training on Emergency Action Levels.

• Task 9.2.A.23 (CAL Item 10 and SIIP Action Plan 8, Strategy 5) (CRAI 3065519 and 3184906) - Administratively control Emergency Preparedness Training similarly to accredited training programs by creation of an Emergency Preparedness Training Program Description which as a minimum places controls on the following:

(1) Description that defines failures and remediation criteria for each Emergency Response Organization position. This would encompass failures during training, drills, exercises, and actual plant events.

(2) Implement a program to track named [a defined set of] Emergency Response Organization team positions to ensure these personnel receive proper training and drill participation on an annual basis.

(3) Develop a plan for drills, specifically on continuance of training drills, in 2009 and beyond, based on performance.

• Task 9.2.A.31 (CAL Item 10 and SIIP Action Plan 8, Strategy 5) (CRAI3152914)

- Ensure that each Emergency Response Organization team has one training drill and one evaluated drill during 2008.

• Task 9.5.1 (CAL Item 10 and SIIP Action Plan 8, Strategy 11) (CRAI 3063488) -

Evaluate implementation of Nuclear Energy Institute Report 99-01 strategy and develop recommendations for presentation to senior leadership.

• Task 9.5.2 (CAL Item 10 and SIIP Action Plan 8, Strategy 11) (CRAI 3063489) -

Present the strategy and development recommendations defined in Task 9.5.1 to senior leadership.

• Task 9.5.5 (CAL Item 10 and SIIP Action Plan 8, Strategy 9) (CRAI 3065531) -

Revise Procedure 21SP-0SK11 to include Emergency Action Levels 7.1, 7.2, and 7.3; Provide information to Operations Training in support of CRAI 3065613 and to Emergency Services Department Training in support of CRAI 3121416.

• Task 12.2.8 (CAL Item 4 and SIIP Action Plan 9, Strategy 4) (CRAI 3094447) -

Develop/implement upper tier documents for major processes.

• Task 12.3.3 (CAL Item 4 and SIIP Action Plan 9, Strategy 7) (CRAI 3062741) -

Based on results of Tasks 12.3.1 and 12.3.2, develop process inventory infrastructure including process owners.

• Task 15.1.9 (CAL Item 3 and SIIP Action Plan 6 Part 2, Strategy 8)

(CRAI 3017938) - Conduct station quality review board for review and approving self-assessment plans and completed reports.

• Task 16.2.A.4.c (CAL Item 4 and SIIP Action Plan 9, Strategy 10)

(CRAI 3062232) - Establish funding and schedule Site Work Management System usability improvements for implementation.

• Task 19.1.14 (CAL Item 5 and SIIP Action Plan 2, Strategy 2) (CRAI 3062082) -

Conduct a focused assessment of the short-term actions implemented in Tasks 19.1.1a-g and incorporate lessons learned.

• Task 20.2.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3068731) -

Complete the Safety Culture Improvement Plan (for Planning/Maintenance).

• Task 20.2.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083263) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.3.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3068723) -

Complete the Safety Culture Improvement Plan (for Finance and Community).

• Task 20.3.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083276) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.4.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3068556) -

Complete the Chemistry Safety Culture Improvement Plan for Chemistry.

• Task 20.4.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3079945) -

Close the Chemistry Safety culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.5.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083287) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.6.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083232) -

Complete the Safety Culture Improvement Plan (Training).

• Task 20.6.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083239) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.7.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083245) -

Complete the Safety Culture Improvement Plan (Operations).

• Task 20.7.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083245) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.8.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083255) -

Complete the Safety Culture Improvement Plan (Project Engineering).

• Task 20.8.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083257) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.9.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3068485) -

Complete the Safety Culture Improvement Plan (RP Operations).

• Task 20.9.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3083269) -

Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.11.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083022) - Complete the Safety Culture Improvement Plan (Work Management).

• Task 20.11.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083225) - Close the Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.12.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083416) - Complete the priority groups Safety Culture Improvement Plan (Procedures and Standards).

• Task 20.12.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083436) - Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.13.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083447) - Complete the priority groups Safety Culture Improvement Plan (Mechanical Design).

• Task 20.13.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083450) - Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

• Task 20.14.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3068723) - Complete the priority groups Safety Culture Improvement Plan (Radiation Services).

• Task 20.14.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083460) - Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

The following tasks could not be completed at the time of the inspection because of task due dates after the inspection period. The inspection team considers these tasks closed based upon the following provisions:

(1) the task was assigned a CRAI with actions that are sufficient to fully address the task,

(2) the CRAI actions are currently on schedule such that they will be completed at the due date,

(3) upon final closure of the tasks, they will be reviewed by the NRC for adequacy.

• Task 1.2.E.22 (CAL Item 5 and SIIP Action Plan 5, Strategy 1)

(CRAI 3065077) - Perform self assessments on all Engineering Programs based on the schedule and criteria identified in Task 15.1.7 and developed as part of interim action for Task 1.2.E.22. Completion date August 27, 2010.

• Task 2.1.D.5.f (CAL Item 8 and SIIP Action Plan 6, Strategy 5) (CRAI 3075719) -

Incorporate expected behaviors from Task 2.1.D.5.b into individual mid-year 2009 Performance Management Processes for Department Leaders and above.

Completion date August 6, 2009.

• Task 2.1.D.5.g (CAL Item 8 and SIIP Action Plan 6, Strategy 5)

(CRAI 3075721) - Incorporate expected behaviors from Task 2.1.D.5.b into all individual 2010 Performance Management Processes. Completion date February 22, 2010.

• Task 3.4.7.i (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047300) - Incorporate changes into the trend program such that line organizations trend their own data and identify developing trends, including the area of human performance on a proactive basis. Completion date June 6, 2009.

• Task 3.4.7.j (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 10)

(CRAI 3047302) - Implement an interactive automated trending program to facilitate identification of developing trends at both the line and site levels.

Completion date September 26, 2009.

• Task 3.6.11 (CAL Item 2 and SIIP Action Plant 14, Strategy 7) (CRAI 3074615) -

Replace K1 relays in the emergency diesel generator control cabinets for all onsite Class 1E emergency diesel generators. Completion date July 7, 2009.

• Task 3.7.5.ii (CAL Item 1 and SIIP Action Plan 15, Focus Area 4)

(CRAI 3014822) - Perform evaluation to determining cause of inadequate Procedure Quality. Completion date June 2, 2010.

• Task 3.7.3.x (CAL Item 1 and SIIP Action Plan 15, Focus Area 2, Strategy 3; and CAL Item 1 and SIIP Action Plan 15, Focus Area 10, Strategy 4)

(CRAI 3145684) - Revise the Design Basis Manuals for systems containing high risk components to incorporate the changes outlined in the revised design basis manual writers guide and other changes that were identified during the project.

Completion date November 25, 2011.

• Task 3.7.3.y (CAL Item 1 and SIIP Action Plan 15, Focus Area 2, Strategy 3; and CAL Item 1 and SIIP Action Plan 15, Focus Area 10, Strategy 4)

(CRAI 3145690) - Create a new topical design basis manual for systems that have nuclear steam supply system design interface requirements and do not have a system Design Basis Manual. Completion date November 25, 2011.

• Task 6.1.1.c (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 1)

(CRAI 3032686) - Conduct effectiveness review or self-assessment on implementation of standards/expectations for leadership fundamentals.

Completion date July 22, 2009.

• Task 6.1.3.c (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 1)

(CRAI 3032692) - Conduct effectiveness review/self-assessment on implementation of engineering human performance tools, standard/expectations for engineering, and engineering fundamentals observations. Completion date April 22, 2009.

• Task 6.2.10 (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 8) (CRAI 3115664) - Develop Integrated Issues Identification Team to be used in conjunction with coach-the-coach program. Completion date June 30, 2009.

• Task 6.5.2.h (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022277) - First quarter 2009, review and determine if additional analysis is required for declining human performance. Completion date May 2, 2009.

• Task 6.5.2.i (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022278) - Second quarter 2009, review and determine if additional analysis is required for declining human performance. Completion date August, 1, 2009.

• Task 6.5.2.j (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022279) - Third quarter 2009, review and determine if additional analysis is required for declining human performance. Completion date November 1, 2009.

• Task 6.5.2.k (CAL Item 4 and SIIP Action Plan 11 Part 1, Strategy 2)

(CRAI 3022280) - Fourth quarter 2009, review and determine if additional analysis is required for declining human performance. Completion date January 25, 2010.

• Task 11.6.13 (CAL Item 1 and SIIP Action Plan 15, Focus Area 2, Strategy 3; CAL Item 1 and SIIP Action Plan 15, Focus Area 10, Strategy 4; and CAL Item 5 and SIIP Action Plan 4, Strategy 5) (CRAI 3115690) - Complete component design basis review project per project schedule. Completion date June 9, 2010.

• Task 20.10.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083294) - Complete the Safety Culture Improvement Plan (Security).

Completion date June 24, 2009.

• Task 20.10.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8)

(CRAI 3083295) - Close the above Safety Culture Improvement Plan based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment.

Completion date June 24, 2009.

The inspectors were made aware of certain tasks where closure packages had gone through and been approved for closure by the closure review board, and subsequently reviewed and considered closed by the NRC, had been changed and sent back through closure review board for another review. The inspectors re-reviewed the following task closure packages:

• Task 3.2.5.c (CAL Item 3 and SIIP Action Plan 6 Part 1, Strategy 6)

(CRAI 3047259) - Implement root and apparent cause review checklists to be used by Performance Improvement Department, Corrective Action Review Board, and Nuclear Assurance Department.

• Task 3.7.3.a (CAL Item 1 and SIIP Action Plan 15, Focus Area 4, Strategy 4)

(CRAI 2833594 ) - Track the need for further evaluation of Combustion Engineering control system Independent Design Review volume II based on completed results of the other Independent Design Review reviews.

• Task 3.7.3.q, 3.7.8.k, 3.7.11.b (CAL Item 1 and SIIP Action Plan 15, Focus Area 2, Strategy 2; and CAL Item 1 and SIIP Action Plan 15, Focus Area 10, Strategy 1) (CRAI 2825641) - Engineering to communicate to all potential design basis manual users, on the possible limitations of the design basis manual and measures to effectively use the design basis manuals. Accuracy of the design basis manuals may not be 100 percent; there may be errors of omission (primarily unincorporated engineering design changes, and other possible omissions) and possibly inaccurate content. Users should "validate" and "quality verify" the information with other resources when possible. Users should also understand the context of set point information; the differences between safety limits, operational bands, instrument inaccuracies, etc.

• Task 3.7.5.a (CAL Item 1 and SIIP Action Plan 15, Focus Area 4, Strategy 4)

(CRAI 2833593) - Track to completion Open Combustion Engineering control system Independent Design Review items.

• Task 3.7.5.e (CAL Item 1 and site integrated improvement plan Action Plan 15, Focus Area 4, Strategy 4) (CRAI 2785329) - Review Containment Systems Independent Design Review to determine if any other design requirement was not incorporated in design documents.

• Task 3.7.5.o (CAL Item 1 and site integrated improvement plan Action Plan 15, Focus Area 4, Strategy 4) (CRDR 2825202) - Document a potential discrepancy on applicability of seismic requirement to containment access purge filter identified during review of Containment Systems.

• Task 3.7.5.p (CAL Item 1 and site integrated improvement plan Action Plan 15, Focus Area 4, Strategy 4) (CRAI 2825460) - Conduct extent of condition review to determine if there were other instances of design or licensing commitments identified that were not effectively translated into design documents (containment systems).

• Task 3.7.5.t (Cal Item 1 and site integrated improvement plan Action Plan 15, Focus Area 4, Strategy 4) (CRAI 2825475) - Conduct extent of condition review to determine if there were other instances of design or licensing commitments identified that were not effectively translated into design documents (fire protection).

• Task 3.7.5.dd (CAL Item 1 and SIIP Action Plan 15, Focus Area 4, Strategy 2; and CAL Item 1 and site integrated improvement plan Action Plan 15, Focus Area 6, Strategy 6) (CRAI 2825630) - Training will develop the Emergency Core Cooling Sump event as a case study emphasizing how the design configuration escaped detection for over 20 years during various missed opportunities.

• Task 4.1.G.4 (CAL Item 8 and SIIP Action Plan 1, Strategy 8) (CRAI 3064339) -

Review Institute of Nuclear Power Operations01-002, "Conduct of Operations,"

and identify the key operations department attributes and behaviors of an operationally focused organization.

• Task 4.1.G.6.a (CAL Item 8 and SIIP Action Plan 1, Strategy 8)

(CRAI 3064340) - Develop Operational Focus training module and perform a Needs Analysis to determine the training required for establishment of an operationally focused organization (Operations).

• Task 4.1.G.6.b (CAL Item 8 and SIIP Action Plan 1, Strategy 8)

(CRAI 3076121) - Develop Operational Focus training module and perform a Needs Analysis to determine the training required for establishment of an operationally focused organization (Maintenance).

• Task 4.1.G.6.c (CAL Item 8 and SIIP Action Plan 1, Strategy 8)

(CRAI 3076123) - Develop Operational Focus training module and perform a Needs Analysis to determine the training required for establishment of an operationally focused organization (Engineering).

• Task 4.1.G.6.d (CAL Item 8 and SIIP Action Plan 1, Strategy 8)

(CRAI 3076124) - Develop Operational Focus training module and perform a Needs Analysis to determine the training required for establishment of an operationally focused organization (Radiation Protection).

• Task 4.1.G.6.e (CAL Item 8 and SIIP Action Plan 1, Strategy 8)

(CRAI 3076126) - The training required for establishment of an operationally

focused organization. Develop Operational Focus training module and perform a Needs Analysis to determine (Chemistry).

• Task 6.1.3.a (CAL Item 4 and site integrated improvement plan Action Plan 11 Part 1, Strategy 1) (CRAI 3032691) - Identify/revise procedures containing direction to use human performance tools associated with engineering tools.

• Task 6.7.7 (CAL Item 1 and SIIP Action Plan 15, Focus Area 6, Strategy 4)

(CRAI 2988515) - Evaluate and implement metrics/indicators to include station performance on and overall health of the Operating Experience program.

• Task 6.7.30 (CAL Item 6 and SIIP Action Plan 6 Part 2, Strategy 3)

(CRAI 3133784) - Develop plan to validate and perform effectiveness reviews on other past high tier operating experience received from Institute of Nuclear Power Operations /NRC.

• Task 9.1.A.6 (CAL Item 10 and SIIP Action Plan 8, Strategy 6) (CRAI 3063200) -

Develop strategies (posters, lanyard cards, etc.) to communicate Emergency Planning Program elements to the line organization.

• Task 9.1.21 (CAL Item 10 and SIIP Action Plan 8, Strategy 6) (CRAI 3076194) -

Implement the communications strategies developed in CRAI 3063200.

• Task 9.2.A.16 (CAL Item 10 and SIIP Action Plan 8, Strategy 4) (CRAI 3065521)

- Ensure the initial training for Emergency Coordinators contains training on Emergency Action Levels and their bases. This includes the Emergency Coordinator in the Control Room and the Emergency Coordinator in the Technical Support Center.

• Task 9.2.22 (CAL Item 10 and SIIP Action Plan 8, Strategy 4) (CRAI 3065522) -

Ensure the continuing training programs for Emergency Coordinator contains biennial training on Emergency Action Levels.

• Task 9.2.23 (CAL Item 10 and SIIP Action Plan 8, Strategy 5) (CRAI 3065519 and 3184906) - Administratively control Emergency Preparedness Training similarly to accredited training programs by creation of an Emergency Preparedness Training Program Description which as a minimum places controls on the following:

(1) Description that defines failures and remediation criteria for each Emergency Response Organization position. This would encompass failures during training, drills, exercises, and actual plant events.

(2) Implement a program to track named [a defined set of] Emergency Response Organization team positions to ensure these personnel receive proper training and drill participation on an annual basis.

(3) Develop a plan for drills, specifically on continuance of training drills, in 2009 and beyond, based on performance.

• Task 9.5.5 (CAL Item 10 and SIIP Action Plan 8, Strategy 9) (CRAI 3065531) -

Revise Procedure 21SP-0SK11 to include Emergency Action Levels 7.1, 7.2, and 7.3; Provide information to Operations Training in support of CRAI3065613 and to Emergency Services Department Training in support of CRAI 3121416.

The inspectors were made aware of certain tasks where interim closure packages had been reviewed and considered closed by the NRC, had been finalized and gone through and been approved for closure by the closure review board. The inspectors re-reviewed the following task closure packages:

• Task 3.6.7 (CAL Item 2 and SIIP Action Plan 14, Strategy 6) (CRAI 2958748) -

Develop and provide training to equipment root cause of failure analysis qualified personnel that will include:

(1) the need to consider all failure modes as part of initial troubleshooting and root cause activities;

(2) reviewing any applicable operating experience as part of the initial troubleshooting and root cause activities;

(3) a discussion of establishing appropriate priority to ensure a quality analysis; and

(4) a discussion of accountability and expectations for both quality and timeliness.

• Task 11.3.14 (CAL Item 12 and SIIP Action Plan 4, Strategy 6) (CRAI 3064843) -

Develop metrics to facilitate and monitor burn-off of temporary installations identified in Task 11.3.13.

• Task 11.8.21 (CAL Item 2 and SIIP Action Plan 14, Strategy 5) (CRAI 3066109) -

Provide training on the systematic problem solving and decision-making methodologies/techniques.

The inspectors considered all of the above tasks closed. For more details, see Section 4OA5.1.b.1.

b. Findings and Observations

(1) Task Closure During the review of the SIIP tasks, the inspectors identified quality issues, including the following:

• Task 2.4.B.4 (Develop/implement a formal Management Succession Plan and associated policy) called for Policy Guide 0308-01, Palo Verde Succession Planning, to include descriptions of rotations outside of employees area of responsibility. The licensees implementation of this task did not include this step. To address this issue PVAR 3284684 was written.

• Task 6.6.1.a (Include human performance PIs to site performance indicators)current plan has 6 human performance PIs as part of the task closure. Palo Verde will be implementing INPO 08-004 Human Performance Key Performance Indicators, General Practices for Tracking, Trending and Communicating Station Human Performance to replace the CAL task human performance PIs in the April-May timeframe. The INPO plan has 12 PIs, though the identical indicators to the current Palo Verde plan have a higher threshold. The final task closure document is to identify the differences.

• Five Tasks contained ambiguous or undefined terms. Specifically:

o Tasks 9.1.A.6 (Develop strategies to communicate Emergency Planning Program elements to the line organization) and 9.1.A.21 (implement the communications

strategies) did not define the program elements to be communicated, did not define the personnel to whom the elements would be communicated, and did not identify the emergency planning performance intended to be enhanced. These tasks were closed based on licensee actions to inform employees who were not members of the emergency response organization about protected area evacuation signals and the site evacuation process; documentation did not establish that emergency planning program elements were intended to be limited to the site evacuation process, or that the line organization was limited to employees who were not members of the licenses emergency response organization. To address this issue PVAR 3279660 was written.

o Task 9.2.A.23 (defines failures and remediation criteria for each Emergency Response Organization position, implement a program to track named [a defined set of] Emergency Response Organization team positions) did not define remediation criteria for Emergency Response Organization performance in that the threshold was not defined for performance in an acceptable manner, and the specific Emergency Response Organization positions that constituted a defined set of team positions was not included in the task description. To address this issue PVAR 3279663 was written.

o Tasks 9.5.1 (Evaluate implementation of Nuclear Energy Institute Report 99-01 strategy and develop recommendations) and 9.5.2 (Present the strategy and development recommendations defined in Task 9.5.1 to senior leadership) did not define senior leadership. These tasks were closed based on actions of the Emergency Preparedness Steering Committee; however, the Steering Committee did not include licensee executives at the Director and Vice President levels. To address this issue PVAR 3279668 was written.

• Task 9.1.A.33 (Create anappropriate number of Emergency Preparedness Training Advisory Committees): The closure documentation did not demonstrate sustainability of licensee actions to close the task in that:

o Documentation was not provided for monthly Training Advisory Committee meetings in Third Quarter 2008, as required by Training Review Group meeting minutes dated June 26, 2008.

o Documentation of the initial membership of the Training Advisory Committees was not provided in the closure package, nor a discussion of when and how the committees membership would be maintained.

o Licensee Procedure 16P-0EP20, Emergency Planning Conduct of Operations, Revision 11, provides for an Industry Peer or subject-matter expert Contract as a member of the Training Review Group when possible. Documentation was not provided for when or how a peer or contractor would be utilized by the Training Review Group, and the licensee subsequently identified this individual as an interim measure, rather than as an ongoing expectation. To address these issues PVAR 3279670 was written.

• Task 9.2.A.15 (Conduct training on Emergency Action Levels with Emergency Coordinator and Emergency Operations Director-qualified individuals): Closure document, Attachment 11, was a Site Work Management System report listing

the On-Shift Emergency Coordinators, Technical Support Center Emergency Coordinators, and Emergency Operation Directors who completed lesson plan NLR33, Emergency Action Levels. Documentation was also provided for each individuals attendance at the eight training sessions associated with lesson plan NLR33. Documentation was not provided that each individual successfully completed the Job Performance Measure or written examination associated with lesson plan NLR33, to verify the report provided as Attachment 11. To address this issue PVAR 3279671 was written.

• Task 9.2.A.23 (define failures and remediation criteria for each Emergency Response Organization position):

o Documentation was not provided for the sustainability of group and emergency response facility tabletops. Specifically, the Drill Plan Letter dated June 6, 2008, stated that emergency response organization performance would be trended and trend results used to develop areas of emphasis for group and emergency response facility tabletop drills, however, the licensee did not provide documentation for who was responsible to perform the trending, the frequency of trending, or how the results would be considered in the scenario development process. The task was closed before the procedure to address these elements was developed and implemented.

o Metrics to measure the effectiveness of Emergency Response Organization remediation actions (e.g. rate of repeat failures, rate of failure on post-remediation examination, etc.) were not developed prior to closing the Task. To address these issues PVAR 3279675 was written.

• Tasks 9.5.1 (Evaluate implementation of Nuclear Energy Institute Report 99-01 strategy and develop recommendations) and 9.5.2 (Present the strategy and development recommendations defined in Task 9.5.1 to senior leadership):

o Closure documentation did not support that a technical evaluation of Nuclear Energy Institute Report 99-01, Revision 5, Emergency Action Levels was performed. Specifically, no documentation was provided that problems were identified with the current, Revision 2, Emergency Action Level scheme, that benefits were identified associated with implementing a Revision 5 classification scheme, that the option of maintaining the present classification scheme was considered, that actual or potential costs related to implementing a new classification scheme were considered or identified, or that potential problems related to implementing a new classification scheme were considered or identified.

o Closure documentation did not support that a strategy was developed for developing and implementing a Nuclear Energy Institute Report 99-01, Revision 5, Emergency Action Level scheme. Specifically, documentation was lacking for an anticipated schedule of necessary activities and milestones, that support for the NRCs licensing activities was identified, that options were identified for when a new classification scheme could be implemented, that related documents and procedures were identified, that training needs associated with implementing a new classification scheme were estimated or identified, or that estimates of the personnel resources associated with developing and implementing a new

classification scheme (including revised and new training lesson plans) were evaluated. To address these issues PVAR 3279678 was written.

• The Nuclear Assurance Departments closure review identified Tasks 9.1.A.6, 9.1.A.21, 9.2.A.16, 9.2.A.22, and 9.2.A.23, as having insufficient information to justify closure of the task activities. Although the licensee addressed the Nuclear Assurance Departments concerns for each tasks, revised Nuclear Assurance Department closure reviews were not generated to document that the licensees subsequent actions were acceptable to the Nuclear Assurance Department. To address this issue PVAR 3279683 was written.

• Task 20.6.1 (Complete the Safety Culture Improvement Plan (Training)) provided little objective evidence that the safety culture in the training department had improved. After being challenged by the inspectors, the licensee was able to produce additional documentation to demonstrate improvement in specific attributes of safety culture. Additionally, the inspectors noted that a large, unmitigated backlog exists in the Training Change System. The backlog contains over two hundred necessary training changes, and the department does not presently have a work-off strategy to accomplish these tasks.

• Task 20.12.2 (Close 20.12.1 based on satisfactory results of the Site Wide Fall 2008 Safety Culture Assessment), did not include an evaluation of managers and contractors feedback. To address this issue, PVAR 3282721 was written.

• During 2008 the licensee created an Emergency Planning Steering Committee to increase managements attention to, and engagement with, site emergency preparedness issues. Draft Revision 3 to the Steering Committee charter eliminates Manager and Director-level individuals as required committee members, with Group Leaders as the highest required level of management.

This change has the potential to reduce the Committees ability to perform its stated mission, as it may reduce the engagement of site Managers and Directors with emergency preparedness. The draft Charter also eliminates the Committees responsibility to review information and sources that may impact the Emergency Preparedness program, a change also having the potential to reduce the Committees effectiveness. To address this issue, PVAR 3279684 was written.

• The licensees Drill and Exercise Performance Indicator was essentially unchanged from 2007, prior to the Site Integrated Improvement Plan, through the end of 2008, after the closure of emergency-preparedness related Site Integrated Improvement Plan tasks:

o The licensees reported Fourth Quarter 2008 Drill and Exercise Performance Indicator was 94.6% (475 successes in 501 opportunities). The licensee provided documentation for three Emergency Response Organization training drills not counted in the Drill and Exercise Performance Indicator, with a cumulative success rate of 76% (25 successes in 33 opportunities). When the training drill data is included with evaluated drills, the effective performance rate is 93.5% (500 successes in 535 opportunities). The NRC 95003 Inspection Team identified a concern that the licensees actual drill and exercise

performance success rate was less than the reported performance indicator value. To address this issue PVAR 3279685 was written.

• The inspectors reviewed the Priority Group Safety Culture Improvement Plans.

As part of this review, the inspectors challenged the licensees position that the individual actions contained within these plans did not need to be considered for sustainability based on the licensees assumption that other SIBP/SIIP actions would ensure their sustainability. Based upon the challenge from the inspectors, the licensee initiated PVAR 3283131 and CRAI 3284978 to review the individual actions for sustainability. The licensee discovered that of the 213 individual actions, only 175 demonstrated sustainability through one or more site-wide SIBP/SIIP actions, policies or procedures. The licensee subsequently generated one PVAR and seven additional CRAIs to ensure the sustainability of the remaining actions.

• Closure package for Task 20.10.1, Safety Culture Findings - Security, documented completion of Action 6.2, to develop and administer a survey designed to identify the security procedures of greatest concern to security officers and leaders with respect to ease of use and quality; however, only 8 of approximately 250 security personnel responded to the informal email survey and the survey could not be considered statistically valid. In addition, the licensee planned to take credit for the identification and correction of security procedural issues by the formation of the security performance improvement team but had failed to specify this responsibility in the teams charter. To address these issues, the licensee initiated PVAR 3290517. This task and the effectiveness task (20.10.2) were closed on an interim basis pending review of a valid survey.

The inspectors also reviewed the SIIP quality performance indicators, interviewed numerous personnel, and reviewed several effectiveness reviews related to CAL SIIP actions.

(2) Metrics and Measures to Monitor Improvement The inspectors also reviewed the SIIP quality performance indicators, interviewed numerous personnel, and reviewed several effectiveness reviews related to CAL SIIP actions. The inspectors noted that the licensee had developed SIIP performance indicators to track the effectiveness of tasks associated with the SIIP Action Plans and the CAL Key Performance Areas. The performance indicators were divided into Operations, such as Operator Workarounds, Long-Term Tagouts, and Operability Determinations; Engineering, including Engineering Work Product Quality and Engineering Program Health Reports; Site Programs and Processes, including Corrective Action Plan Quality Index and Timeliness of Operating Experience Screening; and Organization, including Site Clock Resets and Consequential Human Error Rate.

The inspectors reviewed these performance indicators and determined that the indicators were appropriate and provided useful information. However, the inspectors identified that functional assessment issues were not addressed in the Operability Determination Quality Metric or in any other metric. Also, operability determination issues found during this inspection needed to be added to the Operability Determination Quality Metric. To address these issues, PVAR 3285126 was written.

(3) CAL Item Closure The inspectors reviewed all of the tasks associated with the following key performance areas:

• Key Performance Area 1, address root and contributing causes identified in Palo Verde Nuclear Generating Station evaluations in response to the Yellow finding associated with voided containment sump suction piping for all three units.

• Key Performance Area 3, address problem identification and resolution performance issues.

• Key Performance Area 4, address human performance issues.

• Key Performance Area 5, address problems with the implementation of engineering programs.

• Key Performance Area 7, address issues identified during the 2007 independent safety culture assessment.

• Key Performance Area 8, address problems associated with standards and expectations for performance and holding individuals accountable for nuclear safety.

• Key Performance Area 10, address problems associated with the emergency preparedness program.

During this CAL inspection, the inspectors reviewed and closed the 20 remaining open CAL items associated with Key Performance Area 1, the 21 remaining open CAL items associated with Key Performance Area 3, the 19 remaining open CAL items associated with Key Performance Area 4, the three remaining open CAL items associated with Key Performance Area 5, the 32 remaining open CAL items associated with Key Performance Area 7, the eight remaining open CAL items associated with Key Performance Area 8, and the 12 remaining open CAL items associated with Key Performance Area 10. During the previous four CAL inspections, all other tasks associated with Key Performance Areas 1, 3, 4, 5, 7, 8, and 10 were reviewed and closed. Because all the tasks have been reviewed and closed, Key Performance Areas 1, 3, 4, 5, 7, 8, and 10 are considered completed and closed.

(4) Closure of the Yellow Finding (Closed) Violation (VIO) 05000528,529,530/2004014-01, Failure to Maintain Design Control of Containment Sump Recirculation Piping. Consistent with the guidance provided in NRC Inspection Procedure 95002, Inspection for One Degraded Cornerstone or Any Three White Inputs in a Strategic Performance Area, the inspectors reviewed the root cause and extent of condition evaluations associated with the voided section of the containment sump recirculation piping. Upon completion of their review, the inspectors determined that the evaluation appropriately identified the root and contributing causes for the voided piping, identified appropriate corrective actions to address the root and contributing causes, and included a thorough extent of condition and extent of cause review. The inspectors independently determined that any safety

culture components that contributed to the performance issues were addressed. The inspectors noted that the voided emergency core cooling system piping was evaluated using a systematic method to identify the root and contributing causes, and the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem. The inspectors reviewed the completed corrective actions to verify that qualitative measures had been developed for the corrective actions, and implementation of these actions was sufficient to prevent recurrence. Also, the inspectors reviewed the root cause evaluation to ensure it included consideration of prior occurrences of the problem and knowledge of any applicable operating experience. Because the root cause evaluation has been reviewed in its entirety, and all the completed corrective actions associated with the Yellow finding have been reviewed, the Yellow finding is also considered closed.

(5) Findings

(a) Failure to Thoroughly Evaluate Conditions Adverse Quality for Potential Operability Impacts

Introduction.

The inspectors identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the licensees failure to ensure that degraded and nonconforming conditions associated with safety related systems and systems important to safety were reviewed for potential operability impacts. Specifically, operations personnel failed to ensure all relevant information was reviewed for operability of PVARs associated with the component design basis review project and other site projects, resulting in 97 PVARs that either needed an immediate operability determination or a functional assessment or the information provided was not adequate to give reasonable assurance of operability.

Description.

On February 4, 2009, inspectors noted that PVARs written during the component design basis review either needed an immediate operability determination or a functional assessment or the immediate operability determination/functional assessment needed more information to provide a reasonable assurance of operability.

Procedure 01DP-0AP12, "Palo Verde Action Request Processing," Revision 10, Step 3.5, states, in part, that an evaluation shall be performed for PVARs that have been screened at the operations review step and determined that a control room review is warranted. Procedure 01DP-0AP12 also states that the condition described in the PVAR shall be evaluated by the shift manager for the assessment of potential operability concerns. During the inspectors review, it was noted that the conditions described in the PVARs met this criteria; therefore, these issues were required to be evaluated by the shift manager for the assessment of potential operability concerns. The inspectors questioned operations personnel on whether 17 PVARs either needed an immediate operability determination/functional assessment or the immediate operability determination/functional assessment needed more information to provide a reasonable assurance of operability. Operations personnel reviewed the PVARs and determined that immediate operability determinations/functional assessments were either required or more information was needed in the assessments to support operability. After discussion with the inspectors, operations personnel performed immediate operability determinations/functional assessments on the 17 PVARs in accordance with

Procedure 40DP-9OP26, Operability Determination and Functional Assessments, Revision 22, and determined that all affected components were operable and would be able to perform their functions.

The inspectors questioned the licensee to determine if other PVARs associated with the component design basis review project either needed immediate operability determinations/functional assessments or needed more information to support operability. The licensee developed a team to expand the scope to understand the extent of condition. This team reviewed 925 component design basis review PVARs, 213 PVARs associated with other reviews, and three NRC-identified issues. The licensee determined that a total of 97 PVARs either needed immediate operability determinations/functional assessments or needed more information to support operability. The team reviewed the examples identified by the licensee and noted that the majority of them were issues that had very small impact on safety margins. Although these issues were of small impact, the team communicated with the licensee the importance of thoroughly understanding their safety impact so that adequate margins of safety are always maintained.

In response to the NRC Confirmatory Action Letter (CAL-4-07-004), the licensee took numerous actions to improve their operability determination process. The actions, including procedure changes and training, were complete by June 30, 2008. Twenty of the 97 PVARs that either needed immediate operability determinations/functional assessments or needed more information to support operability occurred after implementation of corrective actions to improve the operability determination process.

The team noted that the majority of examples identified occurred prior to completion of the licensees corrective actions. As the corrective actions were implemented the number of examples significantly reduced indicating that the licensees actions were improving performance.

The licensee performed an apparent cause evaluation and determined that two causes primarily resulted in the inadequate immediate operability determinations/functional assessments. Specifically, lack of procedural quality and guidance, and a lack of knowledge such that problems are thoroughly evaluated, were the primary causes. Due to the operability determination issues raised by the inspectors, the licensee took interim actions to prevent other missed immediate operability determinations/functional assessments including issuing a night order that all component design basis review, Westinghouse, and License Renewal PVARs get a control room review. The licensee developed additional actions to improve the guidance in Procedures 01DP-0AP12 and 40DP-9OP26, and provide additional training as a result of the apparent cause evaluation.

One example of a PVAR written after June 30, 2008, that needed an immediate operability determination but did not receive one and was associated with the knowledge causal factor, includes the following:

On September 23, 2008, the component design basis review project identified in PVAR 3226929 that the Class-1E 125 Vdc distribution system motor operated valve full load and locked rotor current values in the Technical Data Files Drawing do not agree with the calculations used to ensure load limits are not exceeded. This potentially impacts the ability of the Class-1E 125 Vdc distribution system to provide appropriate current to safety-related loads during design basis accidents. In this case, the senior

reactor operator did not recognize that the deficiency had the potential to impact operability or functionality of a Technical Specification system, structure, or component and the PVAR was errantly dispositioned as not identifying a degraded or nonconforming condition and was not reviewed for operability. After inspectors questioned the need for an operability determination for this issue, an immediate operability determination was performed on February 6, 2009, and determined the motor-operated valves were operable.

Analysis.

The performance deficiency associated with this finding was the failure of operations personnel to follow the corrective action program to ensure that degraded and nonconforming conditions associated with safety related systems and systems important to safety were reviewed for operability. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings,"

the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event.

This finding has a crosscutting aspect in the area of problem identification and resolution associated with the corrective action program because nine examples, reflective of current performance, were not thoroughly evaluated such that the resolutions address causes and extent of conditions, as necessary, including properly evaluating for operability conditions adverse to quality P.1(c).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings.

Procedure 01DP-0AP12, "Palo Verde Action Request Processing," Revision 10, Step 3.5 stated, in part, that an evaluation shall be performed for PVARs that have been screened at the operations review step and determined that a control room review is warranted. Procedure 01DP-0AP12 also stated that the condition described in the PVAR shall be evaluated by the shift manager for the assessment of potential operability concerns. Contrary to the above, between December 21, 2006, and January 30, 2009, operations personnel failed to ensure that degraded and nonconforming conditions associated with safety-related systems and systems important to safety were reviewed for operability. Specifically, operations personnel failed to ensure all relevant information was reviewed for operability of PVARs associated with the component design basis review project and other site projects, resulting in failures to perform immediate operability determinations or functional assessments to demonstrate reasonable assurance of operability. Because this finding is of very low safety significance and has been entered into the licensee's corrective action program as PVAR 3281099, this violation is being treated as a noncited violation, consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000528;529;530/2009006-06, Failure to Thoroughly Evaluate Conditions Adverse Quality for Potential Operability Impacts.

(b) Inadequate Procedures for Performing Operability Determinations Introduction and

Description.

As previously discussed, a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings," was identified involving the licensees failure to ensure that degraded and nonconforming conditions associated with safety related systems and systems important to safety were reviewed for potential operability impacts. Of the 97 examples 20 were reflective of current performance since corrective actions to improve the operability evaluation process had been completed. Of the 20 current examples 11 of the 20 errors resulted from a lack of procedural quality and guidance such that a more rigorous procedure, with more in-depth requirements, may have prevented the deficiency.

One example of a PVAR written after June 30, 2008, that needed an immediate operability determination but did not receive one and was associated with the procedural adequacy causal factor, includes the following:

On October 29, 2008, the component design basis review project identified in PVAR 3244919 that several subcomponents for the valve operators for the atmospheric dump valves were classified as nonquality-related, but should have been classified as quality-related or quality-augmented. These subcomponents are necessary for the manual operation of the atmospheric dump valves and are credited in fire protection scenarios as a backup method of closing the valve in the event of a stuck open atmospheric dump valve in a steam generator tube rupture event, and for reactor coolant system cooldown. The PVAR was dispositioned as not identifying a degraded or nonconforming condition, and was not reviewed for operability. After inspectors questioned the need for an operability determination for this issue, an immediate operability determination was performed on February 9, 2009, and determined the atmospheric dump valves were operable. This PVAR was originally labeled as trend code N/A, indicating that there was no need for an operability determination or functional assessment. The licensee determined that the guidance provided in Procedure 40DP-9OP26 was not sufficient relative to the use of the N/A trend code.

Analysis.

The performance deficiency associated with this finding was the failure of operations personnel to follow the corrective action program to ensure that degraded and nonconforming conditions associated with safety related systems and systems important to safety were reviewed for operability. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the reliability of systems that respond to initiating events to prevent undesirable consequences. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings,"

the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its Technical Specification allowed outage time, or screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event.

This finding has a crosscutting aspect in the area of human performance associated with resources because eleven examples, involved inadequate procedural guidance governing the conduct of operability determinations to ensure that conditions adverse to quality were properly evaluated for their potential operability impacts H.2(c).

Enforcement.

Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings.

Procedure 01DP-0AP12, "Palo Verde Action Request Processing," Revision 10, Step 3.5 stated, in part, that an evaluation shall be performed for PVARs that have been screened at the operations review step and determined that a control room review is warranted. Procedure 01DP-0AP12 also stated that the condition described in the PVAR shall be evaluated by the shift manager for the assessment of potential operability concerns. Contrary to the above, between December 21, 2006, and January 30, 2009, operations personnel failed to ensure that degraded and nonconforming conditions associated with safety-related systems and systems important to safety were reviewed for operability. Specifically, operations personnel failed to ensure all relevant information was reviewed for operability of PVARs associated with the component design basis review project and other site projects, resulting in failures to perform immediate operability determinations or functional assessments to demonstrate reasonable assurance of operability. Because this finding is of very low safety significance and has been entered into the licensee's corrective action program as PVAR 3281099, this violation is being treated as a noncited violation, consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000528;529;530/2009006-07, Inadequate Procedures for Performing Operability Determinations.

(c) Failure to Promptly Identify and Correct a Condition Adverse to Quality with the Emergency Core Cooling System Piping

Introduction.

A Green self-revealing noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," was identified for the licensees failure to promptly identify and correct a condition adverse to quality associated with the high pressure safety injection piping. Specifically, the licensee failed to conduct appropriate inspections to ensure that damage due to flow erosion in the high pressure safety injection piping was discovered prior to equipment failure.

Description.

On October 12, 2006, while running high pressure safety injection Pump B, maintenance personnel discovered a small through-wall leak on the high pressure safety injection system Train B piping. The leak was on a common emergency core cooling system recirculation line to the refueling water tank. The line was isolated from the refueling water tank by closing the common recirculation line Valve 1JSIBUV0659, rendering Train B high pressure safety injection, low pressure safety injection, and containment spray inoperable.

During examination of the affected Unit 1 piping, it was discovered that localized cavitation, over a long period of time, caused erosion in isolation Valve 1JSIBUV0667, and in the downstream piping and elbow. Evaluation of this failure confirmed that high pressure safety injection Train B recirculation line is susceptible to incipient, or in some cases, damaging cavitation resulting from operation of the system during certain surveillance tests and when the system is configured to fill the safety injection tanks.

The affected portion of piping was replaced in all three units.

The inspectors reviewed Engineering Evaluation Request 88-SI-119, which assessed the potential affects of high flow in the high pressure safety injection recirculation line, and the maximum allowable flow for testing using the recirculation lines. This

engineering evaluation request stated, in part, that because it cannot be determined conclusively whether the projected cavitation at the bypass valve results in any material damage, a boroscope inspection of the valves and the adjacent piping following the first surveillance tests should be conducted to determine if damage has occurred. The results of these inspections will determine if boroscopic inspections will be required following subsequent surveillance tests. Even though this calculation said that boroscopic inspections should be performed to ensure no damage occurs to the piping and valves, these inspections were never performed. As a result, the flow erosion damage in the high pressure safety injection piping was not discovered until it resulted in a the through-wall leak.

The licensees apparent cause identified other piping that may be susceptible to erosion due to cavitation. The licensee inspected these areas in Unit 1, and found no further indication of erosion. The inspectors reviewed the work orders to inspect the other susceptible piping in Units 2 and 3, and questioned whether piping, that was determined to have similar flow conditions to the high pressure safety injection piping that had the through-wall leak, should be inspected two refueling outages after the high pressure safety injection leak was identified. The licensee determined that the extent of condition inspection of other susceptible piping was not timely, and wrote PVAR 3285128. The licensee later determined the inspection dates were appropriate since the licensee had already inspected similar piping in Unit 1 and found no degradation, and since erosion due to cavitation is a long term issue.

Analysis.

The performance deficiency associated with this finding involved the licensees failure to promptly identify and correct a condition adverse to quality associated with the high pressure safety injection system piping. Specifically, the licensee failed to conduct appropriate inspections to ensure that damage due to flow erosion in the high pressure safety injection piping was discovered prior to equipment failure. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the reliability and availability of systems that respond to initiating events. Using Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because the finding did not result in a loss of system safety function, an actual loss of safety function of a single train for greater than its technical specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. This finding was evaluated as not having a crosscutting aspect because the performance deficiency is not indicative of current performance.

Enforcement.

Title 10 of the Code of Federal Regulation, Part 50, Appendix B, Criterion XVI, "Corrective Action," states, in part, that measures shall be established to assure that conditions adverse to quality are promptly identified and corrected.

Procedure 01PR-0AP04, "Corrective Action Program," Revision 3, stated, in part, that measures will be established to assure that conditions adverse to quality, such as failures, malfunctions, and deficiencies are promptly identified and corrected. Contrary to this, between January 18, 1989 and October 12, 2006, the licensee failed to ensure that select sections of Unit 1 high pressure safety injection Train B piping was inspected to prevent erosion due to cavitation. This resulted in a through-wall leak in the high pressure safety injection Train B recirculation line. Because the finding is of very low safety significance and has been entered into the licensees corrective action program

as CRDR 2932507, this violation is being treated as an noncited violation consistent with Section VI.A.1 of the Enforcement Policy: NCV 05000528;529;530/2009006-08, Failure to Promptly Identify and Correct a Condition Adverse to Quality with the High Pressure Safety Injection System Piping.

(d) Safety Injection Pump Full Flow Recirculation Potential Design Control Issue

Introduction.

The inspectors identified an unresolved item (URI) regarding the potential failure of the containment spray and/or low pressure safety injection pumps during plant operations that put the refueling water tank in full flow recirculation.

Description.

The inspectors reviewed PVAR 3284044 and identified two issues that potentially exist with the ability of the safety injection mini-flow lines to recirculate to the refueling water tank when a safety injection actuation signal is received during accident conditions with either the low pressure safety injection or containment spray pumps in full flow recirculation to the refueling water tank. In some system lineups, operations personnel use the full flow recirculation line to the refueling water tank by opening normally locked-closed manual valves. This full flow recirculation line uses the same return line back to the refueling water tank that the pump mini-flow lines use. The two scenarios of concern are:

1) One of the two containment spray pumps starts, but does not immediately inject into the reactor coolant system due to a small break loss of coolant accident. The head losses in the return line under full flow recirculation to the refueling water tank could induce sufficient backpressure in the mini-flow lines such that the other train of containment spray and both trains of low pressure safety injection potentially run dead-headed, or without any mini-flow. In a related issue, if one of the two low pressure safety injection pumps start, but do not inject immediately, the backpressure in the common recirculation line could dead-head the other train of low pressure safety injection.

2) A scenario in which there is a large break loss of coolant accident, and the refueling water tank depletes relatively quickly. If the manual valves associated with the full flow recirculation line are not shut, an unintended flow diversion out of containment back to the refueling water tank could occur after a recirculation actuation signal.

NRC Bulletin No. 88-04, "Potential Safety Related Pump Loss," described the potential for dead-heading one or more pumps in safety-related systems that have a common mini-flow line. This licensee's evaluation in response to the bulletin failed to consider all emergency core cooling system line-ups, resulting in a failure to evaluate the full flow recirculation lineup to the refueling water tank on the mini-flow lines. Discussions with the licensee and the Westinghouse indicated that this recently discovered information is potentially generic in nature. This URI is being opened to determine if a design control (or other) performance deficiency exists and to determine the significance of any identified performance deficiencies: URI 05000528;529;530/2009006-09, Safety Injection Pump Full Flow Recirculation Potential Design Control Issue.

4OA6 Management Meetings

On February 27, 2009, an exit meeting was conducted. The results of the inspection were discussed with Mr. Bement and other members of the staff. The licensee confirmed that the inspectors had retained no proprietary information during this inspection.

On March 13, 2009, a re-exit meeting was conducted to clarify the characterization and the crosscutting aspects of the findings. The results of the inspection were discussed with Mr. Bement and other members of the staff.

4OA7 Licensee-Identified Violations

The following finding of very low safety significance was identified by the licensee and is a violation of NRC requirements which meets the criteria of Section VI of the NRC Enforcement Policy, NUREG-1600 for being dispositioned as a noncited violation.

• Title 10 of the Code of Federal Regulation 50.73(a)(1) required that LERs be submitted within 60 days after the discovery of the event. Contrary to this requirement, on December 5, 2007, the licensee identified a condition that rendered the containment spray system inoperable but failed to submit the required report within 60 days. LER 05000530/2007-001-00 was submitted to the NRC on April 7, 2008, 120 days after the date of discovery. The licensee had identified seat leakage caused entry of borated water into the Unit 3 containment spray header in April 2007. Subsequent evaporation through the spray nozzles had caused boric acid residue to accumulate in the nozzles. On December 5, 2007, Arizona Public Service Company discovered that two nozzles were obstructed (one obstructed nozzle on each of two separate containment spray Train A headers). Technical Specification Surveillance Requirement 3.6.6.6 required that each spray nozzle be unobstructed. The Limiting Condition for Operation for Technical Specification 3.6.6. allowed one train of containment spray to be inoperable for less than 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> with the unit in Modes 1-4. Unit 3 had been in Mode 1 between April and November of 2007. Since the nozzles had been blocked sometime between April and December of 2007, the two containment spray nozzles were considered blocked for a period of time greater than allowed by Technical Specification 3.6.6, Containment Spray System, Limiting Condition for Operation. The licensee documented this missed LER submittal required by 10 CFR 50.73(a)(2)(i)(B) Any operation or condition prohibited by the plants Technical Specifications, in the corrective action program as PVAR 3288593 and CRDR 3130839. The nozzles were cleaned and restored to service on December 18, 2007.

This finding affects the mitigating systems cornerstone and is greater than minor because the NRC relies on licensees to identify and report conditions or events meeting the criteria specified in the regulations in order to perform its regulatory function. This is a Severity Level IV noncited violation consistent with Section 7.10 and Supplement I, Paragraph D.4, of the NRC Enforcement Policy.

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

A. Amoroso, Engineering Section Leader, Component Engineering Elec/I&C

J. Anderson, Performance Improvement Team Section Lead, CAP 1A

S. Bauer, Director Nuclear Regulatory Affairs

M. Benac, Director Strategic & Long Range Planning and Owner Affairs

D. L. Berg, Engineering Section Leader, E-FIN A

B. Berles, Engineering Department Leader, Component Performance Engineering

P. Borchert, Assistant Plant Manager, Unit 1

W. Borrero, Senior Engineer, Design Mechanical

F. Burdick, Regulatory Affairs Consultant, Compliance

R. Buzard, Compliance Section Leader, Compliance

R. Byers, Primary Plant Event Investigator, CAP 2A

D. Kanitz, Technical Management Assistant, Regulatory Affairs

E. Cesena, Shift Manager, OPS Work Control Unit 1

A. Chan, Site Finance Manager

D. Charles, Primary Plant Event Investigator, CAP 2A

K. Chermansky, Senior Engineer, Systems Engineering-Secondary

D. Coxon, Operations Department Leader, Unit 3

R. Doyle, Senior Engineer, Design Electrical

H. Duarte, Maintenance Team Leader, Electrical Maintenance

H. Durrani, Engineering Section Leader, Design Mechanical BOP

E. Dutton, Director Nuclear Assurance

D. Eastman, Technical Management Assistant, Refuel/Mechanical Maintenance

D. Elkinton, Senior Regulatory Affairs Consultant, Compliance

L. Elliott, Senior Engineer, Component Engineering Elec/I&C

J. Glover, Engineering Section Leader, Systems Engineering-Secondary

M. Grigsby, Operations Department Leader, Site Procedure Standards

D. Henry, Performance Improvement Team Section Lead, CAP 2A

J. Houston, Maintenance Department Leader, Maintenance Programs

R. Hunzelman, Control Room Supervisor, Unit 1

G. Hettel, Director Operations

G. Higgs, Maintenance Manager

M. Hooshmand, Engineering Department Leader, Design Mechanical

M. Hypse, Technical Management Assistant, Design Elec/I&C

T. Johnson, Compliance

M. Karbassian, Director of Nuclear Engineering Design & Technical Services

S. Karimi, Contract Services, IMPACT

J. Kendall, Primary Plant Event Investigator, CAP 2A

R. Kershaw, Senior Engineer, Component Programs

D. Kissinger, Senior Engineer, E-FIN A

R. Kropp, Senior Engineer, Component Programs

R. Laine, Senior Engineer, Component Engineering Mechanical

Attachment 1

M. Lacal, Director Executive Projects and Plant Support

F. Lake, Performance Improvement Team Department Lead, CAP 2A

C. Landstrom, Senior Engineer, System Engineering

H. Leake, Senior Consulting Engineer, Design Elec/I&C

L. Leavitt, Primary Plant Event Investigator, CAP 2A

D. Lorenzi, Senior Engineer, Component Design Basis Review Group

N. Lossing, Engineering Section Leader, System Engineering Elec/I&C

R. Lucero, Supply Chain Manager

D. Macedonia, Engineer II, Design Mechanical BOP

J. R. Maner, Senior Consulting Engineer, Component Optimization

M. McGhee, Operations Department Leader, Operations Support

M. Mclain, Senior Engineer, System Engineer-Primary

A.R. Meeden, Engineering Section Leader, Design Mechanical NSSS

R. Merryman, Ops Standards Section Leader, Operations Site Procedure Standards

J. Molden, Director Nuclear Engineering

H. Mortazavi, Senior Consulting Engineer, Design Mechanical

A. Nelsen, Shift Manager, Operations Work Control

M. Powlikosky, Senior Engineer, System Engineering-Primary

B. Ramey, Nuclear Engineering Department Leader

M. Renfroe, TMA Management, Plant Manager

F. Riedel, TMA Management, Operations

K. Schrecker, Engineering Section Leader, System Engineering

M. Shea, Director IMPACT

J. Skrtich, Control Room Supervisor, Operations Work Control

R.S. Smith, Control Room Supervisor, Unit 3 Operations

T. Smith, Component Optimization PM Program

D. Sollars, Control Room Supervisor, Unit 2 Operations

M. Sontag, Performance Improvement Team Department Lead, IMPACT

D. Steen, Senior Engineer, System Engineering BOP

D. Steinsiek, Nuclear Engineering Department Leader, Program Engineering

K. Sweeney, Nuclear Engineering Department Leader, System Engineering

A. F. Swirbul, Engineering Section Leader, Digital Upgrades Group

R. Timmons, Training Section Leader, License Requalification

C. B. Thiele, Nuclear Engineering Department Leader, Component Design Basis Review Group

J. Taylor, Operations Department Leader

J. Tolar, Senior Engineer, Design Mechanical NSSS

D. Vogt, STA Section Leader, Shift Technical Advisors

L. Weaver, Engineer III, System Engineering Elec/I&C

T. Weber, Regulatory Affairs Department Leader

D. Wheeler, Performance Improvement Team Department Lead CAP 1A

R. Wilferd, Nuclear Engineering Department Leader, Fire Protection

NRC Personnel

Carl Schulten, NRR

Attachment 1

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000530/2009006-01 NCV Inadequate Operability Evaluation for Potential Emergency Diesel Generator Slow Start Issue Section (4OA2)

05000530/2009006-02 NCV Failure to Follow Procedure for Screening Significant Condition Adverse to Quality (Section 4OA2)

05000530/2009006-03 NCV Failure to Perform 10 CFR 50.59 Screenings on Scaffolds Installed for Greater Than 90 Days (Section 4OA2)

05000528,529,530/2009006-05 NCV Failure to Identify and Correct-Related Degradation of Safety-Related Inverters (Section 4OA2)

05000528,529,530/200906-06 NCV Failure to Properly Implement Corrective Action Process for Potential Operability Issues with the Safety Related Systems and Systems Important to Safety (Section 4OA5)

05000528,529,530/200906-07 NCV Inadequate Procedures for Performing Operability Determinations (Section 4OA5)

050005282009006-08 NCV Failure to Promptly Identify and Correct a Condition Adverse to Quality with the Emergency Core Cooling System Piping (Section 4OA5)

Opened

05000530/2009006-04 NOV Failure to Implement Adequate Design Controls (Section 4OA2)

05000528,529,530/200906-09 URI Safety Injection Pump Full Flow Recirculation Potential Design Control Issue

Closed

05000528,529,530/2004-009-00 LER Emergency Core Cooling System Piping Voids May Have Prevented Fulfillment of Safety Function (Section 4OA3)

05000528,529,530/2004-009-01 LER Emergency Core Cooling System Piping Voids May Have Prevented Fulfillment of Safety Function (Section 4OA3)

Attachment 1

05000530/2007-001-00 LER Condition Prohibited by Technical Specifications Resulting from Containment Spray Nozzle Blockage (Section 4OA3)

05000528/2008-002-00 LER Technical Specification - Limiting Condition for Operation 3.0.3 for Greater than 1 Hour (Section 4OA3)

05000529/2008-003-00 LER Technical Specification - Limiting Condition for Operation 3.0.3 for Greater Than 1 Hour (Section 4OA3)

05000529,529,530/2004014-01 VIO Failure to Maintain Design Control of Containment Sump Recirculation Piping (Section 4OA5)