ML081300387
| ML081300387 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 05/09/2008 |
| From: | Hay M NRC/RGN-IV/DRP/RPB-D |
| To: | Edington R Arizona Public Service Co |
| References | |
| IR-08-002 | |
| Download: ML081300387 (79) | |
See also: IR 05000528/2008002
Text
{{#Wiki_filter:UNITED STATES
NU C LE AR RE G ULATO RY C O M M I S S I O N
R E GI ON I V
612 EAST LAMAR BLVD , SU ITE 400
AR LIN GTON , TEXAS 76011-4125
May 20, 2008
Randall K. Edington, Executive Vice President, Nuclear
and Chief Nuclear Officer
Mail Station 7602 Arizona Public Service Company P.O. Box 52034 Phoenix, AZ 85072-2034 SUBJECT: ERRATA FOR PALO VERDE NUCLEAR GENERATING STATION - NRC
INTEGRATED INSPECTION REPORT 05000528/2008002, 05000529/2008002,
AND 05000530/2008002
Dear Mr. Edington: This errata corrects the decision basis for the significance determination for Noncited Violation 05000528; 05000529; 05000530/2008002-04, "Failure To Maintain Adequate Staffing Levels Results in Heavy Use of Overtime to Maintain Adequate Shift Coverage," described in Section 4OA2 of the subject inspection report. Please replace page 4 of the Summary of Findings and page 30 of NRC Inspection Report 05000528/2008002, 05000529/2008002, and 05000530/2008008, dated May 9, 2008, with the enclosed revised pages. We regret any inconvenience this may have caused. In accordance with 10 CFR 2.390 of the NRCs "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room). Should you have any questions concerning this inspection, we will be pleased to discuss them with you.
Sincerely,
/RA/
Michael C. Hay, Chief
Projects Branch D
Division of Reactor Projects
Arizona Public Service Company -2- Dockets: 50-528
50-529
50-530
Licenses: NPF-41
NPF-51
NPF-74
cc w/Enclosure: Douglas K. Porter, Senior Counsel
Steve Olea Southern California Edison Company
Arizona Corporation Commission Law Department, Generation Resources
1200 W. Washington Street P.O. Box 800
Phoenix, AZ 85007 Rosemead, CA 91770
Chairman Aubrey V. Godwin, Director
Maricopa County Board of Supervisors Arizona Radiation Regulatory Agency
301 W. Jefferson, 10th Floor 4814 South 40 Street
Phoenix, AZ 85003 Phoenix, AZ 85040
Scott Bauer, Director Mr. Dwight C. Mims
Regulatory Affairs Vice President, Regulatory Affairs and
Palo Verde Nuclear Generating Station Performance Improvement
Mail Station 7636 Palo Verde Nuclear Generating Station
P.O. Box 52034 Mail Station 7605
Phoenix, AZ 85072-2034 P.O. Box 52034
Phoenix, AZ 85072-2034
Jeffrey T. Weikert
Eric J. Tharp
Assistant General Counsel
Los Angeles Department of Water & Power
El Paso Electric Company
Southern California Public Power Authority
Mail Location 167
P.O. Box 51111, Room 1255-C
123 W. Mills
Los Angeles, CA 90051-0100
El Paso, TX 79901
James Ray Geoffrey M. Cook
Public Service Company of New Mexico Southern California Edison Company
2401 Aztec NE, MS Z110 5000 Pacific Coast Hwy, Bldg. D21
Albuquerque, NM 87107-4224 San Clemente, CA 92672
Arizona Public Service Company -3-
Brian Almon
Robert Henry Public Utility Commission
Salt River Project William B. Travis Building
6504 East Thomas Road P.O. Box 13326
Scottsdale, AZ 85251 1701 North Congress Avenue
Austin, TX 78701-3326
Karen O' Regan Matthew Benac
Environmental Program Manager Assistant Vice President
City of Phoenix Nuclear & Generation Services
Office of Environmental Programs El Paso Electric Company
200 West Washington Street 340 East Palm Lane, Suite 310
Phoenix, AZ 85003 Phoenix, AZ 85004
Arizona Public Service Company -4- Electronic distribution by RIV: Regional Administrator (Elmo.Collins@nrc.gov) DRP Director (Dwight.Chamberlain@nrc.gov) DRS Director (Roy.Caniano@nrc.gov) DRS Deputy Director (Troy.Pruett@nrc.gov) Senior Resident Inspector (Greg.Warnick@nrc.gov) Branch Chief, DRP/D (Michael.Hay@nrc.gov) Senior Project Engineer, DRP/D (Greg.Werner@nrc.gov) Senior Project Engineer, DRP/D (Geoff Miller@nrc.gov) Team Leader, DRP/TSS (Chuck.Paulk@nrc.gov) RITS Coordinator (Marisa.Herrera@nrc.gov) Only inspection reports to the following: DRS STA (Dale.Powers@nrc.gov) J. Adams, OEDO RIV Coordinator (John.Adams@nrc.gov) P. Lougheed, OEDO RIV Coordinator (Patricia.Lougheed@nrc.gov) ROPreports PV Site Secretary (Patricia.Coleman@nrc.gov) SUNSI Review Completed: MCH ADAMS: / Yes No Initials: MCH__ / Publicly Available Non-Publicly Available Sensitive / Non-Sensitive R:\_REACTORS\_PV\2008\PV2008-02Errata-GGW.doc ML
RIV:RI:DRP/D RI:DRP/D RI:DRP/D RI:DRP/D RI:DRP/D RI:DRP/D JHBashore MPCatts JFMelfi GGWarnick RITreadway GBMiller /RA/ /RA/ /RA/ /RA/ MCHay for /RA/ E-mailed /RA/ 05/20/2008 05/19/2008 05/20/2008 05/20/2008 05/19/2008 05/192008 RI:DRP/E C:DRP/D JEJosey MLHay /RA/ /RA/ 05/20/2008 05/19/2008
OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax
and maintenance personnel failed to incorporate the adequate level of detail into their troubleshooting plans for the Unit 3 auxiliary feedwater trip and throttle Valve AFA-HV-0054 when it failed to fully close upon demand from the control room hand switch, and for the Unit 3 log power Channel A when induced noise was present. These issues were entered into the licensee's corrective action program as Palo Verde Action Requests 3120075 and 3118744. This finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because it did not represent a loss of system safety function, an actual loss of safety function of a single train for greater than its technical specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. Both examples have a crosscutting aspect in the area of human performance associated with decision-making because the licensee did not obtain appropriate interdisciplinary input and reviews on safety-significant or risk-significant decisions [H.1(a)]. (Section 1R19)
- Green. The inspectors identified a noncited violation of Technical
Specification 5.2.2.d involving the routine use of excessive overtime for operations personnel that performed safety-related functions. Specifically, between January 1 and December 31, 2007, operations personnel routinely used excessive overtime. This issue was entered into the licensees corrective action program as Condition Report/Disposition Request 3112231. The finding is greater than minor because if left uncorrected the finding would become a more significant safety concern in that the routine use of excessive work hours increases the likelihood of operator errors. Using the Manual Chapter 0609, "Significance Determination Process," Appendix M, the finding is determined to have very low safety significance because there were no recent instances where findings of low to moderate (White) or greater significance were attributed to the increased use of overtime by operating personnel. The finding has a crosscutting aspect in the area of human performance associated with resources because the licensee failed to maintain sufficient qualified operations personnel to maintain working hours within guidelines without the excessive use of overtime [H.2(b)] (Section 4OA2).
- Green. The inspectors identified a non-cited violation of 10 CFR Part 50,
Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure
of engineering personnel to ensure that potentially nonconforming conditions
associated with the Class 1E 125 Vdc system were reviewed for operability.
Specifically, between September 29, 2007 and March 7, 2008, engineering
personnel failed to ensure all relevant information was reviewed for operability
when it was determined that vendor recommended preventative maintenance
tasks were not being performed on the Class 1E 125 Vdc system. This issue
was entered into the licensee's corrective action program as Palo Verde Action
Request 3144707.
-4- Enclosure
Arizona Public Service Company - 30 -
2003 2004 2005 2006 2007
Control 7.32 8.92 10.53 14.13 20.87
Room
Supervisor
Reactor 10.43 13.84 16.25 20.96 27.65
Operator
Shift 8.81 10.29 12.12 17.51 20.28
Manager
Since 2003, overtime, as a percent of regular hours worked, has increased steadily and
substantively for control room operators. The inspectors noted that the increase in
overtime rates for operations department positions appeared to be largely the result of a
decrease in staffing, rather than the result of an increase in the total number of person-
hours expended. The inspectors also noted that the 2007 overtime rates were more
than double the overtime rates of 2003.
During their review the inspectors noted that Technical Specification 5.2.2.d,
Organization - Unit Staff, requires that administrative procedures shall be developed
and implemented to limit the working hours of unit staff that perform safety-related
functions, as well as requiring that the controls shall include guidelines on working hours
that ensure adequate shift coverage shall be maintained without routine heavy use of
overtime. Station procedure 01DP-9EM01, Overtime Limitations, Revision 6, is the
licensees administrative procedure used to control unit staff working hours in
accordance with facility Technical Specifications. Section 2.1 of this procedure requires
that department leaders ensure that adequate shift coverage is maintained without the
routine heavy use of overtime. The objective is to have personnel work a nominal
40-hour week while the plant is operating.
The inspectors determined that the licensee had several missed opportunities to identify
this issue. Specifically, during their review the inspectors noted that the licensee had not
been issuing and reviewing Technical Specification required excess overtime reports
from approximately June 2006 through July 2007. The purpose of these reports was to
facilitate identification of excess overtime usage by site management. However, due to
changing computer software the reports were not generated and reviewed. Also, the
inspector noted that several CRDRs written that identified the metric window for
operations overtime were red for most of 2007. The inspectors determined that these
were indicators of the use of excessive overtime and these indicators were missed by
the licensee.
Analysis. The performance deficiency associated with this finding involved
excessive routine use of heavy amounts of overtime for operations personnel
that perform safety-related functions. The finding is greater than minor because
if left uncorrected the finding would become a more significant safety concern in
that the routine use of excessive work hours increases the likelihood of operator
errors. Using the Manual Chapter 0609, "Significance Determination Process,"
Appendix M, the finding is determined to have very low safety significance
because there were no recent instances where findings of low to moderate
(White) or greater significance were attributed to the increased use of overtime
by operating personnel. The finding has a crosscutting aspect in the area of
human performance associated with resources because the licensee failed to
maintain sufficient qualified operations personnel to maintain working hours
within guidelines without heavy use of overtime [H.2(b)].
UNITE D S TATES
NUC LEAR RE GULATOR Y C OMMIS SI ON
R EG IO N I V
6 12 EAST LAMAR BLVD , SU ITE 4 00
AR L I N GTON , TEXAS 7 601 1- 4125
May 9, 2008
Randall K. Edington, Executive Vice President, Nuclear
and Chief Nuclear Officer
Mail Station 7602 Arizona Public Service Company P.O. Box 52034 Phoenix, AZ 85072-2034 SUBJECT: PALO VERDE NUCLEAR GENERATING STATION - NRC INTEGRATED
INSPECTION REPORT 05000528/2008002, 05000529/2008002, AND
05000530/2008002
Dear Mr. Edington: On March 31, 2008, the U.S. Nuclear Regulatory Commission (NRC) completed an integrated inspection at your Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility. The enclosed integrated report documents the inspection findings, which were discussed on April 16, 2008, with you and other members of your staff. The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your licenses. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel. This report documents five NRC identified findings and two self-revealing findings. These findings were evaluated under the risk significance determination process as having very low safety significance (Green). Because of the very low safety significance of these violations and because they were entered into your corrective action program, the NRC is treating these findings as non-cited violations consistent with Section VI.A.1 of the NRC Enforcement Policy. Two licensee-identified violations, which were determined to be of very low safety significance, are listed in Section 4OA7 of this report. If you contest these non-cited violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington DC 20555-0001; and the NRC Resident Inspector at the Palo Verde Nuclear Generating Station, Units 1, 2, and 3, facility.
Arizona Public Service Company -2- In accordance with 10 CFR 2.390 of the NRCs "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/
Michael C. Hay, Chief
Projects, Branch D
Division of Reactor Projects
Docket Nos. 50-528
50-529
50-530
License Nos. NPF-41
NPF-51
NPF-74
Enclosure: NRC Inspection Report 05000528/2008002, 05000529/2008002, and 05000530/2008002
w/Attachment: Supplemental Information
cc w/enclosure:Steve Olea Arizona Corporation Commission 1200 W. Washington Street Phoenix, AZ 85007 Douglas K. Porter, Senior Counsel Southern California Edison Company Law Department, Generation Resources P.O. Box 800 Rosemead, CA 91770 Chairman Maricopa County Board of Supervisors 301 W. Jefferson, 10th Floor Phoenix, AZ 85003 Aubrey V. Godwin, Director Arizona Radiation Regulatory Agency 4814 South 40 Street Phoenix, AZ 85040
Arizona Public Service Company -3- Scott Bauer, Director Regulatory Affairs Palo Verde Nuclear Generating Station Mail Station 7636 P.O. Box 52034 Phoenix, AZ 85072-2034 Mr. Dwight C. Mims Vice President, Regulatory Affairs and
Performance Improvement
Palo Verde Nuclear Generating Station Mail Station 7605 P.O. Box 52034 Phoenix, AZ 85072-2034 Jeffrey T. Weikert Assistant General Counsel El Paso Electric Company Mail Location 167 123 W. Mills El Paso, TX 79901 Eric J. Tharp Los Angeles Department of Water & Power Southern California Public Power Authority P.O. Box 51111, Room 1255-C Los Angeles, CA 90051-0100 James Ray Public Service Company of New Mexico 2401 Aztec NE, MS Z110 Albuquerque, NM 87107-4224 Geoffrey M. Cook Southern California Edison Company 5000 Pacific Coast Hwy, Bldg. D21 San Clemente, CA 92672 Robert Henry Salt River Project 6504 East Thomas Road Scottsdale, AZ 85251 Brian Almon Public Utility Commission William B. Travis Building P.O. Box 13326 1701 North Congress Avenue Austin, TX 78701-3326
Arizona Public Service Company -4- Karen O' Regan Environmental Program Manager City of Phoenix Office of Environmental Programs 200 West Washington Street Phoenix, AZ 85003 Matthew Benac Assistant Vice President Nuclear & Generation Services El Paso Electric Company 340 East Palm Lane, Suite 310 Phoenix, AZ 85004 Chief, Radiological Emergency Preparedness Section National Preparedness Directorate Technological Hazards Division Department of Homeland Security 1111 Broadway, Suite 1200 Oakland, CA 94607-4052 Chairperson, Radiological Assistance Committee Region IX Federal Emergency Management Agency Department of Homeland Security 1111 Broadway, Suite 1200 Oakland, CA 94607-4052
Arizona Public Service Company -5- Electronic distribution by RIV: Regional Administrator (Elmo.Collins@nrc.gov) DRP Director (Dwight.Chamberlain@nrc.gov) DRS Director (Roy.Caniano@nrc.gov) DRS Deputy Director (Troy.Pruett@nrc.gov) Senior Resident Inspector (Greg.Warnick@nrc.gov) Branch Chief, DRP/D (Michael.Hay@nrc.gov) Senior Project Engineer, DRP/D (Greg.Werner@nrc.gov) Senior Project Engineer, DRP/D (Geoff.Miller@nrc.gov) Team Leader, DRP/TSS (Chuck.Paulk@nrc.gov) RITS Coordinator (Marisa.Herrera@nrc.gov) Only inspection reports to the following: DRS STA (Dale.Powers@nrc.gov) J. Adams, OEDO RIV Coordinator (John.Adams@nrc.gov) P. Lougheed, OEDO RIV Coordinator (Patricia.Lougheed@nrc.gov) ROPreports PV Site Secretary (Patricia.Coleman@nrc.gov) SUNSI Review Completed:__GEW__ADAMS: ;Yes No Initials: __GEW__ ;Publicly Available Non-Publicly Available Sensitive ;Non-Sensitive R:\_REACTORS\_PV\2008\PV2008-002RP-GGW.doc ML )81300387
RIV:RI:DRP/D RI:DRP/D RI:DRP/D SRI:DRP/D SRI:DRP/D SPE:DRP/D JBashore MCatts JFMelfi GGWarnick RTreadway GEWerner /RA/ MHay for /RA/ MHay for /RA/ E-mailed /RA/ /RA/ MCHay for /RA/ 05/8/2008 05/8/2008 05/9/2008 05/8/2008 05/8/2008 05/5/2008 C:DRS/PSB C:DRS/EB2 C:DRS/EB C:DRS/OB C:DRP/D MPShannon LJSmith RLBywater RELantz MHay /RA/ /RA/ DProulx for /RA/ /RA/ MRunyan for /RA/ 05/2/2008 05/2/2008 05/2/2008 05/2/2008 05/8/2008
OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax
U.S. NUCLEAR REGULATORY COMMISSION
REGION IV
Dockets: 50-528, 50-529, 50-530 Licenses: NPF-41, NPF-51, NPF-74 Report: 05000528/2008002, 05000529/2008002, 05000530/2008002 Licensee: Arizona Public Service Company Facility: Palo Verde Nuclear Generating Station, Units 1, 2, and 3 Location: 5951 S. Wintersburg Road
Tonopah, Arizona
Dates: January 1 through March 31, 2008 Inspectors: J. Bashore, Resident Inspector
M. Catts, Resident Inspector
L. Carson II, Senior Health Physics Inspector
P. Elkmann, Emergency Preparedness Inspector
J. Melfi, Resident Inspector
R. Treadway, Senior Resident Inspector
G. Warnick, Senior Resident Inspector
G. Werner, Senior Project Engineer
Approved By: Michael C. Hay, Chief, Project Branch D
Division of Reactor Projects
-1- Enclosure
CONTENTS
SUMMARY OF FINDINGS ...................................................................................................... - 3 - REPORT DETAILS .................................................................................................................. - 7 - REACTOR SAFETY ................................................................................................................ - 7 -
1R04 Equipment Alignment............................................................................................. - 7 - 1R05 Fire Protection........................................................................................................ - 8 - 1R11 Licensed Operator Requalification Program .......................................................... - 8 - 1R12 Maintenance Effectiveness .................................................................................... - 9 - 1R13 Maintenance Risk Assessments and Emergent Work Control .............................. - 9 - 1R15 Operability Evaluations ........................................................................................ - 11 - 1R18 Plant Modifications............................................................................................... - 13 - 1R19 Post-Maintenance Testing ................................................................................... - 14 - 1R20 Refueling and Other Outage Activities ................................................................. - 17 - 1R22 Surveillance Testing............................................................................................. - 18 - 1EP2 Alert Notification System Testing ......................................................................... - 19 - 1EP3 Emergency Response Organization Augmentation Testing ................................ - 19 - 1EP4 Emergency Action Level and Emergency Plan Changes .................................... - 19 - 1EP5 Correction of Emergency Preparedness Weaknesses and Deficiencies ............. - 20 - 1EP6 Drill Evaluation ..................................................................................................... - 21 -
RADIATION SAFETY ............................................................................................................ - 21 -
2OS1 Access Control to Radiologically Significant Areas.............................................. - 21 - 2OS2 As Low As Is Reasonably Achievable (ALARA) Planning And Controls.............. - 22 -
OTHER ACTIVITIES.............................................................................................................. - 23 -
4OA1 Performance Indicator Verification....................................................................... - 23 - 4OA2 Identification and Resolution of Problems............................................................ - 25 - 4OA3 Followup of Events and Notices of Enforcement Discretion ................................ - 39 - 4OA5 Other Activities..................................................................................................... - 44 - 4OA6 Meetings, Including Exit ....................................................................................... - 44 - 4OA7 Licensee-Identified Violations .............................................................................. - 45 -
SUPPLEMENTAL INFORMATION ...............................................................................................1 KEY POINTS OF CONTACT ........................................................................................................1 LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED ............................................................2 LIST OF DOCUMENTS REVIEWED ............................................................................................3 LIST OF ACRONYMS USED................................................................................................. - 30 -
-2- Enclosure
SUMMARY OF FINDINGS
IR 05000528/2008002, 05000529/2008002, 05000530/2008002; 01/01/08 - 03/31/08; Palo Verde Nuclear Generating Station, Units 1, 2, and 3; Integrated Resident and Regional Report; Maintenance Risk Assessments and Emergent Work Control, Operability Evaluations, Post- Maintenance Testing, Identification and Resolution of Problems, Follow-Up of Events. This report covered a 3-month period of inspection by resident inspectors and regional inspectors. The inspection identified nine findings. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management's review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 4, dated December 2006. A. NRC-Identified and Self-Revealing Findings
Cornerstone: Mitigating Systems
* Green. The inspectors identified a non-cited violation of Technical
Specification 5.4.1.a for the failure of operations and engineering personnel to
establish and implement maintenance procedures for inspection and
replacement of items that have a specific lifetime. Specifically, between
February 12, 2007 and March 7, 2008, operations and engineering personnel
failed to inspect or replace the emergency diesel generators fuel oil injection
pump upper O-rings prior to the end of their service life resulting in fuel leakage
and increased unavailability and unreliability of Unit 1 Train A, Unit 2 Train B, and
Unit 3 Train B emergency diesel generators. This issue was entered into the
licensee's corrective action program as Palo Verde Action Request 3143422.
This finding is greater than minor because it is associated with the equipment
performance attribute of the mitigating systems cornerstone and affects the
cornerstone objective of ensuring the availability and reliability of systems that
respond to initiating events to prevent undesirable consequences. Using the
Manual Chapter 0609, "Significance Determination Process," Phase 1
Worksheets, the finding is determined to have very low safety significance
because it did not represent a loss of system safety function, an actual loss of
safety function of a single train for greater than its technical specification allowed
outage time, or screen as potentially risk-significant due to a seismic, flooding, or
severe weather initiating event. This finding has a crosscutting aspect in the
area of problem identification and resolution associated with operating
experience because the licensee failed to use available operating experience,
including vendor recommendations, to implement and institutionalize operating
experience through changes to station processes, procedures, equipment, and
training programs [P.2(b)]. (Section 1R15)
* Green. The inspectors identified two examples of a non-cited violation of 10 CFR
Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," for
the failure of operations, engineering, and maintenance personnel to follow
procedures for troubleshooting failures of safety-related components.
-3- Enclosure
Specifically, between January 8 and January 13, 2008, operations, engineering, and maintenance personnel failed to incorporate the adequate level of detail into their troubleshooting plans for the Unit 3 auxiliary feedwater trip and throttle Valve AFA-HV-0054 when it failed to fully close upon demand from the control room hand switch, and for the Unit 3 log power Channel A when induced noise was present. These issues were entered into the licensee's corrective action program as Palo Verde Action Requests 3120075 and 3118744. This finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because it did not represent a loss of system safety function, an actual loss of safety function of a single train for greater than its technical specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. Both examples have a crosscutting aspect in the area of human performance associated with decision-making because the licensee did not obtain appropriate interdisciplinary input and reviews on safety-significant or risk-significant decisions [H.1(a)]. (Section 1R19)
- Green. The inspectors identified a non-cited violation of Technical
Specification 5.2.2.d involving the routine use of excessive overtime for operations personnel that performed safety-related functions. Specifically, between January 1 and December 31, 2007, operations personnel routinely used excessive overtime. This issue was entered into the licensees corrective action program as Condition Report/Disposition Request 3112231. The finding is greater than minor because if left uncorrected the finding would become a more significant safety concern in that the routine use of excessive work hours increases the likelihood of operator errors. Using the IMC 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because no specific human performance issues due to personnel fatigue were identified that resulted in the degradation or loss of safety function of equipment important to safety. The finding has a crosscutting aspect in the area of human performance associated with resources because the licensee failed to maintain sufficient qualified operations personnel to maintain working hours within guidelines without the excessive use of overtime [H.2(b)]. (Section 4OA2)
- Green. The inspectors identified a non-cited violation of 10 CFR Part 50,
Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure
of engineering personnel to ensure that potentially nonconforming conditions
associated with the Class 1E 125 Vdc system were reviewed for operability.
Specifically, between September 29, 2007 and March 7, 2008, engineering
personnel failed to ensure all relevant information was reviewed for operability
when it was determined that vendor recommended preventative maintenance
tasks were not being performed on the Class 1E 125 Vdc system. This issue
was entered into the licensee's corrective action program as Palo Verde Action
Request 3144707.
-4- Enclosure
This finding is greater than minor because it is associated with the equipment
performance attribute of the mitigating systems cornerstone and affects the
cornerstone objective to ensure the availability and reliability of systems that
respond to initiating events to prevent undesirable consequences. Using the
Manual Chapter 0609, "Significance Determination Process," Phase 1
Worksheets, the finding is determined to have very low safety significance
because it did not represent a loss of system safety function, an actual loss of
safety function of a single train for greater than its technical specification allowed
outage time, or screen as potentially risk-significant due to a seismic, flooding, or
severe weather initiating event. This finding has a crosscutting aspect in the
area of human performance associated with decision-making because
safety-significant decisions were not verified to validate underlying assumptions
and identify unintended consequences [H.1(b)]. (Section 4OA2)
- Green. A self-revealing non-cited violation of Technical Specification 3.7.3.c was
identified for the failure of operations personnel to perform the actions required
for an inoperable main feedwater isolation valve. Specifically, on July 17, 2006,
operations personnel failed to perform actions to place the unit in Mode 3 within
6 hours and Mode 5 within 36 hours, as required by Technical
Specification 3.7.3.c, for an inoperable main feedwater isolation valve that had
not been closed or isolated in 72 hours, as required by Technical
Specification 3.7.3.a. This resulted in main feedwater isolation
Valve 2JSGAUV0174 to steam Generator A exceeding the Technical
Specification 3.7.3 allowed outage time. This issue was entered into the
licensee's corrective action program as Condition Report/Disposition
Request 2915450.
This finding is greater than minor because it is associated with the equipment
performance attribute of the mitigating systems cornerstone and affects the
cornerstone objective to ensure the availability and reliability of systems that
respond to initiating events to prevent undesirable consequences. A Phase 2
analysis was required because the Manual Chapter 0609, "Significance
Determination Process," Phase 1 Worksheets, determined that there was a loss
of main feedwater isolation of a single train to steam Generator A for greater than
the technical specification allowed outage time. Using the Phase 2 Worksheets
associated with a steam generator tube rupture without steam generator
isolation, the finding is determined to have very low safety significance since all
remaining mitigation capability was available or recoverable. (Section 4OA3)
Cornerstone: Barrier Integrity
- Green. The inspectors identified a non-cited violation of 10 CFR Part 50,
Appendix B, Criterion V, "Instructions, Procedures, and Drawings," for the failure
of fuels services personnel to evaluate leaving foreign material in the Unit 2 spent
fuel pool in accordance with procedures, and failed to ensure those procedures
included appropriate quantitative and qualitative acceptance criteria. Specifically,
between October 13, 2006, and January 31, 2008, fuels services personnel used
Procedure 30DP-9MP03, "System Cleanliness and Foreign Material Exclusion
Controls," Revision 6, which did not specify acceptance criteria for time to
perform a functional assessment of foreign material in the spent fuel pool,
resulting in foreign material being left in the spent fuel pool for greater than one
year without an evaluation on affected safety systems. This issue was entered
-5- Enclosure
into the licensee's corrective action program as Palo Verde Action
Request 3126308.
This finding is greater than minor because it is associated with the structure,
systems, and component performance and human performance attributes of the
barrier integrity cornerstone and affects the cornerstone objective to provide
reasonable assurance that physical design barriers (fuel cladding, reactor coolant
system, and containment) protect the public from radionuclide releases caused
by accidents or events. Using the Manual Chapter 0609, "Significance
Determination Process," Phase 1 Worksheets, the finding is determined to have
very low safety significance because the finding did not result in loss of cooling to
the spent fuel pool; the finding did not result from fuel handling errors that caused
damage to the fuel clad integrity or a dropped assembly; and the finding did not
result in a loss of spent fuel pool inventory greater than ten percent of the spent
fuel pool volume. This finding has a crosscutting aspect in the area of human
performance associated with decision-making because the licensee failed to use
conservative assumptions when evaluating degraded and nonconforming
conditions [H.1.(b)]. (Section 4OA2)
* Green. A self-revealing non-cited violation of Technical Specification 5.4.1.a was
identified for the failure of operations personnel to follow procedures.
Specifically, on January 13, 2008, operations personnel failed to properly
implement Procedure 40OP-9PC06, "Fuel Pool Cleanup and Transfer,"
Revision 41, for operating the pool cooling cleanup system, resulting in pool
cooling cleanup Filter PCN-F01B bypass Valve PCN-V061 being improperly
aligned. This resulted in the inadvertent transfer of 300 gallons of spent fuel pool
water to the refueling water tank. This issue was entered into the licensee's
corrective action program as Condition Report/Disposition Request 3121713.
The finding is greater than minor because it is associated with the configuration
control and human performance attributes of the barrier integrity cornerstone and
affects the cornerstone objective to provide reasonable assurance that physical
design barriers (fuel cladding, reactor coolant system, and containment) protect
the public from radionuclide releases caused by accidents or events. Using the
Manual Chapter 0609, "Significance Determination Process," Phase 1
Worksheets, the finding is determined to have very low safety significance
because the finding did not result in loss of cooling to the spent fuel pool; the
finding did not result from fuel handling errors that caused damage to the fuel
clad integrity or a dropped assembly; and the finding did not result in a loss of
spent fuel pool inventory greater than ten percent of the spent fuel pool volume.
This finding has a crosscutting aspect in the area of human performance
associated with work practices because the licensee failed to use adequate
human error prevention techniques, such as pre-job briefings, to ensure that the
pool cooling cleanup system activity was performed safely [H.4(a)].
(Section 4OA3)
B. Licensee-Identified Violations
Violations of very low safety significance that were identified by the licensee have been
reviewed by the inspectors. Corrective actions taken or planned by the licensee have
been entered into the licensee's corrective action program. These violations and
corrective action tracking numbers are listed in Section 4OA7 of this report.
-6- Enclosure
REPORT DETAILS
Summary of Plant Status Unit 1 operated at essentially full power for the entire inspection period. Unit 2 operated at essentially full power for the entire inspection period. Unit 3 began the inspection period shutdown for refueling Outage 3R13. The unit was restarted on January 15, 2008, returned to full power on January 24, 2008, and remained there for duration of the inspection period. 1. REACTOR SAFETY
Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity
1R04 Equipment Alignment (71111.04)
a. Inspection Scope
Partial Walkdown
The inspectors: (1) walked down portions of the three below listed risk important
systems and reviewed plant procedures and documents to verify that critical portions of
the selected systems were correctly aligned; and (2) compared deficiencies identified
during the walk down to the licensee's Updated Final Safety Analysis Report (UFSAR)
and corrective action program (CAP) to ensure problems were being identified and
corrected.
* January 17, 2008, Unit 3, emergency diesel generator (EDG) Train B
* February 20, 2008, Unit 2, essential chilled water, essential spray pond water,
and high pressure safety injection Train A while Train B was out of service
* March 14, 2008, Unit 1, 13.8 kV and 4.16 kV non-class 1E alternating current
power system Train B
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed three samples.
b. Findings
No findings of significance were identified.
-7- Enclosure
1R05 Fire Protection (71111.05)
a. Inspection Scope
Quarterly Inspection
The inspectors walked down the four below listed plant areas to assess the material
condition of active and passive fire protection features and their operational lineup and
readiness. The inspectors: (1) verified that transient combustibles and hot work
activities were controlled in accordance with plant procedures; (2) observed the
condition of fire detection devices to verify they remained functional; (3) observed fire
suppression systems to verify they remained functional and that access to manual
actuators was unobstructed; (4) verified that fire extinguishers and hose stations were
provided at their designated locations and that they were in a satisfactory condition;
(5) verified that passive fire protection features (electrical raceway barriers, fire doors,
fire dampers, steel fire proofing, penetration seals, and oil collection systems) were in a
satisfactory material condition; (6) verified that adequate compensatory measures were
established for degraded or inoperable fire protection features and that the
compensatory measures were commensurate with the significance of the deficiency; and
(7) verified the licensee identified and corrected fire protection problems.
* January 29, 2008, Unit 1, condensate storage pump house and tunnel
* January 29, 2008, Unit 1, spray pond pump house
* February 11, 2008, Unit 3, diesel generator building, 100 foot, 115 foot, and
131 foot elevations
* February 25, 2008, Unit 2, condensate storage pump house and tunnel
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed four samples.
b. Findings
No findings of significance were identified.
1R11 Licensed Operator Requalification Program (71111.11)
a. Inspection Scope
Quarterly Inspection
On February 26, 2008, the inspectors observed testing and training of senior reactor
operators (SROs) and reactor operators (ROs) to identify deficiencies and discrepancies
in the training, to assess operator performance, and to assess the evaluator's critique.
The training Scenario SES-0-07-E--02, "Loss of PKC-M43/Loss of Offsite Power,"
involved four events including: (1) failure of condensate storage tank level instrument;
(2) failure of a steam flow transmitter; (3) loss of Class 1E 125 volts direct current
Bus (PK) C; and (4) loss of offsite power.
-8- Enclosure
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed one sample.
b. Findings
No findings of significance were identified.
1R12 Maintenance Effectiveness (71111.12)
a. Inspection Scope
The inspectors reviewed the two below listed maintenance activities to: (1) verify the
appropriate handling of structure, system, and component (SSC) performance or
condition problems; (2) verify the appropriate handling of degraded SSCs functional
performance; (3) evaluate the role of work practices and common cause problems; and
(4) evaluate the handling of SSC issues reviewed under the requirements of the
Maintenance Rule, 10 CFR Part 50, Appendix B, and the Technical Specifications (TSs).
* January 25, 2008, Units 1, 2, and 3, EDG fuel oil injection pump leakage that
impacted EDG operability as described in Condition Report/Disposition Request
(CRDR) 2950136 and Palo Verde Action Requests (PVARs) 3092611, 3125050,
and 3125979
* February 5, 2008, Unit 3, failure of control element Assembly 26 causing cross
channel comparison failures and control element assembly Calculator 1
deviations
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed two samples.
b. Findings
No findings of significance were identified.
1R13 Maintenance Risk Assessments and Emergent Work Control (71111.13)
a. Inspection Scope
Risk Assessment and Management of Risk
The inspectors reviewed the two below listed assessment activities to verify:
(1) performance of risk assessments when required by 10 CFR 50.65 (a)(4) and licensee
procedures prior to changes in plant configuration for maintenance activities and plant
operations; (2) the accuracy, adequacy, and completeness of the information considered
in the risk assessment; (3) that the licensee recognizes, and/or enters as applicable, the
appropriate licensee-established risk category according to the risk assessment results
and licensee procedures; and (4) the licensee identified and corrected problems related
to maintenance risk assessments.
-9- Enclosure
* January 9, 2008, Unit 2, risk assessment and management during scheduled
implementation of design modification to lower average reactor coolant system
(RCS) temperature by one degree Fahrenheit
* February 17 through March 3, 2008, Units 1, 2 and 3, risk assessment and
management during re-performance of remote shutdown disconnect switch
surveillance tests
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed two samples.
b. Findings
No findings of significance were identified.
a. Inspection Scope
Emergent Work Control
The inspectors: (1) verified that the licensee performed actions to minimize the
probability of initiating events and maintained the functional capability of mitigating
systems and barrier integrity systems; (2) verified that emergent work-related activities
such as troubleshooting, work planning/scheduling, establishing plant conditions,
aligning equipment, tagging, temporary modifications, and equipment restoration did not
place the plant in an unacceptable configuration; and (3) verified the licensee identified
and corrected risk assessment and emergent work control problems. The following
three activities were reviewed:
* January 8, 2008, Unit 3, troubleshooting and repair of nuclear instrument log
Channel A induced noise while EDG Train A was in service
* January 10-17, 2008, Unit 3, auxiliary feedwater (AFW) Train A, trip and throttle
Valve AFA-HV-54, troubleshooting and repair
* March 3, 2008, Unit 1, AFW actuating system for steam Generator (SG) A,
Train B, troubleshooting and repair
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed three samples.
b. Findings
No findings of significance were identified.
- 10 - Enclosure
1R15 Operability Evaluations (71111.15)
a. Inspection Scope
The inspectors: (1) reviewed plant status documents such as operator shift logs,
emergent work documentation, deferred modifications, and night orders to determine if
an operability evaluation was warranted for degraded components; (2) referred to the
UFSAR and design basis documents to review the technical adequacy of licensee
operability evaluations; (3) evaluated compensatory measures associated with
operability evaluations; (4) determined degraded component impact on any TSs; (5)
used the Significance Determination Process, to evaluate the risk significance of
degraded or inoperable equipment; and (6) verified that the licensee has identified and
implemented appropriate corrective actions associated with degraded components. The
following six activities were reviewed:
* January 1-15, 2008, Unit 3, evaluation of dissolved desiccant in the reactor
coolant system (RCS) during heatup
* January 10, 2008, Unit 3, AFW pump Train A operability following
troubleshooting efforts on AFW trip and throttle Valve AFA-HV-0054
* January 10, 2008, evaluation of Unit 3, shutdown cooling heat exchanger Train B
performance degradation
* January 25, 2008, Units 1, 2, and 3, operability assessment associated with
leakage from the EDGs fuel oil injection pumps
* February 17-March 3, 2008, Units 1, 2 and 3, operability evaluation of remote
shutdown disconnect switches
* March 24 - 26, 2008, Units 1 and 2, EDG Train A operability for non optimal field
configuration of overspeed trip air line pressure switch isolation valves
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed six samples.
b. Findings
Introduction. The inspectors identified a Green non-cited violation (NCV) of TS 5.4.1.a
for the failure of operations and engineering personnel to adequately establish and
implement maintenance procedures for inspection and replacement of items that have a
specific lifetime.
Description. Fuel oil leakage from the EDG fuel oil injection pump upper O-rings was
first identified on December 12, 2006, when Unit 1 Train A EDG fuel oil injection
Pump 5L developed a leak from the upper O-ring and was declared inoperable as
documented in CRDR 2950136. On February 12, 2007, the fuel oil injection pump
vendor, Haynes, completed a report and determined the Unit 1 Train A EDG fuel oil
injection pump leakage was due to the pumps O-ring, made from Buna-N material,
having approached the end of its useful life, as documented in 8000865-FA, "Failure
Analysis of Fuel Injection Pump." Also, the licensee performed an apparent cause
- 11 - Enclosure
evaluation on February 20, 2007, as documented in CRDR 2950136 and determined the leakage was due to material aging of the Buna-N rubber. It was determined the shelf life of the Buna-N O-rings was 13 to 15 years and Unit 1 Trains A and B EDGs had pumps with O-rings that were approximately 15 years old. The evaluation also determined the contributing cause was that no preventative maintenance (PM) task existed to inspect and replace the O-rings even though the O-rings have a finite life time. On February 28, 2007, the licensee wrote condition report action item (CRAI) 2976063 to evaluate the Haynes report for eight pumps sent off-site for rework, including the degraded Unit 1 Train A EDG fuel oil injection Pump 5L. The due date for CRAI 2976063 was extended from December 21, 2007, to March 31, 2008, without putting a PM plan or schedule in place for the aging O-rings. Three more leaks occurred on fuel oil injection pumps upper O-rings. On November 13, 2007, Unit 1 Train A EDG fuel oil injection Pump 9L developed a leak of approximately 300 drops per minute (dpm) and was declared inoperable as documented in PVAR 3092611. After this leak, CRAI 3095506 was written on November 16, 2007, to implement replacement of the Buna-N O-rings for all EDGs onsite. However, the inspectors noted that the licensee did not initiate this action and determined the strategy was to replace the fuel oil injection pumps as they leaked. On January 23, 2008, approximately a 61 dpm leak was identified on Unit 2 Train B EDG fuel oil injection Pump 3R and the EDG was declared inoperable. On January 25, 2008, Unit 3 Train B EDG fuel oil injection Pump 5L developed a leak of approximately 200 dpm and the EDG was declared inoperable. The inspectors questioned the licensee on the operability of all EDGs onsite due to the increased fuel oil injection pump leakage and the age of the Buna-N O-rings. On January 25, 2008, the licensee performed an immediate operability determination on the reliability of all EDGs to perform their seven day mission time with the O-ring leakage, as documented in PVAR 3126297. Operations personnel reiterated that the service life of the fuel oil injection pump upper O-rings is 13 to 15 years and that the O-rings on all three units fuel oil injection pumps were reaching, or had reached, the end of their service life resulting in leakage from the O-rings. On January 28, 2008, the licensee completed a more in-depth evaluation in a prompt operability determination, documented in PVAR 3125979. The licensee determined a reasonable expectation of operability of the EDGs based on; 1) the leakage was low pressure; 2) multiple O-ring failures were unlikely to occur on a single EDG during any single EDG start and run; 3) complete failures of the O-rings would not occur; and, 4) the leak rate would not increase over time as determined in Haynes Vendor Report 8001090-Test, dated February 13, 2008. Following the review of the multiple fuel oil injection pump leakage issues, the inspectors noted that a maintenance procedure and had not been implemented for the inspection and replacement of the fuel oil injection pump upper O-rings that had exceeded their service life. On March 7, 2008, the inspectors shared their observations with the licensee who subsequently wrote PVAR 3143422 to develop and implement this PM procedure and schedule. Analysis. The performance deficiency associated with this finding involved the failure of operations and engineering personnel to adequately establish and implement maintenance procedures for inspection and replacement of items that have a specific lifetime; specifically, the EDG fuel oil injection pump upper O-rings. This finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective of ensuring the
- 12 - Enclosure
availability and reliability of systems that respond to initiating events to prevent
undesirable consequences. Using the Manual Chapter 0609, "Significance
Determination Process," Phase 1 Worksheets, the finding is determined to have very low
safety significance because it did not represent a loss of system safety function, an
actual loss of safety function of a single train for greater than its TS allowed outage time,
or screen as potentially risk-significant due to a seismic, flooding, or severe weather
initiating event. This finding has a crosscutting aspect in the area of problem
identification and resolution associated with operating experience because the licensee
failed to use available operating experience, including vendor recommendations, to
implement and institutionalize operating experience through changes to station
processes, procedures, equipment, and training programs [P.2(b)].
Enforcement. Technical Specification 5.4.1.a requires that written procedures be
established, implemented, and maintained covering the activities specified in Regulatory
Guide 1.33, Revision 2, Appendix A, February 1978. Regulatory Guide 1.33
Appendix A, Section 9, "Procedures for Performing Maintenance," Sub-Section "b",
requires that preventative maintenance procedures and schedules be developed to
include inspections of equipment and replacement of items that have a specific lifetime.
Contrary to this, between February 12, 2007 and March 7, 2008, operations and
engineering personnel failed to establish and implement a PM procedure and schedule
to inspect and replace the EDG fuel oil injection pump upper O-rings resulting in fuel
leakage and increased unavailability and unreliability of Unit 1 Train A EDG, Unit 2
Train B EDG, and Unit 3 Train B EDG. Because this finding is of very low safety
significance and has been entered into the licensee's CAP as PVAR 3143422, this
violation is being treated as an NCV consistent with Section VI.A.1 of the NRC
Enforcement Policy: NCV 05000528; 05000529; 05000530/2008002-01, "Failure to
Establish Preventative Maintenance Procedures for Emergency Diesel Generator Fuel
Oil Injection Pump O-rings."
1R18 Plant Modifications (71111.18)
a. Inspection Scope
Temporary Modifications
On March 24, 2008, the inspectors reviewed a temporary modification for Unit 2 to install
an accelerometer on Train A shutdown cooling suction Valve 2JSIAUV0651 from reactor
coolant Loop 1A. The inspectors reviewed the UFSAR, plant drawings, procedure
requirements, operator logs, and TSs to ensure that the temporary modification was
properly implemented. The inspectors verified that: (1) the modification did not have an
effect on system operability/availability; (2) the installation was consistent with
modification documents; (3) the post-installation test results were satisfactory and that
the impact of the temporary modification on permanently installed SSCs were supported
by the test; (4) the modification was identified on control room drawings and that
appropriate identification tags were placed on the affected drawings; (5) the licensee
evaluated the combined effects of temporary modifications; and (6) there were no
temporary modifications installed that have not been evaluated. The inspectors verified
that the licensee identified and implemented any needed corrective actions associated
with temporary modifications.
- 13 - Enclosure
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed one sample.
b. Findings
No findings of significance were identified.
1R19 Post-Maintenance Testing (71111.19)
a. Inspection Scope
The inspectors selected the five below listed post-maintenance test activities of
risk-significant systems or components. For each item, the inspectors: (1) reviewed the
applicable licensing basis and/or design-basis documents to determine the safety
functions; (2) evaluated the safety functions that may have been affected by the
maintenance activity; and (3) reviewed the test procedure to ensure it adequately tested
the safety function that may have been affected. The inspectors either witnessed or
reviewed test data to verify that acceptance criteria were met, plant impacts were
evaluated, test equipment was calibrated, procedures were followed, jumpers were
properly controlled, the test data results were complete and accurate, the test equipment
was removed, the system was properly re-aligned, and deficiencies during testing were
documented. The inspectors also verified the licensee identified and corrected problems
related to post-maintenance testing.
* January 10-17, 2008, Unit 3, AFW Train A, trip and throttle Valve AFA-HV-0054
following troubleshooting and repair when the valve failed to close upon demand
from the control room hand switch
* February 7, 2008, Unit 3, EDG Train A, following replacement of six fuel oil
injection pumps
* February 28, 2008, Unit 2, low pressure safety injection Train B, following
planned maintenance activities to lubricate, clean, and inspect motor operated
valves and change oil in upper and lower motor bearings
* March 21, 2008, Unit 3, EDG Train B, following troubleshooting and repair of a
packing leak on Valve 3PDGBV652
* March 21, 2008, Unit 3, EDG Train B, following replacement of the fuel oil
injection Pump 7L
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed five samples.
- 14 - Enclosure
b. Findings
Introduction. The inspectors identified two examples of a Green NCV of 10 CFR
Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," for the failure
of operations, engineering, and maintenance personnel to follow procedures for
troubleshooting degraded safety-related components.
Description. The first example occurred on January 9, 2008, when the Unit 3 AFW
system, Train A, was started to support retest activities. Following the run, operations
personnel attempted to close trip and throttle Valve AFA-HV-0054 via the hand switch
from the control room. Valve AFA-HV-0054 is the trip and throttle valve that provides
overspeed protection for steam driven AFW Pump AFA-P01. The valve stroked partially
closed and then stopped in mid position. Operations personnel reopened the valve and
contacted outage maintenance and engineering personnel. At the time of the failure, the
AFW system Train A was already considered inoperable during the retest activities on
Valve AFA-HV-0054. Unit 3 was in Mode 3 at normal operating temperature and
pressure.
The valve failure was entered into the CAP on January 9, 2008, as PVAR 3118968. On
January 9, 2008, a Level C troubleshooting plan was developed by valve services
engineering. Work Order (WO) 3118969 was generated to implement the
troubleshooting plan. The WO and troubleshooting plan were reviewed by the on duty
shift manager (SM) and troubleshooting activities were authorized to begin. The
inspectors noted the troubleshooting plan narrowly focused on the torque switch
contacts in the Limitorque valve actuator, while other potential failure mechanisms were
not addressed. On January 9, 2008, maintenance personnel determined that the
resistance across the torque switch contacts was satisfactory, although a small fiber
near the contact device was found. When the fiber was found, troubleshooting was
stopped with the troubleshooting plan only partially completed. Operations and
maintenance personnel were satisfied that the cause of the valves failure to close had
been adequately addressed. On January 10, 2008, following the completion of other
maintenance and required post maintenance testing on the AFW system Train A, the
system was declared operable.
On January 10, 2008, inspectors questioned the licensee's decision-making process and
the adequacy of a Level C troubleshooting plan. Procedure 01DP-9ZZ01, "Systematic
Troubleshooting," Revision 0, requires that a Level A troubleshooting plan be used for
equipment classified as Key-Safety and having an impact on a safety function.
Valve AFA-V-0054 is classified Key-Safety and its failure could adversely impact the
reliability and availability of the steam driven AFW system. Engineering and
maintenance personnel informally determined that a Level C troubleshooting plan would
be sufficient and a valve services engineer subsequently developed the plan. The SM
did not question the level or the rigor of the proposed troubleshooting plan. As a result,
all potential failure mechanisms were not adequately addressed or evaluated. In
addition, the Level C troubleshooting plan that was implemented was not performed in
the field as written. Additionally, Procedure 70DP-0EE01, "Equipment Root Cause of
Failure Analysis," Revision 17, provides guidance for quarantine and control of
equipment failures to preserve physical evidence in order to aid the troubleshooting and
diagnostic efforts. Upon its failure, before initiation of the troubleshooting plan,
Valve AFA-HV-0054 was reopened thereby losing any contact or relay status
information.
- 15 - Enclosure
The issue of not establishing a Level A troubleshooting plan was entered into the CAP as PVAR 3120075 and subsequently addressed in Adverse CRDR 3120574. On January 11, 2008, the troubleshooting plan was revised to eliminate other potential failure mechanisms to ensure increased reliability of Valve AFA-HV-0054. The second example occurred on January 8, 2008, when operations personnel observed that the Unit 3 meter indication for log power Channel A increased approximately two decades while EDG Train A was in operation for a surveillance test. Log power Channel A is one of four log power channels. Two of four log power channels exceeding a trip setpoint would generate a reactor trip. The licensee entered this issue into the CAP as PVAR 3118744 and as adverse CRDR 3119111 on January 9, 2008. The inspection associated with this example is documented in Section 1R13 of this inspection report. On January 10, 2008, the inspectors requested a copy of the troubleshooting plan. The inspectors observed that a formal troubleshooting plan did not exist, and only a Level C one page hand written troubleshooting plan was available for review. The inspectors challenged the licensee regarding the adequacy of the troubleshooting plan for an SSC designated as Key-Safety. Inspectors noted that Procedure 01DP-9ZZ01, "Systematic Troubleshooting," Revision 0, provided guidance that a Level A troubleshooting plan be used for equipment classified as Key-Safety and having an impact on a safety function. Since Unit 3 was shutdown for refueling Outage 3R13, the safety function was not impacted, and a Level B, not a Level C troubleshooting plan, was appropriate. The inspectors noted that engineering and maintenance personnel did not recognize that the equipment reliability classification for log power Channel A was designated as Key-Safety. On January 11, 2008, the licensee developed a formal Level B troubleshooting plan, and implemented corrective maintenance WO 3118787. Troubleshooting determined that this indication deviation would occur whenever the exciter to EDG Train A was producing voltage, whether the generator was running with or without load. Maintenance personnel observed that the log power meter indication returned to normal when a cable within core protection calculator Channel A cabinet was disconnected. Maintenance personnel also confirmed that the noise was coming from the EDG exciter via core protection calculator Channel A, and not from the detector and preamplifier. The indication deviation was determined to be a result of typical noise produced by a generator exciter. The licensee eliminated the noise by installing an instrumentation filter via WO 3120932 on January 13, 2008. The inspectors noted both examples for this issue involved self-imposed schedule pressures during periods of high work activity, which are related to previously identified findings by the NRC and documented as NCV 05000528; 05000529; 05000530/2006003-07 and NCV 05000528; 05000529; 05000530/2006005-09. Analysis. The performance deficiency associated with this finding was the failure of operations, engineering, and maintenance personnel to follow procedures for troubleshooting degraded safety-related components. The finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low
- 16 - Enclosure
safety significance because it did not represent a loss of system safety function, an
actual loss of safety function of a single train for greater than its TS allowed outage time,
or screen as potentially risk-significant due to a seismic, flooding, or severe weather
initiating event. Both examples have a crosscutting aspect in the area of human
performance associated with decision-making because the licensee did not obtain
appropriate interdisciplinary input and reviews on safety-significant or risk-significant
decisions [H.1(a)].
Enforcement. 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and
Drawings," requires that activities affecting quality shall be prescribed by instructions,
procedures, or drawings, and shall be accomplished in accordance with those
instructions, procedures, and drawings. The troubleshooting process of safety-related
equipment needed to mitigate accidents was an activity affecting quality and was
implemented by Procedure 01DP-9ZZ01, "Systematic Troubleshooting," Revision 0.
Procedure 01DP-9ZZ01, Step 3.2.1, requires an engineering troubleshooting plan,
Level A or Level B, when a degraded SSC is classified as Key-Safety. Contrary to the
above, between January 8 and 13, 2008, operations, engineering, and maintenance
personnel failed to enter the appropriate level of troubleshooting plan upon discovery of
degraded conditions that affected SSCs. Specifically, operations, engineering, and
maintenance personnel failed to adequately incorporate the level and detail into their
troubleshooting plans on the Unit 3 AFW trip and throttle Valve AFA-HV-0054 when it
failed to fully close upon demand from the control room hand switch, and on the Unit 3
log power Channel A when induced noise was present on the channel. Because this
finding is of very low safety significance and has been entered into the CAP as
PVARs 3120075 and 3118744, and CRDRs 3120574 and 3119111, respectively, this
violation is being treated as an NCV, consistent with Section VI.A.1 of the NRC
Enforcement Policy: NCV 05000530/2008002-02, "Two Examples of a Failure to
Properly Implement the Systematic Troubleshooting Process."
1R20 Refueling and Other Outage Activities (71111.20)
a. Inspection Scope
Unit 3 Refueling Outage 3R13
The inspectors reviewed the following risk-significant refueling items or outage activities
to verify defense in depth commensurate with the outage risk control plan, compliance
with the TSs, and adherence to commitments in response to Generic Letter 88-17, "Loss
of Decay Heat Removal:" (1) the risk control plan; (2) tagging/clearance activities;
(3) reactor coolant system instrumentation; (4) electrical power; (5) decay heat removal;
(6) spent fuel pool (SFP) cooling; (7) inventory control; (8) reactivity control;
(9) containment closure; (10) reduced inventory or mid-loop conditions; (11) refueling
activities; (12) heatup and cooldown activities; (13) restart activities; and (14) licensee
identification and implementation of appropriate corrective actions associated with
refueling and outage activities. The inspectors' containment inspections included
observations of the containment sump for damage and debris; and supports, braces,
and snubbers for evidence of excessive stress, water hammer, or aging.
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed one sample.
- 17 - Enclosure
b. Findings
No findings of significance were identified.
1R22 Surveillance Testing (71111.22)
a. Inspection Scope
The inspectors reviewed the UFSAR, procedure requirements, and TSs to ensure that
the four below listed surveillance activities demonstrated that the SSCs tested were
capable of performing their intended safety functions. The inspectors either witnessed
or reviewed test data to verify that the following significant surveillance test attributes
were adequate: (1) preconditioning; (2) evaluation of testing impact on the plant;
(3) acceptance criteria; (4) test equipment; (5) procedures; (6) jumper/lifted lead
controls; (7) test data; (8) testing frequency and method to demonstrate TS operability;
(9) test equipment removal; (10) restoration of plant systems; (11) fulfillment of American
Society of Mechanical Engineers Code requirements; (12) updating of performance
indicator (PI) data; (13) engineering evaluations, root causes, and bases for returning
tested SSCs not meeting the test acceptance criteria were correct; (14) reference setting
data; and (15) annunciators and alarms setpoints. The inspectors also verified that the
licensee identified and implemented any needed corrective actions associated with the
surveillance testing.
* January 3-17, 2008, Unit 3, Procedure 73ST-9AF04, "AFA-P01 Full
Flow-Inservice Test," Revision 2
* February 7, 2008, Unit 3, Procedure 40ST-9DG01, "Diesel Generator A Test,"
Revision 32
* February 17, 2008, Unit 3, Procedure 40ST-9ZZ25, "Online Remote Shutdown
Disconnect Switch Operability," Revision 1
* February 28, 2008, Unit 2, Procedure 73ST-9ZZ18, "Main Steam and Pressurizer
Safety Valve Set Pressure Verification," Revision 20
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed four samples.
b. Findings
No findings of significance were identified.
Cornerstone: Emergency Preparedness
- 18 - Enclosure
1EP2 Alert Notification System Testing (71114.02)
a. Inspection Scope
The inspector discussed with licensee staff the status of offsite siren and tone alert radio
systems to determine the adequacy of licensee methods for testing the alert and
notification system in accordance with 10 CFR Part 50 Appendix E. The licensee's alert
and notification system testing program was compared with criteria in NUREG-0654,
"Criteria for Preparation and Evaluation of Radiological Emergency Response Plans and
Preparedness in Support of Nuclear Power Plants," Revision 1, Federal Emergency
Management Agency Report REP-10, "Guide for the Evaluation of Alert and Notification
Systems for Nuclear Power Plants," and the licensee=s current Federal Emergency
Management Agency approved alert and notification system design report.
Documents reviewed by the inspector are listed in the attachment.
The inspector completed one sample.
b. Findings
No findings of significance were identified.
1EP3 Emergency Response Organization Augmentation Testing (71114.03)
a. Inspection Scope
The inspector discussed with licensee staff the status of primary and backup systems for
augmenting the on-shift emergency response staff to determine the adequacy of
licensee methods for staffing emergency response facilities. The inspector reviewed
licensee Procedure EPIP-61, "Emergency Planning Equipment Testing," Revision 5, and
the references listed in the attachment to this report related to the emergency response
organization augmentation system, to evaluate the licensee=s ability to staff the
emergency response facilities in accordance with the licensee emergency plan and the
requirements of 10 CFR Part 50 Appendix E.
Documents reviewed by the inspector are listed in the attachment.
The inspector completed one sample.
b. Findings
No findings of significance were identified.
1EP4 Emergency Action Level and Emergency Plan Changes (71114.04)
a. Inspection Scope
The inspector performed an in-office review of:
* Palo Verde Nuclear Generating Station Emergency Plan, Revision 36, submitted
May 21, 2007
- 19 - Enclosure
* Palo Verde Nuclear Generating Station Emergency Plan, Revision 37, submitted
May 21, 2007
* EPIP-99, "Emergency Plan Implementing Procedures Standard Appendices,"
Revision 16, Appendix P, "Emergency Action Level (EAL) Bases," submitted
November 9, 2007
* Palo Verde Nuclear Generating Station Emergency Plan, Revision 38, submitted
February 11, 2008
These revisions added descriptions to the technical basis for security-related EALs 7-5,
7-6, and 7-7; updated descriptions of the duties of the shift technical advisor and
systems engineering; updated emergency planning zone maps; added public alert and
notification system sirens; updated the locations of offsite reception and care centers;
changed the licensee's computer dose projection system from Mesorem Jr. to Raddose;
updated the locations of telecommunications equipment; added a description of the
transfer of dose projection duties from the control room to other emergency response
facilities; removed the requirement that changes to EALs be approved by offsite officials
in accordance with 10 CFR Part 50, Appendix E; updated emergency planning zone
demographic information; added detail concerning the performance requirements for
licensee dose assessment software; and made minor administrative corrections.
These revisions were compared to their previous revisions, to the criteria of
NUREG-0654, "Criteria for Preparation and Evaluation of Radiological Emergency
Response Plans and Preparedness in Support of Nuclear Power Plants," Revision 1, to
the criteria of Nuclear Energy Institute (NEI) Report 99-01, "Methodology for
Development of Emergency Action Levels," Revisions 2 and 4, and to the standards in
10 CFR 50.47(b) to determine if the revisions adequately implemented the requirements
of 10 CFR 50.54(q). These reviews were not documented in a safety evaluation report
and did not constitute approval of licensee changes; therefore, these revisions are
subject to future inspection.
Documents reviewed by the inspector are listed in the attachment.
The inspectors completed four samples.
b. Findings
No findings of significance were identified.
1EP5 Correction of Emergency Preparedness Weaknesses and Deficiencies (71114.05)
a. Inspection Scope
The inspector reviewed the licensees corrective action program requirements in
Procedures 01DP-0AP12, "Palo Verde Action Request Processing," Revision 4,
and 90DP-0IP10, "Condition Reporting," Revision 36, and other documents listed in the
attachment to this report. The inspector reviewed summaries of 363 CRDRs assigned to
the emergency preparedness department between February 2006 and January 2008,
and selected 20 for detailed review against the program requirements. The inspector
evaluated the response to the corrective action program requests to determine the
licensees ability to identify, evaluate, and correct problems in accordance with the
- 20 - Enclosure
licensee program requirements and 10 CFR 50.47(b)(14) and 10 CFR Part 50,
Appendix E.
Documents reviewed by the inspector are listed in the attachment.
The inspector completed one sample.
b. Findings
No findings of significance were identified.
1EP6 Drill Evaluation (71114.06)
a. Inspection Scope
On March 5, 2008, for the Emergency Response Organization exercise scenario Guide
08-E-AEV-03002 simulator-based training evolution, contributing to Drill/Exercise
Performance and Emergency Response Organization PIs, the inspectors: (1) observed
the training evolution to identify any weaknesses and deficiencies in classification,
notification, and Protective Action Requirements development activities; (2) compared
the identified weaknesses and deficiencies against licensee identified findings to
determine whether the licensee is properly identifying failures; and (3) determined
whether licensee performance is in accordance with the guidance of the NEI 99-02,
"Voluntary Submission of Performance Indicator Data," acceptance criteria.
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed one sample.
b. Findings
No findings of significance were identified.
2. RADIATION SAFETY
Cornerstone: Occupational Radiation Safety
2OS1 Access Control to Radiologically Significant Areas (71121.01)
a. Inspection Scope
This area was inspected to assess the licensee's performance in implementing physical
and administrative controls for airborne radioactivity areas, radiation areas, high
radiation areas, and worker adherence to these controls. The inspector used the
requirements in 10 CFR Part 20, the TSs, and the licensee's procedures required by TSs
as criteria for determining compliance. During the inspection, the inspector interviewed
the radiation protection manager, radiation protection supervisors, and radiation workers.
The inspector performed independent radiation dose rate measurements and reviewed
the following items:
* Performance indicator events and associated documentation packages reported
by the licensee in the Occupational Radiation Safety Cornerstone (two samples)
- 21 - Enclosure
* Controls (surveys, posting, and barricades) of three radiation, high radiation, or
airborne radioactivity areas
* Radiation exposure permits, procedures, engineering controls, and air sampler
locations
* Self-assessments, audits, licensee event reports (LERs), and special reports
related to the access control program since the last inspection
* Licensee actions in cases of repetitive deficiencies or significant individual
deficiencies
* Posting and locking of entrances to all accessible high dose rate-high radiation
areas and very high radiation areas
Documents reviewed by the inspector are listed in the attachment.
The inspector completed seven samples.
b. Findings
No findings of significance were identified.
2OS2 As Low As Is Reasonably Achievable (ALARA) Planning And Controls (71121.02)
a. Inspection Scope
The inspector assessed licensee performance with respect to maintaining individual and
collective radiation exposures ALARA. The inspector used the requirements in 10 CFR
Part 20 and the licensee's procedures required by TSs as criteria for determining
compliance. The inspector interviewed select licensee personnel and reviewed:
* Five outage work activities scheduled during the inspection period and
associated work activity exposure estimates which were likely to result in the
highest personnel collective exposures
* Site-specific ALARA procedures
* ALARA work activity evaluations, exposure estimates, and exposure mitigation
requirements
* Intended versus actual work activity doses and the reasons for any
inconsistencies
* Integration of ALARA requirements into work procedure and radiation work
permit (or radiation exposure permit) documents
* Person-hour estimates provided by maintenance planning and other groups to
the radiation protection group with the actual work activity time requirements
- 22 - Enclosure
* Use of engineering controls to achieve dose reductions and dose reduction
benefits afforded by shielding
* Workers use of the low dose waiting areas
* Records detailing the historical trends and current status of tracked plant source
terms and contingency plans for expected changes in the source term due to
changes in plant fuel performance issues or changes in plant primary chemistry
* Source-term control strategy or justifications for not pursuing such exposure
reduction initiatives
* Specific sources identified by the licensee for exposure reduction actions and
priorities established for these actions, and results achieved against since the
last refueling cycle
* Declared pregnant workers during the current assessment period, monitoring
controls, and the exposure results
* Resolution through the CAP of problems identified through post-job reviews and
post-outage ALARA report critiques
* Corrective action documents related to the ALARA program and follow-up
activities, such as initial problem identification, characterization, and tracking
Documents reviewed by the inspector are listed in the attachment.
The inspector completed 14 samples.
b. Findings
No findings of significance were identified.
4. OTHER ACTIVITIES 4OA1 Performance Indicator (PI) Verification (71151)
a. Inspection Scope
Cornerstone: Initiating Events
The inspectors sampled licensee submittals for the three PIs listed below for the period
January 2007 to December 2007, for Units 1, 2, and 3. The definitions and guidance of
NEI 99-02, "Regulatory Assessment Indicator Guideline," Revision 5, were used to verify
the licensees basis for reporting each data element in order to verify the accuracy of PI
data reported during the assessment period. The inspectors reviewed LERs, monthly
operating reports, and operating logs as part of the assessment. Licensee PI data was
also reviewed against the requirements of Procedures 93DP-0LC09, "Data Collection
and Submittal Using INPO's Consolidated Data Entry System," Revision 7, and
70DP-0PI01, "Performance Indicator Data Mitigating Systems Cornerstone," Revision 3.
* Unplanned Scrams Per 7,000 Critical Hours
- 23 - Enclosure
- Unplanned Scrams With Complications
- Unplanned Power Changes Per 7,000 Critical Hour
Documents reviewed by the inspectors are listed in the attachment. The inspectors completed three samples. Cornerstone: Emergency Preparedness The inspector reviewed licensee evaluations for three emergency preparedness cornerstone PIs for the period of January through December 2007. The definitions and guidance of NEI Report 99-02, "Regulatory Assessment Indicator Guideline," Revisions 3 through 5, and licensee PI Procedure 16DP-0EP19, "Performance Indicator Emergency Preparedness Cornerstone," Revision 6, were used to verify the accuracy of the licensees evaluations for each PI reported during the assessment period. The inspector reviewed a one hundred percent sample of drill and exercise scenarios and licensed operator simulator training sessions, notification forms, and attendance and critique records associated with training sessions, drills, and exercises conducted during the verification period. The inspector reviewed sixteen selected emergency responder qualification, training, and drill participation records. The inspector reviewed alert and notification system testing procedures, maintenance records, and a one hundred percent sample of siren test records.
- Drill and Exercise Performance
- Emergency Response Organization Participation
- Alert and Notification System Reliability
Documents reviewed by the inspectors are listed in the attachment. The inspector completed three samples. Cornerstone: Occupational Radiation Safety The inspector reviewed the Occupational Exposure Control Effectiveness PI and associated licensee documents from October 1 through December 31, 2007. The review included corrective action documentation that identified occurrences in locked high radiation areas (as defined in the licensees TSs), very high radiation areas (as defined in 10 CFR 20.1003), and unplanned personnel exposures (as defined in NEI 99-02, "Regulatory Assessment Indicator Guideline," Revision 5). Additional records reviewed included ALARA records and whole body counts of selected individual exposures. The inspector interviewed the licensee that were accountable for collecting and evaluating the PI data. In addition, the inspector toured plant areas to verify that high radiation, locked high radiation, and very high radiation areas were properly controlled. Performance indicator definitions and guidance contained in NEI 99-02, Revision 5, were used to verify the basis in reporting for each data element. Documents reviewed by the inspectors are listed in the attachment. The inspector completed one sample. Cornerstone: Public Radiation Safety
- 24 - Enclosure
The inspector reviewed the Radiological Effluent TS /Offsite Dose Calculation Manual
Radiological Effluent Occurrences PI and associated licensee documents from
October 1, 2007, through December 31, 2007. Licensee records reviewed included
corrective action documentation that identified occurrences for liquid or gaseous effluent
releases that exceeded PI thresholds and those reported to the NRC. The inspector
interviewed the licensee that was accountable for collecting and evaluating the PI data.
Performance indicator definitions and guidance contained in NEI 99-02, Revision 5, were
used to verify the basis in reporting for each data element.
Documents reviewed by the inspectors are listed in the attachment.
The inspector completed one sample.
b. Findings
No findings of significance were identified.
4OA2 Identification and Resolution of Problems (71152) .1 Routine Review of Identification and Resolution of Problems
The inspectors performed a daily screening of items entered into the licensee's CAP.
This assessment was accomplished by reviewing daily summary reports for CRDRs and
work mechanisms, and attending corrective action review and work control meetings.
The inspectors: (1) verified that equipment, human performance, and program issues
were being identified by the licensee at an appropriate threshold and that the issues
were entered into the CAP; (2) verified that corrective actions were commensurate with
the significance of the issue; and (3) identified conditions that might warrant additional
follow-up through other baseline inspection procedures (IPs).
.2 Selected Issue Follow-up Inspection
a. Inspection Scope
In addition to the routine review, the inspectors selected the three below listed issues for
a more in-depth review. The inspectors considered the following during the review of the
licensee's actions: (1) complete and accurate identification of the problem in a timely
manner; (2) evaluation and disposition of operability/reportability issues;
(3) consideration of extent of condition, generic implications, common cause, and
previous occurrences; (4) classification and prioritization of the resolution of the problem;
(5) identification of root and contributing causes of the problem; (6) identification of
corrective actions; and (7) completion of corrective actions in a timely manner.
* January 29, 2008, Unit 2, foreign material (FM) previously found in the spent fuel
pool (SFP) no longer visible
* February 4, 2008, Units 1, 2, and 3, reviewed unresolved Item 05000528,
05000529, 05000530/2007012-18, "Routine Heavy Use of Overtime," opened
during the IP 95003 Supplemental Inspection for an NRC review of actual hours
worked by operations personnel
- 25 - Enclosure
* February 6-26, 2008, Units 1, 2, and 3, reviewed quality control evaluators
organizational structure
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed three samples.
b. Findings and Observations .1 Foreign Material in the Spent Fuel Pool
Introduction. The inspectors identified a Green NCV of 10 CFR Part 50, Appendix B,
Criterion V, "Instructions, Procedures, and Drawings," for the failure of fuels services
personnel to evaluate leaving foreign material (FM) in the Unit 2 SFP in accordance with
procedures, and failed to ensure those procedures included appropriate quantitative and
qualitative acceptance criteria.
Description. On October 13, 2006, during Unit 2 refueling Outage 2R13, fuels services
personnel were performing under-bundle fuel inspection for fuel being removed from the
reactor when FM was found. The FM appeared to be fixed inside the guardian grid of
the lower end fitting of fuel Assembly P2N111, as documented in CRDR 2932719. A
control room review of CRDR 2932719, performed on October 14, 2006, stated no
additional foreign object search and retrieval (FOSAR) was required to look for the
material.
On November 7, 2006, fuels services personnel initiated CRAI 2940130 to use
Procedure 78DP-9ZZ01, "Foreign Object Search And Retrieval, Remotely Operated
Vehicles, And Submersible Retrieval Tools And Pumps," Revision 0, and a written plan
to attempt to remove the FM from Assembly P2N111 per WO 2940366. Engineering
personnel planned to evaluate leaving the debris in Assembly P2N111, if the removal
attempt was unsuccessful. Fuels services personnel attempted to recover the FM on
January 24, 2008, in accordance with WO 2940366; however, the piece of debris was no
longer visible. On January 24, 2008, PVAR 3126308 documented that the FM may have
been transported to another location in the Unit 2 SFP or RCS, and that the FOSAR
effort would be expanded to the rest of the SFP. The PVAR stated, in part, that if the
FOSAR effort failed to locate and retrieve the debris, then, an evaluation and an
engineering deficiency work order (ENG-DFWO) would be initiated in accordance with
Procedure 30DP-9MP03, "System Cleanliness and Foreign Material Exclusion Controls,"
Revision 6.
On January 24, 2008, inspectors reviewed Procedure 30DP-9MP03, Step 2.9.5, which
states, "that if the FM cannot be retrieved, then the Responsible Leader shall ensure that
an ENG-DFWO has been initiated and dispositioned by the Responsible Engineer
before the system is closed. The ENG-DFWO shall be linked to the CRDR written to
document the loss of FME control." Procedure 81DP-0DC13, "Deficiency Work Order,"
Revision 21, Step 3.2.1 states, "engineering personnel assigned to disposition a
deficiency work order (DFWO) which addresses degraded or nonconforming conditions
to TS equipment or equipment that supports TS equipment should verify an operability
determination or functional assessment has been performed in accordance with
Procedure 40DP-9OP26, Operability Determination and Functional Assessment,
Revision 18." The inspectors observed that Procedure 30DP-9MP03 provided no time
limit acceptance criteria to perform a functional assessment and to write a DFWO, as
- 26 - Enclosure
specified in Procedure 81DP-0DC13, from the time the FM was found in the SFP. Additionally, the inspectors questioned fuels services personnel whether FM in the SFP was a potentially degraded or nonconforming condition and should be evaluated by the control room in accordance with Procedure 01DP-0AP12, "Palo Verde Action Request Processing," Revision 4, and Procedure 40DP-9OP26. Procedure 01DP-0AP12, Step 3.5 states, "A control room review will be performed for PVARs that have been screened at the Operations Review step and determined that a control room review is warranted. The condition described in the PVAR shall be evaluated by the SM for the assessment of potential operability concerns." If a degraded or nonconforming condition exists, Step 3.5.3 states, "the SM shall initiate actions to determine Operability/Functionality per Procedure 40DP-9OP26." On January 27, 2008, operations personnel performed a functional assessment of the effects of SFP FM on SFP cooling and the ability of the SFP to provide a borated water source, as documented in PVAR 3126308. Operations personnel determined the small piece of FM would have little impact on fuel assembly cooling and SFP cooling; and that the FM would not go back into the RCS because under-bundle inspections on fuel bundles are performed before the fuel is put back in the core. The inspectors noted that the FM could have been transferred back into the RCS and affect reactor core fuel assemblies, because at the time the FM was found in October 2006, no under-bundle inspections were performed for fuel going back into the RCS. Consequently, the licensee updated the functional assessment performed in PVAR 3126308 on January 31, 2008, to address the possibility that the FM was transported into the RCS. The licensee determined that due to the size of the FM, about three eights of an inch, that it would not create any operability concerns. The licensee determined that this FM was very similar in shape and size, and was covered by a more limiting evaluation performed for FM found in Unit 3 fuel bundles, as documented in DFWO 2885310. This DFWO, including Westinghouse vendor guidance, determined that the material was flexible graphite (grafoil), which is commonly used in gasket and valve packing material and has been approved for use in the RCS. The licensee wrote WO 3139395 to continue to look for the debris. Analysis. The performance deficiency associated with this finding involved the failure of fuels services personnel to evaluate leaving FM in the Unit 2 SFP in accordance with procedures, and failed to ensure those procedures included appropriate quantitative and qualitative acceptance criteria. The finding is greater than minor because it is associated with the SSC performance and human performance attributes of the barrier integrity cornerstone and affects the cornerstone objective to provide reasonable assurance that physical design barriers (fuel cladding, RCS, and containment) protect the public from radionuclide releases caused by accidents or events. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because the finding did not result in loss of cooling to the SFP; the finding did not result from fuel handling errors that caused damage to the fuel clad integrity or a dropped assembly; and the finding did not result in a loss of SFP inventory greater than ten percent of the SFP volume. This finding has a crosscutting aspect in the area of human performance associated with decision-making because the licensee failed to use conservative assumptions when evaluating degraded and nonconforming conditions [H.1.(b)]. Enforcement. 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those
- 27 - Enclosure
instructions, procedures, and drawings. The control of FM to prevent damage to quality and quality augmented components is implemented by Procedure 30DP-9MP03, "System Cleanliness and Foreign Material Exclusion Controls," Revision 6. Procedure 30DP-9MP03, Step 2.9.5 states, "that if the FM cannot be retrieved, then the Responsible Leader shall ensure that an ENG-DFWO has been initiated and dispositioned by the Responsible Engineer before the system is closed. The ENG-DFWO shall be linked to the CRDR written to document the loss of FME control." Procedure 81DP-0DC13, "Deficiency Work Order," Revision 21, Step 3.2.1, states, "engineering personnel assigned to disposition a DFWO which addresses degraded or nonconforming conditions to TS equipment or equipment that supports TS equipment should verify an operability determination or functional assessment has been performed in accordance with Procedure 40DP-9OP26, 'Operability Determination and Functional Assessment,' Revision 18." Contrary to the above, between October 13, 2006 and January 31, 2008, fuels services personnel failed to evaluate leaving FM in the Unit 2 SFP in accordance with procedures, and failed to ensure those procedures included appropriate quantitative and qualitative acceptance criteria. Specifically, fuels services personnel used Procedure 30DP-9MP03, "System Cleanliness and Foreign Material Exclusion Controls," Revision 6, which did not specify acceptance criteria for time to perform a functional assessment of FM in the SFP, resulting in FM being left in the SFP for greater than one year without an evaluation on affected safety systems. Because this finding is of very low safety significance and has been entered into the licensee's CAP as PVAR 3126308, this violation is being treated as an NCV, consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 0500529/2008002-03, "Inadequate Procedure to Evaluate Foreign Material in the Spent Fuel Pool."
.2 Failure To Maintain Adequate Staffing Levels
Introduction. The inspectors identified a Green noncited violation of Technical
Specification 5.2.2.d involving the routine use of excessive overtime for operations
personnel.
Description. The inspectors reviewed APS payroll data from January 1, 2003 through
December 31, 2007, that summarized the regular and overtime hours worked for each
operations department position. During this review the inspectors noted that the total
number of hours worked annually by operations department personnel remained
relatively constant, or decreased, while the percentage of those total hours that were
worked as overtime increased. As a result, the inspectors determined that the licensee
increasingly relied on the use of overtime to provide the person-hours necessary to
operate the three units.
- 28 - Enclosure
Operator staffing from January 1 for 2003 through December 31, 2007
2003 2004 2005 2006 2007 Control Room Supervisor 25 24 24 23 33 Reactor Operator 43 40 40 39 36 Shift Manager 20 20 19 19 19
- Data is an average number of personnel in the position over the year, taken from APS payroll data.
Operator Hours from January 1, 2003 through December 31, 2007
2003 2004 2005 2006 2007
Control Regular 44411 40415 42407 38713 57367
Room
Overtime 5014 5562 6890 8440 18473
Supervisor
Reactor Regular 75589 71183 71329 67199 62945
Operator Overtime 12161 15206 17888 21740 26854
Shift Regular 35821 33348 33545 32300 32059
Manager Overtime 4870 5294 6273 8726 10035
- Data taken from APS payroll data.
Average regular hours worked by position
2003 2004 2005 2006 2007 Control Room Supervisor 1776 1684 1767 1683 1738 Reactor Operator 1758 1780 1783 1723 1748 Shift Manager 1791 1667 1766 1700 1687
- Hours worked were calculated based on a comparison of the total regular hours worked by personnel in
the position relative to the average number of personnel in that position. The inspectors derived the percent overtime using the following assumptions: 1. A 4 percent correction factor to account for overtime hours worked as part of the normally scheduled shift rotation. 2. A 5 to10 percent correction factor to account for shift turnover. 3. A 75 percent correction factor to exclude overtime worked during refueling outages. The inspectors used the following equation to calculate the percent overtime worked. X=[[((Y*0.96)*0.9)/Z]*100]*0.75
X = Percent overtime
Y = Total overtime hours worked as documented in payroll data
Z = Total regular hours worked as documented in payroll data
- 29 - Enclosure
2003 2004 2005 2006 2007 Control 7.32 8.92 10.53 14.13 20.87 Room Supervisor Reactor 10.43 13.84 16.25 20.96 27.65 Operator Shift 8.81 10.29 12.12 17.51 20.28 Manager
Since 2003, overtime, as a percent of regular hours worked, has increased steadily and substantively for control room operators. The inspectors noted that the increase in overtime rates for operations department positions appeared to be largely the result of a decrease in staffing, rather than the result of an increase in the total number of person- hours expended. The inspectors also noted that the 2007 overtime rates were more than double the overtime rates of 2003. During their review the inspectors noted that Technical Specification 5.2.2.d, Organization - Unit Staff, requires that administrative procedures shall be developed and implemented to limit the working hours of unit staff that perform safety-related functions, as well as requiring that the controls shall include guidelines on working hours that ensure adequate shift coverage shall be maintained without routine heavy use of overtime. Station procedure 01DP-9EM01, Overtime Limitations, Revision 6, is the licensees administrative procedure used to control unit staff working hours in accordance with facility Technical Specifications. Section 2.1 of this procedure requires that department leaders ensure that adequate shift coverage is maintained without the routine heavy use of overtime. The objective is to have personnel work a nominal 40- hour week while the plant is operating. The inspectors determined that the licensee had several missed opportunities to identify this issue. Specifically, during their review the inspectors noted that the licensee had not been issuing and reviewing Technical Specification required excess overtime reports from approximately June 2006 through July 2007. The purpose of these reports was to facilitate identification of excess overtime usage by site management. However, due to changing computer software the reports were not generated and reviewed. Also, the inspector noted that several CRDRs written that identified the metric window for operations overtime were red for most of 2007. The inspectors determined that these were indicators of the use of excessive overtime and these indicators were missed by the licensee. Analysis. The performance deficiency associated with this finding involved excessive routine use of heavy amounts of overtime for operations personnel that perform safety- related functions. The finding is greater than minor because if left uncorrected the finding would become a more significant safety concern in that the routine use of excessive work hours increases the likelihood of operator errors. Using the Manual C Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because no specific human performance issues due to personnel fatigue were identified that resulted in the degradation or loss of safety function of equipment important to safety. The finding has a crosscutting aspect in the area of human performance associated with resources because the licensee failed to maintain sufficient qualified operations personnel to maintain working hours within guidelines without heavy use of overtime [H.2(b)].
- 30 - Enclosure
Enforcement. Technical Specification 5.2.2.d, "Organization-Unit Staff," requires, in part, that administrative procedures be developed and implemented to limit the working hours of unit staff that perform safety-related functions (e.g., licensed SROs, licensed ROs, radiation protection technicians, auxiliary operators and key maintenance personnel). This TS further requires the controls include guidelines on working hours that ensure adequate shift coverage be maintained without the routine heavy use of overtime. Procedure 01DP-9EM01, "Overtime Limitations," Revision 6, is the licensees administrative procedure used to control unit staff working hours. Procedure 01DP- 9EM01 requires, in part, that department leaders ensure that adequate shift coverage is maintained without the routine heavy use of overtime. The objective is to have personnel work a nominal 40-hour week while the plant is operating. Contrary to the above, between January 1 and December 31, 2007, the licensee failed to meet the objective of operations personnel working a nominal 40-hour week while all three units are operating, and has relied upon the excessive use of overtime to maintain adequate shift coverage. Because this finding is of very low safety significance and has been entered into the CAP as CRDR 3112231, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000528; 05000529; 05000530/2008002-04, "Failure To Maintain Adequate Staffing Levels Results In Heavy Use of Overtime to Maintain Adequate Shift Coverage." Unresolved Item 05000528; 05000529; 05000530/2007012-18 is closed.
.3 Quality Control Organizational Structure
Inspectors determined that no findings of significance were identified during the review
of CRDR 3129081. Inspectors reviewed issues regarding warehouse operations as
documented in CRDR 3129081. Inspectors evaluated the organizational structure of the
Quality Control (QC) Evaluators and the effectiveness of the Employee Concerns
Program within the Supply Chain and Stores department. The licensee addressed these
issues in a letter to the NRC dated January 25, 2008. Inspectors evaluated the
adequacy of the licensees response by conducting independent inspections.
Inspectors observed that currently QC Evaluators report directly to the Warehouse
Section Leader-Stores. The Warehouse Section Leader has the responsibility for
receiving, processing, handling, and placing into stores, equipment and components for
use at Palo Verde Nuclear Generating Station (PVNGS). Quality Control Evaluators
perform inspections for quality related equipment and components in this receipt
process. While QC Evaluators are insulated from cost and schedule pressures
associated with the rest of the PVNGS organization, they are subject to the production
pressures and budget constraints within the Supply Chain and Stores department.
Consequently, QC Evaluators do not report to a management level that assures the
required authority and organizational freedom, including sufficient independence from
cost and schedule when opposed to safety considerations.
Inspectors noted that the PVNGS Quality Assurance (QA) program was revised as
specified in licensing document change request (LDCR) 01-F-012. Quality Control
Evaluators were reassigned from the Nuclear Assurance Department to the Strategic
Procurement organization. This change to the QA program was accomplished without
prior NRC review and approval. Justification for changing the QA program without prior
NRC review and approval was described in LDCR 01-012. Regulatory Affairs and NAD
personnel concluded that the change was allowed without prior NRC approval under the
provisions of 10 CFR 50.54(a)(3). The NRC previously approved, with an NRC safety
evaluation, a similar quality assurance program description change to the Beaver Valley
- 31 - Enclosure
Power Station (BVPS). The licensee concluded that the commitments made by Beaver
Valley prior to the program change were the same as PVNGS commitments with respect
to the quality assurance program. The licensee also concluded that all other issues
questioned by the NRC during Beaver Valleys approval process were adequately
addressed in LDCR-01-012. Based on these conclusions, the licensee believed that
they were allowed to change the quality assurance program description in the UFSAR
without prior NRC approval.
Variations in the methods employed to meet the standards of the commitments exist
between the licensee and the BVPS. At the BVPS, in order to provide maximum
independence from production pressures within the Nuclear Procurement Department,
QC Inspectors would report directly to the department manager and would be assigned
a separate budget. At Palo Verde, the QC Evaluators report directly to a front line
supervisor and fall under one common Supply Chain and Stores budget. Although the
commitments themselves may be the same between the two facilities, the methods in
which those commitments are met are different. Inspectors observed that by reporting
directly to a front line supervisor, and being subject to one common budget, the QC
Evaluators may not have an the necessary level of independence from production
pressures within the Supply Chain and Stores department. No findings of significance
were identified since the changes to the UFSAR did not involve a decrease in
commitments to the NRC. The organizational structure for QC Evaluators is being
addressed in the CAP as PVAR 3143574.
.3 Annual Sample: Review of Apparent Cause Evaluations
a. Inspection Scope
The inspectors selected 20 CRDRs and six apparent cause evaluations for detailed
review. The reports were reviewed to ensure that the full extent of the performance
issues were identified, an appropriate evaluation was performed, and appropriate
corrective actions were specified and prioritized. The inspectors evaluated the selected
CRDRs against the requirements of licensee Procedure 90DP-0IP10, "Condition
Reporting," Revision 36.
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed one sample.
b. Findings
No findings of significance were identified.
.4 Multiple/Repetitive Degraded Cornerstone Column and Crosscutting Issues Follow-up
Activities
Quarterly Confirmatory Action Letter Inspection
This inspection was the first in a series of inspections to be performed by the NRC to
assess the progress that PVNGS made with respect to the implementation of their Site
Integrated Improvement Plan (SIIP) and to verify their progress in addressing the
specific actions in the NRC Confirmatory Action Letter (CAL) dated February 15, 2008.
- 32 - Enclosure
During the IP 95003 Supplemental Inspection, the licensee was still in the process of developing the SIIP and only limited progress had been made in completing SIIP tasks. As of November 1, 2007, the licensee had completed 12 closure packages and only 2 had been approved for closure by the Closure Review Board (CRB). On December 31, 2007, PVNGS submitted portions of their SIIP to address Action 5 of the original CAL dated June 21, 2007. Action 5 required the licensee to submit the portions of their improvement plan that impacted the Reactor Safety strategic performance area. The revised CAL, dated February 15, 2008, superseded the CAL dated June 21, 2007. The revised CAL contains a subset of actions delineated in the SIIP that the NRC determined were necessary to address the performance insights identified by PVNGS assessment activities and the IP 95003 Supplemental Inspection. The key performance areas that PVNGS has committed to address are as follows: Yellow and White findings as documented in NRC Inspection Reports 05000528; 05000529; 05000530/2004014 and 2006012, problem identification and resolution issues, human performance issues, engineering programs, review of current equipment evaluations, safety culture, accountability, change management, emergency preparedness, longstanding equipment deficiencies, and backlog. The areas to be inspected are identified in the revised CAL. The licensee submitted a list of the specific tasks, including due dates, associated with the action plans and strategies for each of the CAL items on March 31, 2008. The items selected for this quarterly CAL inspection were based on the completion due dates provided by the licensee from their submittal dated, December 31, 2007.
a. Inspection Scope
The inspectors selected the SIIP tasks listed below for an in-depth review. The
inspectors considered the following during the review of the licensees actions: (1) SIIP
task matches the CRAI description; (2) corrective actions address and correct the SIIP
task; (3) corrective actions address the action plan problem statement and primary
causes; (4) verification of SIIP task completion; (5) timely completion of corrective
actions in accordance with the SIIP schedule; (6) review of metrics and measures for
improved performance; (7) independent verification of improved performance; and
(8) closure of SIIP task in accordance with procedures.
* Task 1.2.E.35 (CAL Item 5 and SIIP Action Plan 5, Strategy 1) (CRAI 3107133)
-based on rankings, each engineering program owner complete a self
assessment
* Task 2.2.B.1 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3062459)
-develop a targeted staffing strategy for operations
* Task 2.2.B.2 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3062460)
-develop a targeted staffing strategy for engineering
* Task 2.2.B.3 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3062461)
-develop a targeted staffing strategy for maintenance
* Task 2.2.B.4 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3062464)
-develop a targeted strategy for radiation protection and chemistry
- 33 - Enclosure
- Task 2.2.B.5 (CAL Item 7 and SIIP Action Plan 12, Strategy 8) (CRAI 3062465)
-develop a targeted staffing strategy for other positions
- Task 3.6.48 (CAL Item 2 and SIIP Action Plan 14, Strategy 2) (CRAI 3104935)
-engineering design change for K1 relay module
- Task 3.6.60 (CAL Item 2 and SIIP Action Plan 14, Strategy 4) (CRAI 3042092)
-identify and classify components in the Class 1E 480V Power Switchgear system
- Task 3.6.62 (CAL Item 2 and SIIP Action Plan 14, Strategy 4) (CRAI 3042095)
-identify and classify components in the PK system
- Task 3.6.64 (CAL Item 2 and SIIP Action Plan 14, Strategy 4) (CRAI 3042098)
-identify and classify components in the AFW system
- Task 3.7.3 f (CAL Item 1 and SIIP Action Plan 15, Focus Area 2)
(CRAI 2785420) -implement design modification work order 2760330 to replace the existing carbon steel parts on the inboard butterfly valves JSIAUV0673 and JSIBUV0675 with stainless steel parts
- Task 3.7.3 p (CAL Item 1 and SIIP Action Plan 15, Focus Area 1)
(CRAI 2785390) -implement design modification for the Unit 1 containment sump suction valves
- Task 5.1.E.3 (CAL Item 3 and SIIP Action Plan 3, Strategy 4) (CRAI 3062967)
-incorporate operability determination in engineering continuing training program requirements
- Task 9.1.A.1 (CAL Item 10 and SIIP Action Plan 8, Strategy 1) (CRAI 3063144) -
implement Policy 1503, Emergency Planning, to require personnel to fill positions within required timeframe
- Task 9.1.A.5 (CAL Item 10 and SIIP Action Plan 8, Strategy 1) (CRAI 3063199)
-revise Policy Guide 150, "Emergency Planning"
- Task 9.1.A.24 (CAL Item 10 and SIIP Action Plan 8, Strategy 8) (CRAI 3077904)
-develop and implement a multi-discipline Emergency Plan Steering Committee
- Task 15.1.10 (CAL Item 3 and SIIP Action Plan 6, Part 2, Strategy 7) (CRAI
3017939) -develop and implement station metrics/indicators associated with self assessments
- Task 15.2.1.b (CAL Item 3 and SIIP Action Plan 6, Part 2, Strategy 7) (CRAI
3017946) -lessons learned and recommendations for incorporation of good
practices into the site work management system
The inspectors considered the following CAL SIIP tasks completed: 2.2.B.1,
2.2.B.2, 2.2.B.3, 2.2.B.4, 2.2.B.5, 3.6.48, 3.6.60, 3.6.64, 3.7.3.p, 9.1.A.1, 9.1.A.5,
9.1.A.24, 15.1.10, and 15.2.1.b.
- 34 - Enclosure
b. Findings .1 Task Closure
Each task within the SIIP required a closure package along with varying levels of
management review for closure based on the priority of the corrective action. The
inspectors reviewed a total of 33 tasks associated with the licensees SIIP. These tasks
were in various stages of the closure process, including some items that were still open.
The SIIP task closure packages were reviewed in accordance with Procedure 01DP-
0AC06, "SIBP/SIIP Process," Revision 3, to determine if PVNGS personnel were
following the closure process. The process has three closure categories:
* Category A - included significant conditions adverse to quality and CAL items
* Category B - included adverse conditions and improvement plan Priority 3
CRAIs
* Category C - included improvement plan Priority 4 CRAIs.
Category A tasks get the most reviews including: the standard CRDR/CRAI closure
process; initiative lead concurs that the action is ready for closure; reviewed and
approved by the CRB; and, independent reviews from senior management led boards.
During the review of the SIIP tasks, the inspectors identified numerous quality issues,
including closure packages for Tasks 3.6.62, 3.7.3.p, 5.1.E.3, and 9.1.A.8, as follows:
* Closure package for Task 3.6.62, "identify and classify components in the PK
system," was inappropriately closed with outstanding reviews not completed to
ensure operability of the PK system. For details, refer to Section .3 below.
* Closure package for Task 3.7.3.p, "implement design modification for the Unit 1
containment sump suction valves," was closed without supporting documentation
to demonstrate that testing had verified the containment sump piping was full of
water after the modifications were completed. This action was completed, but
the completion documentation was missing.
* Closure package for Task 5.1.E.3, "incorporate operability determination in
engineering continuing training program requirements," was submitted without
demonstrating that the training was effective. The inspectors determined that the
submitted package quality failed to meet the purpose to enhance the skill and
knowledge of engineers performing operability determinations. The package
took credit for general engineering lessons learned training that was conducted in
April and May 2007. The CRB also recognized that operability determination
concerns still existed and additional efforts were needed. CRDR 3095373 was
initiated and it contained 24 CRAIs to address the continuing problems with
operability determinations. Additional inspections will be required to close CAL
SIIP Task 5.1.E.3.
The inspectors also reviewed the SIIP quality PIs, interviewed numerous personnel, and
reviewed several Nuclear Assurance evaluations related to CAL SIIP actions. The
licensee has been and continued to provide training to the task owners on
Procedure 01DP-0AC06 closure process, and was also providing coaching to
- 35 - Enclosure
individuals. Packages can be unsatisfactory for many reasons including: improper formatting, missing signatures, incomplete documentation, lack of demonstrated implementation, inadequate corrective actions, and inadequate sustainability requirements. The closure review process was described in Procedure 01DP-0AC06, Appendix L, SIBP/SIIP Action Closure Flowchart, and contained two quality control steps, administrative and preliminary reviews. Numerous packages that were submitted for closure did not meet the closure review checklist criteria and were sent back to the owners for correction prior to CRB review. The licensee was in the early stage of task closure and overall package quality needs to be improved. Nuclear Assurance Evaluation 08-0024, dated March 4, 2008, determined that the backlog of closure reviews and approvals was growing and that the rejection rate was high. As of February 4, 2008, 246 packages were submitted and 145 did not meet the standards during the administrative and preliminary reviews and were returned to the responsible owners. Those owners were provided feedback to improve the quality of the closure packages. During the same time period, the CRB reviewed 55 closure packages and CRB only accepted 40 packages for closure (of those, 30 packages had minor changes that needed to be made and were verified acceptable by the CRB chairman). Approximately 25 percent of the packages submitted to CRB required additional work. In reviewing recent SIIP quality PIs, it appears that package quality was improving, but no trend was available since the indicators were for January and February 2008. For comparison between January and February 2008, document quality was as follows: four packages verses 44 packages were accepted by the CRB without comments; 13 packages verses one package were accepted by the CRB with comments; four packages for both January and February were tabled (not reviewed by the CRB); and five packages verses zero packages were rejected. The inspectors attended several recent CRB meetings and found the packages reviewed to be of higher quality.
.2 Metrics and Measures to Monitor Improvement
During the inspection, the licensee was still in the process of finalizing the SIIP PIs.
These indicators will not be finalized until PVNGS provides details of their actions to
address each item of the CAL dated February 15, 2008, which was submitted to the
NRC on March 31, 2008. The licensee developed eight additional PIs to track the
quality and schedule completion of SIIP tasks. The inspectors reviewed a sample of
these draft PIs and determined that most of the indicators appeared appropriate and
should provide useful information. However, the inspectors determined that not enough
time had passed to assess trends or determine the appropriateness of the goals and
thresholds.
The SIIP PIs used to track the schedule completion of the tasks were somewhat
misleading because they used the site work management system completion dates
verses SIIP completion dates. At the end of the inspection, none of the Category A
closure packages (highest level and includes over 500 CAL SIIP items) were completely
closed. Only 13 of over 500 CAL SIIP items were accepted by the CRB and these had
not received the independent reviews required by Procedure 01DP-0AC06.
- 36 - Enclosure
.3 Failure to Implement Corrective Action Process for Class 1E 125 Vdc System
Introduction. The inspectors identified a Green NCV of 10 CFR Part 50, Appendix B,
Criterion V, "Instructions, Procedures, and Drawings," for the failure of engineering
personnel to ensure that potentially nonconforming conditions associated with the
PK system were reviewed for operability.
Description. On September 22, 2006, a root cause evaluation was documented in
CRDR 2926830 for the Unit 3 EDG K1 contactor repeat failure, as discussed in NRC
Inspection Report 05000528; 05000529; and 05000530/2006012. The licensee's root
cause evaluation stated the root cause to be that the "K1 contactor was treated as a
single reliable component; therefore, subcomponents of the K1 contactor mechanics
were not fully understood. This lack of understanding produced ineffective PM tasks for
the EDG field flash and de-excitation circuits." On May 15, 2007, during the extent of
cause/condition review for CRDR 2926830, the licensee wrote CRAI 3014243 to
address the following in other systems: identify and classify any auxiliary contacts,
relays, starters, or contactors that had moving parts which break or make contacts
and/or had physical adjustments; of those components identified, determine if
dimensional criteria is given for the components as described in the vendor technical
documents (VTDs); and if criteria is given, determine if the criteria is verified through
PM tasks.
On July 20, 2007, the licensee initiated CRAI 3042095 that looked at this extent of cause
for the PK system. The CRAI evaluation identified over 300 relays and starters in the PK
system that either required periodic gap/wipe adjustments in accordance with their
VTDs, but had no PM to verify proper alignment; or had existing PMs, but the VTD
adjustment requirements were not adequately reflected in the PMs. The licensee
dispositioned this as an enhancement to create or modify these PMs, and on
September 29, 2007, wrote CRAI 3069502 to track the completion of the necessary PM
creation and revision tasks.
The Palo Verde Site Integrated Business Plan (SIBP)/SIIP, Initiative 3.6, addressed
corrective actions associated with the EDG K1 Relay. Specifically, Task 3.6.62
addressed the extent of cause/condition to the PK system and performed the actions
specified in CRAI 3042095. During review of the closure documentation associated with
Task 3.6.62 on March 3, 2008, the reviewers concurred with the conclusion of writing
CRAI 3069502 that tracked the creation and modification of PMs for the affected
PK components.
On March 11, 2008, inspectors reviewed SIBP/SIIP Closure Document for Task 3.6.62.
The affected relays and starters in the PK system potentially did not conform to the
vendor technical documents since adjustments were possible, but were not being
verified through PMs. Inspectors questioned whether this constituted a potentially
degraded/nonconforming condition instead of an enhancement as dispositioned in
CRAI 3042095. Procedure 90DP-0IO10, "Condition Reporting," Revision 36,
Step 3.3.1.12 states, in part, that during the course of a CRDR evaluation, if additional
conditions unrelated to the original condition are discovered, a new PVAR for each new
condition shall be initiated and submitted for review in accordance with
Procedure 01DP-0AP12, "Palo Verde Action Request Processing," Revision 4.
Procedure 01DP-0AP12, Step 3.5 states, in part, that the condition described in the
PVAR shall be evaluated by the SM for the assessment of potential operability concerns.
- 37 - Enclosure
Based on the inspectors concerns, the licensee wrote PVAR 3144707 and performed an immediate operability determination in accordance with Procedure 40DP-9OP26, "Operability Determination and Functional Assessment," Revision 18. The immediate operability determination stated the affected PK components were operable based on all surveillances of the associated valves and equipment being current, and that there were no known failures in these control circuits. Analysis. The performance deficiency associated with this finding was the failure of engineering personnel to ensure that potentially nonconforming conditions associated with the PK system were reviewed for operability. This finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective to ensure the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because it did not represent a loss of system safety function, an actual loss of safety function of a single train for greater than its TS allowed outage time, or screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. This finding has a crosscutting aspect in the area of human performance associated with decision-making because safety-significant decisions were not verified to validate underlying assumptions and identify unintended consequences [H.1(b)]. Enforcement. 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings. The resolution of adverse conditions is implemented by Procedure 90DP-0IO10, "Condition Reporting," Revision 36. Procedure 90DP-0IO10, Step 3.3.1.12 states, in part, that during the course of a CRDR evaluation, additional conditions unrelated to the original condition are discovered, a new PVAR for each new condition shall be initiated and submitted for review in accordance with Procedure 01DP-0AP12, "Palo Verde Action Request Processing," Revision 4. Procedure 01DP-0AP12, Step 3.5 states, in part, that the condition described in the PVAR shall be evaluated by the SM for the assessment of potential operability concerns. The assessment of operability of safety-related equipment needed to mitigate accidents is implemented by Procedure 40DP-9OP26, "Operability Determination and Functional Assessment," Revision 18. Contrary to the above, between September 29, 2007 and March 7, 2008, engineering personnel failed to ensure that potentially nonconforming conditions associated with the PK system were reviewed for operability. Specifically, engineering personnel failed to ensure all relevant information was reviewed for operability when it was determined that vendor recommended preventative maintenance tasks were not being performed on PK system. Because this finding is of very low safety significance and has been entered into the CAP as PVAR 3144707, this violation is being treated as an NCV, consistent with Section VI.A.1 of the Enforcement Policy: NCV 05000528; 05000529; 05000530/2008002-05, "Failure to Properly Implement Corrective Action Process for Potential Operability Issues with the Class 1E 125 Vdc System."
.4 Cross-References to Problem Identification and Resolution Observations and Findings
Documented Elsewhere
Section 1R15 describes a finding where operations and engineering personnel failed to
use available operating experience, including vendor recommendations, to implement
- 38 - Enclosure
and institutionalize operating experience through changes to station processes,
procedures, equipment, and training programs.
Section 4OA2.4 describes a finding where CAP personnel failed to ensure a proper
classification and prioritization of two CRDRs. The inspector evaluated the effectiveness
of the licensees problem identification and resolution process with respect to the
following inspection areas:
* Access Control to Radiologically Significant Areas (Section 2OS1)
* ALARA Planning and Controls (Section 2OS2)
4OA3 Follow-up of Events and Notices of Enforcement Discretion (71153)
Event Follow-Up
a. Inspection Scope
The inspectors reviewed the four below listed events and degraded conditions for plant
status and mitigating actions to: (1) provide input in determining the appropriate agency
response in accordance with Management Directive 8.3, "NRC Incident Investigation
Program;" (2) evaluate performance of mitigating systems and licensee actions; and
(3) confirm that the licensee properly classified the event in accordance with EAL
procedures and made timely notifications to NRC and state/governments, as required.
* January 13, 2008, a RO noticed a SFP level change on the control room remote
camera while an auxiliary operator (AO) was performing an evolution on the pool
cooling (PC) system
* January 20-March 15, 2008, Units 1, 2, and 3, design issues with remote
shutdown disconnect switches to the remote shutdown panel
* January 22, 2008, Unit 3, dry cask storage platforms stored in the fuel building
did not meet seismic requirements and could have affected pump room exhaust
air cleanup system Trains A and B
* January 25, 2008, Units 1, 2, and 3, EDG fuel oil injection pump leakage that
impacted EDG operability
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed four samples.
b. Findings
Introduction. A Green self-revealing NCV of TS 5.4.1.a was identified for the failure of
operations personnel to follow procedures, which resulted in an inadvertent transfer of
SFP water to the refueling water tank (RWT).
Description. On January 13, 2008, the Unit 3 control room supervisor directed an AO to
place PC cleanup Filter PCN-F01B in service or standby following filter replacement per
Procedure 40OP-9PC06, "Fuel Pool Clean-up and Transfer," Revision 41. A pre-job
briefing was performed where the pool cooling lineup was discussed. Specifically, it was
- 39 - Enclosure
communicated to the AO that PC cleanup Train A was in service and that PC cleanup Train B was secured. It was, however, noted that PC cleanup Train B had recently been aligned for RWT recirculation/cleanup. The system drawing was not referenced during the pre-job briefing to verify the flowpath and ensure that the current system lineup was understood. The AO made an erroneous assumption during the valve alignment and marked Procedure 40OP-9PC06, Step 10.6.2.3, as not applicable since he believed Filter PCN-F01B Bypass Valve PCN-061 was not in the current flowpath. Step 10.6.2.3 would have closed Valve PCN-V061. This assumption was in error since Valve PCN-V061 was in the current flowpath due to the recent RWT recirculation/cleanup alignment. Consequently, Step 10.6.2.3 was not performed and Valve PCN-V061 was left open. When Step 10.6.2.5 was performed to open cleanup pump cross-tie isolation Valve PCN V045, a flowpath was established from PC cleanup Train A, through PC cleanup Train B to the RWT. The water transfer event was stopped by isolating the flowpath after a RO noticed a SFP level change on the control room remote camera and notified the AO. As a result of the improper alignment, an estimated 300 gallons of SFP inventory was transferred to the RWT. Similar events occurred between April 2003 and April 2006, when valves associated with the SFP were inappropriately positioned, resulting in a loss of SFP inventory. The events were documented in NCVs 05000528; 05000529; 05000530/2004003-09, 05000528/2005003-04, and 05000530/2006003-04. Analysis. The performance deficiency associated with this finding involved operations personnel not following procedures. The finding is greater than minor because it is associated with the configuration control and human performance attributes of the barrier integrity cornerstone and affects the cornerstone objective to provide reasonable assurance that physical design barriers (fuel cladding, reactor coolant system, and containment) protect the public from radionuclide releases caused by accidents or events. Using the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, the finding is determined to have very low safety significance because the finding did not result in loss of cooling to the SFP; the finding did not result from fuel handling errors that caused damage to the fuel clad integrity or a dropped assembly; and the finding did not result in a loss of SFP inventory greater than ten percent of the SFP volume. This finding has a crosscutting aspect in the area of human performance associated with work practices because the licensee failed to use adequate human error prevention techniques, such as pre-job briefings, to ensure that the pool cooling cleanup system activity was performed safely [H.4(a)]. Enforcement. Technical Specification 5.4.1.a requires that written procedures be established, implemented, and maintained covering the activities specified in Regulatory Guide 1.33, Appendix A, February 1978. Regulatory Guide 1.33, Appendix A, Section 3.h, Procedures for Startup, Operation, and Shutdown of Safety-Related PWR Systems, which requires procedures for operating the fuel storage pool purification and cooling system. Procedure 40OP-9PC06, "Fuel Pool Cleanup and Transfer," Revision 41, provided instructions for placing a cleanup filter in service or standby. Contrary to the above, on January 13, 2008, operations personnel failed to properly implement Procedure 40OP-9PC06 for operating the PC cleanup system, resulting in Filter PCN-F01B Bypass Valve PCN-V061 being improperly aligned. This resulted in an inadvertent transfer of SFP water to the RWT. Because this finding is of very low safety significance and has been entered into the licensee's CAP as CRDR 3121713, this
- 40 - Enclosure
violation is being treated as an NCV consistent with Section VI.A.1 of the Enforcement Policy: NCV 05000530/2008002-06, "Failure to Follow Procedures Resulted in Water Transfer from the Spent Fuel Pool." Event Report Reviews
a. Inspection Scope
The inspectors reviewed the four below listed LERs and related documents to assess: (1) the accuracy of the LER; (2) the appropriateness of corrective actions; (3) violations of requirements; and (4) generic issues.
b. Findings .1 (Closed) LER 05000528/2006-003-00, "EDG Actuation on Loss of Power to A Train 4.16
Kilovolt Bus" On May 30, 2006, Unit 1 was defueled, when an invalid load shed signal was received from the balance of plant engineered safety features actuation system load sequencer Train A resulting in a loss of power (LOP) to safety-related electrical Bus PBAS03. Prior to the LOP, EDG Train A had been manually removed from Bus PBAS03 following a maintenance surveillance test and was still operating in a post-run cooldown mode. The normal offsite power source had been restored to Bus PBAS03. The deenergization of Bus PBAS03 caused a valid LOP signal which resulted in EDG Train A receiving a valid emergency run signal. EDG Train A returned to rated frequency and voltage; however, its output breaker did not close because the load sequencer had locked-up, thus, preventing the closure signal to the EDG output breaker. Operations personnel completed actions to isolate the balance of plant engineered safety features actuation system load sequencer Train A, and energize electrical Bus PBAS03 from its normal offsite power supply approximately six hours after the LOP. Through extensive troubleshooting and reviews of previous events caused by the load sequencer, the licensee's investigation determined that the most probable cause for the event was from electrical noise/interference which affected the operation of the load sequencer. Corrective actions included the installation of a design modification to reduce electromagnetic interference in the sequencer. Suspect relays and noise suppression networks were also replaced in the EDG control cabinet, and several connections in the cabinet were reworked to further reduce the electrical noise. The LER was reviewed by the inspectors and no findings of significance were identified and no violation of NRC requirements occurred. The licensee documented the failed equipment in CRDR 2899375. This LER is closed.
.2 (Closed) LER 05000529/2006004-00, "Unit 2 Feedwater Isolation Valve Inoperability
Results in Condition Prohibited by Technical Specifications"
On July 27, 2006, the Unit 2 hydraulic accumulator for main feedwater isolation Valve
(MFIV) 2JSGAUV0174 would not recharge due to a failed four-way valve lodged in the
center block position. Evaluation of the valve concluded this condition would have
prevented fast closure of Valve 2JSGAUV0174 upon receipt of a main steam isolation
signal and had existed since July 13, 2006. This outage time exceeded the time
requirements of TS 3.7.3.c, to place the plant in Mode 3 within 6 hours and Mode 6
within 36 hours. The cause of the TS violation was the failure of operations personnel to
- 41 - Enclosure
identify that Valve 2JSGAUV0174 "N" four-way valve did not return to the standby position following accumulator pressure reduction. The four-way valve was replaced and the MFIV operating procedure was revised to verify the four-way valves return to their required position. The licensee documented the failed equipment in CRDR 2915450. This LER is closed. Introduction. A Green self-revealing NCV of TS 3.7.3.c was identified for the failure of Unit 2 operations personnel to perform the actions specified in TS 3.7.3 for an inoperable MFIV, resulting in MFIV 2JSGAUV0174 to SG 1 exceeding the TS 3.7.3 allowed outage time. Description. On July 27, 2006, operations personnel declared MFIV 2JSGAUV0174 to SG 1 inoperable as a result of the hydraulic accumulator for Valve 2JSGAUV0174 failing to recharge. This failure occurred when the four-way "N" valve for Valve 2JSGAUV0174 became lodged in the center blocked position such that flow to the hydraulic accumulator was blocked. This would have prevented fast closure of Valve 2JSGAUV0174 upon receipt of a main steam isolation signal and had existed since July 13, 2006. The safety function of this MFIV is to provide containment isolation between the steam generators and the feedwater line in the event of a main steam line break, feedwater line break, or loss of reactor coolant accident. The MFIVs isolate main feedwater flow to the secondary side of the SGs following a high energy line break. Closure of the MFIVs terminates flow to both SGs, terminating the event for feedwater line breaks occurring upstream of the MFIVs. The safety function of the MFIV, to provide containment isolation, was not affected since the redundant valve, MFIV 2JSGBUV0132, on the economizer line would have closed. The normal position and the safety position for Valve 2JSGAUV0174 four-way "N" valve is in the open position to port accumulator nitrogen to fast close the MFIVs. Valve 2JSGAUV0174 was declared inoperable on July 27, 2006, and the "N" four-way valve was replaced. Engineering personnel evaluated the accumulator pressure trends and determined the "N" valve had been lodged in the blocked position since the last time operations personnel reduced pressure on July 13, 2006. A root cause investigation was conducted and documented in CRDR 2915450. The root cause investigation identified the cause to be the inability to detect the failure of the four-way "N" valve when using Procedure 40OP-9SG01, "Main Steam," Revision 53. Procedure 40OP-9SG01, Step 4.5, is used to verify the nitrogen precharge of the accumulators by turning the MFIV exercise/accumulator charge test switch to "ACC CH TEST," which shuttles the "N" four-way valve to bleed off accumulator hydraulic fluid. After verifying the nitrogen pre-charge, operations personnel turn the switch back to normal which causes the actuator air operated hydraulic pump to recharge the accumulator. Further, Procedure 40OP-9SG01, Step 4.6.10, is used if pressure becomes too high in the accumulators, then operations personnel reduce pressure by cycling the exercise/accumulator charge test switch to "ACC CH TEST," which cycles the "N" four-way valve to bleed off a slight amount of pressure. This process should automatically return the "N" four-way valve to its required position. Procedure 40OP-9SG01 did not provide a step to verify the position of the "N" four-way valve after cycling the valve. The action to prevent recurrence was to revise the procedure to require verification of hydraulic pump start and accumulator pressure increase greater than 100 pounds per square inch. The ability to increase accumulator pressure indicates the "N" four-way valve has returned to its proper position to support
- 42 - Enclosure
MFIV operation. The direct cause of the failure of Valve 2JSGAUV0174 "N" four-way valve is unknown. This issue is similar to an event from June 1998 when the Unit 3 MFIV 3JSGAUV0177 "N" four-way valve was found lodged in the center blocked position as described in CRDR 380142. Analysis. The performance deficiency associated with this finding involved the failure of operations personnel to perform the actions specified in TS 3.7.3.c. This finding is greater than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the cornerstone objective to ensure the availability and reliability of systems that respond to initiating events to prevent undesirable consequences. A Phase 2 analysis is required because the Manual Chapter 0609, "Significance Determination Process," Phase 1 Worksheets, determined that there was a loss of main feedwater isolation of a single train to SG 1 for greater than the TS allowed outage time. The initiating event likelihood is determined to be three to 30 days since the finding occurred between July 17 and 27, 2006. Using the Phase 2 Worksheets associated with a SG tube rupture without SG isolation, the finding is determined to only affect Sequence 2, with operator action credit reduced to zero, the finding is determined to have very low safety significance since all remaining mitigation capability was available or recoverable. Enforcement. Technical Specification 3.7.3.a requires that with one MFIV inoperable, actions must be taken to close or isolate the inoperable valve within 72 hours. If these actions are not completed, TS 3.7.3.c requires the unit be placed in Mode 3 within 6 hours, and in Mode 5 within 36 hours. Contrary to the above, on July 17, 2006, operations personnel failed to perform the actions specified in TS 3.7.3.c. Specifically, on July 17, 2006, operations personnel failed to perform actions to place the unit in Mode 3 within 6 hours and Mode 5 within 36 hours, as required by TS 3.7.3.c for an inoperable MFIV that had not been closed or isolated in 72 hours, as required by TS 3.7.3.a. This resulted in MFIV 2JSGAUV0174 to SG 1 exceeding the TS 3.7.3 allowed outage time. Because this finding is of very low safety significance and has been entered into the licensee's CAP as CRDR 2915450, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy: NCV 05000529/2008002-07, "Failure to Identify Inoperable Feedwater Isolation Valve Exceeds Technical Specification Allowed Outage Time."
.3 (Closed) 05000529/2006005-00, "Reactor Head Vent Axial Indications Caused by
Degraded Alloy 600 Component"
On October 7, 2006, engineering personnel performing preplanned in-service
examinations of the Unit 2 reactor vessel head vent penetration discovered two axial
indications. Operation personnel entered TS Limiting Condition for Operations 3.4.103,
Condition A, and made an eight hour notification to the NRC for a nonconforming
condition of the RCS. The indications were located on the inner diameter surface of the
pipe adjacent to the J-weld to the head. The licensee determined that these indications
were due to primary water stress corrosion cracking. The licensee removed the flaws by
machining away approximately one inch of the vessel head vent. These removed
indications were similar to indications found on April 23, 2005, during the previous
refueling outage. This issue was previously noted on LER 05000529/2005001, and the
licensee's corrective actions at that time included machining the inside surface of the
pipe, and verifying no indications by examination. The LER was reviewed by the
- 43 - Enclosure
inspectors and no findings of significance were identified and no violations of NRC
requirements occurred. The licensee documented the problem in CRDR 2931237. This
LER is closed.
.4 (Closed) LER 05000529/2006006-00 and 05000529/2006-01, "Technical
Specification 3.7.7 Violation for an Inoperable Essential Cooling Water Heat Exchanger"
The event described in this LER was previously discussed in NRC Inspection
Report 05000528/2006011; 05000529/2006011; 05000530/2006011, and documented
as NCV 0500529/2006011-01, EW Train 2B Inoperable Longer than Allowed Outage
Time. The inspectors reviewed this LER and its supplement and no additional findings
were identified. This LER is closed.
Personnel Performance
a. Inspection Scope
On January 3, 2008, inspectors reviewed the pressurizer level decrease to below TS
limits during the performance of AFW Pump AFA-P01 full flow testing on Unit 3. The
inspectors: (1) reviewed operator logs, plant computer data, and/or strip charts for the
below listed evolutions to evaluate operator performance in coping with nonroutine
events and transients; (2) verified that operator actions were in accordance with the
response required by plant procedures and training; and (3) verified that the licensee
has identified and implemented appropriate corrective actions associated with personnel
performance problems that occurred during the nonroutine evolutions sampled.
Documents reviewed by the inspectors are listed in the attachment.
The inspectors completed one sample.
b. Findings
No findings of significance were identified.
4OA5 Other Activities
a. Inspection Scope
The inspectors reviewed the Institute of Nuclear Power Operations assessment dated
July 2007.
b. Findings
No findings of significance were identified.
4OA6 Meetings, Including Exit
On January 8, 2008, the inspectors presented the inspection results of the in-office
review of licensee changes to the emergency plan and EALs, on a telephonic exit, to Mr.
E. ONeil, Department leader, Emergency Preparedness, and other members of the
licensee's management staff at the conclusion of the inspection. The licensee
acknowledged the findings presented.
- 44 - Enclosure
On February 15, 2008, the inspectors presented the occupational radiation safety
inspection results to Mr. L. Cortopassi, Plant Manager, and other members of the
licensee's management staff at the conclusion of the inspection. The licensee
acknowledged the findings presented.
On February 15, 2008, the inspectors presented the biennial emergency preparedness
inspection results to Mr. R. Edington, Executive Vice President, Nuclear, and Chief
Nuclear Officer, and other members of the licensee's management staff at the
conclusion of the inspection. The licensee acknowledged the findings presented.
On March 12, 2008, the inspectors presented the inspection results of the in-office
review of licensee changes to the emergency plan, on a telephonic exit, to Mr. E. ONeil,
Department leader, Emergency Preparedness, and other members of the licensee's
management staff at the conclusion of the inspection. The licensee acknowledged the
findings presented.
On April 16, 2008, the inspectors presented the inspection results to Mr. R. Edington,
Executive Vice President, Nuclear, and Chief Nuclear Officer, and other members of the
licensee's management staff at the conclusion of the inspection. The licensee
acknowledged the findings presented.
The inspectors noted that while proprietary information was reviewed, none would be
included in this report.
4OA7 Licensee-Identified Violations
The following violations of very low safety significance (Green) were identified by the
licensee and are a violation of NRC requirements that meet the criteria of Section VI.A.1
of the NRC Enforcement Policy, NUREG-1600, to be dispositioned as NCVs.
* 10 CFR 50.54(q) of Title 10 of the Code of Federal Regulations requires the
licensee to follow their emergency plan. Contrary to the above, between 2002
and 2007, training personnel did not administer annual emergency preparedness
training to all employee site badge holders, as required by Section 8.1.1 of the
Emergency Plan. The finding was entered into the CAP as CRDR 2966025. The
finding is of very low safety significance because it is associated with Planning
Standards 50.47(b)(7) and 50.47(b)(15), is not a functional failure of the planning
standards because all employees received initial general emergency
preparedness training, and means existed to inform holders of site badges about
the actions they should take during an emergency.
* Technical Specification Surveillance Requirement 3.3.11.2 requires that each
remote shutdown system disconnect switch and control circuit is verified capable
of performing the intended function. Contrary to the above, between
January 20, 2008 and March 15, 2008, Procedure 40ST-9ZZ20, "Remote
Shutdown Disconnect Switch and Control Circuit Operability," Revision 10, did
not verify all circuit paths associated with each disconnect switch were
adequately tested. This issue affected all the disconnect switches to the remote
shutdown panel.
- 45 - Enclosure
The licensee entered into TS Surveillance Requirement 3.0.3 for a missed
surveillance, performed a risk evaluation, and tested the most risk-significant
disconnect switches to verify that these disconnect switches could perform their
intended function. Of the risk-significant disconnect switches tested, the licensee
identified that one disconnect switch associated with Unit 1 AFW pump to SG 1
block Valve AFB-UV-34 would not have been capable of performing it's intended
function due to an electrical jumper installed in the closing circuit. This valve is in
the flow path from the motor driven AFW pump to SG 1. However, the potential
failure of this valve would not have affected the ability to maintain a shutdown
condition, because the flowpath to the SG 2 was not affected. The finding was
entered into the CAP as PVARs 3129077, 3135575, 3136664, 3138937 and
3144595. Using Manual Chapter 0609, "Significance Determination Process,"
Appendix F, "Fire Protection Significance Determination Process," the finding is
determined to have very low safety significance because at Step 1.3, Qualitative
Screening Approach, the finding only affected the ability to reach and maintain a
cold shutdown condition.
ATTACHMENT: SUPPLEMENTAL INFORMATION
- 46 - Enclosure
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee G. Andrews, Director, Performance Improvement S. Bauer, Department Leader, Regulatory Affairs J. Bayless, Senior Engineer R. Bement, Vice President, Nuclear Operations P. Borchert, Unit 1 Assistant Plant Manager P. Brandjes, Department Leader, Maintenance J. Bungard, Radiological Engineer R. Buzard, Section Leader, Compliance D. Carnes, Unit 2 Assistant Plant Manager P. Carpenter, Department Leader, Operations R. Cavalieri, Director, Outages K. Chavet, Senior Consultant, Regulatory Affairs L. Cortopossi, Plant Manager, Nuclear Operations D. Coxon, Unit Department Leader, Operations E. Dutton, Acting Director of Nuclear Assurance D. Elkington, Consultant, Regulatory Affairs T. Engbring, Senior Engineer J. Gaffney, Director, Radiation Protection T. Gray, Department Leader, Radiation Protection K. Graham, Department Leader, Fuel Services M. Grigsby, Unit Department Leader, Operations D. Hautala, Senior Engineer, Regulatory Affairs R. Henry, Site Representative, SRP J. Hesser, Vice President, Engineering G. Hettel, Director, Operations A. Huttie, Director, Emergency Services R. Indap, Senior Engineer M. Karbasian, Director, Design Engineering W. Lehman, Senior Engineer J. McDonnell. Department Leader, Radiation Protection S. McKinney, Department Leader, Operations Support J. Mellody, Department Leader, PV Communications E. ONeil, Department leader, Emergency Preparedness F. Poteet, Senior ISI Engineer M. Radspinner, Section Leader, Systems Engineering T. Radtke, General Manager, Emergency Services and Support H. Ridenour, Director, Maintenance F. Riedel, Technical Management Assistant, Nuclear Operations S. Sawtschenko, Department Leader, Emergency Preparedness J. Scott, Section Leader, Nuclear Assurance M. Shea, Director, IMPACT E. Shouse, Representative, El Paso Electric M. Sontag, Department Leader, Performance Improvement J. Summy, Director, Plant Engineering K. Sweeney, Department Leader, Systems Engineering
A-1 Attachment
J. Taylor, Nuclear Project Manager, PNM J. Taylor, Unit Department Leader, Operations D Vogt, Section Leader, Operations Shift Technical Advisor J. Waid, Director, Nuclear Training T. Weber, Section Leader, Regulatory Affairs J. Wood, Department Leader, Nuclear Training Department T. Young, Director, Communications Nuclear Regulatory Commission M. Runyan, Senior Reactor Analyst, Region IV
LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED
Opened and Closed
05000528; 05000529; NCV Failure to Establish Preventative Maintenance Procedures
05000530/2008002-01 for Emergency Diesel Generator Fuel Oil Injection Pump
O-rings (Section 1R15)
05000530/2008002-02 NCV Two Examples of a Failure to Properly Implement the
Systematic Troubleshooting Process (Section 1R19)
05000529/2008002-03 NCV Inadequate Procedure to Evaluate Foreign Material in the
Spent Fuel Pool (Section 4OA2)
05000528; 05000529; NCV Failure to Maintain Adequate Staffing Levels Results in
05000530/2008002-04 Heavy Use of Overtime to Maintain Adequate Shift
Coverage (Section 4OA2)
05000528; 05000529; NCV Failure to Properly Implement Corrective Action Process
05000530/2008002-05 for Potential Operability Issues with the Class 1E
125 V DC System (Section 4OA2)
05000530/2008002-06 NCV Failure to Follow Procedures Resulted in Water Transfer
from the Spent Fuel Pool (Section 4OA3)
05000529/2008002-07 NCV Failure to Identify Inoperable Feedwater Isolation Valve
Exceeds Technical Specification Allowed Outage Time
(Section 4OA3)
A-2 Attachment
Closed
05000528/2006003-00 LER EDG Actuation on Loss of Power to A Train 4.16KV Bus
(Section 4OA3)
05000529/2006004-00 LER Unit 2 Feedwater Isolation Valve Inoperability Results in
Condition Prohibited by Technical Specifications
(Section 4OA3)
05000529/2006005-00 LER Reactor Head Vent Axial Indications Caused by Degraded
Alloy 600 Component (Section 4OA3)
05000529/2006006-00 LER Technical Specification 3.7.7 Violation for an Inoperable
and Essential Cooling Water Heat Exchanger (Section 4OA3)
05000529/2006006-01
05000528; 05000529; URI Routine Heavy Use of Overtime (Section 4OA2)
05000530/2007012-18
Discussed None
LIST OF DOCUMENTS REVIEWED
In addition to the documents called out in the inspection report, the following documents were selected and reviewed by the inspectors to accomplish the objectives and scope of the inspection and to support any findings: Section 1R04: Equipment Alignment Procedures
NUMBER TITLE REVISION
40OP-9DG02 Emergency Diesel Generator B, Appendix A - DG 51
"B" Valve Checklist
40OP-9DG02 Emergency Diesel Generator B, Appendix B - DG 51
"B" Electrical Checklist
40OP-9ZZ04 Plant Startup Mode 2 To Mode 1 53
40OP-9ZZ05 Power Operations 123
40OP-9NA03 13.8 kV Electrical System (NA) 30
A-3 Attachment
Drawings
NUMBER TITLE REVISION
02-M-ECP-001 P&I Diagram, Essential Chilled Water System 29
02-M-SIP-002 P&I Diagram, Essential Spray Pond System, Sheet 1 40
of 3
02-M-SIP-001 P&I Diagram, Safety Injection & Shutdown Cooling 37
System
13-E-MAA-001 Main Single Line Diagram 23
01-M-DGP-001 P&I Diagram Diesel Generator System, Sheets 49
1through 9
13-E-MAA-001 Main Single Line Diagram 23
Work Orders
3025982 3025983
Miscellaneous Schedulers Evaluation for Palo Verde Unit 1, week of March 10, 2008 Schedulers Evaluation for Palo Verde Unit 1, week of January 14, 2008 System Health Report, January 1, 2007, through June 30, 2007 Section 1R05: Fire Protection Procedures
NUMBER TITLE REVISION 14DP-0FP33 Control of Transient Combustibles 15 14DP-0FP33 Control of Transient Combustibles 16 14FT-9FP42 Monthly Portable Fire Extinguisher Inspection 9
Miscellaneous Technical Requirements Manual 3.11, Revision 44 PVNGS Pre-Fire Strategies Manual, Revision 19 UFSAR Appendix 9B, Fire Protection Evaluation Report, Revision 14 UFSAR Section 9.5.1, Fire Protection Evaluation Report, Revision 14 Section 1R11: Licensed Operator Requalification Program Procedures
NUMBER TITLE REVISION
EPIP-01 Satellite Technical Support Center Actions 24
A-4 Attachment
EPIP-99 Emergency Plan Implementing Procedure, 20
Appendices A and P
PVARs
3137869 3139481 3139486
Miscellaneous SES-0-07-E-02, Loss of PKC-M43/LOOP, Licensed Operator Continuing Training Simulator Evaluation Guide Licensed Operator Continuing Training 2008 Weekly Schedule Cycle NLR08-02, Revision 0 Simulator Evaluation Summary Sheet, Crew 33, Cycle NLR08-02 Section 1R12: Maintenance Effectiveness Procedures
NUMBER TITLE REVISION 70DP-0MR01 Maintenance Rule 17 01DP-9ZZ01 Systematic Troubleshooting 1
PVARs
2951473 2954664 2963881 2964221 2995235 2988892 3005648 3027524 3053912 3092611 3093774 3094517 3114753 3119518 3119520 3124489 3125979 3126297 3130468
CRDRs
2945319 3095450
Work Orders
3092613 3129996
Miscellaneous January 1-June 30, 2007,DG- Diesel Generator System Health Report Maintenance Rule System Basis Document, DG - Diesel Generator, Revision 3 Reactor Protection System Health Report Westinghouse Drawing E-14273-435-501, APC - Channel A Wiring Diagram, Sheet 3 of 11, Revision 5 Westinghouse Drawing E-14273-435-501, APC - Channel A Wiring Diagram, Sheet 4 of 11, Revision 5
A-5 Attachment
Section 1R13: Maintenance Risk Assessments and Emergent Work Control Procedures
NUMBER TITLE REVISION
01DP-9ZZ01 Systematic Troubleshooting 1
01DP-9ZZ01 Systematic Troubleshooting 0
30DP-9MT03 Assessment and Management of Risk When 10
Performing Maintenance in Modes 1 - 4
70DP-0MR01 Maintenance Rule 11
70DP-0RA05 Assessment and Management of Risk When 6
Performing Maintenance in Modes 1 and 2
86TD-0EE01 Reliability Centered Maintenance System Review 9
Process
86TD-0EE02 Equipment Reliability Classification Process 1
Drawings
NUMBER TITLE REVISION
01-E-AFB-005 Elementary Diagram Auxiliary Feedwater System Iso 9
Valves Pump B to SG-1 & SF-2 1J-AFB-HV-34,
Sheet 1 of 2
01-E-PKB-001 Elementary Diagram 125V DC Class 1E Power 12
System DC Cont Center 1E-PKB-M42, 125V DC
Battery 1E-PKB0F12, Sheet 2
01-E-SAF-001 Sheets 3 and 4, Control Wiring Diagram Engineered 3
Safety Features Actuation System NSSS, ESFAS
Alarms
01-E-SBF-006 Sheets 3 and 4, Control Wiring Diagram Plant 6
Protection System Channel B, Part 4
03-E-AFB-003 Elementary Diagram Auxiliary Feedwater System 5
Aux FDW Regulating Valve Pump B to SG-1 & 2 3J-
AFB-HV-31, Sheet 2
03-E-AFB-007 Elementary Diagram Auxiliary Feedwater System, 8
Aux FDW Turbine Trip & Throttle Valve 3J-AFA-HV-
0054 & Thermocouples
03-J-AFA-HV-54 Control Logic Diagram Aux. Feedwater Pump A 1
Turbine Trip & Throttle Valve J-AFA-HV-54
13-VTD-E146- ESFAS Auxiliary Relay Cabinet Assembly Manual 4
0006
A-6 Attachment
NP-1516 4" - 900# ASA Trip Throttle Valve TDP Mechanism B
With SMB 000 Limitorque Operator, Hard Packing,
Double Leakoff, Strainer, Mech. Trip, (2) Limit
Switches, Solenoid
PVARs
3140408 3120075 3118968 3119426 3119964 3118744 3129956 3135143 3140246 3143624
CRDRs
3140975 3120574 3120411 3121467 3119111
CRAIs
3136090
Work Orders
3140409 3118969 2980775 2992529 3120932 3133493 3135731 3140249
Miscellaneous CHAR Services Power Point Presentation, Reduction of Electrical Noise Interference with Palo Verde Log Amp 3A Due to Operation of DG3A, January 13, 2008 Control Room Alarm Typer Printout, January 9, 2008 Unit 2 Control Room Logs, January 9, 2008 Section 1R15: Operability Evaluations Procedures
NUMBER TITLE REVISION
40DP-9OP26 Operability Determination and Functional 18
Assessment
40ST-9DG02 Diesel Generator B Test 36
Drawings
NUMBER TITLE REVISION
01-E-AFB-005 Elementary Diagram Auxiliary Feedwater System Iso 9
Valves Pump B to SG-1 & SF-2 1J-AFB-HV-34,
Sheet 1 of 2
01-E-PKB-001 Elementary Diagram 125V DC Class 1E Power 12
System DC Cont Center 1E PKB M42, 125V DC
Battery 1E-PKB0F12, Sheet 2
A-7 Attachment
03-E-AFB-003 Elementary Diagram Auxiliary Feedwater System 5
Aux FDW Regulating Valve Pump B to SG 1 & 2 3J-
AFB-HV-31, Sheet 2
03-M-DGP-001 P & ID Diagram, Control Air Diesel Generator 44
System, Sheet 8
PVARs
2951473 2954664 2988892 3005648 3027524 3053912 3093774 3119518 3119520 3125979 3126297 3125050 3092611 3129956 3135143 3140246 3143624 3118968 3148305 3150570
CRDRs
2945319 3095450 3095505 2950136 3149153
CRAIs
2950256 2976063 3095506 3126903 2950257 3104314 3009278
Work Orders
3107411 3133493 3135731 3140249 3104640 3111422 3148320
Miscellaneous 13-JC-DF-202, Diesel Fuel Oil Storage Tank Level Instrument Uncertainty Calculation, Revision 6 13-JC-DF-202, Diesel Fuel Oil Storage Tank Level Instrument Uncertainty Calculation, Revision 6 8000865-FA, Haynes Vendor Report - Failure Analysis of Fuel Injection Pump, Revision 0 8001090-Test, Haynes Vendor Report - Special Testing of Fuel Injection Pump, Revision 0 Appendix C data sheets, September 29, 2001 and March 30, 2003 to 73DP 9ZZ10, "Guidelines for Heat Exchanger Thermal Performance Analysis," Revision 5 Letter, James S. Olszewski to James McDowell, "APS Fall, 2007 Desiccant Issue CAPS Root Cause Analysis Report," February 7, 2008 PROTO-HX 4.10 Data sheet, dated January 4, 2008 Regulatory Guide 1.33, Appendix A, Section 9, Procedures for Performing Maintenance, Revision 2
A-8 Attachment
Section 1R18: Plant Modifications Procedures
NUMBER TITLE REVISION 81DP-ODC17 Temporary Modification Control 20
PVARs
3109083
CRAIs
2779043
Work Orders
3112242
Miscellaneous Impact Review Form for Temporary Modification Work Order 3112242 S-07-0451, 50.59 screening for Temporary Modification Work Order 3112242 Temporary Modification 3112242 Temporary Modification 2862207 Updated Final Safety Analysis Report, Section 3.9.2, Dynamic System Analysis and Testing, Revision 14 Updated Final Safety Analysis Report, Section 5.4.7, Residual Heat Removal System, Revision 14 Final Safety Analysis Report, Section 14.B 11.3.2, Pipe Shock and Vibration Testing, Revision 14 Section 1R19: Post-Maintenance Testing Procedures
NUMBER TITLE REVISION
01DP-9ZZ01 Systematic Troubleshooting 0
86TD-0EE01 Reliability Centered Maintenance System Review 9
Process
86TD-0EE02 Equipment Reliability Classification Process 1
40OP-9CH12 Refueling Water Tank Operations 27
A-9 Attachment
73ST-2XI12 Safety Injection Train B Emergency Core Cooling 21
System Throttle Valves-Inservice Test
39MT-9ZZ32 Motor Operated Valve Diagnostic Testing 9
39MT-9ZZ02 PM or EQ Inspection of the Generic Letter 89-10 21
Limitorque SMB/SB Motor Operated Valve Actuators
40ST-9DG01 Diesel Generator A Test 32
30DP-9MP01 Conduct of Maintenance 52
40DP-9OP02 Conduct of Shift Operations 37
70DP-0EE01 Equipment Root Cause of Failure Analysis 17
01DP-0AP12 Palo Verde Action Request Processing 4
90DP-0IP10 Condition Reporting 36
40ST-9DG02 Diesel Generator B Test 36
Drawings
NUMBER TITLE REVISION
03-E-AFB-007 Elementary Diagram Auxiliary Feedwater System, 8
Aux FDW Turbine Trip & Throttle Valve 3J-AFA-HV-
0054 & Thermocouples
NP-1516 4" - 900# ASA Trip Throttle Valve TDP Mechanism B
With SMB 000 Limitorque Operator, Hard Packing,
Double Leakoff, Strainer, Mech. Trip, (2) Limit
Switches, Solenoid
03-J-AFA-HV-54 Control Logic Diagram Auxiliary Feedwater Pump A 1
Turbine Trip & Throttle Valve J-AFA-HV-54
03-M-DGP-001 P & ID Diagram, Control Air Diesel Generator 44
System, Sheet 8
PVARs
3120075 3118968 3127568 3126297 3125979 3149118 3148305 3149003
CRDRs
3120574 3149153
CRAIs
3129614 3140483
A-10 Attachment
Work Orders
3118969 3124794 3021681 3021678 3052999 3021829 3021791 3021675 2855497 3127795 3120015 3149122 3149370 3017284 3148320 2983608
Miscellaneous 3JAFAHV0054 Troubleshooting Game Plan, January 9, 2008 Palo Verde Nuclear Generating Station Design Basis Manual-Auxiliary Feedwater System, Revision 16 Technical Specification 3.7.5, Auxiliary Feedwater System Technical Specification Bases B3.7.5, Auxiliary Feedwater System Palo Verde Nuclear Generating Station Surveillance Package Review Sheet Prompt Operability Determination, PVAR 3125979/3126297, EDG Fuel Pump Leakage, Revision 0 Engine Combustion Report APS Emergency Diesel Generator, 3A, February 7, 2008 U3-Diesel 3A Jerk Pump Replacement Schedule, February 4, 2008 Jerk Pump Inspection Checklist, January 2008 Emergency Diesel Generator Emergency Pump Monitoring Test 3JAFAHV0054 Level C Troubleshooting Game Plan, Revision1, January 11, 2008 Review of 3JAFAHV0054 Troubleshooting Game Plan, January 10, 2008 Emergency Diesel Generator Emergency Pump Monitoring Test Engine Combustion Report APS Emergency Diesel Generator, 3B, March 21, 2008 VTD-C628-00051, Cooper Energy Instruction Manual For KSV Turbocharged Diesel Generating Unit For Nuclear Power Plant Emergency Stand-By Service, Revision 11 Section 1R20: Refueling and Other Outage Activities Procedures
NUMBER TITLE REVISION
70DP-0RA01 Shutdown Risk Assessments 22
40DP-9ZZ01 Containment Entry in Modes 1 Thru 4 27
40DP-9ZZ01 Containment Entry in Modes 1 Thru 4 28
A-11 Attachment
72OP-9RX01 Calculation of Estimated Critical Condition 20 40OP-9ZZ03 Reactor Startup 46
Permits
139182 137458 139567 139608 139609 143273 143274 142827 143405 143462 143494 145574 146500
Section 1R22: Surveillance Testing Procedures
NUMBER TITLE REVISION
40ST-9DG01 Diesel Generator A Test 32
40ST-9ZZ25 Online Remote Shutdown Disconnect Switch 1
Operability
73DP-9ZZ14 Surveillance Testing 9
73ST-9AF04 AFA-P01 Full Flow - Inservice Test 2
73ST-9ZZ18 Main Steam and Pressurizer Safety Valve Set 20
Pressure Verification
73DP-9XI01 Pump and Valve Inservice Testing Program - 22
Component Tables
PVARs
3140020 3117353 3120075 3128646 3134489
Work Orders
3131169 3006277 3108764
Miscellaneous NUREG-1482, Guideline for Inservice Testing at Nuclear Power Plants, Revision 1 ASME/ANSI OM-1990, Operation and Maintenance of Nuclear Power Plants PVNGS Surveillance Test Package Review Sheet Technical Specification 3.7.1.1, Main Steam Safety Valves Section 1EP2: Alert Notification System Testing Procedures
NUMBER TITLE REVISION
EPIP-8 Emergency Planning Administration 19
EPIP-61 Emergency Planning Equipment Testing 5
A-12 Attachment
16DP-0EP20 Emergency Planning Conduct of Operations 9
Miscellaneous Palo Verde Nuclear Generating Station Remote Control Siren System Operating Manual, Revisions 8 and 9 Section 1EP3: Emergency Response Organization Augmentation Testing Procedures
NUMBER TITLE REVISION
EPIP 7 Telecommunications 17
EPIP Standard Appendices, Appendix H, Autodialer 18
EPIP 99
Activation
Miscellaneous Call-In Drill Evaluation Reports: January 18, 2007; January 16, 2007; March 16, 2007; May 17, 2007; June 26, 2007; July 31, 2007; August 21, 2007; September 26, 2007; October 23, 2007; November 14, 2007; and December 27; 2007 Section 1EP4: Emergency Action Level and Emergency Plan Changes Procedures
NUMBER TITLE REVISION EPIP-99 EPIP Standard Appendices 16
Miscellaneous Nuclear Energy Institute Report 99-01, Methodology for Development of Emergency Action Levels 2 and 4 NUREG 0654, Criteria for Preparation and Evaluation of Radiological Emergency Response Plans and Preparedness in Support of Nuclear Power Plants, Revision 1 Palo Verde Nuclear Generating Station Emergency Plan, Revision 36, submitted May 21, 2007 Palo Verde Nuclear Generating Station Emergency Plan, Revision 37, submitted May 21, 2007 Section 1EP5: Correction of Emergency Preparedness Weaknesses and Deficiencies Procedures
NUMBER TITLE REVISION
60DP-0QQ19 Internal Audits 18
A-13 Attachment
Palo Verde Section 100, PVNGS Self-Assessment and 5 Nuclear Benchmarking Policy Generating Station Policy 120 EPIP-1 Satellite Technical Support Center Actions 20 and 21
PVARs
2870126 2807473 2870126 2914362 2973337 2976699 2981306 2981606 2981932 3028784 3046518 3048866 3051083 3053838 3053838 3089226 3104356 3107606 3107851 3133068 3133077 3132912
CRDRs
2966025 2966067 2976703 2981615 3014284 3015235 3080366
Work Orders
3009109 095917
Miscellaneous Audit Report 2006-001, "Emergency Planning," March 18, 2006 Audit Report 2007-001, "Emergency Planning," March 7, 2007 Self Assessment EP-06-01: Review EPIPs to ensure Changes meet 50.54Q Requirements, January 10, 2006 Self Assessment (Item 2905254): Autodialer Issue Self Assessment, August 6, 2006 Self Assessment (Item 2926340): Emergency Planning Generic Training Requirements, November 6, 2006 Self Assessment (Item 2947666): Contingency Plans to cope with Problems Encountered during Natural Disasters Self Assessment (Item 2949664): Quarterly Communications Surveillance Self Assessment (Item 2950098): Annual Validation of the Emergency Response Organization Database Self Assessment (Item 2950117): Assessment Summary of PVNGS Master List of Emergency Planning Objectives Self Assessment (Item 2951677): Review of On-site Emergency Kits for Outdated Radioactive Material Labels, December 19, 2006
A-14 Attachment
Self Assessment (Item 2952177): Comparison of EPIP and Emergency Planning Procedure Phone Numbers, January 2, 2007 Self Assessment (Item 2957279): STSC Communicator Annual Training Documentation for EP Performance Indicators Self Assessment (Item 2991885): EPlan Pager Validations, May 29, 2007 Self Assessment (Item 3067589): Results from Benchmarking Trips to Improve OSC Performance, October 24, 2007 Self Assessment (Item 3084358): SAMG Training Self Assessment (Item 3084363): Review EPIP-99 Appendix D, Notification, October 19, 2007 Self Assessment (Item 3084379): STARS Review of EPIP-09, Revision 10, October 18, 2007 Self Assessment (Item 3083981): PIs, October 18, 2007 Self Assessment (Item 3108037): Dose Model Assessment Report, December 12, 2007 Self Assessment (Item 3108421): Benchmark of EP Emergency Notification Form, December 12, 2007 Drill Evaluation Reports: 2006: February 15, March 29, June 16, June 28, July 12, September 7 (06-D-FAC-09007), September 27, December 1, December 6, and December 7 (06-D-ENV- 12011); 2007: January 31, February 15, March 7, March 29 and 30, April 19, May 3, July 19, August 16, September 13, September 14, October 18, and October 19 Design Change Request QF-1093 Section 1EP6: Drill Evaluation Procedures
NUMBER TITLE REVISION
EPIP 03 Technical Support Center Actions 46
EPIP 04 Emergency Operations Facility Actions 41
EPIP 14 Dose Assessment 7
EPIP-99 Emergency Plan Implementing Procedure Standard 19
Appendices, Appendix A, Emergency Action Levels
EPIP-99 Emergency Plan Implementing Procedure Standard 19
Appendices, Appendix B, Protective Action
Recommendations
EPIP-99 Emergency Plan Implementing Procedure Standard 19
Appendices, Appendix D, Notifications
A-15 Attachment
EPIP-99 Emergency Plan Implementing Procedure Standard 19
Appendices, Appendix O, Recovery Organization
EPIP-99 Emergency Plan Implementing Procedure Standard 19
Appendices, Appendix S, Consideration for the use
of Fire Streams/Sprays to Reduce Plume Activity
PVARs
3171747 3142619
CRDRs
3143064 3143276
CRAIs
3150447
Miscellaneous Palo Verde Nuclear Generating Station Emergency Planning Form EP-0541, Palo Verde NAN Emergency Message Form Palo Verde Dose Assessment Forecast Palo Verde Nuclear Generating Station Emergency Planning Form EP-0012, Emergency Action Log NRC Form 361, Reactor Plant Event Notification Worksheet Palo Verde Nuclear Generating Station Annual Objective Evaluations Palo Verde Nuclear Generating Station Biennial Objective Evaluations 2008 Emergency Preparedness Evaluated Scenario 08-AEV-03002
A-16 Attachment
Section 2OS1: Access Control to Radiologically Significant Areas Procedures
NUMBER TITLE REVISION
75DP-0RP01 Radiation Protection Program Overview 6
75DP-0RP02 Radiation Contamination Control 8
75DP-0RP03 ALARA Program Overview 3
75DP-9RP01 Radiation Exposure and Access Control 10
75RP-0RP01 Radiological Posting and Labeling 24
75RP-9RP01 Radiation Exposure and Access Control 10
75RP-9RP07 Radiological Surveys and Air Sampling 15
75RP-9RP10 Conduct of Radiation Protection Operations 24
75RP-9OP02 Control of High Radiation Areas, Locked High 2
Radiation Areas and Very High Radiation Areas
PVARs
3132404 3105482 3116100 3119691 3125775 3125779
Radiation Exposure Permits
3-1393 3-3002 3-3003 3-3006 3-3015 3-6000 3-6001 3-6003
3-6005 3-6006 3-6007 3-6009 3-6010 3-6011 3-6012 3-6013
A-17 Attachment
Section 2OS2: As Low As Is Reasonably Achievable (ALARA) Planning And Controls Procedures
NUMBER TITLE REVISION
75DP-0RP01 Radiation Protection Program Overview 6
75DP-0RP02 Radiation Contamination Control 8
75DP-0RP03 ALARA Program Overview 3
75DP-9RP01 Radiation Exposure and Access Control 10
75RP-0RP01 Radiological Posting and Labeling 24
75RP-9RP01 Radiation Exposure and Access Control 10
75RP-9RP07 Radiological Surveys and Air Sampling 15
75RP-9RP10 Conduct of Radiation Protection Operations 24
75RP-9OP02 Control of High Radiation Areas, Locked High 2
Radiation Areas and Very High Radiation Areas
PVARs
3132404 3105482 3116100 3119691 3125775 3125779
Radiation Exposure Permits
3-1393 3-3002 3-3003 3-3006 3-3015 3-6000 3-6001 3-6003 3-6005 3-6006 3-6007 3-6009 3-6010 3-6011 3-6012 3-6013
Section 4OA1: Performance Indicator Verification Procedures
NUMBER TITLE REVISION
93DP-0LC09 Data Collection and Submittal Using INPOs 7
Consolidated Data Entry System
EPIP 99 EPIP Standard Appendices, Appendix A, Emergency 18
Action Levels
EPIP 99 EPIP Standard Appendices, Appendix B, Protective 18
Action Recommendations
EPIP 99 EPIP Standard Appendices, Appendix D and P 18
A-18 Attachment
Miscellaneous Nuclear Energy Institute 99-02, Regulatory Assessment Indicator Guideline, Revision 5 Palo Verde Units 1 - 3, Performance Indicator View Report, Unplanned Scrams With Complications, January - December, 2007 Palo Verde Units 1 - 3, Performance Indicator View Report, Unplanned Power Changes per 7000 Critical Hours, January - December, 2007 Palo Verde Units 1-3, PI View Report, Unplanned Scrams per 7000 Critical Hours, January - December, 2007 Palo Verde Units 1 - 3, Operating Data Reports, January - December, 2007 Palo Verde Units 1 - 3, 24 Month Power History Report, February 2006 - February 2008 LER 05000529/2007003, Manual Reactor Trip due to Increased Steam Generator Sodium Levels from Failed Heat Exchanger Plug, Revision 0 LER 05000529/2007001, Completion of a Shutdown Required by Technical Specification 3.5.3, Condition C, Revision 0 LER 05000528/2007006, Required Shutdown due to Inoperable Steam Admission Bypass Supply Valve to Auxiliary Feedwater Pump, Revision 0 Palo Verde Nuclear Generating Station Emergency Plan, Revision 37 Section 4OA2: Identification and Resolution of Problems (71152) Procedures
NUMBER TITLE REVISION
01DP-0AC06 Site Integrated Business Plan/Site Integrated 3
Improvement Plan Process
01DP-0AP12 Palo Verde Action Request Processing 4
01DP-0AP16 PVNGS Self-Assessment and Benchmarking 0
01DP-OEM09 Employee Concerns Program 0
12DP-0MC48 Quality Receiving Checklist Development 1
12DP-0MC46 Receipt Inspection 4
12DP-0MC29 Warehouse Discrepancy Notice 18
12DP-0MC25 Stores 22
12DP-0MC 50 Control And Use Of The Metallurgist Pro-Alloy 3
Analyzer
A-19 Attachment
30DP-9MP03 System Cleanliness and Foreign Material Exclusion 6
Controls
32MT-9ZZ06 Testing and Calibration of the 12IFC53A & 53B and 4
the 77A & 77B Time Overcurrent Relays
32MT-9ZZ98 Testing and Recalibration Of The GR5 Ground Fault 1
Relay
40DP-9OP19 Locked Valve, Breaker, and Component Tracking 88 40OP-9SI02 Recovery From Shutdown Cooling to Normal 61
Operating Lineup
40ST-9SI04 Containment Spray Valve Verification 5 40ST-9SI09 Emergency Core Cooling System Systems Leak 24
Test
41AL-1RK1C Alarm Response Procedure for 480 Volt 1E Trouble 36 60DP-0QQ21 Qualification and Certification Of Inspection 5
Personnel
60DP-0QQ23 Qualification and Certification Of Inspection 1
Personnel
73DP-0AP05 Engineering Programs Management and Health 3
Reporting
78DP-9ZZ01 Foreign Object Search and Retrieval, Remotely 0
Operated Vehicles, And Submersible Retrieval Tools
and Pumps
81DP-0DC13 Deficiency Work Order 21 90DP-0IP10 Condition Reporting 36 ECP 01 Employee Concerns Program Guideline 1 ECP 02 Employee Concerns Program Guideline 8 ECP 03 Employee Concerns Program Guideline 4 01DP-OEM09 Employee Concerns Program 0 60DP-0QQ21 Qualification and Certification Of Inspection 5
Personnel
0DP-0QQ23 Nuclear Assurance Stop Work And Escalation 1
Processes
12DP-0MC48 Quality Receiving Checklist Development 1 12DP-0MC46 Receipt Inspection 4 12DP-0MC29 Warehouse Discrepancy Notice 18
A-20 Attachment
12DP-0MC25 Stores 22
12DP-0MC 50 Control And Use Of The Metallurgist Pro-Alloy 3
Analyzer
ECP 01 Employee Concerns Program Guideline 1
ECP 02 Employee Concerns Program Guideline 8
ECP 03 Employee Concerns Program Guideline 4
16DP-0EP20 Emergency Planning Conduct of Operations 9
01DP-9EM01 Overtime Limitations 6
Drawings
NUMBER TITLE REVISION
01-E-PGB-008 Elementary Diagram 480V Class 1E Power System 4
Load Centers 1E-PGA-L35 & 1E-PGB-L36 480V
Main FDR Breakers
01-J-RKS-0001 Annunciator/Electronic Isolation List 22
PVARs
2982198 3124586 3126308 3128719 3143447 2982198 3128719 3110619 3072309
CRDRs
2726509 2913790 2926830 3048870 2984206 3129081 2883793 2932719 2984206 3127014 3144707 2883793 2984206 3129081 3065644 3130583 3095373 3098690 3110358 3130576 3090963 3112991 3112231 3075207 3058809 3075207 3030699 3039642 2984254 3075207 2859635 2774488 2870654 2908560 3030505
Work Orders
026318 026440 244627 2760330 2767628 2767631 2767649 2767650
2792424 2792442 2792443 2836046 2836047 2836050 2836051 2869753
2869762 2869769 2869770 2885310 2940366 3139395
A-21 Attachment
CRAIs
2785390 2785420 2933567 2938874 2940130 3014243 3017939 3017946 3042092 3042095 3042098 3065077 3069502 3086662 3086672 3104091 3104935 3126034 3100375 3123378 3126171 3129886 3075208 3090964 3116079 2987384 2993402 2993405 3020782 2874473 2844961 2779868
Site Integrated Improvement Plan Tasks
1.2.A.3 1.4.2 3.4.7.d 3.6.5 3.6.55 3.7.2.d 3.7.2.h 3.7.5.f 3.7.5.l 3.7.9.g 4.1.F.30 4.4.20 6.1.11 6.7.13 6.11.2.a 8.4.4 9.2.A.15 11.3.1 11.9.A.4.d 11.9.A.5.d
Quality Assurance Program Documents
Palo Verde Final Safety Analysis Report, Chapter 17.2B, Revision 11, June 2001 Palo Verde Final Safety Analysis Report, Chapter 1.8, Revision 12, June, 2003 ANSI N18.7-1976/ANS 3.2, Administrative Controls and Quality Assurance for the Operational Phase of Nuclear Power Plants
NAD Audit Reports Nuclear Assurance Evaluation Report, ER 08-0003, January 8-10, 2008 Nuclear Assurance Evaluation Report ER 06-0012, January 6, 2006 Procurement and Material Control Audit 2007-005 Procurement and Material Control Audit 2007-003 Procurement and Material Control Audit 2007-004 SIBP/SIIP Closure Documents
Task 3.6.48 Closure Document, February 19, 2008 Task 3.6.64 Closure Document, February 19, 2008 Task 15.1.10 Closure Document, February 19, 2008 Task 3.6.60 Closure Document, February 20, 2008 Task 3.7.3.f Closure Document, February 26, 2008 Task 3.7.3.p Closure Document, February 26, 2008 Task 1.2.E.35 Closure Document, February 27, 2008 Task 3.6.62 Closure Document, March 3, 3008 Task 6.7.13 Closure Document, October 31, 2007 Task 15.2.1.b Closure Document, February 5, 2008
Miscellaneous Nuclear Assurance Department Noteworthy Station Quality Issue: Warehouse Operations Nuclear Assurance Department Station Quality Issue: Warehouse Operations
A-22 Attachment
List of Warehouse Discrepancy Notices for 2005, 2006, 2007 Palo Verde Nuclear Generating Station 2007 Synergy NSCA Integrated Issues Resolution Process brochure List of Warehouse Receipt Inspection Condition Report Disposition Request for 2005, 2006, 2007 Warehouse Operations and Human Performance Issues Event Date: March 16, 2007 Apparent Cause Evaluation Report Employee Concerns Program files July 1, 2001 through July 1, 2007 APS Investigation Results And Response To Allegation RIV-2007-A-0129
Licensing Document Change Request 01-F-012
Regulatory Guide 1.33 Revision 2, February 1976, Quality Assurance Program Requirements (Operation) Business Supply Chain and Stores Organization Chart, January 8, 2008 Attendance Record, Senior Management Meeting with warehouse personnel, Jan 18, 2008 Palo Verde Job Description, Evaluator Senior, November 7, 1996 Palo Verde Job Description, Evaluator II, February 2, 2006 Palo Verde Job Description, Storekeeper Senior, October 22, 1993 Palo Verde Job Description, Storekeeper, June 11, 2007 Training and Qualification records for QC Evaluators from 2005 through 2007 Quality Receiving Checklist 50051659302001 TD0961476 Quality Receiving Checklist 50051630202002 TD0961583 Quality Receiving Checklist 50051630202003 TD0961585 Unit 2 Operator Logs, January 23, 2008 13-VTD-G080-00008, General Electric Time Overcurrent Relays Types IFC51A And 51B, IFC 53A And 53B, IFC77A And 77B, Revision 3 13-VTD-G080-0246-1, General Electric Instructions For Undervoltage Relays Types IAV54 & IAV55, Revision 0 1EPGAL35B1*27X* Relay, Component Data Sheet - Bus Undervoltage Relay
A-23 Attachment
Air Operated Valves Program Summary, January 1, 2007 through June 30, 2007 Engineering Design Change 2007-0048, Change 13-VTD-P292-00004 For Cleaning and Inspecting the EDG K1 Contactor DC Coil Auxiliary Contact Module, Revision 3 10 CFR 50.54(q) EAL Change Process Guidance APS letter 102-05789-RKE/CJS to the NRC dated December 31, 2007 Emergency Preparedness Steering Committee Charter, Revision 2 Emergency Preparedness Steering Committee Minutes dated November 29, 2007, December 20, 2007, January 11, 2008, and February 15, 2008 Engineering Training Course NGT90, "Industry and Engineering Events 1Q2007" Nuclear Assurance Evaluation Report 07-0168 SIIP Performance Indicators dated March 5, 2008 Palo Verde Nuclear Generating Station Position Paper titled "Control Room Staffing and Overtime" APS payroll data From January 2003 through December 2007 SECY-01-0113, "Fatigue of Workers at Nuclear Power Plants" Section 4OA3: Follow-up of Events and Notices of Enforcement Discretion Procedures
NUMBER TITLE REVISION
01DP-0AP09 Procedure Use and Adherence 7
40OP-9PC06 Fuel Pool Cleanup and Transfer 41
40OP-9SG01 Main Steam 53
70DP-0EE01 Equipment Root Cause of Failure Analysis 17
73ST-9SG01 Main Steam Isolation Valves - Inservice Test 31
73ST-9XI16 Economizer Feedwater Isolation Valves - Inservice 27
Test
90DP-0IP10 Condition Reporting 36
93DP-0LC17 10CFR 50.59 and 72.48 Guidance Manual 4
30DP-9MP03 System Cleanliness and Foreign Materials Exclusion 11
Controls
74DP-9CY03 Chemistry Control Instruction 5
A-24 Attachment
74DP-9CY04 System Chemistry Specification 52 74DP-9CY04 System Chemistry Specification 35
Drawings
NUMBER TITLE REVISION 01-M-SGP-002 P&I Diagram, Main Steam System 45 Westinghouse Head Vent Line Repair Layout 2 Drawing 8255C49 Westinghouse Vent Pipe Repair Palo Verde 2 Reactor Vessel Head 0 Drawing,10005D 69 Westinghouse Replacement Guide Cone Palo Verde 1, 2 & 3 0 Drawing Reactor Vessel 10008C66
PVARs
2954664 2963881 2995235 3005648 3053912 3092611 3093774 3119520 3125050 3125979 3126297 2954664 2963881 2995235 3005648 3053912 3092611 3093774 3119520 3125050 3125979 3126297 3121048 3120070 2793806 2764549
CRDRs
2900393 3033543 2984700 2990092 3005058 3033623 3032677 2974523 2929277 2904740 2906158 2945319 2950136 3095450 117037 2604468 2915450 2928540 3121713 2897810 2905161 2902498 2928230 2913430
CRAIs
2928802 2946438 3121046 2937383, 2921512 2921404 2921856 2921406 2837083 2938381 2921501 2910704 2905572 2940200 2921504 2921508 2921521 2921515 2921517 2921513 2921508 2921501 2921403 2909939 2910010 2910041 2910020 2910103
Work Orders
2304865 3032675 2913678 2917854 2897128 2897130 2901582 2897078
2896333 2897080 2901584 2901583 2901582 2901581 2901580 2901579
2900133 2900132 2900131 2900130 2900129 2900128 2805530 2805528
A-25 Attachment
2804567 2804566 2898679 2898676 2805524 2897083 2898681 2898682 2804562 2804563 2805523 2917539 2717779 2793837
Miscellaneous 10 CFR 50.73, Licensee Event Report System ASCE/SEI43-05, Seismic Design Criteria for Structures, Systems, and Components in Nuclear Facilities Calculation 13-CC-ZF-0085C, DCS Stairway Sliding Evaluation - ASCE 43-05 Section A.1, Revision 0 Unit 2 Control Room Logs, July 27, 2006 Unit 3 Control Room Logs, January 13 and 22, 2008 Licensee Event Report 50-529/2006-004-00 Technical Specification 3.7.3, Main Feedwater Isolation Valves UFSAR Section 6.5.1, Engineered Safety Feature Filtration System BOP Chemistry Optimization Plan - Fourth Quarter 2007 Units 1, 2, and 3 Spray Ponds Train A and B Chemistry Parameter Data, January 2006 through January 2008 Units 1, 2, and 3 Trains A and B Essential Cooling Water Heat Exchanger Performance Data, January 2004 through November 2007 Root Cause Evaluation for CRDR Number 2897810, Loss of Thermal Performance of the Essential Cooling Water and Emergency Diesel Generator Intercooler Heat Exchangers, dated November 15, 2006 Action 2924010 N001-0302-00434, Westinghouse Correspondence LTR-SGDA-06-159, "Review of the Modified Excavation and Weld Repair of the Palo Verde Unit 2 Reactor Vessel Closure Head Vent Line" Westinghouse Calculation CN-EMT-02-36, Vent Line Modification Evaluation, Revision 3 Westinghouse Calculation CN-SGDA-05-32, PV2 RVCH Vent Pipe Repair, Revision 1 Westinghouse Calculation CN-EMT-02-27, Replacement Guide Cone Weld Strength Evaluation, Revision 3 ENS# 42886 10 CFR 50.59 Evaluation S-06-0485
A-26 Attachment
LIST OF ACRONYMS USED
AFW Auxiliary Feedwater ALARA As-Low-As-Is-Reasonably-Achievable AO Auxiliary Operator BVPS Beaver Valley Power Station CAL Confirmatory Action Letter CAP Corrective Action Program CFR Code of Federal Regulations CRAI Condition Report Action Item CRB Closure Review Board CRDR Condition Report Disposition Request CRS Control DFWO Deficiency Work Order DIWO Design Implementation Work Order DPM Drops Per Minute EAL Emergency Action Level EDG Emergency Diesel Generator ENG-DFWO Engineering Deficiency Work Order FM Foreign Material FOSAR Foreign Object Search and Retrieval IP Inspection Procedure LDCR Licensing Document Change Request LER Licensee Event Report LOP Loss of Power MFIV Main Feedwater Isolation Valve NEI Nuclear Energy Institute NCV Non-Cited Violation NRC U.S. Nuclear Regulatory Commission PC Pool Cooling PI Performance Indicator PK Class 1E 125 Vdc System PM Preventative Maintenance PVAR Palo Verde Action Request PVNGS Palo Verde Nuclear Generating Station QA Quality Assurance QC Quality Control RCS Reactor Coolant System RO Reactor Operators RPCS Reactor Power Cutback System RWT Refueling Water Tank SBC Steam Bypass Control SFP Spent Fuel Pool SIBP Site Integrated Improvement Plan SIIP Site Integrated Improvement Plan SM Shift Manager SSC Structures, Systems, and Components SG Steam Generator SRO Senior Reactor Operators TLI Turbine Load Index TS Technical Specification UFSAR Updated Final Safety Analysis Report
- 30 - Enclosure
VTD Vendor Technical Document WO Work Order
- 31 - Enclosure
}}