IR 05000275/2003007
| ML033110669 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon  |
| Issue date: | 11/07/2003 |
| From: | William Jones NRC/RGN-IV/DRP |
| To: | Rueger G Pacific Gas & Electric Co |
| References | |
| FOIA/PA-2011-0221 IR-03-007 | |
| Download: ML033110669 (36) | |
Text
ber 7, 2003
SUBJECT:
DIABLO CANYON POWER PLANT - NRC INTEGRATED INSPECTION REPORT 05000275/2003007 AND 05000323/2003007
Dear Mr. Rueger:
On September 27, 2003, the U.S. Nuclear Regulatory Commission completed an inspection at your Diablo Canyon Power Plant, Units 1 and 2, facility. The enclosed integrated report documents the inspection findings that were discussed on October 3, 2003, with Mr. James R. Becker and members of your staff.
This inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.
Within these areas, the inspection consisted of selected examination of procedures and representative records, observations of activities, and interviews with personnel.
Based on the results of this inspection, the NRC has identified four issues that were evaluated under the risk significance determination process as having very low safety significance (Green). The NRC has also determined that violations are associated with these issues.
These violations are being treated as noncited violations (NCVs), consistent with Section VI.A of the Enforcement Policy. These NCVs are described in the subject inspection report. If you contest the violations or significance of these NCVs, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission, Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at the Diablo Canyon Power Plant.
Pacific Gas and Electric Company operated under voluntary bankruptcy proceedings during this inspection period. The NRC has monitored plant operations, maintenance, and planning to better understand the impact of the financial situation and how it relates to your responsibility to safely operate the Diablo Canyon reactors. NRC inspections, to date, have confirmed that you are operating these reactors safely and that public health and safety is assured.
Pacific Gas and Electric Company -2-In accordance with 10 CFR 2.790 of the NRCs "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/
William B. Jones, Chief Project Branch E Division of Reactor Projects Dockets: 50-275 50-323 Licenses: DPR-80 DPR-82
Enclosure:
Inspection Report 05000275/2003007 and 05000323/2003007 w/attachment: Supplemental Information
REGION IV==
Dockets: 50-275, 50-323 Licenses: DPR-80, DPR-82 Report: 05000275/2003007 05000323/2003007 Licensee: Pacific Gas and Electric Company Facility: Diablo Canyon Power Plant, Units 1 and 2 Location: 7 1/2 miles NW of Avila Beach Avila Beach, California Dates: June 29 through September 27, 2003 Inspectors: D. L. Proulx, Senior Resident Inspector T. W. Jackson, Resident Inspector J. F. Melfi, Resident Inspector, Palo Verde D. E. Dumbacher, Reactor Engineer L. Ellershaw, Senior Reactor Inspector Approved By: W. B. Jones, Chief, Project Branch E Division of Reactor Projects Enclosure
SUMMARY OF FINDINGS
IR 05000275/2003-007, 05000323/2003-007; 06/29/03 - 09/27/03; Diablo Canyon Power Plant
Units 1 and 2; Equipment Alignment, Personnel Performance Related to Nonroutine Plant Evolutions and Events, Problem Identification and Resolution, and Event Followup.
This report covered a 13-week period of inspection by resident inspectors a regional inspector and an announced inspection by a resident inspector. The NRC identified four Green noncited violations. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609 Significance Determination Process. Findings for which the Significance Determination Process does not apply may be Green or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 3, dated July 2000.
NRC-Identified and Self-Revealing Findings
Cornerstone: Mitigating Systems
- Green.
The inspectors identified a noncited violation of 10 CFR Part 50,
Appendix B, Criterion V for Pacific Gas and Electric Companys failure to utilize acceptance criteria and instructions for Diesel Engine Generator 2-2 auto-voltage regulator card inspection. Specifically, instructions were not utilized for the inspection of the voltage regulating card and during the inspection a degraded solder joint was not identified. Subsequently, an NRC inspector noted the degraded solder joint prior to the voltage regulator card being returned to service.
The finding impacted the mitigating systems cornerstone and was more than minor when assessed using Inspection Manual Chapter 0612, Appendix E.
Similar to Example 4.a, inspection of the Diesel Engine Generator 2-2 voltage regulator card revealed degraded solder joints similar to those identified on Diesel Engine Generator 1-3. This condition was determined to be the apparent cause for the Diesel Engine Generator 1-3 slow voltage rise times (greater than the Technical Specification when the diesel engine was started). Using the Significance Determination Process Phase 1 Worksheet in Inspection Manual Chapter 0609, the finding is of very low safety significance since there was no loss of an actual safety function, no loss of a safety-related train for greater than the diesel engine generator Technical Specification allowed outage time, and the finding is not potentially risk significant due to a seismic, fire, flooding, or severe weather initiating event (Section 1R04).
- Green.
The inspectors identified a noncited violation of 10 CFR Part 50,
Appendix B, Criterion XVI for failure to promptly identify and correct lube oil carbonization in two diesel engine generators. This failure resulted in increased lube oil filter differential pressures and partial clogging of pre-circulation lube oil lines for Diesel Engine Generators 1-3 and 2-2.
The finding impacted a mitigating system cornerstone and was more than minor when assessed using Inspection Manual Chapter 0612, Appendix E. In
Example 3.b, a discrepancy between an actual condition and the design was more than minor if the operation of the system was adversely affected. With respect to this finding, the carbonized oil clogged the pre-circulation lube oil line resulting in Diesel Engine Generator 1-3 unavailability. The finding is of very low safety significance since there was no loss of an actual safety function, no loss of a safety-related train for greater than the diesel engine generator Technical Specification allowed outage time and the finding is not potentially risk significant due to a seismic, fire, flooding, or severe weather initiating event (Section 1R15).
- Green.
The inspectors identified a noncited violation of 10 CFR Part 50,
Appendix B, Criterion XVI for the failure to promptly identify and correct a degraded mechanical governor on Diesel Engine Generator 2-2. This failure caused the degraded governor to remain in service for over 6 months.
The finding impacted the mitigating systems cornerstone and was more than minor when assessed using Inspection Manual Chapter 0612, Appendix E. In Example 4.g, the failure to correct a condition adverse to quality was more than minor because the degraded governor had more than a minor impact on safety and continued operation could impact the diesel engine generators ability to provide emergency power. Using the Significance Determination Process Phase I Worksheet in Inspection Manual Chapter 0609, the finding is determined to be of very low safety significance. The inspectors determined that there was no loss of an actual safety function, no loss of a safety-related train for greater than the diesel engine generator Technical Specification allowed outage time and the finding is not potentially risk significant due to a seismic, fire, flooding, or severe weather initiating event (Section 4OA2).
- Green.
The inspectors identified two examples of a noncited violation of Technical Specification Section 5.4.1.d, for the failure to establish, implement, and maintain adequate procedures covering fire protection program implementation.
The first example involved Pacific Gas and Electric Companys failure to adequately implement fire protection program requirements specified in Calculation M-944 10 CFR 50 Appendix R, Alternate Shutdown Methodology Time and Manpower Study/Safe Shutdown System Considerations.
Specifically, in a control room fire scenario requiring control room evacuation and remote shutdown, operators failed to complete actions required for achieving safe shutdown specified in Procedure OP AP-8A, "Control Room Inaccessibility Hot Standby," within the times assumed in Calculation M-944.
This finding was greater than minor because it impacted the mitigating systems cornerstone and adversely affected the ability of Pacific Gas and Electric Company to manually operate certain components required for safe shutdown within the analyzed times. Specifically, in a simulated field walkdown, operators were not able to establish auxiliary feedwater within 30 minutes as required by analysis nor close a stuck open power operated relief valve within 5 minutes.
The inspectors used Appendix F of Manual Chapter 0609 and determined that the inability to perform the safe shutdown procedures required a Phase 2 and Phase 3 analysis in the significance determination process. Further analysis of the ignition frequencies and the potential heatup of the core in this degraded condition, revealed that this finding was of very low safety significance (Section 4OA3).
The second example involved Pacific Gas and Electric Companys failure to adequately implement fire protection program requirements for a fire in the control room requiring control room evacuation and remote shutdown.
Specifically, Pacific Gas and Electric Company failed to provide adequate information in Procedure OP AP-8A, "Control Room Inaccessibility Hot Standby," or on the Unit 2 hot shutdown panel concerning the correct hot shutdown panel switch positions of certain components required for safe shutdown.
Consequently, in performing a walkdown of Procedure OP AP-8A, operators failed to transfer control of the auxiliary feedwater throttle valves and steam generator atmospheric dump valves from the control room to the hot shutdown panel.
This finding was greater than minor because it impacted the mitigating systems cornerstone and adversely affected the ability of the operators to take control of certain components required for safe shutdown. Specifically, information identifying the correct hot shutdown panel switch positions for the auxiliary feedwater throttle valves and steam generator atmospheric dump valves were not provided to the operators. During a control room fire requiring use of the remote shutdown, if the component switches were not placed in the correct positions, the throttle and dump valves would have remained vulnerable to fire damage that could cause spurious operation. The inspectors used Appendix F of Manual Chapter 0609 and determined that the inability to perform the safe shutdown procedures required a Phase 2 and Phase 3 analysis in the significance determination process. Further analysis of the ignition frequencies and the potential heatup of the core in this degraded condition, revealed that this finding was of very low safety significance (Section 4OA3).
Licensee-Identified Violations
None.
REPORT DETAILS
Summary of Plant Status
Diablo Canyon Unit 1 began this inspection period at 50 percent power to support cleaning of the circulating water tunnels. On June 30, 2003, operators increased Unit 1 reactor power and achieved 100 percent power on July 1. Unit 1 remained at 100 percent power through the end of the inspection period.
Diablo Canyon Unit 2 began this inspection period at 100 percent power and remained at that power level through the end of the inspection period.
REACTOR SAFETY
Cornerstones: Initiating Events, Mitigating Systems, and Barrier Integrity
1R01 Adverse Weather
a. Inspection Scope
The inspectors performed reviews of the design features, equipment, and plant preparation for protecting mitigating systems from the adverse effects of Pacific Ocean storms. These storms, in conjunction with kelp and other aquatic plants, can obstruct the intake traveling screens, causing a trip of the circulating water pumps and loss of the main condenser. In the past, these storms, also known as kelp attacks, have caused the shutdown of both units within a short period of time due to a loss of the normal heat sink. Three sample reviews of the traveling screen, screen refuse, and circulating water systems were conducted to support the scope of this inspection activity. In addition, the inspectors reviewed the lesson plans for operator simulator/classroom training to combat the effects of Pacific Ocean swells.
.1 Units 1 and 2 Bar Rack/Traveling Screen System
a. Inspection Scope
Bar racks and traveling screens prevent solid material (kelp, trash, etc.) from being ingested into the pumps located at the intake. The inspectors walked down the bar racks and traveling screens at the intake structure to observe their operational readiness. During the walkdown, the inspectors verified the condition of mechanical and electrical components, lubrication, and the functionality of support systems. Additionally, the inspectors reviewed corrective action documents related to the traveling screen system, to verify that appropriate corrective maintenance was being pursued. The inspectors used the following documentation:
- Work Order R0249277, "Unit 1 Traveling Screen Preventative Maintenance"
- Action Request (AR) A0566580, "Provide Redundant Screen Start Circuit for Pressure Switch PS-168"
- Procedure AR PK13-01, Bar Racks/Screens (Unit 1), Revision 18
- Procedure AR PK13-01, Bar Racks/Screens (Unit 2), Revision 7
- Drawing OVID 106717, Saltwater System,
- Sheet 7, Revision 119
- Sheet 7A, Revision 116
b. Findings
No findings of significance were identified.
.2 Units 1 and 2 Screen Refuse System
a. Inspection Scope
The screen refuse system pumps kelp and other material that has been washed from the traveling screens back to the ocean, at a location away from the intake. The inspectors walked down the screen refuse system at the intake structure to observe their operational readiness. During the walkdown, the inspectors verified the condition of mechanical and electrical components.
b. Findings
No findings of significance were identified.
.3 Units 1 and 2 Circulating Water System
a. Inspection Scope
The circulating water system pumps cooling water from the Pacific Ocean to the main condensers. Each main condenser is supplied by two circulating water pumps. The inspectors walked down the circulating water pumps and reviewed the corrective action program for any outstanding corrective maintenance that might impact the circulating water system during Pacific Ocean storms.
b. Findings
No findings of significance were identified.
1R04 Equipment Alignments
Partial System Walkdowns This inspection performed two partial system walkdowns.
.1 Unit 2 Diesel Engine Generator (DEG) Automatic Voltage Regulator Card (AVR)
Inspections
a. Inspection Scope
On July 9, 2003, Pacific Gas and Electric Company initiated actions to inspect the AVR cards for all DEGs, with the exception of DEG 1-3 which had been previously replaced.
Maintenance and engineering staff were verifying the absence of cracked solder joints on the DEGs AVR card in response to cracked solder joints that caused DEG 1-3 to have a slow voltage rise time. The inspectors observed the AVR cards for poor solder joints and any other condition that could affect DEG voltage.
b. Findings
Introduction.
A Green noncited violation was identified by the inspectors for the failure to accomplish DEG AVR card inspections in accordance with procedures, as required by 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings.
Specifically, Pacific Gas and Electric Company did not utilize the acceptance criteria and instructions referenced in the work order to identify degraded solder joints on the DEG 2-2 AVR card. Degraded solder joints on AVR cards has led to slow voltage rise times on the other DEGs.
Description.
On February 23, 2003, DEG 1-3 experienced a slow voltage rise time of 53 seconds as compared to the Technical Specification required rise time of 13 seconds. Following an in-depth troubleshooting process, Pacific Gas and Electric Company narrowed the probable cause of the slow voltage rise time to the AVR card.
Pacific Gas and Electric Company sent the AVR card to MPR Associates to perform additional testing. As described in MPR Report, Failure Analysis of Basler Circuit Card Model 32101-109, dated May 2, 2003, the laboratory could not re-create a failure in the AVR card, however, they did identify several cold solder joints associated with a magnetic amplifier on the circuit card. Approximately two months later, MPR Associates confirmed through further board tests that cold solder joints associated with the amplifier could cause a slow voltage rise time.
The inspectors followed the troubleshooting activities and technical analysis related to DEG 1-3. The inspection activities and findings associated with DEG 1-3 are documented in NRC Inspection Report 05000275; 323/2003-006, issued on July 28, 2003. The inspectors questioned the extent of condition of the degraded solder joints found in DEG 1-3 relative to the other five DEGs, which had AVR cards of similar age and design. Subsequently, Pacific Gas and Electric Company initiated AVR card inspections beginning July 9, 2003.
On August 1, 2003, Pacific Gas and Electric Company inspected the AVR card associated with DEG 2-2. Following the inspection, the inspectors reviewed the card for degraded solder joints. The inspectors noted one solder joint that appeared lifted from the board. Pacific Gas and Electric Company had noticed some resin on the solder joint, but did not identify the gap between the solder joint and the board. The inspectors also questioned whether the solder joints for the magnetic amplifier had minute cracks that appeared as a hair-line circle around the center of the solder joint. Upon recognizing the gap between the solder joint and the board, Pacific Gas and Electric Company initiated actions to remove the board, repair it, and place it back into service.
Subsequent solder work on the board confirmed that the one questioned solder joint had broken away from the board. Although it had an electrical connection, the joint would most likely fail later in life. It was also identified during solder work that the solder joints for the magnetic amplifier were cold solder joints, similar to those found on the DEG 1-3 AVR card.
Instructions for the DEG AVR card inspections were provided in Work Order C0183306, which referenced Procedure MP I-2.17-1, Guidelines for Printed Circuit Board Repair and Modification, Revision 1, as additional guidance. The work order stated that if any solder joints are found or suspected to be degraded, determine and remove the AVR card. Procedure MP I-2.17-1 stated acceptance criteria for solder joints to be
- (1) no cold solder joints,
- (2) no flux or residue left on the joints, and
- (3) free of projections, bridging, fractures, voids and porosity. The solder joints noted by the inspectors did not meet the acceptance criteria of Procedure MP I-2.17-1 since one solder joint was fractured from the board and the solder joints at the magnetic amplifier were cold solder joints. The inspectors determined that Pacific Gas and Electric Company would not have identified the degraded solder joints outside of the inspectors questioning the condition of those joints. Following the identification of the degraded solder joints on DEG 2-2 AVR card, Pacific Gas and Electric Company initiated AR A0587851 to pull the AVR cards from previously inspected DEGs and inspect/repair those cards in the instrumentation and control shop. Minor solder joint repairs were made to these cards; however, no cold solder joints were found. None of the solder joints that were repaired, including those on DEG 2-2 AVR card, would have prevented the DEGs from performing their safety function.
Analysis.
The performance deficiency associated with this finding is a failure to follow instructions and procedures for determining degraded solder joints on DEG AVR cards.
The finding impacted the mitigating systems cornerstone and was more than minor when assessed using Inspection Manual Chapter 0612, Appendix E, Example 4.a.
Similar to Example 4.a, the subsequent solder work on the DEG 2-2 card revealed degraded solder joints similar to those on DEG 1-3, which was the apparent cause for the DEG 1-3 slow voltage rise time. Using the Significance Determination Process (SDP) Phase I Worksheet in Inspection Manual Chapter 0609, the inspectors determined that there was no loss of an actual safety function, no loss of a safety-related train for greater than the DEG Technical Specification allowed outage time, and the finding is not potentially risk significant due to a seismic, fire, flooding, or severe weather initiating event. Therefore, using the SDP Phase I Worksheet, the finding is determined to be of very low safety significance.
Enforcement.
10 CFR Part 50, Appendix B, Criterion V states, in part, that activities affecting quality shall be prescribed by documented instructions, and the instructions shall include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily accomplished. The activities shall be accomplished in accordance with these instructions, procedures, or drawings. Contrary to the above, Pacific Gas and Electric Company failed to identify degraded solder joints on the DEG 2-2 AVR card as described by the given instructions and acceptance criteria. Outside of NRC inspector questioning, the degraded AVR card would have been left in service. Because the failure to accomplish the inspections using appropriate instructions and acceptance criteria was determined to be of very low safety significance and has been entered into the corrective action program as AR A0587851, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy: NCV 50-323/03-07-01, Failure to Follow Instructions and Acceptance Criteria During Diesel Engine Generator Automatic Voltage Regulator Card Inspections.
.2 Unit 2 Atmospheric Dump Valves
a. Inspection Scope
On September 9, 2003, while Atmospheric Dump Valve MS-2-PCV-22 was being calibrated, the inspectors performed a partial system walkdown of the remaining Unit 2 atmospheric dump valves. The inspectors observed valve alignment, material condition of air supplies, labeling, lubrication, and structural support. The inspectors used Procedure OP C-2:I, Main Steam and Steam Dump Systems - Alignment Verification for Plant Startup, Revision 19, for reference during the inspection.
b. Findings
No findings of significance were identified.
1R05 Fire Protection
.1 Routine Observations
a. Inspection Scope
The inspectors performed eight fire protection walkdowns to assess the material condition of plant fire detection and suppression, fire seal operability, and proper control of transient combustibles. The inspectors used Section 9.5 of the Final Safety Analysis Report (FSAR) Update as guidance. The inspectors considered whether the suppression equipment and fire doors complied with regulatory requirements and conditions specified in Procedures STP M-69A, Monthly Fire Extinguisher Inspection, Revision 33, STP M-69B, Monthly CO2 Hose Reel and Deluge Valve Inspection, Revision 14, STP M-70C, Inspection/Maintenance of Doors, Revision 8, and OM8.ID4, Control of Flammable and Combustible Materials, Revision 10. Specific risk-significant areas inspected included:
- Units 1 and 2 Diesel Generator Rooms in Turbine Building
- Units 1 and 2 Radiologically Controlled Areas of Auxiliary Building
- Units 1 and 2 Switchgear Rooms of Auxiliary Building
- Units 1 and 2 Intake Structure
b. Findings
No findings of significance were identified.
1R11 Licensed Operator Requalification
a. Inspection Scope
The inspectors witnessed operator performance in the simulator during routine training and requalification examinations. The inspectors verified the crews ability to meet the objectives of the training scenario, and attended the post-scenario critique to verify that crew weaknesses were identified and corrected by Pacific Gas and Electric Company.
On September 24, 2003, the inspectors observed one simulator scenario associated with a loss-of-coolant accident, transfer to cold leg recirculation, and clogging of the emergency core cooling sump.
The inspectors used Procedures AP-17, EOP E-0, Reactor Trip or Safety Injection, Revision 27, EOP E-1, Loss of Reactor or Secondary Coolant, Revision 18, E1.3 "Transfer to Cold Leg Recirculation, Revision 19, and ECA 1.1 "Loss of Emergency Cooling Recirculation," Revision 15, to support the inspection activities.
b. Findings
No findings of significance were identified.
1R12 Maintenance Effectiveness
a. Inspection Scope
The inspectors reviewed Pacific Gas and Electric Companys Maintenance Rule implementation for two equipment performance problems. The inspectors assessed whether the equipment was properly placed into the scope of the rule, whether the failures were properly characterized, and whether goal setting was recommended, if required. Procedure MA1.ID17, Maintenance Rule Monitoring Program, Revision 10, was used as guidance. The inspectors reviewed the following action requests:
- A0588886, Maintenance Rule Quarterly Report Recommendations
- A0587463, Maintenance Rule Performance Criteria, Goal Setting Review for Auxiliary Saltwater Pumps
b. Findings
No findings of significance were identified.
1R13 Maintenance Risk Assessments and Emergent Work Control
.1 Risk Assessments
a. Inspection Scope
The inspectors reviewed daily work schedules and compensatory measures to confirm that Pacific Gas and Electric Company had performed proper risk management for routine work. The inspectors considered whether risk assessments were performed according to their procedures and whether Pacific Gas and Electric Company had properly used their risk categories, preservation of key safety functions, and implementation of work controls. The inspectors used Procedure AD7.DC6, On-line Maintenance Risk Management, Revision 7, as guidance. The inspectors specifically observed the following five work activities during the inspection period:
- Unit 1, Auxiliary Saltwater Pump 1-2 maintenance outage window and a Control Room Ventilation System damper cover installation on July 8
- Unit 1, Auxiliary Saltwater Pump 1-1 maintenance outage window coincident with a California Independent System Operator Restricted Generator Maintenance Notice and maintenance on a 230 kV transmission line from the Morro Bay Power Plant to the Midway switchyard on July 15
- Unit 1, Residual Heat Removal Pump 1-2 maintenance outage window and the trip risk associated with lightning strikes near the plant on July 30
- Unit 2, Startup power crosstied with Unit 1 and DEG 2-1 Automatic Voltage Regulator Card inspection on August 6
- Unit 1, Centrifugal Charging Pump 1-1 seal replacement, using one-time special 7-day completion time
b. Findings
No findings of significance were identified.
.2 Emergent Work
a. Inspection Scope
The inspectors observed emergent work activities to verify that actions were taken to minimize the probability of initiating events, maintain the functional capability of mitigating systems, and maintain barrier integrity. The scope of work activities reviewed includes troubleshooting, work planning, plant conditions and equipment alignment, tagging and clearances, and temporary modifications. The following two activities were observed during this inspection period:
- Unit 2, Pinhole leak on Centrifugal Charging Pump 2-2 discharge line on July 13-14, 2003
- Unit 1, Leak from Pressurizer Safety Valve 8010B Loop Seal Drain to the Pressurizer Relief Tank and possible installation of a new valve on the drain line, July 17-18, 2003
b. Findings
No findings of significance were identified.
1R15 Operability Evaluations
a. Inspection Scope
The inspectors reviewed operability evaluations and supporting documents to determine if the associated systems could meet their intended safety functions despite the degraded status. The inspectors reviewed the applicable Technical Specifications, Codes/Standards, and FSAR Update sections in support of this inspection. The inspectors reviewed the following 10 ARs and operability evaluations:
- Unit 1, Diesel Engine Generator (DEG) 1-3 and DEG 2-2 low lube oil pressure (ARs A0584297 and A0589167)
- Unit 1, DEG 1-2 friction plate from exhaust support missing (AR A0576937)
- Unit 1, Evaluate Unit 1 Core Thermocouple Indications (AR A0590156)
- Unit 2, Degraded Battery Charger Solder Connections (AR A0588992)
- Unit 1, Evaluate Reactor Coolant System Boundaries versus Pressure Isolation Valve (PIV) Locations (AR A0588974)
- Units 1 and 2, Postulated Rupture Locations in Piping Systems (AR A0588546)
- Unit 2, Breaker 52-PY-2229 Failed to Deenergize Upon Manual Opening (AR A0590196)
- Unit 2, Delay in Backup Air Bottle for Valve PCV-22 Opening (A0590110)
- Units 1 and 2, Environmental Qualification of Containment Fan Cooler Unit Power Cables (AR A0590124)
- Unit 1, Auxiliary Feedwater Level Control Valve Area Temperature Exceeded 103 degrees F (AR A0588990)
b. Findings
.1 Diesel Engine Generator Lube Oil Coking
Introduction.
A Green noncited violation was identified by the inspectors for the failure to promptly identify and correct a condition adverse to quality, in accordance with 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Actions. Specifically, Pacific Gas and Electric Company recognized carbonized oil depositing on the DEG lube oil filter in previous refueling outages but failed to address the nonconforming condition.
This failure resulted in increased amounts of carbonized oil particles collecting in the lube oil filter and clogging the pre-circulation lube oil lines.
Description.
Diablo Canyon Power Plant, Units 1 and 2, utilize three DEGs per unit as a source of emergency power. The lube oil for the DEGs is maintained above 90EF by an electric pre-circulating lube oil pump and an electric lube oil heater. Once the DEGs are running, the pre-circulating lube oil pump and heater are bypassed and the lube oil is circulated by a shaft-driven lube oil pump. The pre-circulating lube oil pump is constantly running in standby and the heater is controlled by a thermostat. While the DEG is in standby, the lube oil travels, in order, from the lube oil reservoir, through the pre-circulating lube oil pump, the electric heater, the lube oil filter, the lube oil heat exchanger, lube oil strainers, and engine components and back to the lube oil reservoir.
As discussed in AR A0353620, the DEG lube oil heaters were inspected and carbonized oil debris was found accumulated on the heater elements during Refueling Outage 2R6.
Pacific Gas and Electric Company determined that the carbonized oil debris was created by the high temperature of heater elements and replaced the Unit 1and 2 DEG heater elements in Refueling Outages 1R8 and 2R8, respectively. The old heater had six elements with a total heat output of 12 kW, and the new heater had nine elements while maintaining the total heat output of 12 kW. The increased surface area for heat transfer reduced the carbonization of the lube oil. The heater element change was documented in Replacement or New Part Evaluation E-8521, Revision 0. Pacific Gas and Electric Company reported that the heater element change reduced the amount of carbonized oil. However, in the 9th through 11th cycle refueling outages, Pacific Gas and Electric Company reported that carbonized oil was found in the lube oil filter and on the pre-circulating electric heater.
On May 19, 2003, operators noticed that the standby lube oil pressure for DEG 1-3 was 9.5 psig when it is required to be at least 10 psig. Maintenance technicians adjusted the pressure on the pre-circulating lube oil pump to maintain 10 psig. On June 7 operators noticed that the pre-circulating lube oil pressure was 9 psig and that the lube oil filter differential pressure was 5 psid. The differential lube oil pressure for the other DEGs ranged approximately from 0 to 3 psig. On June 10 the lube oil pressure dropped to 7 psig and maintenance technicians attempted to adjust the pre-circulating lube oil pump pressure without success. Suspecting a problem with the pre-circulating lube oil pump, Pacific Gas and Electric Company replaced the pump on June 16. Following the lube oil pump replacement, the lube oil pressure did not change. The following day, a troubleshooting team was organized to identify the cause of the lower pre-circulating lube oil pressure. The troubleshooting team determined that the lube oil line was restricted somewhere downstream of the electric heater. On June 25 maintenance technicians discovered that the flow path through Flow Switch FS-46 was blocked by granulated chunks of carbonized oil from the electric heater. This restriction decreased the flow into the crankcase oil galleries. The heater elements were removed and found to have carbonized oil deposits over 25 percent of the element surface area. Pacific Gas and Electric Company determined that this was half of what they would normally see at the end of an operating cycle. Upon finding the presence of carbonized oil deposits, the inspectors questioned Pacific Gas and Electric Company concerning the operability of the other DEGs. Pacific Gas and Electric Company considered the DEGs operable since the standby lube oil temperature was above 90EF and did not feel that a prompt operability assessment, per Procedure OM7.ID12, Operability Determination, Revision 7A, was required.
On August 25, 2003, operators noticed that the standby lube oil pressure for DEG 2-2 was 9 psig when it is required to be at least 10 psig. Pacific Gas and Electric Company determined that the decreased standby lube oil pressure was attributed to carbonized lube oil partially clogging the pre-circulation lube oil line. The inspectors again questioned Pacific Gas and Electric Company concerning the operability of the DEGs with respect to the carbonized lube oil deposits. Subsequently, a prompt operability assessment was completed on August 29 and documented in AR A0589167.
The cause of the carbonized oil deposits was attributed to the design and the amount of heat the electric lube oil heater generated. The inspectors requested DEG design information that would support a certain amount of carbonized lube oil in the lube oil system. However, Pacific Gas and Electric Company stated that there was no design information that considered carbonized lube oil in the lube oil system. Additionally, Pacific Gas and Electric Company stated that the DEG vendor, Fairbanks-Morse, would only provide limited support with this issue since it was a problem they had rarely, if ever, encountered.
The inspectors determined that Pacific Gas and Electric Company did not promptly identify and correct the deposition of carbonized lube oil in the DEG lube oil system.
Specifically, Pacific Gas and Electric Company observed carbonized lube oil on the pre-circulation electric heater elements and the lube oil filter during the 9th through 11th cycle refueling outages, but did not take corrective action. Additionally, the inspectors determined that Pacific Gas and Electric Company did not promptly address the operability of the other five DEGs when the cause of DEG 1-3 low standby lube oil pressure was identified on June 25, 2003. Using the guidance in Generic Letter 91-18, Revision 1, Information to Licensees Regarding NRC Inspection Manual Section on Resolution of Degraded and Nonconforming Conditions, the inspectors determined the deposition of carbonized lube oil in the DEG lube oil system was a nonconforming condition since operating experience demonstrated a design inadequacy. Pacific Gas and Electric Company assumed that operability of the DEGs was not impacted since the standby lube oil temperature had not fallen below 90EF. However, Pacific Gas and Electric Company did not consider a technical basis for why the carbonized lube oil could not impact lube oil flow thorough the lube oil filter, the potential for crushing the lube oil filter due to a high differential pressure across it as a result of carbonized lube oil deposits, and the potential to transfer carbonized lube oil to engine parts if the lube oil bypass valve is actuated. Pacific Gas and Electric Company subsequently addressed these issues in a prompt operability assessment. The inspectors concluded that the DEGs were operable per the prompt operability assessment discussed in AR A0589167.
Pacific Gas and Electric Companys short-term corrective actions included inspections of the pre-circulation lube oil line for blockage on the remaining DEGs. Long-term corrective actions will be determined through Quality Evaluation Q0012333.
Analysis.
The performance deficiency associated with this finding is the failure to promptly identify and correct a nonconforming condition, which resulted in a blocked lube oil line and carbonized oil accumulating on the lube oil filters. The finding impacted the mitigating systems cornerstone and was more than minor when assessed using Inspection Manual Chapter 0612, Appendix E. In Example 3.b, a discrepancy between an actual condition and the design was more than minor if the operation of the system was adversely affected. With respect to this finding, the carbonized oil clogged the pre-circulation lube oil line requiring DEG 1-3 unavailability to clean the line. Additionally, the carbonized lube oil caused an increase in lube oil filter differential pressure, but not to the point of causing the DEG to be inoperable. Using the SDP Phase I Worksheet in Inspection Manual Chapter 0609, the inspectors determined that there was no loss of an actual safety function, no loss of a safety-related train for greater than the DEG Technical Specification allowed outage time, and the finding is not potentially risk significant due to a seismic, fire, flooding, or severe weather initiating event. Therefore, using the SDP Phase I Worksheet, the finding is determined to be of very low safety significance.
Enforcement.
10 CFR Part 50, Appendix B, Criterion XVI states, in part, that conditions adverse to quality, such as nonconformances, are to be promptly identified and corrected. Contrary to the above, Pacific Gas and Electric Company failed to evaluate and correct the formation of carbonized lube oil that partially blocked a pre-circulation lube oil line and increased the lube oil filter differential pressure for DEGs 1-3 and 2-2.
Because the failure to promptly identify and correct this nonconformance was determined to be of very low safety significance and has been entered into the corrective action program as Quality Evaluation Q0012333, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy:
NCV 50-275/03-07-02, Failure to Promptly Identify and Correct Diesel Engine Generator Lube Oil Carbonization.
1R16 Operator Workarounds
a. Inspection Scope
(71111.16)
The inspectors reviewed Pacific Gas and Electric Companys documented actions in which degraded conditions or changes to accident analyses required additional operator action beyond that credited in the design basis to compensate for these conditions.
Pacific Gas and Electric Company tracked two types of conditions: operator burdens and operator workarounds.
Pacific Gas and Electric Company defined an operator burden as a manual action taken to compensate for degraded equipment that affected normal operation of a unit. Pacific Gas and Electric Company had 17 operator burdens.
Pacific Gas and Electric Company defined an operator workaround as a manual action taken to compensate for a degraded condition required for response to abnormal or emergency operating procedures. Pacific Gas and Electric Company had 17 active operator workarounds. The inspectors assessed the cumulative affect of the operator workarounds to determine if operators would be overly taxed with working around numerous degraded conditions that would complicate an abnormal or emergency condition.
The inspectors reviewed Pacific Gas and Electric Companys program for tracking the operator workarounds and restoring the applicable systems to full qualification, to determine if Pacific Gas and Electric Company appropriately managed these items.
None of the operator workarounds involved risk-significant actions.
b. Findings
No findings of significance were identified.
1R19 Postmaintenance Testing
a. Inspection Scope
The inspectors reviewed postmaintenance tests for selected risk-significant systems to verify their operability and functional capability. As part of the inspection process, the inspectors witnessed and/or reviewed the postmaintenance test acceptance criteria and results. The test acceptance criteria was compared to the Technical Specifications and the FSAR Update. Additionally, the inspectors verified that the test was adequate for the scope of work, the test was performed as prescribed, jumpers and test equipment were properly removed after the test, and test equipment range, accuracy, and calibration were consistent for the application. The following five corrective maintenance activities were reviewed by the inspectors:
- Unit 1, Nuclear Instrument Channel NI-41 deviations in delta current on June 29 (AR A0585719)
- Unit 1, Accumulator 1-4 Pressure Channel PI-967 power supply repair on July 9 (AR A0586143)
- Unit 2, Eagle 21 Loop Calculation Processor Non-Volatile Read-Only Memory replacement on July 10 (Work Order R0218775)
- Unit 2, Charging Pump 2-1 outlet pressure valve to PI-142D repair on July 13-14 (AR A0586663, A0586677, A0586678)
- Unit 2, Safety Injection Pump 2-1 panel meter and motor protection relay calibration on September 9 (Work Orders R0148806 and R0198142)
b. Findings
No findings of significance were identified.
1R22 Surveillance Testing
a. Inspection Scope
The inspectors evaluated six routine surveillance tests to determine if Pacific Gas and Electric Company complied with the applicable Technical Specification requirements to demonstrate that equipment was capable of performing its intended safety functions and operational readiness. The inspectors performed a technical review of the procedure, witnessed portions of the surveillance test, and reviewed the completed test data. The inspectors also considered whether proper test equipment was utilized, preconditioning occurred, test acceptance criteria agreed with the equipment design basis, and equipment was returned to normal alignment following the test. The following tests were evaluated during the inspection period:
- Procedure STP P-CCP-21, "Routine Surveillance Test of Centrifugal Charging Pump 2-1," Revision 14, on July 14 for Unit 2
- Procedure STP P-AFW-12, Routine Surveillance Test of Motor-driven Auxiliary Feedwater Pump 1-2, Revision 10, on August 26 for Unit 1
- Procedure STP V-3M1B, Exercising Valve RHR-FCV-641B RHR Pump 2 Recirculation Valve, Revision 3, on August 27 for Unit 2
- Procedure STP P-RHR-22, "Routine Surveillance Test of RHR Pump 2-2, Revision 11A, on August 27 for Unit 2
- Procedure STP P-SIP-21, Routine Surveillance Test of Safety Injection Pump 2-1, Revision 15, on September 10 for Unit 2
b. Findings
No findings of significance were identified.
CORNERSTONE: Emergency Preparedness
1EP6 Emergency Preparedness Evaluation
a. Inspection Scope
The inspectors witnessed the operator simulator session that included emergency plan implementation conducted on July 17, 2003. The scenario simulated sabotage on the facility, coupled with a loss of offsite power and damage to safe shutdown plant components. The scenario continued with a large loss of reactor coolant, subsequent damage to fission product barriers, core damage and a radiological release to the environment to demonstrate Pacific Gas and Electric Company's capabilities to implement the emergency plan. This resulted in declaration of a Notification of Unusual Event and notification of state and local authorities, declaration of an Alert and Site Area Emergency. The inspectors witnessed the operator performance in the control room (i.e., simulator), and from the Technical Support Center. The inspectors also attended Pacific Gas and Electric Companys self-critique of the scenario. The following procedures were used to evaluate the performance:
- EP G-1, "Emergency Classification and Emergency Plan Activation," Revision 32
- EP G-2, "Interim Emergency Response Organization," Revision 26
- EP G-3, "Notification of Off-Site Agencies and Emergency Response Organization Personnel," Revision 40
- EP EF-1, "Activation and Operation of the Technical Support Center,"
Revision 30.
The inspectors evaluated performance by focusing on the risk-significant activities of classification and notification. No protective action recommendations were required by this drill. The inspectors also assessed personnel recognition of abnormal plant conditions, the transfer of emergency responsibilities between facilities, communications, and the overall implementation of the emergency plan.
b. Findings
No findings of significance were identified.
OTHER ACTIVITIES
4OA1 Performance Indicator Verification
.1 Reactor Safety Performance Indicator Verification
a. Inspection Scope
The inspectors reviewed the following 3 performance indicators for the period from the second quarter of 2002 through the first quarter of 2003 to assess the accuracy and completeness of the indicator. The inspectors reviewed plant operating logs and Pacific Gas and Electric Companys monthly operating reports to support this inspection. The inspectors used NEI 99-02, Regulatory Assessment Performance Indicator Verification, Revision 2, as guidance for this inspection.
- Safety System Failures (Units 1 and 2)
- Reactor Coolant System Identified Leak Rate (Units 1 and 2)
- Reactor Coolant System Activity (Units 1 and 2)
b. Findings
No findings of significance were identified.
4OA2 Identification and Resolution of Problems
.1 Diesel Engine Generator (DEG) 2-2 Load Swings
a. Inspection Scope
The inspectors reviewed the performance and maintenance activities associated with DEG 2-2 for the past year. In particular, the inspectors reviewed the following DEG 2-2 events:
- August 31, 2002, DEG 2-2 experienced spurious load swings during a surveillance test (AR A0564160)
- December 21, 2002, DEG 2-2 experienced spurious load swings of greater magnitude than those seen on August 31 (AR A0564160)
- February 24, 2003, DEG 2-2 failed to shutdown following a 24-hour surveillance test (AR A0576118)
- March 8, 2003, previous corrective actions for DEG 2-2 were evaluated and determined to be adequate (AR A0577700)
- March 20, 2003, DEG 2-2 experienced load swings that progressively became worse resulting in the replacement of the governor (AR A0579038)
- April 22, 2003, the oil sample from the replaced DEG 2-2 governor indicated high ferrous particle count that was less than 10 micron in size (AR A0581548)
The inspectors reviewed each event to determine if complete and accurate identification of the problem was made in a timely manner and commensurate with its safety significance. Inspectors also evaluated issues regarding maintenance effectiveness, operability, reportability, generic implications, and extent of condition. For identified problems, the inspectors reviewed their classification and prioritization, root and contributing causes, and corrective actions. The inspectors used Procedure STP M-9A, Diesel Engine Generator Routine Surveillance Test, Revision 64, during the inspection.
b. Findings
Introduction.
A Green noncited violation was identified by the inspectors for Pacific Gas and Electric Company's failure to identify and correct a condition adverse to quality as required by 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Actions. This failure caused DEG 2-2 to remain in service for over 6 months with a degraded governor.
Description.
From August 31, 2002, to March 20, 2003, operators reported increasing difficulty in maintaining the required surveillance test load on DEG 2-2. The DEGs are required by Technical Specification Surveillance Requirement 3.8.1.3 to be synchronized to its associated vital electric bus and loaded for greater than or equal to 60 minutes. The load was required to remain between 2340 kW and 2600 kW, however, momentary transients outside the load range do not invalidate the test.
On August 31, 2002, operators initiated AR A0564160, in accordance with Procedure STP M-9A, Step 12.4.15, when DEG 2-2 experienced spurious load changes that were greater than 0.2 MW, peak-to-peak during the performance of the surveillance test, STP M-9A. Pacific Gas and Electric Company's engineering staff reviewed the control room strip charts and determined that the spurious load changes were the result of bus frequency variations.
On December 20, 2002, operators recorded, in AR A0564160, that the load for DEG 2-2 needed to be adjusted significantly more often to remain in the required load range during the performance of surveillance test Procedure STP M-9A. Operators also reported that a rain storm was the most likely cause for the load disturbances.
On February 24, 2003, at the end of a 24-hour load run on DEG 2-2, the diesel engine failed to shutdown from both in the control room and at the local control panel.
Operators subsequently tripped DEG 2-2 using the over speed trip handle, which tripped the fuel racks and shutdown the DEG. The engineering staff evaluated the DEG and determined that the most likely cause of the DEG 2-2 failure to shutdown was a problem with the shutdown solenoid located inside the mechanical governor. Pacific Gas and Electric Companys maintenance personnel removed the solenoid and evaluated its condition. The solenoid coil passed the continuity checks, which were found to match readings on a similar solenoid. Additionally, it was documented in AR A0576118 that the solenoid valve could be heard to transition when the coil was energized/de-energized. The engineering staff determined that the cause of the problem was associated with the shutdown solenoid valve not opening. Subsequently, maintenance personnel replaced the shutdown solenoid valve inside the mechanical governor.
On March 8, 2003, AR A0577700 was initiated by engineering staff to further evaluate the failure of DEG 2-2 to shutdown and the resulting troubleshooting process. The engineering staff concluded that the troubleshooting and other actions taken correctly identified the shutdown failure of DEG 2-2 to be a failed solenoid coil and/or a stuck solenoid shutdown valve. The engineering staff stated that a further concern did not exist as supported by the parts replaced, inspections performed, evaluation of the mechanical governor performance, and an oil analysis review.
On March 20, 2003, operators initiated AR A0579038 and reported DEG 2-2 load swings during performance of surveillance test Procedure STP M-9A. The operators stated that the DEG response to manual operation was sluggish and toward the end of the test run, the load swings began small and progressively became worse. The load swings ranged from 2.3 MW to 2.6 MW. These load swings did not result in the DEG failing the surveillance test. Maintenance personnel replaced the mechanical governor and no abnormal load swings were observed.
On April 22, 2003, AR A0581548 was initiated to document the results from the oil sample that was taken when the DEG 2-2 mechanical governor was replaced in March.
The results indicated a high concentration of iron particles (46 ppm) less than 10 microns in size. Subsequent visual inspections by a governor refurbishing shop detected corrosion (rust) inside the governor housing. Pacific Gas and Electric Company reviewed the governor oil analysis results from the other five DEGs. Each of the oil analysis results were satisfactory. Pacific Gas and Electric Company is currently reviewing the cause of the corrosion in DEG 2-2 mechanical governor through Quality Evaluation Q0012327.
The inspectors determined that there were several opportunities to identify and correct the degraded mechanical governor for DEG 2-2 that existed from August 31, 2002, to its eventual replacement after the March 12, 2003, surveillance test. The engineering staff attributed the cause of the load swings to other phenomenon such as bus frequency variations and rainstorms without adequately verifying that these phenomenon existed and what their impact would be on DEG 2-2 during its surveillance test. Additionally, Pacific Gas and Electric Company did not thoroughly troubleshoot the cause for the failure of DEG 2-2 to shutdown which resulted in the degraded governor remaining in service for approximately another month.
Analysis.
The performance deficiency associated with this finding is a failure to promptly identify and correct a condition adverse to quality, which led to DEG 2-2 remaining in service with a degraded mechanical governor. The finding impacted the mitigating systems cornerstone and was more than minor when assessed using Inspection Manual Chapter 0612, Appendix E, Example 4.g. In Example 4.g, the failure to correct a condition adverse to quality was more than minor because the degraded governor had more than a minor impact on safety and continued operation could impact the DEGs ability to provide emergency power. Using the SDP Phase I Worksheet in Inspection Manual Chapter 0609, the inspectors determined that there was no loss of an actual safety function, no loss of a safety-related train for greater than the DEG Technical Specification allowed outage time, and the finding is not potentially risk significant due to a seismic, fire, flooding, or severe weather initiating event. Therefore, using the SDP Phase I Worksheet, the finding is determined to be of very low safety significance.
Enforcement.
10 CFR Part 50, Appendix B, Criterion XVI states, in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and nonconformances are promptly identified and corrected. Contrary to the above, Pacific Gas and Electric Companys failure to promptly identify and correct the degraded mechanical governor on DEG 2-2 resulted in a condition where the DEG speed control was adversely effected. Because the failure to promptly identify and correct the degraded mechanical governor was determined to be of very low safety significance and has been entered into the corrective action program as Quality Evaluation Q0012327, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy: NCV 50-323/03-07-03, Failure to Promptly Identify and Correct a Degraded Mechanical Governor.
.2 Cross-References to Problem Identification and Resolution Findings Documented
Elsewhere.
Section 1R15 describes Pacific Gas and Electric Companys failure to promptly identify and correct lube oil carbonization in two diesel engine generators.
4OA3 Event Followup
.1 (Closed) Lincesee Event Report (LER) 50-323/02-002-00 and -01: Unit 2 manual
reactor trip due to loss of main feedwater to a steam generator.
This LER and revision discussed a manual reactor trip in response to a loss of feedwater to one steam generator because of a blown fuse in a solenoid valve. This event was discussed in detail in NRC Special Inspection Report 50-323/2002-07 and was identified as a noncited violation. This LER and subsequent revision provided no new information that would change the disposition of this issue. This item is closed.
.2 (Closed) LER 50-275; 323/02-001-00: Technical Specification violation due to
nonconservative steam generator narrow range level setpoints.
This LER discussed a Technical Specification violation due to nonconservative steam generator narrow range level setpoints that was identified following a reactor trip in response to a loss of feedwater to one steam generator. This event was discussed in detail in NRC Special Inspection Report 50-323/2002-07, and a non-cited violation was issued. This LER provided no new information that would change disposition of this issue. This item is closed.
.3 (Closed) LER 50-275/02-004-00: Automatic reactor trip on low steam generator level
due to low main feedwater flow.
This LER discussed a reactor trip on loss of feedwater to one steam generator when the feed regulating valve closed. The feedwater regulating valve closed because of a loss of instrument air when a test connection broke off that was disoriented. This event was discussed in detail in NRC Inspection Report 50-275; 323/2002-03. No findings of significance were identified. This LER provided no new information that would affect disposition of this issue. This item is closed.
.4 (Closed) LER 50-323/02-004-00: Manual start of Unit 2 auxiliary feedwater pumps due
to ocean debris and high swells.
This LER discussed an event in which high Pacific Ocean swells resulted in high debris loading on the circulating traveling screens with Unit 2 at 20 percent power. In response to this condition, operators rapidly tripped the main turbine and circulating water pumps, shut down the reactor, and manually started the auxiliary feedwater pumps, as required by procedure. This event was discussed in detail in NRC Inspection Report 50-275; 323/2002-05. No findings of significance were identified. This LER provided no new information that would affect disposition of this issue. This item is closed.
.5 (Closed) Unresolved Item (URI)50-275;323/03-02-03: Two examples of a noncited
violation of Technical Specification 5.4.1.d for inadequate fire protection implementation procedure.
a. Inspection Scope
The inspectors reviewed the systems required to achieve alternative safe shutdown to determine if Pacific Gas and Electric Company had properly identified the components and systems necessary to achieve and maintain safe shutdown conditions from the remote shutdown panel and alternate shutdown locations. The team focused on the adequacy of the systems to perform reactor pressure control, reactor makeup, decay heat removal, process monitoring, and support system functions. The team also reviewed Abnormal Operating Procedure OP AP-8A, "Control Room Inaccessibility -
Establishing Hot Standby," Revision 13, which would be used by operators to shut down the reactor in the event of a control room fire that required evacuation and remote shutdown. The actions of licensed and non licensed personnel were timed as they stepped through the procedure to determine its adequacy of the procedure to direct safe shutdown from remote shutdown locations.
b. Findings
b.1 Timing of Operator Actions in Control Room Fire and Evacuation Scenario
Introduction:
The inspectors identified a noncited violation of Technical Specification 5.4.1.d, for failure to establish, implement, and maintain an adequate procedure covering fire protection program implementation. Specifically, in a control room fire scenario requiring control room evacuation and remote shutdown, operators failed to complete the actions specified in procedure OP AP-8A, "Control Room Inaccessibility Hot Standby," within the times assumed in Calculation M-944 10 CFR Part 50 Appendix R, Alternate Shutdown Methodology Time and Manpower Study/Safe Shutdown System Considerations.
Description Pacific Gas and Electric Company's safe shutdown analysis Calculation M-944 10 CFR Part 50 Appendix R, Alternate Shutdown Methodology Time and Manpower Study/Safe Shutdown System Considerations, credited operator action to mitigate the effects of a control room fire. The analysis assumed operator action to: (1)trip the reactor immediately;
- (2) close power operated relief valve(s) within 5 minutes; (3)trip charging pump(s) within 11 minutes;
- (4) initiate auxiliary feedwater within 30 minutes;
- (5) initiate auxiliary salt water within 72 minutes; and
- (6) trip pressurizer heaters within 2.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />. The analysis also credited non-time critical actions to align auxiliary feedwater to the raw water storage reservoir and trip a safety injection pump within 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />.
The inspectors timed operator actions required to be performed within 30 minutes, assuming a fire in the control room requiring control room evacuation. The only action assumed to be taken in the control room was the reactor trip. These actions were simulated in the plant with no actual indications. The inspectors found that the operators could take the power operated relief to the emergency close position within 5 minutes, as required by Calculation M-944. However, because routine reactor trip actions inside the control room were not credited, operators took 48 minutes to initiate auxiliary feedwater. This was well outside the 30 minutes required in Calculation M-944.
As a result of this finding, Pacific Gas and Electric Company immediately initiated night orders to inform operators of the time critical actions required in a control room fire and evacuation scenario. In addition, the night orders emphasized the need to properly prioritize the time critical actions out of sequence. The team determined that Pacific Gas and Electric Companys immediate actions were adequate to ensure that a current safety issue did not exist.
Pacific Gas and Electric Company noted that specific indications at the hot shutdown panel were not available for use during the timing evolution, and believed that this resulted in a scenario artificiality that slowed the actual operator response. Pacific Gas and Electric Company re-performed the scenario on April 4, 2003, using the hot shutdown panel simulator to allow the shift foreman indications to properly diagnose and react to the event. During the simulator run on the simulator alternative shutdown panel, operators took 18 minutes to close the power-operated relief valve and 42 minutes to initiate auxiliary feedwater (versus 5 minutes and 30 minutes assumed in Calculation M-944, respectively). Pacific Gas and Electric Company initiated action request AR0580357 to place this item into the corrective action system.
Analysis:
This finding was greater than minor because it impacted the mitigating systems cornerstone and adversely affected the ability of Pacific Gas and Electric Company to manually operate certain components required for safe shutdown within the analyzed times. Specifically, in a simulated field walkdown, operators were not able to establish auxiliary feedwater within 30 minutes nor close a stuck open power operated relief valve (PORV) within 5 minutes as required by analysis. The inspectors used Appendix F of Manual Chapter 0609 and determined that the inability to perform the safe shutdown procedures required a Significance Determination Process Phase 2 and Phase 3 analysis. Further analysis of the ignition frequencies and the potential heatup of the core in this degraded condition, revealed that this finding was of very low safety significance.
Enforcement:
Section 5.4.1.d of the Technical Specifications, states, in part, that procedures shall be established, implemented, and maintained covering fire protection program implementation. Contrary to the above, in a control room fire scenario requiring control room evacuation and remote shutdown, Pacific Gas and Electric Company failed to adequately implement fire protection program requirements.
Specifically, operators failed to complete the actions specified in Procedure OP AP-8A, "Control Room Inaccessibility Hot Standby," within the times assumed in Calculation M-944, 10 CFR 50 Appendix R, Alternate Shutdown Methodology Time and Manpower Study/Safe Shutdown System Considerations. This is the first example of violation of Technical Specifications Section 5.4.1.d. Because the failure to implement and maintain an adequate safe shutdown procedure was determined to be of very low safety significance and has been entered into the corrective action program as A0580357, this example of violation is being treated as an noncited violation, consistent with Section VI.A of the NRC Enforcement Policy: NCV 50-275; 323/03-07-04, Timing of Operator Actions In Control Room Fire and Evacuation Scenario (Example 1).
b.2 Lack of Guidance On Taking Local Control at the Hot Shutdown Panel
Introduction:
The inspectors identified a noncited violation of Technical Specification 5.4.1.d, for failure to establish, implement, and maintain an adequate procedure covering fire protection program implementation. Specifically, Pacific Gas and Electric Company failed to provide adequate information in Procedure OP AP-8A, "Control Room Inaccessibility Hot Standby," concerning how to take local control of certain components required for safe shutdown.
Description:
Procedure OP AP-8A directed operators to take control of certain components at the hot shutdown panel. On the Unit 2 hot shutdown panel, operators had two switch positions to choose from: "AUTO" or "MANUAL." The procedure failed to provide the operators with adequate direction concerning how to take control of these components at the hot shutdown panel. The inspectors noted that operator aids which were provided on the Unit 1 hot shutdown panel were missing from the Unit 2 hot shutdown panel. The missing operator aids equated the "AUTO" position to control room operation and the "MANUAL" position to local or hot shutdown panel control. The team noted that these missing operator aids impacted the crews response to the fire scenario, in that the operators did not take local control of several parameters during the scenario. Thus, the automatic controls for several parameters, such as auxiliary feedwater throttle valve positions and steam generator atmospheric dump valve positions would have remained in the control room positions, still exposed to fire damage and possible spurious operation.
With no procedural guidance concerning how to take local control, coupled with missing operator aids, the team concluded that Procedure OP AP-8A was not adequate to ensure operators could take local control of certain components required for safe shutdown. Pacific Gas and Electric Company initiated action request AR 0573063 to replace the missing operator aids for Valves PCV-19, PCV-20, PCV-21, PCV-22, HCV-142, and FCV-128. The operator aids were reinstalled on February 4, 2003; therefore, no safety issue existed at the conclusion of the inspection.
Analysis:
This finding was greater than minor because it impacted the mitigating systems cornerstone and adversely affected the ability of Pacific Gas and Electric Company to take control of certain components required for safe shutdown.
Specifically, operator aids identifying the correct panel switch positions for the auxiliary feedwater throttle valves and steam generator atmospheric dump valves were missing from the Unit 2 hot shutdown panel. During a control room fire and remote shutdown, if not placed in the correct positions, these components would have remained vulnerable to fire damage that could cause spurious operation.
Enforcement:
Technical Specification Section 5.4.1.d, states, in part, that procedures shall be established, implemented, and maintained covering fire protection program implementation. Contrary to the above, in a control room fire scenario requiring control room evacuation and remote shutdown, Pacific Gas and Electric Company failed to adequately implement fire protection program requirements. Specifically, Pacific Gas and Electric Company failed to provide adequate information in Procedure OP AP-8A, "Control Room Inaccessibility Hot Standby," concerning how to take local control of certain components required for safe shutdown. Consequently, in stepping through Procedure OP AP-8A, operators failed to transfer control of certain safe shutdown components from the control room to the hot shutdown panel. This is a second example of violation of Technical Specification Section 5.4.1.d. Because the failure to have appropriate instructions for taking local control of the hot shutdown panel was determined to be of very low safety significance and has been entered into the corrective action program as A0573063, this violation is being treated as an NCV, consistent with Section VI.A of the NRC Enforcement Policy: NCV 50-275; 323/03-07-04, Lack of Guidance On Taking Local Control at the Hot Shutdown Panel (Example 2).
40A4 Cross-Cutting Aspects of Findings
.1 Cross-Reference to Human Performance Findings Documented Elsewhere
Section 1R04 describes that Pacific Gas and Electric Company failed to utilize acceptance criterion and instructions for inspections of a diesel engine generator voltage regulator card.
Section 40A3 describes two findings were Pacific Gas and Electric Company personnel failed to adequately implement fire protection programs requirements.
40A5 Other Evaluation of Diablo Canyon Safety Condition in Light of Financial Conditions
a. Inspection Scope
Due to Pacific Gas and Electric Companys financial condition, Region IV initiated special review processes for Diablo Canyon. The resident inspectors continued to evaluate the following factors to determine whether the financial condition and power needs of the station impacted plant safety. The factors reviewed included:
- (1) impact on staffing,
- (2) corrective maintenance backlog,
- (3) corrective action system backlogs,
- (4) changes to the planned maintenance schedule,
- (5) reduction in outage scope,
- (6) availability of emergency facilities and operability of emergency sirens, and
- (7) grid stability (i.e., availability of offsite power to the switchyard, status of the operating reserves, and main generator Volt-Ampere reactive loading).
Additionally, the resident inspectors observed the energy supply and operating reserves available in the California market. Inspectors have also increased attention to areas such as employee morale, Pacific Gas and Electric Company activities, and specific technical issues.
b. Findings
No findings of significance were identified.
40A6 Management Meetings
Exit Meeting Summary
The resident inspection results were presented on October 3, 2003, to Mr. James R. Becker, Vice President - Diablo Canyon Operations and Station Director, and other members of Pacific Gas and Electric Companys management. Pacific Gas and Electric Company acknowledged the findings presented.
The inspectors asked Pacific Gas and Electric Company whether any materials examined during the inspection should be considered proprietary. Proprietary information was reviewed by the inspectors and left with Pacific Gas and Electric Company at the end of the inspection.
ATTACHMENT:
SUPPLEMENTAL INFORMATION
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Pacific Gas and Electric Company personnel
- G. Anderson, Assistant Manager, Operations
- J. Becker, Vice President - Diablo Canyon Operations and Station Director
- C. Belmont, Director, Nuclear Quality, Analysis, and Licensing
- S. Chesnut, Director, Engineering Services
- S. David, Manager, Operations
- J. Hays, Director, Maintenance Services
- S. Ketelsen, Manager, Regulatory Services
- T. King, Manager, Learning Services
- M. Lemke, Manager, Emergency Preparedness
- D. Oatley, Vice President and General Manager, Diablo Canyon
- P. Roller, Director, Operations Services
- J. Tompkins, Director, Site Services
- L. Womack, Vice President Nuclear Services
Attachment
LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED
Opened and Closed
50-323/03-07-01 NCV Failure to Follow Instructions and Acceptance Criteria
During Diesel Engine Generator Automatic Voltage
Regulator Card Inspections (Section 1R04.1)
50-275/03-07-02 NCV Failure to Promptly Identify and Correct Diesel Engine
Generator Lube Oil Carbonization (Section 1R15)
50-323/03-07-03 NCV Failure to Promptly Identify and Correct a Degraded
Mechanical Governor (Section 4OA2)
50-275; 323/03-07-04 NCV Two examples of a violation of Technical Specification 5.4.1.d for inadequate fire protection implementation
procedure. (Section 4OA3.5)
Closed
50-323/02-002-00 LER Unit 2 manual reactor trip due to loss of main feedwater to
and -01 a steam generator (Section 4OA3.1)
50-275; 323/ LER Technical Specification violation due to nonconservative
2-001-00 steam generator narrow range level setpoints
(Section 4OA3.2)
50-275/02-004-00 LER Automatic reactor trip on low steam generator level due to
low main feedwater flow (Section 4OA3.3)
50-323/02-004-00 LER Manual start of Unit 2 auxiliary feedwater pumps due to
ocean debris and high swells (Section 4OA3.4)
50-275;323/0302-03 URI Two examples of a violation of Technical Specification 5.4.1.d for inadequate fire protection implementation
procedure (Section 4OA3.5)
Attachment
LIST OF ACRONYMS
AR action request
AVR automatic voltage regulator
DEG diesel engine generator
CFR Code of Federal Regulations
FSAR Final Safety Analysis Report
INPO Institute for Nuclear Power Operations
LOCA loss-of-coolant accident
LER Licensee Event Report
NCV noncited violation
NRC Nuclear Regulatory Commission
PARS Publicly Available Records System
PIV pressure isolation valve
PORV power-operated relief valve
SDP Significance Determination Process
URI unresolved item
Attachment
Attachment