IR 05000528/2004013: Difference between revisions
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
StriderTol (talk | contribs) (StriderTol Bot change) |
||
Line 1: | Line 1: | ||
{{IR-Nav| site = 05000528 | year = 2004 | report number = 013 | {{Adams | ||
| number = ML043380346 | |||
| issue date = 12/03/2004 | |||
| title = IR 05000528-04-013, 05000529-04-013, 05000530-04-013, on 09/07/2004 - 10/26/2004; Palo Verde Nuclear Generating Station, Units 1, 2, and 3; Augmented Inspection Followup | |||
| author name = Chamberlain D | |||
| author affiliation = NRC/RGN-IV/DRS | |||
| addressee name = Overbeck G | |||
| addressee affiliation = Arizona Public Service Co | |||
| docket = 05000528, 05000529, 05000530 | |||
| license number = NPF-041, NPF-051, NPF-074 | |||
| contact person = | |||
| case reference number = EA-04-222 | |||
| document report number = IR-04-013 | |||
| document type = Inspection Report, Letter | |||
| page count = 51 | |||
}} | |||
{{IR-Nav| site = 05000528 | year = 2004 | report number = 013 }} | |||
=Text= | |||
{{#Wiki_filter:ber 3, 2004 | |||
==SUBJECT:== | |||
PALO VERDE NUCLEAR GENERATING STATION, UNITS 1, 2, AND 3 - | |||
NRC AUGMENTED INSPECTION TEAM FOLLOWUP REPORT 05000528/2004013; 05000529/200413; 05000530/2004013 | |||
==Dear Mr. Overbeck:== | |||
From September 24 through October 26, 2004, the Nuclear Regulatory Commission (NRC) | |||
conducted an onsite Augmented Inspection Team followup inspection at your Palo Verde Nuclear Generating Station, Units 1, 2, and 3. The enclosed report documents the inspection findings, which were discussed following completion of the inspection on October 26, 2004, with you, Mr. Levine, Senior Vice President of Generation, and other members of your staff. | |||
This inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license. | |||
The team reviewed selected procedures and records, observed activities, and interviewed personnel. | |||
This inspection report documents ten findings of very low safety significance (Green). The NRC also determined that violations of NRC requirements were associated with these findings. | |||
However, because of the very low safety significance and because the findings were entered into your corrective action program, the NRC is treating these findings as noncited violations consistent with Section V1.A of the NRC Enforcement Policy. If you contest the violations or significance of these noncited violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C. 20555-0001, with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission, Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555-0001; and the NRC Resident Inspector at the Palo Verde Nuclear Generating Station facility. | |||
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room). | |||
Arizona Public Service Company -2-Should you have any questions concerning this inspection, we will be pleased to discuss them with you. | |||
Some of the material enclosed herewith contains exempt information in accordance 10 CFR 2.390(d)(1). Therefore, the applicable material will not be made available electronically for public inspection in the NRC Public Document Room or from the NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html. | |||
Sincerely, | |||
/RA/ | |||
Dwight D. Chamberlain, Director, Division of Reactor Safety Dockets: 50-528; 50-529; 50-530 Licenses: NPF-41; NPF-51; NPF-74 | |||
===Enclosure:=== | |||
NRC Inspection Report 05000528/2004013; 05000529/200413; 05000530/2004013 | |||
REGION IV== | |||
Dockets: 50-528; 50-529; 50-530 Licenses: NPF-41; NPF-51; NPF-74 Report No.: 05000528/2004013; 05000529/2004013; 05000530/2004013 Licensee: Arizona Public Service Company Facility: Palo Verde Nuclear Generating Station, Units 1, 2, and 3 Location: 5951 S. Wintersburg Road Tonopah, Arizona Dates: September 7 through October 26, 2004 Team J. I. Tapia, P.E., Senior Reactor Inspector Leader: Engineering Branch Inspectors: R. E. Lantz, Senior Emergency Preparedness Inspector, Operations Branch G. G. Warnick, Resident Inspector, Projects Branch D M. R. Cox, Resident Inspector, Region I Accompanied F. Baxter, P. E., Electrical Engineer, Beckman and Associates By: | |||
Approved By: Jeff Clark, P. E., Branch Chief Engineering Branch, Division of Reactor Safety I Enclosure | |||
=SUMMARY OF FINDINGS= | |||
IR 05000528/2004-013, 05000529/2004-013, 05000530/2004-013; 9/7-10/26/2004; Palo Verde | |||
Nuclear Generating Station, Units 1, 2, and 3; Augmented Inspection Followup On June 14, 2004, a ground-fault occurred on a 230kV transmission line, approximately 47 miles from the Palo Verde Nuclear Generating Station. A failure in the protective relaying precluded isolation of the ground-fault from the local grid for approximately 38 seconds and caused a loss-of-offsite power and a trip of all three Palo Verde Nuclear Generating Station units. The Unit 2 Train A emergency diesel generator started, but did not complete the load sequence process due to a failed diode in the exciter rectifier circuit. This resulted in the Train "A" engineered safeguards features busses de-energizing and limiting the availability of certain safety equipment. Because of this failure, the licensee elevated the emergency declaration for Unit 2 to an Alert. All three units were safely shutdown and stabilized under hot shutdown conditions. | |||
An NRC Augmented Inspection Team (AIT) was dispatched to the site and found that the licensees response to the event was generally acceptable, although complicated by a number of equipment failures, procedure issues, and human performance issues with diverse apparent causes and with varying degrees of significance (See NRC Inspection Report 05000528/2004012; 05000529/2004012; 05000530/2004012). The issues requiring additional followup were identified and tracked as unresolved items in the AIT report. The AIT reviewed the licensees immediate corrective actions prior to restart of the units, including actions to improve the independence and reliability of offsite power sources, and found those actions appropriate for continued operation of the units. The AIT charter in accordance with NRC Inspection Procedure 93800, did not direct the team to address compliance or assess significance of findings and observations. | |||
This report documents the followup inspection, which was conducted to address the unresolved issues identified during the AIT inspection. The inspection covered a 2-week period of inspection onsite by a team of four inspectors and one consultant. Ten findings of very low safety significance (Green) were identified. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using IMC 0609, Significance Determination Process, (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, | |||
Revision 3, dated July 200 | |||
===NRC-Identified and Self-Revealing Findings=== | |||
===Cornerstone: Reactor Safety=== | |||
: '''Green.''' | |||
A noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, was identified because the licensee failed to adequately implement their corrective action program when an emergency diesel-generator excitation circuit failed. | |||
The failure precluded the emergency diesel-generator from achieving rated voltage within the required time period. The licensee subsequently repaired the circuit (Section 2.4). | |||
III | |||
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and did not result in the actual loss of a safety function at the time. | |||
C | |||
: '''Green.''' | |||
A noncited violation of 10 CFR Part 50, Appendix B, Criterion V, Procedures, was identified because the licensee failed to follow the procedure for dispositioning a degraded condition for continued use. Specifically, the licensee failed to address a degraded main generator excitation limiter circuit into the work control process via the appropriate procedure to ensure that it was appropriately evaluated and processed. | |||
This circuit, in part, prevents an overpower condition in the reactor fuel (Section 2.5). | |||
The finding was greater than minor because it was associated with the human performance attribute of the barrier integrity cornerstone and impacted the cornerstone objective to provide reasonable assurance that physical design barriers, in this case the fuel cladding, protect the public from radionuclide releases caused by accidents or events. | |||
C | |||
: '''Green.''' | |||
A noncited violation of 10 CFR Part 50, Appendix B, Criterion V, Procedures, with two examples, was identified because the licensee failed to implement contingency actions when two circuit breakers failed to operate during recovery operations in Units 1 and 3. Specifically, operators deviated from the emergency operating procedure for loss-of-offsite power/loss of forced circulation when they initiated maintenance on the two failed breakers instead of performing the contingency actions prescribed by the procedure. In addition, for Unit 1, the procedure was inadequate because it did not list all available contingency actions available to operators for restoring power to the electrical bus. The licensee is currently reviewing these procedures through their corrective action program (Section 2.8). | |||
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and redundancy existed in other electrical buses. | |||
C | |||
: '''Green.''' | |||
A noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, was identified by the team because the licensee failed to implement timely corrective actions to ensure that the feedwater system was operated in a manner that would minimize the possibility of thermally induced vibration that could affect auxiliary feedwater system operability (Section 2.9). | |||
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and because no transient occurred that necessitated implementation of the needed corrective actions. | |||
IV | |||
C | |||
: '''Green.''' | |||
A noncited violation of Technical Specification 5.4.1 was identified because the licensee implemented an inadequate emergency operating procedure. Specifically, the procedure failed to provide direction to maintain turbine-driven auxiliary feedwater pumps operable following a main steam isolation signal. The licensee is currently reviewing this procedure through their correction action program (Section 3.1). | |||
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and because the turbine-driven auxiliary feedwater pumps did not become inoperable. | |||
C | |||
: '''Green.''' | |||
A noncited violation of 10 CFR 50.65(a)(4), Maintenance Rule, was identified because the licensee failed to perform a risk assessment. Specifically, the licensee inappropriately decided to begin draining the Unit 1 turbine-driven auxiliary feedwater pump steam traps first, without addressing the higher risk profile in Unit 2, which resulted from having an inoperable emergency diesel generator. Draining the Unit 2 traps first would have restored the auxiliary feedwater pump and lowered overall Unit 2 risk (Section 3.1). | |||
The finding was greater than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and because the turbine-driven auxiliary feedwater pumps were not needed. | |||
: '''Green.''' | |||
A noncited violation of Technical Specification 5.4.1 was identified because the licensee failed to follow emergency operating procedures. Specifically, the control room operator and an auxiliary operator performed the incorrect steps in Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, | |||
Revision 10. The Unit 2, Positive Displacement Charging Pump E was temporarily lost due to these human performance errors and resulted in a total loss of Unit 2 charging flow for a short period (Section 3.2). | |||
The finding was greater than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and did not result in the actual loss of a safety function and no significant delays occurred that adversely impacted operator response to the event. | |||
C | |||
: '''Green.''' | |||
A noncited violation of 10 CFR 50.54(q) was identified because the licensee failed to follow the emergency plan when they did not adequately maintain facilities required for emergency response. Specifically, the Technical Support Center emergency diesel generator failed because a test switch was not returned to its proper position following maintenance 6 days prior to the event. As a result, the emergency response organization assembled in the alternate Technical Support Center. This resulted in some confusion and posed some unique challenges to the emergency response organization (Section 3.4). | |||
V | |||
The finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem. | |||
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 8 is considered a failure to comply with Planning Standard 8 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter MC 0609 Appendix B, §2.0. Therefore, the finding is of very low safety significance. | |||
C | |||
: '''Green.''' | |||
A noncited violation of 10 CFR 50.54(q) was identified because the licensee failed to follow the emergency plan when they did not ensure that adequate command and control was established during the event. Specifically, the licensee did not follow Emergency Plan Implementing Procedure 01, Satellite Technical Support Center Actions, which requires that for multiple unit events, the Unit 1 shift manager is responsible for initially classifying and declaring the emergency and assuming the position of the on-shift emergency coordinator. As a result, each of the units respective shift managers initially assumed the role of emergency coordinator, resulting in notification irregularities to state and local officials (Section 3.5). | |||
The finding is more than minor because it is related to the Emergency Preparedness cornerstone attribute of response organization performance, and affects the cornerstone objective in that command and control challenges resulting in inaccurate communications to the offsite officials could potentially affect the ability to ensure that adequate measures would be taken to protect the public health and safety. | |||
: '''Green.''' | |||
A noncited violation of 10 CFR 50.54(q) was identified because the licensee failed to follow the emergency plan. Specifically, the licensee failed to meet minimum staffing goals of Table 1, Minimum Staffing Requirements for PVNGS for Nuclear Power Plant Emergencies, following the Alert declaration on June 14, 2004 (Section 3.5). | |||
This finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem. | |||
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 2 is considered a failure to comply with Planning Standard 2 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609 Appendix B, §2.0. Therefore, the finding is of very low safety significance. | |||
VI | |||
=REPORT DETAILS= | |||
===1. INTRODUCTION=== | |||
On June 14, 2004, a ground-fault on a 230kV transmission line, approximately 47 miles from the Palo Verde Nuclear Generating Station (PVNGS), caused a loss-of-offsite power and a concurrent trip of all three PVNGS units. Because of the loss-of-offsite power, the licensee declared a Notice of Unusual Event for all three units at approximately 7:50 a.m. MST. The Unit 2 Train A emergency diesel generator started, but failed early in the load sequence process due to a failed diode in the exciter rectifier circuit. This resulted in the Train "A" engineered safeguards features busses de-energizing, which limited the availability of certain safety equipment for operators. | |||
Because of this failure, the licensee elevated the emergency declaration for Unit 2 to an Alert at 7:54 a.m. MST. All three units were safely shutdown and stabilized under hot shutdown conditions. | |||
An NRC Augmented Inspection Team (AIT) was dispatched to the site and found that the licensees response to the event was generally acceptable, although complicated by a number of equipment failures, procedure issues, and human performance issues with diverse apparent causes and with varying degrees of significance. The issues requiring additional followup were identified and tracked as unresolved items (See NRC Inspection Report 05000528/2004012; 05000529/2004012; 05000530/2004012). The AIT reviewed immediate corrective actions prior to restart of the units, including actions to improve the independence and reliability of offsite power sources and found those actions appropriate for continued operation of the units. | |||
This followup inspection was conducted to address the unresolved issues identified during the AIT inspection. The inspection addressed compliance and assessed the significance of the AIT observations. | |||
===2. SYSTEM PERFORMANCE AND DESIGN ISSUES=== | |||
A number of unresolved items associated with system performance and design issues were revealed during and following the event. Each of these issues was inspected to assess the licensees effectiveness in determining the root and contributing causes, the extent of condition, and corrective actions. | |||
2.1 (Closed) Unresolved Item 05000528; -529; -530/2004012-001: Offsite Power Reliability and Independence Issues | |||
====a. Inspection Scope==== | |||
On June 14, 2004, a phase to ground fault occurred on the Westwing to Liberty 230kV transmission line. The fault was not quickly cleared due to a relay failure in the protection scheme, which caused Breaker WW1022 to not receive a trip signal. Since the fault was not cleared, it continued to be fed by three 525/230kV transformers in the Westwing substation. The Westwing substation is connected to the PVNGS switchyard by two 525kV transmission lines. The fault continued to be fed for approximately 38 seconds and resulted in all transmission lines to the PVNGS switchyard (PL) being disconnected. This caused a generator trip, reactor trip, and loss-of-offsite power at all three PVNGS units. It was also noted that the three tie lines between the PVNGS and Hassayampa switchyards were the first 525kV lines to trip. | |||
This unresolved item was identified by the AIT to facilitate the review of the root and contributing causes of the ground fault failing to isolate from the grid and to evaluate the protective tripping of the Hassayampa to PVNGS transmission lines. This review also included an evaluation of the extent of condition associated with other potential design issues that could adversely impact the independence and reliability of offsite power to PVNGS, and to assess the corrective actions for any issues identified. | |||
The inspectors reviewed the licensees root-cause evaluation to determine if the problems which caused the failure to clear the fault and subsequent loss-of-offsite power were well understood. The inspectors also evaluated the corrective actions to ensure they were appropriate to correct the root and contributing causes of the event. | |||
In addition the inspectors reviewed the 10 CFR 50.59 screen for the addition of the three 525kV lines connecting the Hassayampa and PVNGS switchyards, and the associated protective relaying for the transmission lines. | |||
====b. Observations and Findings==== | |||
The inspectors found that the root and contributing causes identified by the licensee were appropriate. The root cause of the failure to clear the fault was a defective AR relay (fast acting, auxiliary relay), which provided trip signals for Breakers WW1022 and WW1026. This relay acts as a contact multiplier (i.e. provides multiple outputs based on a single input) in the protective scheme and is used to send trip signals to its two associated breakers. Two of the four contacts failed to shut on demand, which resulted in Breaker WW1022 not receiving a trip signal. The relay has been sent back to the vendor for failure analysis to determine the reason for the contacts failing to shut. A breaker failure signal was not actuated since it was fed from the same failed contacts. | |||
Breaker WW1026 opened, isolating the fault on the Liberty side, but since Breaker WW1022 stayed shut the fault continued to be fed from the Westwing 525kV switchyard. | |||
Contributing causes included a lack of redundancy to preclude a single failure from preventing a breaker trip, relay testing did not ensure each contact shut, and no overcurrent protection was provided on the 525/230kV transformers in the Westwing substation. It was also determined that the three Hassayampa-PL transmission lines tripped on negative-sequence current, a protection system feature, which is designed to provide open pole protection for the lines. Open pole refers to a fault in which one phase of the breaker fails to shut. | |||
The inspectors determined that the corrective actions were appropriate to address the root and contributing causes of the event. In addition, the enhancements to the off-site transmission network were effective in improving the reliability and independence of the off-site electrical grid. The inspectors reviewed the extent of condition review and determined that it was sufficient in scope to ensure that PVNGS should not be challenged by a similar uncleared fault at neighboring switchyards. The inspectors found the following actions had been implemented by the licensee and transmission system owners: | |||
* Removed negative sequence relay protection of the PL-Hassayampa 525kV tie lines. Open pole protection was still provided by a phase current mismatch protection scheme. This scheme is less likely to spuriously trip due to external faults. | |||
* Added redundant AR relays for Deer Valley - WW and Liberty - WW 230kV lines. | |||
No other lines were found which had single point failure vulnerabilities because of the AR relays. | |||
* Added phase distance and overcurrent relay protection on two of the three 525/230kV transformer (the third will be added when the transformer is replaced)and the single 525/345kV transformer at Westwing to provide backup protection against a similar fault. | |||
* A study was performed, which showed that the fault current at the Westwing substation was the most severe. A sustained fault at any other substation would not have a similar impact at PVNGS. | |||
* The transmission system owners reviewed prints to ensure no single point failures on 230kV systems at Rudd, Cairina, Browning, Deers and Silvervine substations. | |||
* Change functional testing for relays to ensure that all contacts actuate. | |||
The inspectors determined that the 10 CFR 50.59 evaluation for the addition of the three Hassayampa-PL tie lines did not identify a vulnerability in the protective relay scheme, which surfaced during the June 14, 2004, event. Required design reviews for protective schemes look at system coordination and the response to numerous types of transients. | |||
As a standard practice, consistent with NERC reliability requirements, extended faults were not considered a plausible event during design development and review by the transmission system operator because of the extremely low probability of occurrence. | |||
This type of failure, which revealed this vulnerability, had never occurred before on the transmission system associated with PVNGS. | |||
The three tie lines each used multifunction relays for their primary and backup protective relaying schemes. The multifunction relays used for backup protection included a negative sequence relay to provide open pole protection. This particular function was not needed for this application since a backup for pole protection is not required. Since the function was already available the transmission and distribution designers added the function and adjusted the settings high enough that it would not normally trip. During the grid event, the three tie lines tripped due to actuation of the negative sequence relays. It was determined that this was a correct, but not as planned operation. The function tripped at the appropriate settings, however, it was not intended to protect against this type of fault. The tripping of these lines resulted in the PVNGS units being isolated from nearby generation facilities and further increased the severity of the transient and degradation of voltage at the PVNGS switchyard. | |||
Based on the above stated facts the inspectors determined that the 10 CFR 50.59 review was adequate in depth and scope even though it did not identify this vulnerability in the protective relay scheme. The transmission system operator determined that the protective scheme would be more reliable without the negative sequence relays and this feature was removed from the protective scheme as a corrective action following the event. | |||
The inspectors also noted that the description of the transmission lines contained in Updated Final Safety Analysis Report, Sections 8.1 and 8.2, were not consistent with the as-built design and differed from section to section. The licensee generated appropriate documentation to address these discrepancies. | |||
2.2 (Closed) Unresolved Item 05000528/2004012-002: Unit 1 Atmospheric Dump Valve Failure | |||
====a. Inspection Scope==== | |||
The augmented inspection team identified a unresolved item concerning reactor operator observations that Atmospheric Dump Valve ADV-185 failed to operate properly while being remote-manually operated from the control room during the performance of Procedure 40EP-9EO10, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, following the Unit 1 loss-of-offsite power event. This issue was made unresolved pending review of the root and contributing causes of the valve failure; the licensees extent of condition; and assessment of the effectiveness of corrective actions implemented by the licensee. | |||
The inspectors reviewed troubleshooting activities and trend plots, and interviewed licensee personnel to evaluate the cause determination performed by the licensee. The team also reviewed the adequacy of corrective actions taken for problems identified. | |||
====b. Observations and Findings==== | |||
The licensee was unable to recreate the anomalies reported by operations through extensive troubleshooting and trend plot review illustrating atmospheric dump valve movement. A thorough troubleshooting plan was implemented by engineering and maintenance personnel to evaluate the system. No anomalies were discovered during numerous valve strokes and no leaks were detected on instrument air fittings and associated parts. Nonetheless, the positioner was replaced and calibrated since a review of equipment history revealed that the old positioner was slightly aged with 6 years of service. | |||
The team noted that the conclusions made by engineering and maintenance personnel did not correlate with the observations made by the reactor operator. Additionally, the engineering evaluation did not include a disposition of the reactor operators statements concerning operation of the valve. Following the NRCs identification of this oversight, engineering and maintenance personnel met with the reactor operator to properly disposition statements made after the Unit 1 loss-of-offsite power event. The trend plots illustrating valve movement were compared with the reactor operators recollection of his post-trip actions. Recorded trend data and reactor operator recollection indicate that the atmospheric dump valve was being operated in the area of 5 percent to regulate reactor coolant system temperature and pressure. When atmospheric dump valves are given a change in demand, they tend to overshoot, then undershoot the setpoint prior to settling at the final position. If the valve is desired to be 5 percent open to match a low primary heat load, the valve may trigger the closed limit switch, giving the operator the impression that the valve has closed, when in fact, the valve is still hunting to attain a final position. It was apparent the normal hunting toward final valve position following adjustments led the reactor operator to believe that the valve was drifting closed. | |||
Although troubleshooting was extensive and no physical anomalies were identified, the valve positioner was replaced and calibrated as a conservative measure. However, the engineering evaluation that provided the basis for closure of the condition report/disposition request did not include a disposition of the reactor operators statements concerning the operation of the valve. Following the NRCs identification of this oversight, engineering and maintenance personnel met with the reactor operator to properly disposition statements made concerning the valve operation. As a result of this followup to address all pertinent facts concerning valve operation, it was determined that the reactor operator did not understand the valve response at low primary heat loads. | |||
This determination identified an area for improvement in operator training. Corrective actions were subsequently developed to address this training need. | |||
The failure to disposition and resolve the reactor operators statements concerning operation of the valve resulted in the failure to identify a condition adverse to quality regarding operation of the atmospheric dumps. This failure also represented an ineffective problem identification and resolution cross-cutting aspect that resulted in the failure to identify the need for additional operator training. Following identification of this oversight, the licensee developed corrective actions to train operators on atmospheric dump valve response with low primary heat loads. We determined that your initial corrective actions were ineffective and failed to be in accordance with Criterion XVI of Appendix B to 10 CFR Part 50. However, because there was no actual problem with the valve, this is a finding of minor significance and is not subject to enforcement action in accordance with Section VI of the NRCs Enforcement Policy. | |||
2.3. (Closed) Unresolved Item 05000528/2004012-003: Unit 1 Letdown System Failure to Isolate | |||
====a. Inspection Scope==== | |||
During the June 14, 2004, loss-of-offsite power event, the Unit 1 letdown system did not operate as expected when fluid temperatures exceeded the alarm setpoint. The letdown system bypassed the ion exchanger and the filter at 140EF, as expected. However, a temporary modification to bypass a flow sensor resulted in the system failing to isolate when needed. The letdown system response had apparently not been anticipated by the engineers designing the temporary modification, and operators were unaware of the systems response to a loss-of-offsite power. This issue was made unresolved to review the root and contributing causes of the failure of the letdown system to isolate, review the licensees extent of condition, and assess the effectiveness of corrective actions implemented by the licensee. | |||
The inspectors reviewed the design control measures used when designing Temporary Modification 2594804, interviewed licensee personnel, and reviewed the adequacy of corrective actions taken for problems identified. | |||
====b. Observations and Findings==== | |||
The AIT review of the letdown system response determined that inadequate design control may have resulted in the overheating of a system designed for low temperature operation. The team also reviewed the apparent cause evaluation, system evaluation, and immediate corrective actions taken, which included removal of the temporary modification. The inspectors noted during the AIT follow-up inspection that the root-cause evaluation performed by the licensee was consistent with the observations made by the NRC augmented inspection team. Additionally, the inspectors determined that associated corrective actions were appropriate and also timely. | |||
Procedure 81TD-0EE10, Plant Design and Modification, was used to design Temporary Modification 2594804 to address inadvertent Unit 1 letdown isolation events that have occurred over the past several years. The inadvertent isolation events were temperature initiated by a nuclear cooling water flow switch during the cooler months when the nuclear cooling water is low and the nuclear cooling water letdown heat exchanger outlet valve is controlling near its closed position. In addition, there were indications that the nuclear cooling water flow switch was isolating at flow rates above the required setpoint. The combination of these two conditions has resulted in inadvertent isolation of letdown flow in Unit 1 due to the generation of the isolation signal to Valve CH-523. The temporary modification to bypass the nuclear cooling water flow switch, which generates the isolation signal, was originally installed in 1999, removed during Refueling Outage U1R10 following corrective actions, then reinstalled following another inadvertent isolation that occurred on March, 27, 2003. The inspectors noted that engineers failed to adequately consider letdown system response to a loss of power when performing the 10 CFR 50.59 screening. The failure to perform an adequate 10 CFR 50.59 screening resulted in the system failing to isolate when needed following the loss-of-offsite power event. This constitutes a finding of minor significance since it is not associated with any of the Reactor Safety, Radiation Safety, or Safeguards Cornerstones. The minor finding is not subject to enforcement action in accordance with Section VI of the NRCs Enforcement Policy. However, this finding involved human performance cross-cutting aspects because the letdown system response was not anticipated by the engineers designing the temporary modification and by operators when responding to a system loss of power. | |||
2.4 (Closed) Unresolved Item 05000529/2004012-004: Unit 2, Train "A" Emergency Diesel Generator Failure | |||
====a. Inspection Scope==== | |||
During the June 14 loss-of-offsite power event the Unit 2, Train A emergency diesel generator started but failed during the loading sequence due to a diode failure in the emergency diesel generator excitation circuitry. This resulted in a loss of power to the Train A engineered safeguard busses, which limited the availability of certain safety equipment to the operators. Due to the failure of Train A emergency diesel generator coincident with a loss-of-offsite power, the licensee declared an Alert based on the criteria in the emergency action level matrix. | |||
This unresolved item was identified by the AIT to facilitate the review of the root and contributing causes of the failure of the diode in Phase B of the Unit 2, Train A emergency diesel generator voltage regulator exciter circuit, review the extent of condition and assess the effectiveness of corrective actions implemented by the licensee. | |||
The inspectors reviewed the licensee root-cause report and the troubleshooting action plan to ensure the cause of the failure was well understood and the corrective actions appropriate to address the cause of the failure. The inspectors also reviewed available industry experience data on diode failures and data from the testing that the licensee performed on the excitation circuit. This review also included the diode failure analysis report from an outside vendor and the manner in which the licensee incorporated the reported data. | |||
====b. Observations and Findings==== | |||
Each emergency diesel generator has two redundant excitation system bridge circuits. | |||
By design, one bridge circuit is in use and the alternate is an installed spare. However, the alternate bridge circuit had not been tested as part of the preventive maintenance or routine surveillance programs. The licensee determined that they should periodically swap and test these spare systems based on owners group and industry experience. | |||
The Unit 2, Train A emergency diesel generator bridge had been changed to the alternate circuit during the 2003 outage and had been in service for approximately 70 hours of operation before the failure. The Unit 1, Train A emergency diesel generator had also been swapped to its alternate bridge during the last outage for that unit and has been operating satisfactorily since placed in service. | |||
The inspectors determined that the licensees root-cause evaluation appropriately determined that the failure was due to a manufacturing defect that resulted in infant mortality of the part. This evaluation was justified by: | |||
A large design margin for overcurrent. | |||
Same type of diode in the other emergency diesel generators with no failures in 20 years. | |||
Low failure incidence in the industry. | |||
The inspectors reviewed the available test data, the failure analysis report from the vendor and industry operating experience to ensure that the licensees classification of infant mortality was appropriate. The failed diode was rated for 275 amps and 1000 volts for repetitive peak reverse overvoltage. The licensee performed testing, which showed that the maximum reverse voltage applied to any of the diodes in the bridge circuit was 316 volts, well below the maximum rating. The testing was performed during engine startup, steady state run and shutdown. Although the testing did not repeat the conditions on the emergency diesel generator at the time of failure, the inspectors determined that the testing provided reasonable assurance that there were no anomalies in the circuit, which would cause an overvoltage condition. The risk associated with securing the safety bus to allow for testing during loading conditions was not justified during reactor power operations. The licensee stated that additional testing would be performed during the outage while loading the emergency diesel generator to provide baseline data. In addition, the essential service water pump, which was loading onto the emergency diesel generator at the time of the failure was evaluated to ensure that no electrical fault on that motor precipitated the failure. The licensee also performed testing on all the spare diodes in stock to ensure they were free of manufacturing defects and to provide baseline data for later stock. | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance was identified because the licensee failed to adequately address an alternate bridge circuit failure in their corrective action program. Specifically, the licensee failed to implement effective corrective actions for the failure because troubleshooting to identify a root cause was not conducted. | |||
=====Description.===== | |||
The failure occurred in the Unit 3, Train A emergency diesel generator during testing on April 19, 2003. At that time, the machine failed to achieve rated voltage within the required time period. This failure occurred immediately after the excitation bridge was swapped to its alternate circuit. This emergency diesel generator was the first one to be transferred to the alternate bridge. The corrective actions for the failure were simply to place the normal bridge back in service and to place the bridge swap in a different maintenance procedure to allow more time for testing should problems occur. No troubleshooting was performed to determine why the alternate bridge failed and the problem was not captured in the corrective action program. These oversights resulted in a missed opportunity, which may have identified deficiencies with the alternate bridge circuits due to the extended time that had been installed but not used. Although the alternate bridge circuits were installed but not normally in service, they are safety-related components. | |||
=====Analysis.===== | |||
The inspectors determined that this finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process, Phase 1 Worksheet, the finding was determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function at the time. | |||
Problem identification and resolution cross-cutting aspects of this finding were related to failing to thoroughly investigate the failure of safety-related equipment and failing to capture the issue in the corrective action program. | |||
=====Enforcement.===== | |||
Criterion XVI, Appendix B of 10 CFR Part 50 requires, in part, that conditions adverse to quality be promptly identified and corrected. Contrary to the above, PVNGS failed to adequately address the alternate bridge circuit failure in the corrective action program. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-01). | |||
2.5 Plant Response to Loss-of-Offsite Power Event (Closed) Unresolved Item 05000530/2004012-005: Unit 3 Main Turbine Bypass Valve Control System Operation | |||
====a. Inspection Scope==== | |||
During the June 14, 2004, loss-of-offsite power event, Unit 3 experienced an automatic main steam-line isolation. Licensee personnel attributed the automatic isolation to a steam bypass control system anomaly that caused all the bypass valves to open simultaneously, suddenly decreasing main steam line pressure, and causing a main steam isolation. The apparent cause of the "anomaly" was the result of a momentary loss of power to the control system being re-energized in the automatic mode, vice manual. According to the licensee engineers, this power loss initiated a 30-second timer that disconnected the valve control signals from the control cabinet. When the 30-second timer completed, all eight valves modulated open in about 14 seconds. This issue was made unresolved to review the root and contributing causes of the automatic main steam-line isolation in Unit 3, which appeared contrary to the expected response described in the plant safety analysis, the extent of condition, and the effectiveness of corrective actions implemented by the licensee. | |||
The inspectors reviewed troubleshooting activities, trend plots, and interviewed licensee personnel to evaluate the root-cause determination and extent of condition review performed by the licensee. The team also reviewed the adequacy of corrective actions taken for problems identified. | |||
====b. Observations and Findings==== | |||
The inspectors noted that the licensee initially determined that the cause for the steam bypass control system operation, which appeared contrary to the expected response described in the plant safety analysis, was because of an initiating fault with the main generator control circuit. Further, the licensee believed that if the control circuit had functioned as expected, the steam bypass control system would have been de-energized on the loss-of-offsite power and reverted to manual control with no output, consistent with the description in the plant safety analysis. The main generator control circuit failure is discussed in the following section of this report (URI 05000530/2004012-006). | |||
During interviews with engineering personnel, the inspectors learned that the licensee believed that there may be other power transient scenarios where the steam bypass control system could behave similarly, which was contrary to the initial condition report/disposition request disposition. Even though the main generator control circuit failure contributed to the difference in the Unit 3 steam bypass control system response, emergency diesel generator start time and/or the nature of the grid disturbance could cause the same steam bypass control system response following a loss-of-offsite power. This previously undisclosed behavior of the steam bypass control system response following a loss-of-offsite power represents a nonconformance between the as-built plant response and that which is described in the design basis. Following the interviews with engineering personnel, a condition report/disposition request was initiated to further analyze and resolve the nonconforming condition. The failure to identify and correct the nonconforming condition constituted a violation of Criterion XVI of Appendix B to 10 CFR Part 50. However, this finding is of minor significance because a subsequent safety assessment showed that the consequence of a steam bypass control system anomaly following a loss-of-offsite power remains bounded by the consequences documented in the plant safety analysis. This minor finding is not subject to enforcement action in accordance with Section IV or VI of the NRCs Enforcement Policy. Problem identification and resolution cross-cutting aspects were associated with this finding because additional transient responses were identified but not captured in the corrective action program. | |||
(Closed) Unresolved Item 05000530/2004012-006: Unit 3, Main Generator Excitation Controls Failure | |||
====a. Inspection Scope==== | |||
The June 14, 2004, electrical fault caused voltage to be reduced at the PVNGS switchyard. As the voltage collapsed the three generators boosted their reactive power output in an attempt to increase switchyard voltage. The generator reactive power output is increased by raising the generator field current. During normal operation this is accomplished automatically by the voltage regulation system. The extended fault resulted in a collapse of the transmission system and the three PVNGS units to be isolated. The resultant load rejection caused the turbine to trip on overspeed, the generators to trip on low volts/hertz ratio and the reactor coolant pumps tripped on undervoltage. This is the as expected sequence of events for a load rejection scenario. | |||
In Unit 3, it was determined that the reactor coolant pumps remained connected to the substation bus for approximately 5 seconds longer than at the other two units. This resulted in the Unit 3 reactor coolant pumps operating at a higher frequency, due to an increase in generator frequency following the load rejection, with a resultant increase in core flow to 108.2 percent of design flow. The increased flow caused a power excursion to 109 percent reactor power and a subsequent reactor trip due to a variable over-power-trip signal. Units 1 and 2 were not exposed to this condition since their respective reactor coolant pumps tripped on undervoltage before the generator frequency ramped upward. | |||
The licensee determined that the cause of the anomaly in Unit 3 was due to a failure of the main generator excitation limiter circuitry, which caused the field excitation and, therefore, output voltage to remain high during the transient. The high voltage output delayed component load shedding since it increased the time to reach the undervoltage relay setpoints. | |||
This unresolved item was identified by the AIT to review the root and contributing causes of Unit 3 main generator excitation controls response since it operated differently than at Units 1 and 2, and may have contributed to the variable overpower trip at Unit 3. In addition it includes an evaluation of the licensees extent of condition review and any associated corrective actions. | |||
The inspectors reviewed the licensee root-cause report to ensure the cause of the transient was well understood and the corrective actions appropriate to address the cause of the failure. In addition the inspectors reviewed the generator transient response at each unit. The inspectors also evaluated a vendor analysis related to core flow at Unit 3 during the event and associated impacts due to hydraulic considerations and industry operating experience associated with overfrequency events. | |||
====b. Observations and Findings==== | |||
The inspectors found that the root-cause identified by the licensee was appropriate. | |||
The licensee determined that the transient differences at Unit 3 were due to a failure of the main generator excitation limiter circuit, in that, it failed to switch to a lower controlling reference signal at the appropriate time resulting in higher bus voltages. The main generator excitation limiter circuit is designed to protect the generator from excessive field currents (If) by providing two functions. A field current limit is set at 1.4 per unit (PU) of 100 percent of the rated field current. This limit protects solid state components in the excitation bridge circuit. The second function is a generator field maximum excitation limit, which is designed to protect the generator field from overheating. When If exceeds 1.05 PU for a specified time period, this function reduces the field current limit from 1.4 to 1.01 PU. This limit is held until If is less than 1.05 and the ac regulator is calling for a lower signal than 1.01 PU. Once this occurs, normal field regulation is resumed and the field current limit is reset. If this does not occur in a preset time then control is switched to a fixed field regulator set at 1.0 PU and the generator will be tripped if If cannot be reduced to less than 1.05 within 4 seconds. | |||
From a review of the field currents of the Unit 3 generator, it was found that main generator excitation limiter attempted, but failed, to switch the reference limit from 1.4 to 1.01 PU as required. A work order was written to repair the circuit, however, the licensee decided to wait until the next outage to perform the repairs since it could not be performed with the generator on line. An engineering position paper was attached to the associated condition report/disposition request to provide justification for this decision. It determined that delaying the repair was acceptable based on the most probable cause of the failure and that redundant protection would protect the main generator from damage. | |||
The inspectors reviewed the hydraulic analysis that stated the reactor coolant system flow during the transient reached a maximum of 108.2 percent of design flow. This was determined to be acceptable since it did not exceed the evaluated limit of 110.4 percent of designed volumetric flow and, therefore, would have no negative impact on continued operation with respect to fuel grid-to-rod fretting, vessel hydraulic uplift forces and fuel mechanical design. This review also determined that acceptable margins were maintained during overspeed operation of the reactor coolant pumps. | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance was identified, in that, the licensee failed to follow their procedure for dispositioning a degraded condition for continued use using engineering work orders. Specifically, the licensee failed to place the degraded main generator excitation limiter circuit into the work control process via the appropriate procedure, 81DP-0DC13, Deficiency Work Order, to ensure that it was appropriately evaluated and processed. This was determined to be a violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings. | |||
=====Description.===== | |||
Following the identification of the degraded circuit, the licensee wrote a work order to repair the Unit 3 excitation circuitry. In addition a white paper was attached to the associated condition report/disposition request to provide a basis for delaying the repair until the next outage. The inspectors review of this issue noted that the degraded condition was not processed in accordance with the licensee procedure that provides guidelines for correcting such a condition through engineering work orders. | |||
This work control process is contained in Procedure 81DP-0DC13, Deficiency Work Order. This process is used to ensure that all deficiencies involving structures, systems and components are resolved in order to maintain PVNGS in its designed and documented condition. It requires a description of actions taken to resolve the degraded condition. In addition it requires analysis and the appropriate screening to confirm it will still perform its intended function if corrective actions are to be delayed. | |||
This failure to utilize required procedures represents the human factor contributing aspects of this finding. | |||
=====Analysis.===== | |||
The inspectors determined that this was a performance deficiency since the licensee failed to address the degraded condition through the required process. | |||
Traditional enforcement does not apply since there were no actual safety consequences or potential for impacting the NRCs regulatory function and the finding was not the result of any willful violation of NRC requirements or PVNGS procedures. The examples in Inspection Manual Chapter 0612, Appendix E, were not applicable to this finding. | |||
This finding is greater than minor since it is associated with the human performance attribute of the barrier integrity cornerstone and impacted the cornerstone objective to provide reasonable assurance that physical design barriers, in this case, the fuel cladding, protect the public from radionuclide releases caused by accident or events. | |||
The inspectors noted that the licensee evaluation to delay repairs only addressed concerns with generator protection. This limited approach results from the problem identification and resolution cross-cutting aspects of this finding. While the core hydraulics evaluation determined that no adverse consequences occurred during the June 14, 2004, event based on actual plant conditions at that time, it was not a bounding analysis to verify that flow limits would not be exceeded if a similar event occurred again with the system in a degraded condition. No evaluation was performed that considered setpoint tolerances on the turbine generator to verify flow limits would be bounded below the maximum evaluated limits. In addition, no evaluation looked at the potential impacts due to the associated power excursion and the consequences on departure from nucleate boiling. The licensees evaluation also did not take into account other plant issues that occurred during the transient because of the degraded condition, such as the unanticipated main steam isolation. The inspectors determined that this could impact the fuel cladding design barrier since the degraded main generator excitation limiter circuit caused reactor power and flow to exceed the normal operating design parameters during the event. The basis for the power limit is to protect against positive reactivity excursions, which could result in departure from nucleate boiling and excessive fuel clad temperatures. The flow limit is to ensure hydraulic forces are kept low enough to prevent core mechanical tilt which could cause unanalyzed flux distributions. While an evaluation was not performed, the licensee indicated that preliminary, qualitative assessment indicated the evaluation would demonstrate that no design limits would be exceeded. | |||
The inspectors conducted a Phase 1 SDP screening and determined that the issue was of a very low safety significance since during the June 14 event the reactor protection system operated correctly to mitigate the transient, hydraulic limits were not exceeded and no mitigating systems were rendered inoperable. | |||
=====Enforcement.===== | |||
Criterion V of Appendix B, 10 CFR Part 50 states, in part, that activities affecting quality shall be prescribed by procedures and be accomplished in accordance with these procedures. Contrary to this, PVNGS failed to process the degraded condition of the main generator excitation limiter circuit through the appropriate work control procedures. Due to this, a rigorous evaluation was not performed to ensure that continued operation would not challenge the plant systems or operators. | |||
Because the violation was of very low safety significance and has been entered into the licensees corrective actions program this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy: | |||
(NCV 05000528,529,530/2004013-02). | |||
2.6 (Closed) Unresolved Item 05000530/2004012-007: Unit 3, Reactor Coolant Pump 2B Lift Oil Pump Breaker | |||
====a. Inspection Scope==== | |||
The Unit 3 reactor coolant pump Oil Lift Pump 2B tripped from actuation of its thermal overload relay during restart activities. Troubleshooting, revealed that the motor was drawing a current in excess of its nameplate current and that the thermal overload relay sizing was not in accordance with the current for this motor as required by Calculation 13-EC-PH-250, Overload Relay Heater Sizing Criteria. | |||
Troubleshooting performed subsequent to the trip showed that the motor was drawing a full load current from 11.2 to 10.7 A. The Lift Pump 2B motor thermal overload relay heater had been sized for a nameplate full load current of 10.9 A, in accordance with Calculation 13-EC-PH-250. The motor was replaced in 2000 with a substitute motor having a nameplate full load current of 10.5 A. | |||
====b. Observations and Findings==== | |||
The inspectors determined that the nameplate full load current, the frame size, the NEMA Design Code, and the insulation class of the motor were different than the original design specification. The motor had been replaced, but not with a one-for-one replacement. The differences had not been recognized or evaluated for acceptability by the licensee. In addition, Calculation 13-EC-PH-250 sizes the thermal overload relay heaters based on motor full-load current; therefore, it also became apparent that when the motor was changed, the thermal overload relay heater had not been resized to match the new motor current. Nonetheless, failure to resize the overload relay heater had little to do with the trip experienced; resizing would have only exacerbated the problem by tripping the motor even sooner. | |||
Although it was at first believed that the Reactor Coolant Pump 2B oil lift pump motor tripped because of improper thermal overload relay sizing, it was subsequently determined that the trip resulted from internal motor or pump malfunction. | |||
Based on the inspectors identification that the motor replaced in 2000 was a motor having different parameters from the existing motor, the licensee issued Condition Report/Disposition Request 2737748. This corrective action document cited a failure to meet Section 6.14.3 of Procedure 30DP-9WP02, Work Document Development and Control, which called for the use of approved design output documents when ordering materials (parts) to verify that material (APN) being ordered is the correct material to ensure configuration control is maintained. We found that the lift oil pump motor had been replaced in 2000 with a substitute motor that was not the same type as originally installed and that, as a result, the replacement motor was drawing current in excess of its rating and, therefore, tripped on this occasion. We determined that licensee staff failed to follow the procedure for Work Document Development and Control when replacing the motor in 2000. The procedure specifically requires that approved design output documents be used when ordering materials and parts to verify that material being ordered is the correct material in order to ensure that configuration control is maintained. This procedural requirement was not followed when ordering the replacement motor. This constitutes a finding of minor significance since it is not associated with a safety-related component and, therefore, did not result in a violation of NRC requirements. | |||
The inspectors also noted that Reactor Coolant Pump 2B oil lift pump motor had previously tripped as a result of actuation of its thermal overload relay on July 28, 2003. | |||
Troubleshooting through Work Order 2623530 identified that the motor was Drawing 12.4 A; however, 2 days later, when additional testing was performed under the same work order, the motor current was noted to be between 8.4 and 8.9 A. Since the value was below the full-load current, the licensee concluded that the problem no longer existed and declared the motor operable following a 30 minute run. The inspectors determined that this decision was premature and constituted ineffective corrective action. Since there was clear indication of some form of intermittent motor/pump malfunction from the earlier tests, the failure to pursue this issue and determine a root cause represented a problem identification and resolution cross-cutting aspect of this finding. | |||
2.7 (Open) Unresolved Item 05000528; -529; -530/2004012-008: Unit 3, Low Pressure Safety Injection System In-Leakage | |||
====a. Inspection Scope==== | |||
The augmented inspection team identified a unresolved item associated with Valve RCEV-217, a 14-inch Borg-Warner check valve that began to leak and pressurized the safety injection header to Reactor Coolant Loop 2A in Unit 3, that moderately challenged operators by the unnecessary distraction from emergency procedures. This issue was made unresolved to review the root and contributing causes, extent of condition, and corrective actions associated with the Borg-Warner safety injection check valve leakage; to review the effectiveness of prior corrective actions for previous check valve leakage issues; to evaluate the adequacy of the in-service testing program for demonstrating check valve operability; and to assess the licensees use of industry operating experience and generic communications. | |||
The inspectors reviewed condition report/disposition request evaluations for the check valve leakage identified during the loss-of-offsite power event, and past Borg-Warner check valve deficiencies; interviewed engineering personnel; and reviewed in-service testing program documents. | |||
====b. Observations and Findings==== | |||
The inspectors determined that the cause evaluation and corrective actions taken were based on sound technical information acquired through review of plant response and operator interviews following the loss-of-offsite power event; historical review of in-service testing results; industry operating experience; and plant operating experience from previous check valve leakage issues. | |||
However, as-found alignment and seat conditions of the valve will provide a better understanding as to the cause of this leakage and the adequacy of proposed corrective actions. Therefore, this unresolved item will remain open pending review of maintenance activities on Valve RCEV-217 during Refueling Outage U3R11. | |||
2.8 (Closed) Unresolved Item 05000528; -529; -530/2004012-009: Units 1 and 3, General Electric Magna Blast Breaker Failures | |||
====a. Inspection Scope==== | |||
Subsequent to the loss-of-offsite power event, operators attempted to restore power from the 525kV Switchyard to the station buses. This required operators to close 13.8kV Circuit Breakers 1ENANS06K (on Intermediate Bus IE-NAN-S06) and 3ENANS05D (on Intermediate Bus 3E-NAN-S05). When the operators in Units 1 and 3 each attempted to close the respective breakers, the breakers failed to close. | |||
====b. Observations and Findings==== | |||
The inspectors reviewed the root and contributing causes of the failure, the extent of condition determination, the effectiveness of corrective actions and the results of inspections and tests, which were conducted on the breakers in an attempt to identify the cause of the failure to close. | |||
Circuit breaker components, as well as control circuit components, were tested to determine possible weak links that could have contributed to the failure. Although no direct cause was identified, the licensee determined that the most probable cause was the anti-pump relay, which was identified as an exposed, non-hermetically sealed unit that had exhibited high resistance because of dirt and dust contamination. Appropriate corrective actions were planned to address this problem. Short-term corrective actions included replacement of the anti-pump relay with hermetically sealed units. Long-term corrective action will provide an air-conditioned filtered environment for the 13.8kV switchgear. | |||
The inspectors also reviewed operator actions taken in response to the failure of the circuit breakers to close. Operators in both units elected to conduct maintenance on the circuit breakers, which subsequently resulted in successful closures. However, by initiating maintenance, any evidence that may have determined the cause of failure was obliterated. | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance, with two examples, was identified, in that, the operators failed to implement contingency actions when the breakers did not close. Specifically, operators deviated from Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, when they initiated maintenance on the two failed 13.8kV breakers instead of performing the contingency actions prescribed by the procedure. Additionally, for Unit 1, Emergency Operating Procedure 40EP-9EO07, was inadequate because it did not list all possible contingency actions available to operators. These two issues were determined to be a violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings. | |||
=====Description.===== | |||
The licensee issued Condition Report/Disposition Request 2739844 to document that emergency operating procedure contingency actions were not followed in the case of one breaker and that available contingency actions were not included in the procedure, for the other breaker. In the first case, operators deviated from the Loss of Offsite Power/Loss of Forced Circulation procedure when they did not perform the required contingency actions prescribed by the procedure when the circuit breaker did not close. Instead, operators initiated maintenance on the failed breaker in an attempt to repair it. In the second case, all available Unit 1 contingency actions to restore power to the electrical bus were not listed in the procedure. In the first case, the violation represents a failure to follow procedures and the second example represents an inadequate procedure. | |||
=====Analysis.===== | |||
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. The inspectors conducted a Phase 1 significance determination procedure screening and determined that the issue was of a very low safety significance. The failure to follow a procedure and the inadequate procedure did not result in the loss of a safety function. | |||
The inspectors noted human performance cross-cutting aspects that contributed to this finding because operators in two units failed to follow contingency actions in emergency operating procedures. | |||
=====Enforcement.===== | |||
Criterion V of Appendix B to 10 CFR Part 50 states, in part, that activities affecting quality shall be prescribed by procedures and be accomplished in accordance with these procedures. Contrary to the above, Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, was not followed by operators and was also inadequate for use in Unit 1 because it did not list all available contingency actions to restore power to the electrical bus when the breaker failed to close. Because both examples of the violation are of very low safety significance and have been entered into the licensees corrective action program, they are being treated as a noncited violations consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-03). | |||
2.9 (Closed) Unresolved Item 05000528; -529; -530/2004012-010: Auxiliary Feedwater (AFW) System Performance | |||
====a. Inspection Scope==== | |||
The AIT identified unresolved item to review the root and contributing causes, extent of condition, and corrective actions associated with the design and operation of the auxiliary feedwater system as a result of a thermally induced vibration that occurred when operators placed the non-essential auxiliary feedwater system into service and which may also have involved procedural issues. | |||
The inspectors reviewed the licensees evaluation and proposed corrective actions documented in Condition Report/Disposition Request 2715731, interviewed engineering and operations personnel, and performed system walkdowns to determine the adequacy of the licensees response. | |||
====b. Observations and Findings==== | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance was identified for untimely implementation of corrective actions to ensure that the feedwater system was operated in a manner that would minimize the possibility of thermally induced vibration that could affect auxiliary feedwater system operability. | |||
=====Description.===== | |||
The licensee was not able to conclude that the thermally induced vibration was initiated by the starting of the non-essential auxiliary feedwater pump or if portions of the non-essential auxiliary feedwater train were involved. The inspectors were unable to support the licensees conclusion based on independent review of the facts. The inspectors observed that inaccurate information was used in the condition report/disposition request analysis, which could have impacted the results of the evaluation. Specifically, the evaluation stated that the auxiliary operator heard the loud banging associated with a hydraulic transient from both the 100 and 140 foot elevations of the turbine building, which helped the operator conclude that the source of the noise was in the vicinity of Heater Drain Tank B. Independent interviews conducted by the inspectors revealed that the auxiliary operator only heard the loud noise from the 140 foot elevation of the turbine building, from the direction of Heater Drain Tank B. | |||
The inspectors noted that from where the operator was standing, the sounds were also in the same direction of portions of the non-essential auxiliary feedwater train. | |||
Moreover, the auxiliary operator could not conclusively determine the source of the loud noises associated with the thermally induced vibration through direct observation. | |||
Following a report of the loud noises, engineers performed a system walkdown and identified a pipe clamp that had rotated on a vertical section of auxiliary feedwater piping. This evidence of physical damage to the potentially affected piping was not used by the licensee to form the basis for the inconclusive evaluation. Based on the facts considered, the inspectors concluded that the thermally induced vibrations most likely occurred in portions of the non-essential auxiliary feedwater train. | |||
As was determined by the AIT, the transient was not severe enough to cause an operability concern. The licensee has proposed corrective actions to develop procedural guidance for restoration of condensate and feedwater systems, after a reactor trip with a loss-of-offsite power, to minimize water hammer events. At the time of the inspection there was nothing to preclude a repetition of this event since corrective actions were not scheduled for implementation until the end of November. Although the inspectors determined the proposed corrective actions were appropriate, the prioritization and timeliness of these proposed actions were inadequate because of the human factors cross-cutting aspects that did not consider this event attributable to placement of the non-class auxiliary feedwater pump in service. | |||
=====Analysis.===== | |||
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment reliability. Using the Significance Determination Process Phase 1 worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function. The inspectors noted human performance cross-cutting aspects that contributed to this finding, in that, engineering personnel did not properly use all information and facts available to determine if a condition adverse to quality existed that warranted prompt corrective actions. | |||
=====Enforcement.===== | |||
Criterion XVI of Appendix B to 10 CFR Part 50, Corrective Action, requires, in part, that conditions adverse to quality are promptly identified and corrected. | |||
Contrary to the above, the licensee failed to promptly implement corrective actions for a condition that could impact the reliability of the auxiliary feedwater system in a timely manner. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-04). | |||
===3. HUMAN PERFORMANCE AND PROCEDURAL ASPECTS OF THE EVENT=== | |||
A number of unresolved items associated with human performance and procedures were revealed during and following the event. Each of these issues was inspected to assess the licensees effectiveness in determining the root and contributing causes, the extent of condition, and corrective actions. | |||
3.1 (Closed) Unresolved Item 05000528; -529; -530/2004012-011: Auxiliary Feedwater System Operation | |||
====a. Inspection Scope==== | |||
During the loss-of-offsite power event, control room operators manually initiated a main steam isolation signal actuation in accordance with Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, Step 6. In addition to closing the main steam isolation valves, this step also causes closure of drains associated with two critical steam traps required to maintain operability of the turbine-driven auxiliary feedwater pump. With the steam traps unavailable, condensate could accumulate in the steam lines and lead to an overspeed trip of the turbine during startup. Guidance established in Normal Operating Procedure 40OP-9SG01, Main Steam, Revision 37, provides the necessary instructions for manually draining those sections of piping necessary to maintain operability of the pump. | |||
The AIT identified a unresolved item to review the root and contributing causes, extent of condition, and corrective actions associated with emergency operating procedure implementation, the availability of equipment to accomplish manual drains on the turbine-driven auxiliary feedwater system, and the decision-making process for implementing manual drain procedures. | |||
The inspectors reviewed the licensees evaluation and associated corrective actions documented in Condition Report/Disposition Request 2719463; interviewed engineering and operations personnel; and performed walkdowns of the equipment to accomplish manual drains on the turbine-driven auxiliary feedwater system. | |||
====b. Observations and Findings==== | |||
===.1 Inadequate Emergency Operating Procedure=== | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance was identified for an inadequate procedure in that emergency operating procedures failed to provide direction to maintain turbine-driven auxiliary feedwater pump operability following a main steam isolation signal. | |||
=====Description.===== | |||
The AIT noted that the emergency operating procedure did not caution the operators that a main steam isolation signal would potentially make the turbine-driven auxiliary feedwater pump inoperable. The emergency operating procedure also did not direct the operators to implement the applicable sections of Normal Operating Procedure 40OP-9SG01, Main Steam, Revision 37, which provide the necessary instructions for manually draining those sections of piping necessary to maintain operability of the pump. This procedure requires that the piping associated with the critical steam traps be blown down every 2 hours until a dry steam condition is reached and then every 6 hours thereafter. On the day of the event, operators did not commence actions to drain the associated piping until 11 hours after the reactors tripped. | |||
=====Analysis.===== | |||
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process Phase 1 worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function of a single auxiliary feedwater train for greater than the technical specification allowed outage time. The inspectors noted problem identification and resolution aspects that contributed to this finding associated with incomplete corrective actions to provide instructions for manually draining those sections of piping necessary to maintain pump operability. The licensee revised surveillance and operating procedures in response to turbine-driven auxiliary feedwater pump overspeed events in 1990, but overlooked the need to provide the necessary instructions in emergency operating procedures. | |||
=====Enforcement.===== | |||
Technical Specification 5.4, Procedures, requires that written procedures be established, implemented, and maintained covering the applicable procedures recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978. Regulatory Guide 1.33, Appendix A, Section 6c, recommends procedures for combating loss of electrical power events. Contrary to the above, Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, was inadequate, in that, the emergency operating procedure did not caution the operators that a main steam isolation signal would potentially make the turbine-driven auxiliary feedwater pumps inoperable and did not provide the necessary instructions to maintain operability of the pump. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-05). | |||
===.2 Failure to Manage Risk Profile=== | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance was identified for failing to perform a risk assessment to address the changed plant conditions following the June 14, 2004, loss-of-offsite power event, which affected the licensees decision making process for prioritization of equipment restoration. | |||
=====Description.===== | |||
Following the loss-of-offsite power event and stabilization of the units, the licensee did not immediately recognize that a risk evaluation needed to be performed. A management review team was held on the afternoon of June 14 to discuss current unit status and prioritize recovery efforts. The licensee failed to recognize that a risk management action level of RED existed on Unit 2 in accordance with Procedure 30DP-9MT03, Assessment and Management of Risk When Performing Maintenance in Modes 1 - 4, Revision 8. The RED risk management action level condition on Unit 2 was because of the combination of having both the turbine-driven auxiliary feedwater pump and Emergency Diesel Generator 2A inoperable. | |||
Procedure 30DP-9MT03 delineates management controls for a RED risk management action level, which includes focusing with the highest priority, to restore out-of-service equipment rapidly. Management inappropriately decided to begin draining the Unit 1 turbine-driven auxiliary feedwater pump steam traps first, without consideration of the higher risk configuration of Unit 2. | |||
The AIT noted that with Unit 2 having only one of two emergency diesel generators available, it was a more prudent decision to restore the Unit 2 turbine-driven auxiliary feedwater pump to service first. The inspectors observed during the followup inspection that this NRC identified issue was not addressed by the licensees corrective action program. The licensee initiated Condition Report/Disposition Request 2735952 to evaluate this issue following the inspectors identification of this oversight. | |||
=====Analysis.===== | |||
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process Phase 1 Worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function of a single auxiliary feedwater train for greater than the technical specification allowed outage time. The inspectors noted human performance cross-cutting aspects associated with poor decision making that contributed to the cause of this finding. The inspectors also noted problem identification and resolution aspects associated with this finding in that this NRC identified issue was not addressed in the licensees corrective action program until prompted by the NRC. | |||
=====Enforcement.===== | |||
Requirements for monitoring the effectiveness of maintenance at nuclear power plants, 10 CFR Part 50.65, Section (a)(4), states, in part, that before performing maintenance activities, the licensee shall assess and manage the increase in risk that may result from the proposed maintenance activities. Contrary to the above, the licensee did not assess and manage the increase in risk associated with the prioritization of equipment restoration and unit recovery. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-06). | |||
3.2 (Closed) Unresolved Item 05000529/2004012-012: Unit 2, Train E Positive Displacement Charging Pump Trip | |||
====a. Inspection Scope==== | |||
The AIT identified a unresolved item associated with multiple operator errors that occurred during Unit 2 charging pump operations. The errors occurred while implementing Procedures 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, and 40EP-9EO10, Standard Appendices, Revision 33, in response to the June 14, 2004, loss-of-offsite power event. This issue was made unresolved to review the root and contributing causes, extent of condition, and corrective actions associated with the operator errors. | |||
The inspectors reviewed the licensees evaluation and associated corrective actions documented in Condition Report/Disposition Requests 2716521 and 2716806. | |||
====b. Observations and Findings==== | |||
=====Introduction.===== | |||
A noncited violation of very low safety significance was identified for multiple examples of failing to follow emergency operating procedures. | |||
=====Description.===== | |||
The AIT reviewed the emergency operating procedures and the control room operator response to the loss-of-offsite power event with respect to the charging pumps to determine the effect on the response to the event. The team also interviewed plant personnel and reviewed Condition Report/Disposition Requests 2716521 and 2716806 regarding the activities surrounding the charging pump operation. The licensees evaluation of the condition was consistent with observations made by the AIT. | |||
No additional observations were identified associated with the operators response following the loss-of-offsite power event. | |||
The inspectors observed that the evaluations performed by the licensee only addressed operational issues with the auxiliary operator. The NRC identified issue associated with procedural adherence by the control room supervisor was not addressed by the licensees corrective action program. The licensee initiated Condition Report/Disposition Request 2736503 to evaluate this issue following the inspectors identification of this oversight. The licensees evaluation concluded that the control room supervisor did not implement step 11 of Procedure 40EP-9EO07 in accordance with expectations. Sub-step 11b should have been performed when step 11a was actually implemented. The licensee also concluded that the error was the result of the control room supervisors mind set of unavailability of motor operated valves without power. | |||
=====Analysis.===== | |||
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process Phase 1 worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone, did not result in the actual loss of a safety function, and no significant delays occurred that adversely impacted operator response to the loss-of-offsite power event. The inspectors noted human performance cross-cutting aspects that contributed to this finding in that operators lacked attention to detail when implementing emergency operating procedures. The inspectors also noted problem identification and resolution aspects associated with this finding in that this NRC identified issue was not addressed in the licensees corrective action program until prompted by the NRC. | |||
=====Enforcement.===== | |||
Technical Specification 5.4, Procedures, requires that written procedures be established, implemented, and maintained covering the applicable procedures recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978. | |||
Regulatory Guide 1.33, Appendix A, Section 6c, recommends procedures for combating loss of electrical power events. Contrary to the above, operations personnel failed to properly implement Emergency Operating Procedures 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, and 40EP-9EO10, Standard Appendices, Revision 33. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-07). | |||
3.3 (Closed) Unresolved Item 05000528; -529; -530/2004012-013: Entry Into Technical Specification Action Statements | |||
====a. Inspection Scope==== | |||
The AIT identified an unresolved item to review how technical specifications are used during and following an event in which emergency operating procedures were used. | |||
Specifically, the team observed that technical specification limiting conditions for operation were not started until the applicable step in the emergency operating procedure was reached to assess limiting conditions for operation. The inspectors evaluated control room log entries associated with the plant trips caused by the loss-of-offsite power. The inspectors also assessed the operator response as it related to the required entry into Technical Specification Action Statements. The inspection was accomplished through a review of documents and interviews with operators and engineering staff. | |||
====b. Observations and Findings==== | |||
The team found that in each of the following examples, the time of entry into the limiting conditions for operation did not reflect the time of discovery of the inoperability of the affected components. | |||
A review of the Unit 2 control room log entries disclosed that operators exited the emergency operating procedure at 5:10 a.m. MST on June 15, 2004. Co-incident with this log entry were the entries into Technical Specifications Limiting Conditions for Operation 3.7.5 for an inoperable turbine-driven auxiliary feedwater pump and Limiting Conditions for Operation 3.8.1 for an inoperable Train "A" emergency diesel generator. | |||
The emergency diesel generator was not operable shortly after the reactor trip because a failed diode in the exciter prevented it from accepting loads from the load sequencer (Section 2.4). When the manual main steam isolation signal actuation occurred, the turbine-driven auxiliary feedwater system steam trap drains were isolated, which could cause the turbine-driven auxiliary feedwater pump to become inoperable without manual action to drain the associated piping within 2 hours (Sections 2.9 and 3.1). The manual action did not occur until approximately 11 hours after the main steam isolation signal actuation. Consequently, both components were considered inoperable prior to exiting the emergency operating procedure. | |||
During the plant transient, the battery chargers to the Unit 2 A and C Vital 125 V batteries were not operable for approximately 2 hours when the Train "A" electrical bus was not powered by either offsite power or the emergency diesel generator. Technical Specification 3.8.4 requires that, within 1 hour, battery cell parameters be verified to meet Table 3.8.6-1, Category "A" limits when the required battery charger is inoperable. | |||
The batteries were discharged for 110 minutes until offsite power was restored to the electrical bus and the battery charger. The entry into the required technical specification action was not documented in the control room log and the action to verify battery cell parameters was not taken until approximately 5 hours after the battery charger became inoperable. Additionally, the batteries were declared operable solely on the restoration of offsite power to the bus and battery charger and without any surveillance to verify compliance with the technical specification. | |||
The Unit 3 Loop 2A Safety Injection Check Valve SIE-V217, is a 14-inch swing check valve. At 10:12 a.m. on June 14, an alarm indicating back leakage through this check valve was received. Alarm Response Procedure 40AL-9RK2B, requires that, when indicated pressure is greater than 1850 psig, Low Pressure Safety Injection Train "B" be declared inoperable and Technical Specification 3.5.3 be entered. At 8:44 p.m. on June 14, 1850 psig was exceeded. Entry into Technical Specification 3.5.3 was logged as being the time that the loss-of-offsite power emergency operating procedure was exited, 12:40 a.m. on June 15, 2004, and not at the time that 1850 psig was exceeded. | |||
The Normal Operating Procedure 40DP-9OP02, Conduct of Shift Operations, Revision 28, requires that when reliable plant indication identifies a condition that requires entry into a technical specification condition, the applicable condition shall be entered immediately. The logging of entry into the applicable limiting conditions for operation after the time of discovery created the potential for failing to meet technical specification requirements. | |||
As a result, the 72 hour clock for the inoperable Unit 3 diesel-generator was not started until 22 hours after it was known to have become inoperable. Although the 72 hour clock was not violated and technical specifications were satisfied, this failure to enter the limiting conditions for operation resulted from inadequate guidance in your Emergency Operating Procedure Users Guide Procedure. We determined that this procedure did not adequately ensure compliance with technical specification requirements. If this erroneous practice were to be left uncorrected, the implementation of this procedure could result in exceeding allowed outage times. This represents an inadequate procedure and a failure to meet the requirements of 10 CFR Part 50, Appendix B, Criterion V. Because the technical specifications were not violated, this is a minor finding, which is not subject to enforcement action in accordance with Section IV or VI of the NRCs Enforcement Policy. However, human performance cross-cutting aspects contributed to this finding because operators lacked attention to detail when implementing technical specification requirements. | |||
3.4 (Closed) Unresolved Item 05000528; -529; -530/2004012-014: Technical Support Center Emergency Diesel Generator Trip | |||
====a. Inspection Scope==== | |||
: | |||
The AIT identified that during the loss-of-offsite power event of June 14, 2004, the technical support center emergency diesel generator failed to supply power as designed to the technical support center and other plant loads. The inspectors reviewed the investigation report associated with Condition Report/Disposition Request 2715749, Emergency Response Organization Response to Reactor Trips on 06/14/04, dated September 3, 2004. The inspectors reviewed the corrective actions identified, and the status of completion of the corrective actions. The inspectors evaluated the completeness of those corrective actions against the standards of condition report/disposition request procedure and 10 CFR Part 50, Appendix E. The inspector also reviewed other documents as listed in the Appendix to this report. | |||
=====Introduction.===== | |||
The inspectors identified a noncited violation of 10 CFR 50.54(q) for failure to follow the emergency plan. Specifically, the finding involved the failure of the technical support center to be available to the Emergency Response Organization for performance of their emergency response functions during the ALERT declared on June 14, 2004. | |||
=====Description.===== | |||
On June 8, 2004, while conducting Work Order 2623863, Monthly Inspection of Technical Support Center DG Battery and Battery Charger, the maintenance technician identified a need to revise the work order due to excessive corrosion on the post and connector on the No.1 battery negative connection. Approval of the scope change, which involved removing the battery connector, and cleaning the negative terminal and connector, and starting the emergency diesel generator for a re-test, was approved by the electrical maintenance team leader. However, detailed retest procedure steps were not added to the work order to conduct a retest of the technical support center emergency diesel generator following the completion of the battery terminal maintenance. | |||
Nuclear Administrative and Technical Manual Procedure 30DP-9MP01, Conduct of Maintenance, Revision 36, step 3.4.7, requires that Changes, amendments, additions of modifications to work order documents shall be accomplished in agreement with 30DP-9WP02, Work Document Development and Control. Revision 34, step 6.2.1 of 30DP-9WP02, requires that Changes to Work Orders (Including Pen and Ink changes)shall be handled as described in Appendix O - Work Order Change. Appendix O, step 2.2.1.2, requires that the scope change be approved by the team leader and operations. | |||
A retest was required by the approved change to the work order, however, the retest procedure was not written in detail to ensure restoration of the technical support center diesel generator to a standby lineup. When the maintenance technician completed the retest of the emergency diesel generator, he failed to restore the idle/run switch to the run position, as would have been required as the last step in the retest procedure for starting the technical support center emergency diesel generator. The retest procedure would not meet the criteria for tool pouch work, as described in Appendix D to Procedure 30DP-9MP01 and, therefore, would require a formal procedure to be used and in-hand when conducting the maintenance. | |||
Failure to restore the idle/run switch to the run position disabled the auto-load feature of the emergency diesel generator. Following a demand signal, the emergency diesel generator would come up to idle speed, but would not load onto the E-NGN-L50 bus and, therefore, the loads powered from that bus remained de-energized. Some of the loads on that bus include the technical support center computers, lighting and receptacles, the technical support center emergency diesel generator cooling fan, and some lighting at the primary access point. Without power to the technical support center emergency diesel generator cooling fan, the emergency diesel generator would approach an automatic shutdown condition within a few minutes of starting because of high temperature in the emergency diesel generator cooling water. | |||
Following the loss-of-offsite power on June, 14, 2004, the technical support center emergency diesel generator started and ran at idle speed, but did not load onto the L50 bus. Within several minutes, the emergency diesel generator automatically shutdown because of high cooling water temperature. The combination of the loss-of-offsite power and the failure of the technical support center emergency diesel generator resulted in no power to the technical support center until offsite power was restored. | |||
This failure resulted from human performance aspects that contributed to this finding. | |||
The licensee took several corrective actions related to this issue. On June 16, 2004, Work Order 2715869 was completed to verify the operability of the technical support center emergency diesel generator. The event was reported in the Electrical Industry Events for operational experience benefit of other power plants, and maintenance technicians were provided specific coaching and positive discipline. The licensee identified 2 additional actions related to maintenance processes to be completed to prevent recurrence of this issue and one additional corrective action to provide an alarm at the technical support center annunciator panel when the technical support center emergency diesel generator is in other than an automatic start and load configuration. | |||
All of the corrective actions were assigned completion dates prior to December 3, 2004, and an effectiveness review was assigned a completion date of August 31, 2005. The inspectors concluded the corrective actions taken were adequate to address the issue, and when completed, the identified corrective actions were adequate to prevent recurrence. | |||
=====Analysis.===== | |||
The failure of the technical support center emergency diesel generator to be maintained in an operable condition was a performance deficiency, in that, adequate facilities and equipment for emergency response are required to be maintained to allow emergency responders to perform their emergency functions and current licensee procedures were adequate to prevent the maintenance error that resulted in disabling the technical support center emergency diesel generator. The failure to maintain the technical support center emergency diesel generator in an operable condition also had human performance/cross cutting aspects that are noted in Section 4OA4 of this report. | |||
The issue is more than minor because it is associated with the Emergency Preparedness Cornerstone attribute of Facilities and Equipment, and affected the cornerstone objective in that loss of an emergency facility and associated emergency equipment would affect the licensees ability to implement adequate measures to protect the health and safety of the public. | |||
The finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem. | |||
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 8 is considered a failure to comply with Emergency Planning Standard 8 during an actual event implementation. The event was a declared an Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609 Appendix B, §2.0. Therefore, the finding is of very low safety significance. | |||
=====Enforcement.===== | |||
10 CFR 50.54(q) provides, in part, that "[a] licensee authorized to possess and operate a nuclear power reactor shall follow . . . emergency plans which meet the standards in [section] 50.47(b). . . ." 10 CFR. 50.47(b) requires that the onsite emergency response plans for nuclear power reactors must meet each of 16 planning standards, of which, Standard 8 states, in part, the ". . . facilities and equipment required for emergency response shall be maintained. . . ." | |||
Contrary to the above, on June 14, 2004, following the loss-of-offsite power and failure of the technical support center emergency diesel generator to supply power to the technical support center, the technical support center was not available to support the emergency response organization. Failure of the technical support center to be available resulted in additional challenges to the emergency response organization to coordinate site-wide efforts to respond to the event and provide accurate and timely notification to the state and local officials. | |||
Because the failure to implement the emergency planning standard was of very low safety significance and has been entered into the licensees corrective action program as Condition Report/Disposition Request 2715726, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-08). | |||
3.5 (Closed) Unresolved Item 05000528; -529; -530/2004012-015: Emergency Response Organization Issues | |||
====a. Inspection Scope==== | |||
The AIT identified three issues associated with this unresolved item. The first was the accuracy and timeliness of notification of state and local officials of the emergency events at the PVNGS. The second was the unavailability of radiological dose projection computers during the loss-of-offsite power event. The third was the delays in notification of the PVNGS emergency response organization and the subsequent failure to meet emergency facility staffing timeliness goals. The inspectors reviewed the investigation report associated with Condition Report/Disposition Request 2715749, Emergency Response Organization (ERO) Response to Reactor Trips on 06/14/04, dated September 3, 2004. The inspectors reviewed the corrective actions identified and the status of completion of the corrective actions. The inspectors evaluated the completeness of those corrective actions against the standards of the condition report/disposition request procedure and 10 CFR Part 50, Appendix E. The inspector also reviewed other documents as listed in the Appendix to this report. | |||
====b. Observations and Findings==== | |||
===.1 Timely and Accurate Notification of State and Local Officials=== | |||
=====Introduction.===== | |||
The inspectors identified a noncited violation of 10 CFR 50.54(q) for failure to follow the emergency plan. Specifically, the licensee failed to ensure that adequate command and control was established during the June 14, 2004, Alert event where multiple units were in a declared emergency condition. Failure to establish adequate command and control resulted in the notification irregularities to the state and local officials. | |||
=====Description.===== | |||
On June 14, 2004, at 7:41am local time, PVNGS, Units 1, 2 and 3 experienced a loss of all offsite power, resulting in automatic reactor trips at all three units. | |||
Unit 1 plant equipment responded as expected. With a loss-of-offsite power continuing for greater than 15 minutes, Unit 1 met the conditions of Emergency Action Level 2-1 for declaration of a Notification of Unusual Event at approximately 7:56 a.m. local time. | |||
Unit 2 plant equipment responded as expected with the exception of the Emergency Diesel Generator 2A, which failed to maintain output voltage and was manually tripped by the Unit 2 operators at 7:50 a.m. local time. With a loss-of-offsite power continuing for greater than 15 minutes, and the failure of one emergency diesel generator to supply power to its respective electrical bus, Unit 2 met the conditions of Emergency Action Level 2-3 to declare an Alert at 7:56 a.m. local time. | |||
Unit 3 plant equipment responded as expected, with the exception of the steam bypass control system, which resulted in an automatic main steam isolation. With a loss-of-offsite power continuing for greater than 15 minutes, Unit 3 met the conditions of Emergency Action Level 2-1 for declaration of a Notification of Unusual Event at approximately 7:56 a.m. local time. The failure of the steam bypass control system did not affect the applicable emergency classification. | |||
Each of the units respective shift managers initially assumed the role of the emergency coordinator. Unit 2 declared an ALERT at 7:54 a.m. local time, and Units 1 and 3 both declared a Notification of Unusual Event at approximately 7:58 a.m. local time. The emergency coordinator for Unit 2 initiated notifications to the state and locals of the ALERT at Unit 2 at 7:59 a.m. local time, but did not include status of the other units. | |||
The emergency coordinator in Unit 3 initiated notifications to the state and locals of the Notification of Unusual Event at Unit 3 at 8:18 a.m. local time, but did not include status of the other units. The emergency coordinator in Unit 1, in consultation with the site shift manager and the Unit 2 operations director, decided that since Unit 2 was in a condition requiring the highest level of declaration of the three units, that the site shift manager should go to the Unit 2 control room and relieve the Unit 2 shift manager as the emergency coordinator. The site shift manager relieved the Unit 2 shift manager as the emergency coordinator at 8:03 a.m. local time. The Unit 2 operations director also stated that Unit 2 would assume the site emergency plan communication requirements. | |||
Based on this communication, the Unit 1 shift manager did not initiate notification of the Notification of Unusual Event declaration at Unit 1, believing that the emergency coordinator in Unit 2 would accomplish the notification. The site manager, as emergency coordinator located in Unit 2, focused on Unit 2 events and did not initiate offsite communications for the other units. Consequently, the state and local officials were not notified of the Notification of Unusual Event at Unit 1. This resulted from a failure to recognize procedural requirements and is a human performance cross-cutting aspect that contributed to this finding. | |||
The declaration of ALERT in Unit 2 required notification of the emergency response organization and activation of the emergency facilities at PVNGS. While the emergency facilities were being staffed, the emergency coordinator in Unit 2 downgraded the Unit 2 classification to an Notification of Unusual Event at 9:51 a.m. local time. When the emergency operation facility was staffed and ready to perform its emergency functions, the Emergency Operations Facility director relieved as the site emergency coordinator. | |||
The site emergency coordinator then initiated the notification to the state and local authorities of the downgrade to an Notification of Unusual Event for Unit 2 at 10:05 a.m., | |||
but did not include status of the other units. This notification erroneously included Emergency Action Levels 2-3 and 8-2, none of which were met at the time of the downgrade. Emergency Action Level 2-3 is an ALERT based on ac power capability to essential busses reduced to a single power source for greater than 15 minutes and Emergency Action Level 8-2 is an Unusual Event based on other conditions existing, which in the judgement of the Emergency Director, warrant declaration of an Unusual Event. The site emergency coordinator terminated the event at 12:07 p.m. local time, and initiated the notification to state and local officials at 12:15 p.m. local time. This final notification also incorrectly included Emergency Action Levels 2-1 and 8-2 as being met at the time of the notification, when they were not. | |||
The licensee took several corrective actions related to this issue. On July 1, 2004, an operations night order was issued to all three units, which summarized the human performance issues associated with the emergency plan activities of June 14, 2004, and also emphasized that the Emergency Plan Implementing Procedure 01, Satellite Technical Support Center Actions, states that for multiple unit events, the Unit 1 shift manager is responsible for initially classifying and declaring the emergency, and assuming the position of the on-shift emergency coordinator. An Operation News Flash issued on July 13, 2004, repeated the message from the July 1, 2004, operations night order. | |||
The licensee identified six additional actions to be completed to prevent recurrence of this issue, and six additional corrective actions to improve communications reliability both between onsite emergency facilities and with the state and local authorities. All of the corrective actions were assigned completion dates prior to December 2004, and an effectiveness review was assigned a completion date of August 31, 2005. The inspectors concluded the corrective actions taken were adequate to address the issue, and when completed, the identified corrective actions were adequate to prevent recurrence. | |||
=====Analysis.===== | |||
Emergency planning standard 10 CFR 50.47(b)(1) states, in part, that responsibility for emergency response is assigned . . . . Section 6.2 of the PVNGS emergency plan requires that, for multiple unit events, the Unit 1 shift manager will assume the position of the onshift emergency coordinator. Emergency Plan Implementing Procedure-01, Section 3.1, states the same requirement as the emergency plan. The onshift emergency coordinator maintains that position until relieved by the emergency facility director in the technical support center or the emergency operations facility. | |||
The issue is a performance deficiency, in that, the licensee failed to follow the emergency plan and Emergency Plan Implementing Procedure 01, in that, the Unit 1 shift manager did not assume the responsibility of the emergency coordinator, resulting in miscommunication between the three units and the offsite officials. The finding is more than minor because it is related to the emergency preparedness cornerstone attribute of response organization performance, and affects the cornerstone objective in that command and control challenges resulting in inaccurate communications to the offsite officials could potentially affect the ability to ensure that adequate measures would be taken to protect the public health and safety. | |||
This finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem. | |||
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 1 is considered a failure to comply with Planning Standard 1 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609, Appendix B, §2.0. Therefore, the finding is of very low safety significance (Green.) | |||
Enforcement . 10 CFR 50.54(q) provides, in part, that "[a] licensee authorized to possess and operate a nuclear power reactor shall follow . . . emergency plans which meet the standards in [section] 50.47(b). . . ." 10 CFR. 50.47(b) requires that the onsite emergency response plans for nuclear power reactors must meet each of 16 planning standards, of which, standard | |||
: (1) states, in part, the ". . . responsibility for emergency response is assigned." The licensees Emergency Plan, Section 6.2, states, For those situations involving more than one unit, the Unit 1 shift manager is responsible for initially classifying and declaring the emergency and assuming the position of the On shift emergency coordinator. | |||
Contrary to the above, on June 14, 2004, following the loss-of-offsite power and other complications to each of the 3 PVNGS units, the Unit 1 shift manager did not assume the emergency coordinator position. Failure to implement the emergency plan requirement affected the ability to coordinate licensee response to the event and provide accurate and timely notification to the state and local officials. | |||
Because the failure to implement the emergency plan requirement was of very low safety significance and has been entered into the licensees corrective action program as Condition Report/Disposition Request 2715749, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy: | |||
(NCV 05000528,529,530/2004013-09). | |||
===.2 Unavailability of the Radiological Dose Projection Computers=== | |||
No significant findings were identified. | |||
Because of the loss-of-offsite power on June 14, 2004, power to the electrical receptacles in the satellite technical support centers of each unit was unavailable. The dose assessment computers in the satellite technical support centers were normally powered by these receptacles. Therefore, without operator action, the dose assessment computers in the satellite technical support centers were not available. | |||
Additionally, the dose assessment computer in the technical support center was unavailable because of the failure of the technical support center emergency diesel generator to supply power to the technical support center. The licensee did identify, during the event, that power was available to the operations support building, and that a computer was available for dose assessment and could be accessed in a timely manner. During the Alert of June 14, 2004, there was no need for dose assessment and the ability to use the operations support building facility to support dose assessment was not demonstrated. The licensee also identified that power was available to certain electrical receptacles in the satellite technical support centers, and that simple operator action to unplug the dose assessment computers and plug them into the energized receptacles was all that was required to restore dose assessment capability in the satellite technical support centers. Dose assessment capability was always available from the emergency operations facility, where the dose assessment computers are equipped with uninterruptible power supplies. | |||
The licensee took adequate corrective actions related to this issue. Uninterruptible power supplies were added to each of the satellite technical support center dose assessment computers. Procedure revisions have been identified to ensure identification of those dose assessment computers with uninterruptible power supplies and ensure quarterly testing of the computers and power supplies. The procedure revisions were assigned a completion date of September 30, 2004, and an effectiveness review was assigned a completion date of August 31, 2005. | |||
===.3 Emergency Response Organization Notification and Staffing Delays=== | |||
=====Introduction.===== | |||
The inspectors identified a noncited violation of 10 CFR 50.54(q) for failure to follow the emergency plan. Specifically, the licensee failed to meet minimum staffing goals of Table 1, Minimum Staffing Requirements for PVNGS for Nuclear Power Plant Emergencies, following the Alert declaration on June 14, 2004. | |||
=====Description.===== | |||
On June 14, 2004, at 7:41 a.m. local time, PVNGS Units 1, 2 and 3 experienced a loss of all offsite power, resulting in automatic reactor trips at all three units. With a loss-of-offsite power continuing for greater than 15 minutes, all three units met the conditions of Emergency Action Level 2-1 for declaration of a Notification of Unusual Event at approximately 7:56 a.m. local time. Unit 2 additionally lost voltage output on the Emergency Diesel Generator 2A, resulting in loss of power to its respective electrical bus. With a loss-of-offsite power continuing for greater than 15 minutes, and the failure of one emergency diesel generator to supply power to its respective electrical bus, Unit 2 met the conditions of Emergency Action Level 2-3 to declare an Alert at 7:56 a.m. local time. | |||
The emergency coordinator, based on imminent conditions, declared an Alert at Unit 2 at 7:54 a.m. local time, and commenced notifications to the emergency response organization (these positions are identified in the Emergency Plan, Table 1,) and the state and local officials. The emergency coordinator attempted to initiate a WPAGER system notification to the essential Emergency Response Organization personnel as required by Emergency Plan Implementing Procedure-99, EPIP Standard Appendices, Appendix D, Notification. However, due to the loss-of-offsite power, the WPAGER computers in the control room were not powered. The emergency coordinator did not attempt to initiate the autodialer system, which was available and was the primary method to notify all Emergency Response Organization personnel of the event classification and the need to report to their respective emergency facilities as required by Emergency Plan Implementing Procedure-01 and Emergency Plan Implementing Procedure-99, Appendix D. After approximately 31 minutes from the Alert declaration, the WPAGER system notification was initiated from an operations support building computer, which had not lost power as a result of the loss-of-offsite power. The licensee indicated that based on the average amount of time required for the communicator to initiate an autodialer notification following an Alert declaration, the actual delay in initiating the Emergency Response Organization notification was approximately 16 minutes. | |||
The licensee noted in the root-cause report that the emergency coordinator chose not to use the autodialer system because he believed he was following the guidance in Emergency Plan Implementing Procedure 01, which stated, During off-normal hours, if an Alert or higher is declared then direct activation fo the Autodialer IAW EPIP-99 Standard Appendix - H Autodialer Activation. The emergency coordinator failed to realize that Monday, June 14, 2004, was a work day off for non-operations shift personnel and, therefore, should have been considered as off-normal hours. | |||
Additionally, the onshift Emergency Response Organization staff was not aware that the autodialer system also provided an automatic initiation of the WPAGER system. | |||
As a result of the delay in notification to the essential Emergency Response Organization members, and no notification to the non-essential Emergency Response Organization members, emergency facility staffing goals for essential Emergency Response Organization members were not met, and many of the non-essential Emergency Response Organization members never responded to the Alert declaration, as required by the facility emergency plan. | |||
The technical support center was staffed with all of its emergency plan required essential responders approximately 2 hours and 6 minutes after the Alert declaration. | |||
Only five of the required six radiation protection technicians were identified as having responded to the operations support center. Other radiation protection technicians were identified as being onsite, but they did not report to and staff the operations support center and would have, therefore, not been immediately available to perform work functions. Many of the non-essential support positions for the designated essential responders did not staff their respective emergency facilities during the event. | |||
The licensee took several corrective actions related to this issue. On July 1, 2004, and again on July 13, 2004, operations night orders were issued to all three units, which summarized the human performance issues associated with the emergency plan activities of June 14, 2004. Emergency Plan Implementing Procedures 01 and 03 were revised to remove the qualifier that the autodialer should be activated during off-normal hours, now requiring its activation for any Alert declaration or higher. | |||
The licensee identified eight additional corrective actions including training on multiple unit events, clarification of the use of the autodialer as a backup to the WPAGER system, and addition of uninterruptible power supplies to select computers and the site pager tower. All of the corrective actions were assigned completion dates prior to December 13, 2004, and an effectiveness review was assigned a completion date of August 31, 2005. The inspectors concluded the corrective actions taken were adequate to address the issue, and when completed, the identified corrective actions were adequate to prevent recurrence. | |||
=====Analysis.===== | |||
Emergency planning standard 10 CFR 50.47(b)(2) states, in part, that process for timely augmentation of on-shift staff is established and maintained . . . . | |||
Table 1 of the PVNGS emergency plan establishes the staffing requirements for augmenting emergency responders. For the technical support center electrical engineer position and the six operations support center radiation protection technicians, Table 1, requires a maximum response time of 2 hours. The Emergency Plan, Section 6.2, also requires that the entire Emergency Response Organization be notified of the declared emergency and to respond to their designated emergency facilities. | |||
The issue is a performance deficiency in that the licensee failed to follow the emergency plan and Emergency Plan Implementing Procedures 01 and 03, in that, the entire Emergency Response Organization was not notified and all essential positions were not staffed in the required times of Table 1. The finding is more than minor because it is related to the emergency preparedness cornerstone attribute of response organization performance, and affects the cornerstone objective in that failure to fully staff the emergency response facilities could potentially affect the ability to ensure that adequate measures would be taken to protect the public health and safety. | |||
This finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem. | |||
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 2 is considered a failure to comply with Planning Standard 2 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609, Appendix B, §2.0. Therefore, the finding is of very low safety significance. | |||
Enforcement . 10 CFR 50.54(q) provides in part that "[a] licensee authorized to possess and operate a nuclear power reactor shall follow . . . emergency plans which meet the standards in [section] 50.47(b). . . ." 10 CFR. 50.47(b) requires that the onsite emergency response plans for nuclear power reactors must meet each of 16 planning standards, of which, standard | |||
: (2) states, in part, the ". . .process for timely augmentation of onshift staff is established and maintained . . . ." The licensees Emergency Plan, Table 1, establishes that the technical support center electrical engineer position and the six operations support center radiation protection technicians positions be filled within 2 hours of the declaration of an Alert or higher. The Emergency Plan, Section 6.3, also requires that the entire Emergency Response Organization be notified of the declared emergency and to respond to their designated emergency facilities. | |||
Contrary to the above, on June 14, 2004, following the loss-of-offsite power and declaration of an Alert at Unit 2, the entire Emergency Response Organization was not notified of the declared event, and the technical support center electrical engineer, and one of the six required radiation protection technicians, did not report to their emergency facilities within 2 hours of the Alert declaration. | |||
Because the failure to implement the emergency plan requirement was of very low safety significance and has been entered into the licensees corrective action program as Condition Report/Disposition Request 2715749, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528, 529, 530/2004013-10). | |||
==OTHER ACTIVITIES== | |||
{{a|4OA2}} | |||
==4OA2 Problem Identification and Resolution== | |||
Section 2.2 (Unresolved Item 02) describes a finding where the licensee failed to adequately resolve reactor operator comments concerning the apparent malfunction of an atmospheric dump valve and consequently failed to identify needed operator training. | |||
Section 2.4 (Unresolved Item 04) describes a finding where the licensee failed to thoroughly investigate the failure of safety related equipment and did not conduct troubleshooting nor capture the failure in the corrective action program. | |||
Section 2.5 (Unresolved Item 05) describes a finding where licensee personnel became aware of other power transient scenarios where the steam bypass control system could fail to perform as expected and this information was not incorporated in the licensees corrective action program. | |||
Section 2.5 (Unresolved Item 06) describes a finding where the licensees evaluation to delay repairs on a failed main generator excitation control circuit only addressed concerns with generator protection. The evaluation was limited in scope and failed to consider all possible impacts. | |||
Section 2.6 (Unresolved Item 07) describes a finding where a previous trip of a reactor coolant pump lift oil pump motor was discounted without adequate basis. The licensee concluded that the problem no longer existed and declared the motor operable following a 30 minute run. This ineffective corrective action led to a subsequent failure. Since there was clear indication of some form of intermittent motor/pump malfunction from the earlier tests, the failure to pursue this issue and determine a root cause was not adequate. | |||
Section 2.9 (Unresolved Item 10) describes a finding where prioritization and timeliness of proposed corrective actions were inadequate because the engineers did not consider the event attributable to the identified root cause, which was the effect of the placement of the non-class auxiliary feedwater pump in service. | |||
Section 3.1 (Unresolved Item 11) describes a finding where the licensee revised surveillance and operating procedures in response to turbine-driven auxiliary feedwater pump overspeed events in 1990, but overlooked the need to provide the necessary instructions in emergency operating procedures. | |||
Sections 3.1 (Unresolved Item 11) and 3.2 (Unresolved Item 12) describe findings where the NRC identified issues were not addressed in the licensees corrective action program until prompted by the NRC. | |||
{{a|4OA3}} | |||
==4OA3 Event Followup== | |||
(Closed) Licensee Event Report 05000528/2004-006-00: Loss-of-Offsite Power - Three Unit Trip. | |||
The June 14, 2004, a loss-of-offsite power and subsequent reactor trip of all three PVNGS plants was the subject of this licensee event report. The licensee event report was reviewed by the inspectors and no findings of significance were identified. Because the event was the subject of an NRC AIT, and because no additional significant information was contained in the licensee event report, this licensee event report is closed. | |||
{{a|4OA4}} | |||
==4OA4 Human Performance== | |||
Section 2.3 (Unresolved Item 03) describes a finding where engineering personnel did not properly consider a loss of power to the letdown system when designing a temporary modification and, as a result, the system failed to isolate when needed. | |||
Section 2.5 (Unresolved Item 06) describes a finding where required procedures for the dispositioning of a degraded condition for continued use were not utilized and, as a result, a rigorous evaluation was not performed to ensure that continued operation would not challenge plant systems or operators. | |||
Section 2.8 (Unresolved Item 09) describes a finding where operators in two units failed to follow contingency actions in Emergency Operating Procedures. | |||
Section 2.9 (Unresolved Item 10) describes a finding where engineering personnel did not properly use all information and facts available to identify a condition adverse to quality that warranted prompt corrective actions. | |||
Section 3.1 (Unresolved Item 11) describes a finding where poor management decision making led to the failure to assess and manage the increase in risk associated with the prioritization of equipment restoration and unit recovery. | |||
Section 3.2 (Unresolved Item 12) describes a finding where operators lacked attention to detail when implementing procedures that resulted in a failure to follow several emergency operating procedure steps during the response to the loss-of-offsite power event. | |||
Section 3.3 (Unresolved Item 13) describes a finding where operators lacked attention to detail when implementing technical specification requirements. | |||
Section 3.4 (Unresolved Item 14) describes a finding where detailed retest procedure steps were not added to a work order to conduct a re-strike test of the technical support center emergency diesel generator following the completion of the battery terminal maintenance. This error led to the failure of the technical support center emergency diesel generator to function when needed. | |||
Section 3.5 (Unresolved Item 15) describes a finding where a failure to recognize procedural requirements for a single emergency coordinator led to notification irregularities to state and local officials. | |||
{{a|4OA6}} | |||
==4OA6 Exit Meeting== | |||
On September 24, 2004, the team presented the preliminary results from the ongoing AIT followup. On October 26, 2004, the AIT Followup Inspection Team Leader presented the results of the inspection in a meeting held at the PVNGS Nuclear Generating Station site to Mr. G. Overbeck and Mr. J. Levine, and other members of his staff. Mr. Overbeck acknowledged the inspection findings. Proprietary information reviewed by the team was returned to the facility. | |||
ATTACHMENT 1 | |||
=SUPPLEMENTAL INFORMATION= | |||
==KEY POINTS OF CONTACT== | |||
Licensee | |||
Scott Bauer, Department Leader, Regulatory Affairs | |||
David Crozier, Program Leader, Emergency Planning | |||
Carl Churchman, Director, Steam Generator Replacement | |||
Jim Levine, Executive Vice President, Generation | |||
Gregg Overbeck, Senior Vice President, Nuclear | |||
David Mauldin, Vice President, Nuclear Engineering & Support | |||
Paul Mueller, Senior Electrical Engineer | |||
John Holmes, Section Leader, Electrical Engineering | |||
Steven Kestler, Electrical Engineer | |||
Bajranga Aggarwal, Systems Engineer | |||
John Hesser, Director, Emergency Services | |||
Larry Leavitt, Senior Program Advisor, Emergency Services | |||
Randy Sorensen, Department Leader, Chemistry | |||
Terry Radtke, Director, Operations | |||
Fred Riedel, Director, Nuclear Training | |||
Brian Ramey, Section Leader, Reliability Engineering | |||
Mark Van Dop, Department Leader, Design Engineering | |||
Mike Hodge, Engineer | |||
Jim McDowell, Investigator, Nuclear Assurance Department | |||
Kevin Sweeney, Section Leader, System Engineering | |||
Don Wheeler, Section Leader, Nuclear Assurance Department | |||
Mike Winsor, Director, Engineering | |||
Pete Borchert, Director, Work Management | |||
Dave Smith, Plant Manager, Operations | |||
Michael Sontag, Department Leader, Nuclear Assurance Department | |||
John Scott, Department Leader, Nuclear Assurance Department | |||
Steve Kesler, Section Leader, Design Engineering | |||
Dan Marks, Section Leader, Regulatory Affairs | |||
Craig Seaman, Director, Nuclear Fuel Management | |||
Ken Manne, Senior Attorney | |||
Michael Grigsby, Department Leader, Operations | |||
Donald Vogt, Section Leader, Operations | |||
Mark McGhee, Department Leader, Operations | |||
Don Straka, Senior Consultant, Regulatory Affairs | |||
Harvey Leake, Senior Consultant, Design Engineering | |||
John Hesser, Director, Emergency Services | |||
1-A Attachment 1 | |||
ITEMS CLOSED | |||
05000528/2004012-01; URI Corrective actions to improve the reliability and | |||
05000529/2004012-01; independence of offsite power (Section 2.1). | |||
05000530/2004012-01 | |||
05000528/2004012-02 URI Unit 1 Atmospheric Dump Valve 185 Failure | |||
(Section 2.2). | |||
05000528/2004012-03 URI Unit 1 Letdown System Failure to Isolate | |||
(Section 2.3). | |||
05000529/2004012-04 URI Unit 2 Train "A" Emergency Diesel Generator | |||
Failure (Section 2.4). | |||
05000530/2004012-05 URI Unit 3 Main Turbine Bypass Valve Control System | |||
Operation (Section 2.5). | |||
05000530/2004012-06 URI Unit 3 Main Generator Excitation Controls and | |||
Variable Overpower Trip on June 14, 2004 | |||
(Section 2.5). | |||
05000530/2004012-07 URI Unit 3 Reactor Coolant Pump 2B Lift Oil Pump | |||
Motor Breaker Thermal Overload Sizing | |||
(Section 2.6). | |||
05000528/2004012-09; URI Magna-Blast Circuit Breaker Reliability | |||
05000529/2004012-09; (Section 2.8). | |||
05000530/2004012-09 | |||
05000528/2004012-10; URI Auxiliary Feedwater System Operational Issues | |||
05000529/2004012-10; (Section 2.9). | |||
05000530/2004012-10 | |||
05000528/2004012-11; URI Turbine-driven Auxiliary Feedwater System Drains, | |||
05000529/2004012-11; Design Control, and Procedures (Section 3.1). | |||
05000530/2004012-11 | |||
05000529/2004012-12 URI Unit 2 Charging Pump Operations Errors | |||
(Section 3.2). | |||
05000528/2004012-13; URI Use of Plant Technical Specifications | |||
05000529/2004012-13; (Section 3.3). | |||
05000530/2004012-13 | |||
05000528/2004012-14; URI Technical Support Center Emergency Diesel | |||
05000529/2004012-14; Generator Failure (Section 3.4). | |||
05000530/2004012-14 | |||
2-A Attachment 1 | |||
ITEMS OPENED and CLOSED | |||
Opened and Closed | |||
05000528/2004013-001 NCV Failure to address EDG circuit failure (Section 2.4). | |||
05000529/2004013-001 | |||
05000530/2004013-001 | |||
05000528/2004013-002 NCV Failure to evaluate main generator excitation limiter | |||
05000529/2004013-002 circuit problems (Section 2.5). | |||
05000530/2004013-002 | |||
05000528/2004013-003 NCV Failure to follow and inadequate EOP (Section 2.8). | |||
05000529/2004013-003 | |||
05000530/2004013-003 | |||
05000528/2004013-004 NCV Failure to implement corrective actions for AFW | |||
05000529/2004013-004 (Section 2.9). | |||
05000530/2004013-004 | |||
05000528/2004013-005 NCV Inadequate EOP for AFW operation (Section 3.1). | |||
05000529/2004013-005 | |||
05000530/2004013-005 | |||
05000528/2004013-006 NCV Failure to manage station risk (Section 3.1). | |||
05000529/2004013-006 | |||
05000530/2004013-006 | |||
05000528/2004013-007 NCV Failure to properly implement LOOP EOP (Section 3.2). | |||
05000529/2004013-007 | |||
05000530/2004013-007 | |||
05000528/2004013-008 NCV Technical Support Center unavailable (Section 3.4). | |||
05000529/2004013-008 | |||
05000530/2004013-008 | |||
05000528/2004013-009 NCV Failure properly implement emergency plan (Section 3.5). | |||
05000529/2004013-009 | |||
05000530/2004013-009 | |||
05000528/2004013-010 NCV Untimely augmentation of emergency personnel | |||
05000529/2004013-010 (Section 3.5). | |||
05000530/2004013-010 | |||
3-A Attachment 1 | |||
==DOCUMENTS REVIEWED== | |||
}} |
Latest revision as of 13:14, 15 March 2020
ML043380346 | |
Person / Time | |
---|---|
Site: | Palo Verde ![]() |
Issue date: | 12/03/2004 |
From: | Chamberlain D Division of Reactor Safety IV |
To: | Overbeck G Arizona Public Service Co |
References | |
EA-04-222 IR-04-013 | |
Download: ML043380346 (51) | |
Text
ber 3, 2004
SUBJECT:
PALO VERDE NUCLEAR GENERATING STATION, UNITS 1, 2, AND 3 -
NRC AUGMENTED INSPECTION TEAM FOLLOWUP REPORT 05000528/2004013; 05000529/200413; 05000530/2004013
Dear Mr. Overbeck:
From September 24 through October 26, 2004, the Nuclear Regulatory Commission (NRC)
conducted an onsite Augmented Inspection Team followup inspection at your Palo Verde Nuclear Generating Station, Units 1, 2, and 3. The enclosed report documents the inspection findings, which were discussed following completion of the inspection on October 26, 2004, with you, Mr. Levine, Senior Vice President of Generation, and other members of your staff.
This inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.
The team reviewed selected procedures and records, observed activities, and interviewed personnel.
This inspection report documents ten findings of very low safety significance (Green). The NRC also determined that violations of NRC requirements were associated with these findings.
However, because of the very low safety significance and because the findings were entered into your corrective action program, the NRC is treating these findings as noncited violations consistent with Section V1.A of the NRC Enforcement Policy. If you contest the violations or significance of these noncited violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C. 20555-0001, with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission, Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555-0001; and the NRC Resident Inspector at the Palo Verde Nuclear Generating Station facility.
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Arizona Public Service Company -2-Should you have any questions concerning this inspection, we will be pleased to discuss them with you.
Some of the material enclosed herewith contains exempt information in accordance 10 CFR 2.390(d)(1). Therefore, the applicable material will not be made available electronically for public inspection in the NRC Public Document Room or from the NRC's document system (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html.
Sincerely,
/RA/
Dwight D. Chamberlain, Director, Division of Reactor Safety Dockets: 50-528; 50-529; 50-530 Licenses: NPF-41; NPF-51; NPF-74
Enclosure:
NRC Inspection Report 05000528/2004013; 05000529/200413; 05000530/2004013
REGION IV==
Dockets: 50-528; 50-529; 50-530 Licenses: NPF-41; NPF-51; NPF-74 Report No.: 05000528/2004013; 05000529/2004013; 05000530/2004013 Licensee: Arizona Public Service Company Facility: Palo Verde Nuclear Generating Station, Units 1, 2, and 3 Location: 5951 S. Wintersburg Road Tonopah, Arizona Dates: September 7 through October 26, 2004 Team J. I. Tapia, P.E., Senior Reactor Inspector Leader: Engineering Branch Inspectors: R. E. Lantz, Senior Emergency Preparedness Inspector, Operations Branch G. G. Warnick, Resident Inspector, Projects Branch D M. R. Cox, Resident Inspector, Region I Accompanied F. Baxter, P. E., Electrical Engineer, Beckman and Associates By:
Approved By: Jeff Clark, P. E., Branch Chief Engineering Branch, Division of Reactor Safety I Enclosure
SUMMARY OF FINDINGS
IR 05000528/2004-013, 05000529/2004-013, 05000530/2004-013; 9/7-10/26/2004; Palo Verde
Nuclear Generating Station, Units 1, 2, and 3; Augmented Inspection Followup On June 14, 2004, a ground-fault occurred on a 230kV transmission line, approximately 47 miles from the Palo Verde Nuclear Generating Station. A failure in the protective relaying precluded isolation of the ground-fault from the local grid for approximately 38 seconds and caused a loss-of-offsite power and a trip of all three Palo Verde Nuclear Generating Station units. The Unit 2 Train A emergency diesel generator started, but did not complete the load sequence process due to a failed diode in the exciter rectifier circuit. This resulted in the Train "A" engineered safeguards features busses de-energizing and limiting the availability of certain safety equipment. Because of this failure, the licensee elevated the emergency declaration for Unit 2 to an Alert. All three units were safely shutdown and stabilized under hot shutdown conditions.
An NRC Augmented Inspection Team (AIT) was dispatched to the site and found that the licensees response to the event was generally acceptable, although complicated by a number of equipment failures, procedure issues, and human performance issues with diverse apparent causes and with varying degrees of significance (See NRC Inspection Report 05000528/2004012; 05000529/2004012; 05000530/2004012). The issues requiring additional followup were identified and tracked as unresolved items in the AIT report. The AIT reviewed the licensees immediate corrective actions prior to restart of the units, including actions to improve the independence and reliability of offsite power sources, and found those actions appropriate for continued operation of the units. The AIT charter in accordance with NRC Inspection Procedure 93800, did not direct the team to address compliance or assess significance of findings and observations.
This report documents the followup inspection, which was conducted to address the unresolved issues identified during the AIT inspection. The inspection covered a 2-week period of inspection onsite by a team of four inspectors and one consultant. Ten findings of very low safety significance (Green) were identified. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using IMC 0609, Significance Determination Process, (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process,
Revision 3, dated July 200
NRC-Identified and Self-Revealing Findings
Cornerstone: Reactor Safety
- Green.
A noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, was identified because the licensee failed to adequately implement their corrective action program when an emergency diesel-generator excitation circuit failed.
The failure precluded the emergency diesel-generator from achieving rated voltage within the required time period. The licensee subsequently repaired the circuit (Section 2.4).
III
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and did not result in the actual loss of a safety function at the time.
C
- Green.
A noncited violation of 10 CFR Part 50, Appendix B, Criterion V, Procedures, was identified because the licensee failed to follow the procedure for dispositioning a degraded condition for continued use. Specifically, the licensee failed to address a degraded main generator excitation limiter circuit into the work control process via the appropriate procedure to ensure that it was appropriately evaluated and processed.
This circuit, in part, prevents an overpower condition in the reactor fuel (Section 2.5).
The finding was greater than minor because it was associated with the human performance attribute of the barrier integrity cornerstone and impacted the cornerstone objective to provide reasonable assurance that physical design barriers, in this case the fuel cladding, protect the public from radionuclide releases caused by accidents or events.
C
- Green.
A noncited violation of 10 CFR Part 50, Appendix B, Criterion V, Procedures, with two examples, was identified because the licensee failed to implement contingency actions when two circuit breakers failed to operate during recovery operations in Units 1 and 3. Specifically, operators deviated from the emergency operating procedure for loss-of-offsite power/loss of forced circulation when they initiated maintenance on the two failed breakers instead of performing the contingency actions prescribed by the procedure. In addition, for Unit 1, the procedure was inadequate because it did not list all available contingency actions available to operators for restoring power to the electrical bus. The licensee is currently reviewing these procedures through their corrective action program (Section 2.8).
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and redundancy existed in other electrical buses.
C
- Green.
A noncited violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, was identified by the team because the licensee failed to implement timely corrective actions to ensure that the feedwater system was operated in a manner that would minimize the possibility of thermally induced vibration that could affect auxiliary feedwater system operability (Section 2.9).
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and because no transient occurred that necessitated implementation of the needed corrective actions.
IV
C
- Green.
A noncited violation of Technical Specification 5.4.1 was identified because the licensee implemented an inadequate emergency operating procedure. Specifically, the procedure failed to provide direction to maintain turbine-driven auxiliary feedwater pumps operable following a main steam isolation signal. The licensee is currently reviewing this procedure through their correction action program (Section 3.1).
The finding was greater than minor because it was associated with the equipment performance attributes of the mitigating systems cornerstone and affected the associated cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and because the turbine-driven auxiliary feedwater pumps did not become inoperable.
C
- Green.
A noncited violation of 10 CFR 50.65(a)(4), Maintenance Rule, was identified because the licensee failed to perform a risk assessment. Specifically, the licensee inappropriately decided to begin draining the Unit 1 turbine-driven auxiliary feedwater pump steam traps first, without addressing the higher risk profile in Unit 2, which resulted from having an inoperable emergency diesel generator. Draining the Unit 2 traps first would have restored the auxiliary feedwater pump and lowered overall Unit 2 risk (Section 3.1).
The finding was greater than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and because the turbine-driven auxiliary feedwater pumps were not needed.
- Green.
A noncited violation of Technical Specification 5.4.1 was identified because the licensee failed to follow emergency operating procedures. Specifically, the control room operator and an auxiliary operator performed the incorrect steps in Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation,
Revision 10. The Unit 2, Positive Displacement Charging Pump E was temporarily lost due to these human performance errors and resulted in a total loss of Unit 2 charging flow for a short period (Section 3.2).
The finding was greater than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. The finding had very low significance because it only affected the mitigating systems cornerstone and did not result in the actual loss of a safety function and no significant delays occurred that adversely impacted operator response to the event.
C
- Green.
A noncited violation of 10 CFR 50.54(q) was identified because the licensee failed to follow the emergency plan when they did not adequately maintain facilities required for emergency response. Specifically, the Technical Support Center emergency diesel generator failed because a test switch was not returned to its proper position following maintenance 6 days prior to the event. As a result, the emergency response organization assembled in the alternate Technical Support Center. This resulted in some confusion and posed some unique challenges to the emergency response organization (Section 3.4).
V
The finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem.
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 8 is considered a failure to comply with Planning Standard 8 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter MC 0609 Appendix B, §2.0. Therefore, the finding is of very low safety significance.
C
- Green.
A noncited violation of 10 CFR 50.54(q) was identified because the licensee failed to follow the emergency plan when they did not ensure that adequate command and control was established during the event. Specifically, the licensee did not follow Emergency Plan Implementing Procedure 01, Satellite Technical Support Center Actions, which requires that for multiple unit events, the Unit 1 shift manager is responsible for initially classifying and declaring the emergency and assuming the position of the on-shift emergency coordinator. As a result, each of the units respective shift managers initially assumed the role of emergency coordinator, resulting in notification irregularities to state and local officials (Section 3.5).
The finding is more than minor because it is related to the Emergency Preparedness cornerstone attribute of response organization performance, and affects the cornerstone objective in that command and control challenges resulting in inaccurate communications to the offsite officials could potentially affect the ability to ensure that adequate measures would be taken to protect the public health and safety.
- Green.
A noncited violation of 10 CFR 50.54(q) was identified because the licensee failed to follow the emergency plan. Specifically, the licensee failed to meet minimum staffing goals of Table 1, Minimum Staffing Requirements for PVNGS for Nuclear Power Plant Emergencies, following the Alert declaration on June 14, 2004 (Section 3.5).
This finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem.
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 2 is considered a failure to comply with Planning Standard 2 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609 Appendix B, §2.0. Therefore, the finding is of very low safety significance.
VI
REPORT DETAILS
1. INTRODUCTION
On June 14, 2004, a ground-fault on a 230kV transmission line, approximately 47 miles from the Palo Verde Nuclear Generating Station (PVNGS), caused a loss-of-offsite power and a concurrent trip of all three PVNGS units. Because of the loss-of-offsite power, the licensee declared a Notice of Unusual Event for all three units at approximately 7:50 a.m. MST. The Unit 2 Train A emergency diesel generator started, but failed early in the load sequence process due to a failed diode in the exciter rectifier circuit. This resulted in the Train "A" engineered safeguards features busses de-energizing, which limited the availability of certain safety equipment for operators.
Because of this failure, the licensee elevated the emergency declaration for Unit 2 to an Alert at 7:54 a.m. MST. All three units were safely shutdown and stabilized under hot shutdown conditions.
An NRC Augmented Inspection Team (AIT) was dispatched to the site and found that the licensees response to the event was generally acceptable, although complicated by a number of equipment failures, procedure issues, and human performance issues with diverse apparent causes and with varying degrees of significance. The issues requiring additional followup were identified and tracked as unresolved items (See NRC Inspection Report 05000528/2004012; 05000529/2004012; 05000530/2004012). The AIT reviewed immediate corrective actions prior to restart of the units, including actions to improve the independence and reliability of offsite power sources and found those actions appropriate for continued operation of the units.
This followup inspection was conducted to address the unresolved issues identified during the AIT inspection. The inspection addressed compliance and assessed the significance of the AIT observations.
2. SYSTEM PERFORMANCE AND DESIGN ISSUES
A number of unresolved items associated with system performance and design issues were revealed during and following the event. Each of these issues was inspected to assess the licensees effectiveness in determining the root and contributing causes, the extent of condition, and corrective actions.
2.1 (Closed) Unresolved Item 05000528; -529; -530/2004012-001: Offsite Power Reliability and Independence Issues
a. Inspection Scope
On June 14, 2004, a phase to ground fault occurred on the Westwing to Liberty 230kV transmission line. The fault was not quickly cleared due to a relay failure in the protection scheme, which caused Breaker WW1022 to not receive a trip signal. Since the fault was not cleared, it continued to be fed by three 525/230kV transformers in the Westwing substation. The Westwing substation is connected to the PVNGS switchyard by two 525kV transmission lines. The fault continued to be fed for approximately 38 seconds and resulted in all transmission lines to the PVNGS switchyard (PL) being disconnected. This caused a generator trip, reactor trip, and loss-of-offsite power at all three PVNGS units. It was also noted that the three tie lines between the PVNGS and Hassayampa switchyards were the first 525kV lines to trip.
This unresolved item was identified by the AIT to facilitate the review of the root and contributing causes of the ground fault failing to isolate from the grid and to evaluate the protective tripping of the Hassayampa to PVNGS transmission lines. This review also included an evaluation of the extent of condition associated with other potential design issues that could adversely impact the independence and reliability of offsite power to PVNGS, and to assess the corrective actions for any issues identified.
The inspectors reviewed the licensees root-cause evaluation to determine if the problems which caused the failure to clear the fault and subsequent loss-of-offsite power were well understood. The inspectors also evaluated the corrective actions to ensure they were appropriate to correct the root and contributing causes of the event.
In addition the inspectors reviewed the 10 CFR 50.59 screen for the addition of the three 525kV lines connecting the Hassayampa and PVNGS switchyards, and the associated protective relaying for the transmission lines.
b. Observations and Findings
The inspectors found that the root and contributing causes identified by the licensee were appropriate. The root cause of the failure to clear the fault was a defective AR relay (fast acting, auxiliary relay), which provided trip signals for Breakers WW1022 and WW1026. This relay acts as a contact multiplier (i.e. provides multiple outputs based on a single input) in the protective scheme and is used to send trip signals to its two associated breakers. Two of the four contacts failed to shut on demand, which resulted in Breaker WW1022 not receiving a trip signal. The relay has been sent back to the vendor for failure analysis to determine the reason for the contacts failing to shut. A breaker failure signal was not actuated since it was fed from the same failed contacts.
Breaker WW1026 opened, isolating the fault on the Liberty side, but since Breaker WW1022 stayed shut the fault continued to be fed from the Westwing 525kV switchyard.
Contributing causes included a lack of redundancy to preclude a single failure from preventing a breaker trip, relay testing did not ensure each contact shut, and no overcurrent protection was provided on the 525/230kV transformers in the Westwing substation. It was also determined that the three Hassayampa-PL transmission lines tripped on negative-sequence current, a protection system feature, which is designed to provide open pole protection for the lines. Open pole refers to a fault in which one phase of the breaker fails to shut.
The inspectors determined that the corrective actions were appropriate to address the root and contributing causes of the event. In addition, the enhancements to the off-site transmission network were effective in improving the reliability and independence of the off-site electrical grid. The inspectors reviewed the extent of condition review and determined that it was sufficient in scope to ensure that PVNGS should not be challenged by a similar uncleared fault at neighboring switchyards. The inspectors found the following actions had been implemented by the licensee and transmission system owners:
- Removed negative sequence relay protection of the PL-Hassayampa 525kV tie lines. Open pole protection was still provided by a phase current mismatch protection scheme. This scheme is less likely to spuriously trip due to external faults.
- Added redundant AR relays for Deer Valley - WW and Liberty - WW 230kV lines.
No other lines were found which had single point failure vulnerabilities because of the AR relays.
- Added phase distance and overcurrent relay protection on two of the three 525/230kV transformer (the third will be added when the transformer is replaced)and the single 525/345kV transformer at Westwing to provide backup protection against a similar fault.
- A study was performed, which showed that the fault current at the Westwing substation was the most severe. A sustained fault at any other substation would not have a similar impact at PVNGS.
- The transmission system owners reviewed prints to ensure no single point failures on 230kV systems at Rudd, Cairina, Browning, Deers and Silvervine substations.
- Change functional testing for relays to ensure that all contacts actuate.
The inspectors determined that the 10 CFR 50.59 evaluation for the addition of the three Hassayampa-PL tie lines did not identify a vulnerability in the protective relay scheme, which surfaced during the June 14, 2004, event. Required design reviews for protective schemes look at system coordination and the response to numerous types of transients.
As a standard practice, consistent with NERC reliability requirements, extended faults were not considered a plausible event during design development and review by the transmission system operator because of the extremely low probability of occurrence.
This type of failure, which revealed this vulnerability, had never occurred before on the transmission system associated with PVNGS.
The three tie lines each used multifunction relays for their primary and backup protective relaying schemes. The multifunction relays used for backup protection included a negative sequence relay to provide open pole protection. This particular function was not needed for this application since a backup for pole protection is not required. Since the function was already available the transmission and distribution designers added the function and adjusted the settings high enough that it would not normally trip. During the grid event, the three tie lines tripped due to actuation of the negative sequence relays. It was determined that this was a correct, but not as planned operation. The function tripped at the appropriate settings, however, it was not intended to protect against this type of fault. The tripping of these lines resulted in the PVNGS units being isolated from nearby generation facilities and further increased the severity of the transient and degradation of voltage at the PVNGS switchyard.
Based on the above stated facts the inspectors determined that the 10 CFR 50.59 review was adequate in depth and scope even though it did not identify this vulnerability in the protective relay scheme. The transmission system operator determined that the protective scheme would be more reliable without the negative sequence relays and this feature was removed from the protective scheme as a corrective action following the event.
The inspectors also noted that the description of the transmission lines contained in Updated Final Safety Analysis Report, Sections 8.1 and 8.2, were not consistent with the as-built design and differed from section to section. The licensee generated appropriate documentation to address these discrepancies.
2.2 (Closed) Unresolved Item 05000528/2004012-002: Unit 1 Atmospheric Dump Valve Failure
a. Inspection Scope
The augmented inspection team identified a unresolved item concerning reactor operator observations that Atmospheric Dump Valve ADV-185 failed to operate properly while being remote-manually operated from the control room during the performance of Procedure 40EP-9EO10, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, following the Unit 1 loss-of-offsite power event. This issue was made unresolved pending review of the root and contributing causes of the valve failure; the licensees extent of condition; and assessment of the effectiveness of corrective actions implemented by the licensee.
The inspectors reviewed troubleshooting activities and trend plots, and interviewed licensee personnel to evaluate the cause determination performed by the licensee. The team also reviewed the adequacy of corrective actions taken for problems identified.
b. Observations and Findings
The licensee was unable to recreate the anomalies reported by operations through extensive troubleshooting and trend plot review illustrating atmospheric dump valve movement. A thorough troubleshooting plan was implemented by engineering and maintenance personnel to evaluate the system. No anomalies were discovered during numerous valve strokes and no leaks were detected on instrument air fittings and associated parts. Nonetheless, the positioner was replaced and calibrated since a review of equipment history revealed that the old positioner was slightly aged with 6 years of service.
The team noted that the conclusions made by engineering and maintenance personnel did not correlate with the observations made by the reactor operator. Additionally, the engineering evaluation did not include a disposition of the reactor operators statements concerning operation of the valve. Following the NRCs identification of this oversight, engineering and maintenance personnel met with the reactor operator to properly disposition statements made after the Unit 1 loss-of-offsite power event. The trend plots illustrating valve movement were compared with the reactor operators recollection of his post-trip actions. Recorded trend data and reactor operator recollection indicate that the atmospheric dump valve was being operated in the area of 5 percent to regulate reactor coolant system temperature and pressure. When atmospheric dump valves are given a change in demand, they tend to overshoot, then undershoot the setpoint prior to settling at the final position. If the valve is desired to be 5 percent open to match a low primary heat load, the valve may trigger the closed limit switch, giving the operator the impression that the valve has closed, when in fact, the valve is still hunting to attain a final position. It was apparent the normal hunting toward final valve position following adjustments led the reactor operator to believe that the valve was drifting closed.
Although troubleshooting was extensive and no physical anomalies were identified, the valve positioner was replaced and calibrated as a conservative measure. However, the engineering evaluation that provided the basis for closure of the condition report/disposition request did not include a disposition of the reactor operators statements concerning the operation of the valve. Following the NRCs identification of this oversight, engineering and maintenance personnel met with the reactor operator to properly disposition statements made concerning the valve operation. As a result of this followup to address all pertinent facts concerning valve operation, it was determined that the reactor operator did not understand the valve response at low primary heat loads.
This determination identified an area for improvement in operator training. Corrective actions were subsequently developed to address this training need.
The failure to disposition and resolve the reactor operators statements concerning operation of the valve resulted in the failure to identify a condition adverse to quality regarding operation of the atmospheric dumps. This failure also represented an ineffective problem identification and resolution cross-cutting aspect that resulted in the failure to identify the need for additional operator training. Following identification of this oversight, the licensee developed corrective actions to train operators on atmospheric dump valve response with low primary heat loads. We determined that your initial corrective actions were ineffective and failed to be in accordance with Criterion XVI of Appendix B to 10 CFR Part 50. However, because there was no actual problem with the valve, this is a finding of minor significance and is not subject to enforcement action in accordance with Section VI of the NRCs Enforcement Policy.
2.3. (Closed) Unresolved Item 05000528/2004012-003: Unit 1 Letdown System Failure to Isolate
a. Inspection Scope
During the June 14, 2004, loss-of-offsite power event, the Unit 1 letdown system did not operate as expected when fluid temperatures exceeded the alarm setpoint. The letdown system bypassed the ion exchanger and the filter at 140EF, as expected. However, a temporary modification to bypass a flow sensor resulted in the system failing to isolate when needed. The letdown system response had apparently not been anticipated by the engineers designing the temporary modification, and operators were unaware of the systems response to a loss-of-offsite power. This issue was made unresolved to review the root and contributing causes of the failure of the letdown system to isolate, review the licensees extent of condition, and assess the effectiveness of corrective actions implemented by the licensee.
The inspectors reviewed the design control measures used when designing Temporary Modification 2594804, interviewed licensee personnel, and reviewed the adequacy of corrective actions taken for problems identified.
b. Observations and Findings
The AIT review of the letdown system response determined that inadequate design control may have resulted in the overheating of a system designed for low temperature operation. The team also reviewed the apparent cause evaluation, system evaluation, and immediate corrective actions taken, which included removal of the temporary modification. The inspectors noted during the AIT follow-up inspection that the root-cause evaluation performed by the licensee was consistent with the observations made by the NRC augmented inspection team. Additionally, the inspectors determined that associated corrective actions were appropriate and also timely.
Procedure 81TD-0EE10, Plant Design and Modification, was used to design Temporary Modification 2594804 to address inadvertent Unit 1 letdown isolation events that have occurred over the past several years. The inadvertent isolation events were temperature initiated by a nuclear cooling water flow switch during the cooler months when the nuclear cooling water is low and the nuclear cooling water letdown heat exchanger outlet valve is controlling near its closed position. In addition, there were indications that the nuclear cooling water flow switch was isolating at flow rates above the required setpoint. The combination of these two conditions has resulted in inadvertent isolation of letdown flow in Unit 1 due to the generation of the isolation signal to Valve CH-523. The temporary modification to bypass the nuclear cooling water flow switch, which generates the isolation signal, was originally installed in 1999, removed during Refueling Outage U1R10 following corrective actions, then reinstalled following another inadvertent isolation that occurred on March, 27, 2003. The inspectors noted that engineers failed to adequately consider letdown system response to a loss of power when performing the 10 CFR 50.59 screening. The failure to perform an adequate 10 CFR 50.59 screening resulted in the system failing to isolate when needed following the loss-of-offsite power event. This constitutes a finding of minor significance since it is not associated with any of the Reactor Safety, Radiation Safety, or Safeguards Cornerstones. The minor finding is not subject to enforcement action in accordance with Section VI of the NRCs Enforcement Policy. However, this finding involved human performance cross-cutting aspects because the letdown system response was not anticipated by the engineers designing the temporary modification and by operators when responding to a system loss of power.
2.4 (Closed) Unresolved Item 05000529/2004012-004: Unit 2, Train "A" Emergency Diesel Generator Failure
a. Inspection Scope
During the June 14 loss-of-offsite power event the Unit 2, Train A emergency diesel generator started but failed during the loading sequence due to a diode failure in the emergency diesel generator excitation circuitry. This resulted in a loss of power to the Train A engineered safeguard busses, which limited the availability of certain safety equipment to the operators. Due to the failure of Train A emergency diesel generator coincident with a loss-of-offsite power, the licensee declared an Alert based on the criteria in the emergency action level matrix.
This unresolved item was identified by the AIT to facilitate the review of the root and contributing causes of the failure of the diode in Phase B of the Unit 2, Train A emergency diesel generator voltage regulator exciter circuit, review the extent of condition and assess the effectiveness of corrective actions implemented by the licensee.
The inspectors reviewed the licensee root-cause report and the troubleshooting action plan to ensure the cause of the failure was well understood and the corrective actions appropriate to address the cause of the failure. The inspectors also reviewed available industry experience data on diode failures and data from the testing that the licensee performed on the excitation circuit. This review also included the diode failure analysis report from an outside vendor and the manner in which the licensee incorporated the reported data.
b. Observations and Findings
Each emergency diesel generator has two redundant excitation system bridge circuits.
By design, one bridge circuit is in use and the alternate is an installed spare. However, the alternate bridge circuit had not been tested as part of the preventive maintenance or routine surveillance programs. The licensee determined that they should periodically swap and test these spare systems based on owners group and industry experience.
The Unit 2, Train A emergency diesel generator bridge had been changed to the alternate circuit during the 2003 outage and had been in service for approximately 70 hours8.101852e-4 days <br />0.0194 hours <br />1.157407e-4 weeks <br />2.6635e-5 months <br /> of operation before the failure. The Unit 1, Train A emergency diesel generator had also been swapped to its alternate bridge during the last outage for that unit and has been operating satisfactorily since placed in service.
The inspectors determined that the licensees root-cause evaluation appropriately determined that the failure was due to a manufacturing defect that resulted in infant mortality of the part. This evaluation was justified by:
A large design margin for overcurrent.
Same type of diode in the other emergency diesel generators with no failures in 20 years.
Low failure incidence in the industry.
The inspectors reviewed the available test data, the failure analysis report from the vendor and industry operating experience to ensure that the licensees classification of infant mortality was appropriate. The failed diode was rated for 275 amps and 1000 volts for repetitive peak reverse overvoltage. The licensee performed testing, which showed that the maximum reverse voltage applied to any of the diodes in the bridge circuit was 316 volts, well below the maximum rating. The testing was performed during engine startup, steady state run and shutdown. Although the testing did not repeat the conditions on the emergency diesel generator at the time of failure, the inspectors determined that the testing provided reasonable assurance that there were no anomalies in the circuit, which would cause an overvoltage condition. The risk associated with securing the safety bus to allow for testing during loading conditions was not justified during reactor power operations. The licensee stated that additional testing would be performed during the outage while loading the emergency diesel generator to provide baseline data. In addition, the essential service water pump, which was loading onto the emergency diesel generator at the time of the failure was evaluated to ensure that no electrical fault on that motor precipitated the failure. The licensee also performed testing on all the spare diodes in stock to ensure they were free of manufacturing defects and to provide baseline data for later stock.
Introduction.
A noncited violation of very low safety significance was identified because the licensee failed to adequately address an alternate bridge circuit failure in their corrective action program. Specifically, the licensee failed to implement effective corrective actions for the failure because troubleshooting to identify a root cause was not conducted.
Description.
The failure occurred in the Unit 3, Train A emergency diesel generator during testing on April 19, 2003. At that time, the machine failed to achieve rated voltage within the required time period. This failure occurred immediately after the excitation bridge was swapped to its alternate circuit. This emergency diesel generator was the first one to be transferred to the alternate bridge. The corrective actions for the failure were simply to place the normal bridge back in service and to place the bridge swap in a different maintenance procedure to allow more time for testing should problems occur. No troubleshooting was performed to determine why the alternate bridge failed and the problem was not captured in the corrective action program. These oversights resulted in a missed opportunity, which may have identified deficiencies with the alternate bridge circuits due to the extended time that had been installed but not used. Although the alternate bridge circuits were installed but not normally in service, they are safety-related components.
Analysis.
The inspectors determined that this finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process, Phase 1 Worksheet, the finding was determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function at the time.
Problem identification and resolution cross-cutting aspects of this finding were related to failing to thoroughly investigate the failure of safety-related equipment and failing to capture the issue in the corrective action program.
Enforcement.
Criterion XVI, Appendix B of 10 CFR Part 50 requires, in part, that conditions adverse to quality be promptly identified and corrected. Contrary to the above, PVNGS failed to adequately address the alternate bridge circuit failure in the corrective action program. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-01).
2.5 Plant Response to Loss-of-Offsite Power Event (Closed) Unresolved Item 05000530/2004012-005: Unit 3 Main Turbine Bypass Valve Control System Operation
a. Inspection Scope
During the June 14, 2004, loss-of-offsite power event, Unit 3 experienced an automatic main steam-line isolation. Licensee personnel attributed the automatic isolation to a steam bypass control system anomaly that caused all the bypass valves to open simultaneously, suddenly decreasing main steam line pressure, and causing a main steam isolation. The apparent cause of the "anomaly" was the result of a momentary loss of power to the control system being re-energized in the automatic mode, vice manual. According to the licensee engineers, this power loss initiated a 30-second timer that disconnected the valve control signals from the control cabinet. When the 30-second timer completed, all eight valves modulated open in about 14 seconds. This issue was made unresolved to review the root and contributing causes of the automatic main steam-line isolation in Unit 3, which appeared contrary to the expected response described in the plant safety analysis, the extent of condition, and the effectiveness of corrective actions implemented by the licensee.
The inspectors reviewed troubleshooting activities, trend plots, and interviewed licensee personnel to evaluate the root-cause determination and extent of condition review performed by the licensee. The team also reviewed the adequacy of corrective actions taken for problems identified.
b. Observations and Findings
The inspectors noted that the licensee initially determined that the cause for the steam bypass control system operation, which appeared contrary to the expected response described in the plant safety analysis, was because of an initiating fault with the main generator control circuit. Further, the licensee believed that if the control circuit had functioned as expected, the steam bypass control system would have been de-energized on the loss-of-offsite power and reverted to manual control with no output, consistent with the description in the plant safety analysis. The main generator control circuit failure is discussed in the following section of this report (URI 05000530/2004012-006).
During interviews with engineering personnel, the inspectors learned that the licensee believed that there may be other power transient scenarios where the steam bypass control system could behave similarly, which was contrary to the initial condition report/disposition request disposition. Even though the main generator control circuit failure contributed to the difference in the Unit 3 steam bypass control system response, emergency diesel generator start time and/or the nature of the grid disturbance could cause the same steam bypass control system response following a loss-of-offsite power. This previously undisclosed behavior of the steam bypass control system response following a loss-of-offsite power represents a nonconformance between the as-built plant response and that which is described in the design basis. Following the interviews with engineering personnel, a condition report/disposition request was initiated to further analyze and resolve the nonconforming condition. The failure to identify and correct the nonconforming condition constituted a violation of Criterion XVI of Appendix B to 10 CFR Part 50. However, this finding is of minor significance because a subsequent safety assessment showed that the consequence of a steam bypass control system anomaly following a loss-of-offsite power remains bounded by the consequences documented in the plant safety analysis. This minor finding is not subject to enforcement action in accordance with Section IV or VI of the NRCs Enforcement Policy. Problem identification and resolution cross-cutting aspects were associated with this finding because additional transient responses were identified but not captured in the corrective action program.
(Closed) Unresolved Item 05000530/2004012-006: Unit 3, Main Generator Excitation Controls Failure
a. Inspection Scope
The June 14, 2004, electrical fault caused voltage to be reduced at the PVNGS switchyard. As the voltage collapsed the three generators boosted their reactive power output in an attempt to increase switchyard voltage. The generator reactive power output is increased by raising the generator field current. During normal operation this is accomplished automatically by the voltage regulation system. The extended fault resulted in a collapse of the transmission system and the three PVNGS units to be isolated. The resultant load rejection caused the turbine to trip on overspeed, the generators to trip on low volts/hertz ratio and the reactor coolant pumps tripped on undervoltage. This is the as expected sequence of events for a load rejection scenario.
In Unit 3, it was determined that the reactor coolant pumps remained connected to the substation bus for approximately 5 seconds longer than at the other two units. This resulted in the Unit 3 reactor coolant pumps operating at a higher frequency, due to an increase in generator frequency following the load rejection, with a resultant increase in core flow to 108.2 percent of design flow. The increased flow caused a power excursion to 109 percent reactor power and a subsequent reactor trip due to a variable over-power-trip signal. Units 1 and 2 were not exposed to this condition since their respective reactor coolant pumps tripped on undervoltage before the generator frequency ramped upward.
The licensee determined that the cause of the anomaly in Unit 3 was due to a failure of the main generator excitation limiter circuitry, which caused the field excitation and, therefore, output voltage to remain high during the transient. The high voltage output delayed component load shedding since it increased the time to reach the undervoltage relay setpoints.
This unresolved item was identified by the AIT to review the root and contributing causes of Unit 3 main generator excitation controls response since it operated differently than at Units 1 and 2, and may have contributed to the variable overpower trip at Unit 3. In addition it includes an evaluation of the licensees extent of condition review and any associated corrective actions.
The inspectors reviewed the licensee root-cause report to ensure the cause of the transient was well understood and the corrective actions appropriate to address the cause of the failure. In addition the inspectors reviewed the generator transient response at each unit. The inspectors also evaluated a vendor analysis related to core flow at Unit 3 during the event and associated impacts due to hydraulic considerations and industry operating experience associated with overfrequency events.
b. Observations and Findings
The inspectors found that the root-cause identified by the licensee was appropriate.
The licensee determined that the transient differences at Unit 3 were due to a failure of the main generator excitation limiter circuit, in that, it failed to switch to a lower controlling reference signal at the appropriate time resulting in higher bus voltages. The main generator excitation limiter circuit is designed to protect the generator from excessive field currents (If) by providing two functions. A field current limit is set at 1.4 per unit (PU) of 100 percent of the rated field current. This limit protects solid state components in the excitation bridge circuit. The second function is a generator field maximum excitation limit, which is designed to protect the generator field from overheating. When If exceeds 1.05 PU for a specified time period, this function reduces the field current limit from 1.4 to 1.01 PU. This limit is held until If is less than 1.05 and the ac regulator is calling for a lower signal than 1.01 PU. Once this occurs, normal field regulation is resumed and the field current limit is reset. If this does not occur in a preset time then control is switched to a fixed field regulator set at 1.0 PU and the generator will be tripped if If cannot be reduced to less than 1.05 within 4 seconds.
From a review of the field currents of the Unit 3 generator, it was found that main generator excitation limiter attempted, but failed, to switch the reference limit from 1.4 to 1.01 PU as required. A work order was written to repair the circuit, however, the licensee decided to wait until the next outage to perform the repairs since it could not be performed with the generator on line. An engineering position paper was attached to the associated condition report/disposition request to provide justification for this decision. It determined that delaying the repair was acceptable based on the most probable cause of the failure and that redundant protection would protect the main generator from damage.
The inspectors reviewed the hydraulic analysis that stated the reactor coolant system flow during the transient reached a maximum of 108.2 percent of design flow. This was determined to be acceptable since it did not exceed the evaluated limit of 110.4 percent of designed volumetric flow and, therefore, would have no negative impact on continued operation with respect to fuel grid-to-rod fretting, vessel hydraulic uplift forces and fuel mechanical design. This review also determined that acceptable margins were maintained during overspeed operation of the reactor coolant pumps.
Introduction.
A noncited violation of very low safety significance was identified, in that, the licensee failed to follow their procedure for dispositioning a degraded condition for continued use using engineering work orders. Specifically, the licensee failed to place the degraded main generator excitation limiter circuit into the work control process via the appropriate procedure, 81DP-0DC13, Deficiency Work Order, to ensure that it was appropriately evaluated and processed. This was determined to be a violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings.
Description.
Following the identification of the degraded circuit, the licensee wrote a work order to repair the Unit 3 excitation circuitry. In addition a white paper was attached to the associated condition report/disposition request to provide a basis for delaying the repair until the next outage. The inspectors review of this issue noted that the degraded condition was not processed in accordance with the licensee procedure that provides guidelines for correcting such a condition through engineering work orders.
This work control process is contained in Procedure 81DP-0DC13, Deficiency Work Order. This process is used to ensure that all deficiencies involving structures, systems and components are resolved in order to maintain PVNGS in its designed and documented condition. It requires a description of actions taken to resolve the degraded condition. In addition it requires analysis and the appropriate screening to confirm it will still perform its intended function if corrective actions are to be delayed.
This failure to utilize required procedures represents the human factor contributing aspects of this finding.
Analysis.
The inspectors determined that this was a performance deficiency since the licensee failed to address the degraded condition through the required process.
Traditional enforcement does not apply since there were no actual safety consequences or potential for impacting the NRCs regulatory function and the finding was not the result of any willful violation of NRC requirements or PVNGS procedures. The examples in Inspection Manual Chapter 0612, Appendix E, were not applicable to this finding.
This finding is greater than minor since it is associated with the human performance attribute of the barrier integrity cornerstone and impacted the cornerstone objective to provide reasonable assurance that physical design barriers, in this case, the fuel cladding, protect the public from radionuclide releases caused by accident or events.
The inspectors noted that the licensee evaluation to delay repairs only addressed concerns with generator protection. This limited approach results from the problem identification and resolution cross-cutting aspects of this finding. While the core hydraulics evaluation determined that no adverse consequences occurred during the June 14, 2004, event based on actual plant conditions at that time, it was not a bounding analysis to verify that flow limits would not be exceeded if a similar event occurred again with the system in a degraded condition. No evaluation was performed that considered setpoint tolerances on the turbine generator to verify flow limits would be bounded below the maximum evaluated limits. In addition, no evaluation looked at the potential impacts due to the associated power excursion and the consequences on departure from nucleate boiling. The licensees evaluation also did not take into account other plant issues that occurred during the transient because of the degraded condition, such as the unanticipated main steam isolation. The inspectors determined that this could impact the fuel cladding design barrier since the degraded main generator excitation limiter circuit caused reactor power and flow to exceed the normal operating design parameters during the event. The basis for the power limit is to protect against positive reactivity excursions, which could result in departure from nucleate boiling and excessive fuel clad temperatures. The flow limit is to ensure hydraulic forces are kept low enough to prevent core mechanical tilt which could cause unanalyzed flux distributions. While an evaluation was not performed, the licensee indicated that preliminary, qualitative assessment indicated the evaluation would demonstrate that no design limits would be exceeded.
The inspectors conducted a Phase 1 SDP screening and determined that the issue was of a very low safety significance since during the June 14 event the reactor protection system operated correctly to mitigate the transient, hydraulic limits were not exceeded and no mitigating systems were rendered inoperable.
Enforcement.
Criterion V of Appendix B, 10 CFR Part 50 states, in part, that activities affecting quality shall be prescribed by procedures and be accomplished in accordance with these procedures. Contrary to this, PVNGS failed to process the degraded condition of the main generator excitation limiter circuit through the appropriate work control procedures. Due to this, a rigorous evaluation was not performed to ensure that continued operation would not challenge the plant systems or operators.
Because the violation was of very low safety significance and has been entered into the licensees corrective actions program this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy:
(NCV 05000528,529,530/2004013-02).
2.6 (Closed) Unresolved Item 05000530/2004012-007: Unit 3, Reactor Coolant Pump 2B Lift Oil Pump Breaker
a. Inspection Scope
The Unit 3 reactor coolant pump Oil Lift Pump 2B tripped from actuation of its thermal overload relay during restart activities. Troubleshooting, revealed that the motor was drawing a current in excess of its nameplate current and that the thermal overload relay sizing was not in accordance with the current for this motor as required by Calculation 13-EC-PH-250, Overload Relay Heater Sizing Criteria.
Troubleshooting performed subsequent to the trip showed that the motor was drawing a full load current from 11.2 to 10.7 A. The Lift Pump 2B motor thermal overload relay heater had been sized for a nameplate full load current of 10.9 A, in accordance with Calculation 13-EC-PH-250. The motor was replaced in 2000 with a substitute motor having a nameplate full load current of 10.5 A.
b. Observations and Findings
The inspectors determined that the nameplate full load current, the frame size, the NEMA Design Code, and the insulation class of the motor were different than the original design specification. The motor had been replaced, but not with a one-for-one replacement. The differences had not been recognized or evaluated for acceptability by the licensee. In addition, Calculation 13-EC-PH-250 sizes the thermal overload relay heaters based on motor full-load current; therefore, it also became apparent that when the motor was changed, the thermal overload relay heater had not been resized to match the new motor current. Nonetheless, failure to resize the overload relay heater had little to do with the trip experienced; resizing would have only exacerbated the problem by tripping the motor even sooner.
Although it was at first believed that the Reactor Coolant Pump 2B oil lift pump motor tripped because of improper thermal overload relay sizing, it was subsequently determined that the trip resulted from internal motor or pump malfunction.
Based on the inspectors identification that the motor replaced in 2000 was a motor having different parameters from the existing motor, the licensee issued Condition Report/Disposition Request 2737748. This corrective action document cited a failure to meet Section 6.14.3 of Procedure 30DP-9WP02, Work Document Development and Control, which called for the use of approved design output documents when ordering materials (parts) to verify that material (APN) being ordered is the correct material to ensure configuration control is maintained. We found that the lift oil pump motor had been replaced in 2000 with a substitute motor that was not the same type as originally installed and that, as a result, the replacement motor was drawing current in excess of its rating and, therefore, tripped on this occasion. We determined that licensee staff failed to follow the procedure for Work Document Development and Control when replacing the motor in 2000. The procedure specifically requires that approved design output documents be used when ordering materials and parts to verify that material being ordered is the correct material in order to ensure that configuration control is maintained. This procedural requirement was not followed when ordering the replacement motor. This constitutes a finding of minor significance since it is not associated with a safety-related component and, therefore, did not result in a violation of NRC requirements.
The inspectors also noted that Reactor Coolant Pump 2B oil lift pump motor had previously tripped as a result of actuation of its thermal overload relay on July 28, 2003.
Troubleshooting through Work Order 2623530 identified that the motor was Drawing 12.4 A; however, 2 days later, when additional testing was performed under the same work order, the motor current was noted to be between 8.4 and 8.9 A. Since the value was below the full-load current, the licensee concluded that the problem no longer existed and declared the motor operable following a 30 minute run. The inspectors determined that this decision was premature and constituted ineffective corrective action. Since there was clear indication of some form of intermittent motor/pump malfunction from the earlier tests, the failure to pursue this issue and determine a root cause represented a problem identification and resolution cross-cutting aspect of this finding.
2.7 (Open) Unresolved Item 05000528; -529; -530/2004012-008: Unit 3, Low Pressure Safety Injection System In-Leakage
a. Inspection Scope
The augmented inspection team identified a unresolved item associated with Valve RCEV-217, a 14-inch Borg-Warner check valve that began to leak and pressurized the safety injection header to Reactor Coolant Loop 2A in Unit 3, that moderately challenged operators by the unnecessary distraction from emergency procedures. This issue was made unresolved to review the root and contributing causes, extent of condition, and corrective actions associated with the Borg-Warner safety injection check valve leakage; to review the effectiveness of prior corrective actions for previous check valve leakage issues; to evaluate the adequacy of the in-service testing program for demonstrating check valve operability; and to assess the licensees use of industry operating experience and generic communications.
The inspectors reviewed condition report/disposition request evaluations for the check valve leakage identified during the loss-of-offsite power event, and past Borg-Warner check valve deficiencies; interviewed engineering personnel; and reviewed in-service testing program documents.
b. Observations and Findings
The inspectors determined that the cause evaluation and corrective actions taken were based on sound technical information acquired through review of plant response and operator interviews following the loss-of-offsite power event; historical review of in-service testing results; industry operating experience; and plant operating experience from previous check valve leakage issues.
However, as-found alignment and seat conditions of the valve will provide a better understanding as to the cause of this leakage and the adequacy of proposed corrective actions. Therefore, this unresolved item will remain open pending review of maintenance activities on Valve RCEV-217 during Refueling Outage U3R11.
2.8 (Closed) Unresolved Item 05000528; -529; -530/2004012-009: Units 1 and 3, General Electric Magna Blast Breaker Failures
a. Inspection Scope
Subsequent to the loss-of-offsite power event, operators attempted to restore power from the 525kV Switchyard to the station buses. This required operators to close 13.8kV Circuit Breakers 1ENANS06K (on Intermediate Bus IE-NAN-S06) and 3ENANS05D (on Intermediate Bus 3E-NAN-S05). When the operators in Units 1 and 3 each attempted to close the respective breakers, the breakers failed to close.
b. Observations and Findings
The inspectors reviewed the root and contributing causes of the failure, the extent of condition determination, the effectiveness of corrective actions and the results of inspections and tests, which were conducted on the breakers in an attempt to identify the cause of the failure to close.
Circuit breaker components, as well as control circuit components, were tested to determine possible weak links that could have contributed to the failure. Although no direct cause was identified, the licensee determined that the most probable cause was the anti-pump relay, which was identified as an exposed, non-hermetically sealed unit that had exhibited high resistance because of dirt and dust contamination. Appropriate corrective actions were planned to address this problem. Short-term corrective actions included replacement of the anti-pump relay with hermetically sealed units. Long-term corrective action will provide an air-conditioned filtered environment for the 13.8kV switchgear.
The inspectors also reviewed operator actions taken in response to the failure of the circuit breakers to close. Operators in both units elected to conduct maintenance on the circuit breakers, which subsequently resulted in successful closures. However, by initiating maintenance, any evidence that may have determined the cause of failure was obliterated.
Introduction.
A noncited violation of very low safety significance, with two examples, was identified, in that, the operators failed to implement contingency actions when the breakers did not close. Specifically, operators deviated from Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, when they initiated maintenance on the two failed 13.8kV breakers instead of performing the contingency actions prescribed by the procedure. Additionally, for Unit 1, Emergency Operating Procedure 40EP-9EO07, was inadequate because it did not list all possible contingency actions available to operators. These two issues were determined to be a violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings.
Description.
The licensee issued Condition Report/Disposition Request 2739844 to document that emergency operating procedure contingency actions were not followed in the case of one breaker and that available contingency actions were not included in the procedure, for the other breaker. In the first case, operators deviated from the Loss of Offsite Power/Loss of Forced Circulation procedure when they did not perform the required contingency actions prescribed by the procedure when the circuit breaker did not close. Instead, operators initiated maintenance on the failed breaker in an attempt to repair it. In the second case, all available Unit 1 contingency actions to restore power to the electrical bus were not listed in the procedure. In the first case, the violation represents a failure to follow procedures and the second example represents an inadequate procedure.
Analysis.
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. The inspectors conducted a Phase 1 significance determination procedure screening and determined that the issue was of a very low safety significance. The failure to follow a procedure and the inadequate procedure did not result in the loss of a safety function.
The inspectors noted human performance cross-cutting aspects that contributed to this finding because operators in two units failed to follow contingency actions in emergency operating procedures.
Enforcement.
Criterion V of Appendix B to 10 CFR Part 50 states, in part, that activities affecting quality shall be prescribed by procedures and be accomplished in accordance with these procedures. Contrary to the above, Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, was not followed by operators and was also inadequate for use in Unit 1 because it did not list all available contingency actions to restore power to the electrical bus when the breaker failed to close. Because both examples of the violation are of very low safety significance and have been entered into the licensees corrective action program, they are being treated as a noncited violations consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-03).
2.9 (Closed) Unresolved Item 05000528; -529; -530/2004012-010: Auxiliary Feedwater (AFW) System Performance
a. Inspection Scope
The AIT identified unresolved item to review the root and contributing causes, extent of condition, and corrective actions associated with the design and operation of the auxiliary feedwater system as a result of a thermally induced vibration that occurred when operators placed the non-essential auxiliary feedwater system into service and which may also have involved procedural issues.
The inspectors reviewed the licensees evaluation and proposed corrective actions documented in Condition Report/Disposition Request 2715731, interviewed engineering and operations personnel, and performed system walkdowns to determine the adequacy of the licensees response.
b. Observations and Findings
Introduction.
A noncited violation of very low safety significance was identified for untimely implementation of corrective actions to ensure that the feedwater system was operated in a manner that would minimize the possibility of thermally induced vibration that could affect auxiliary feedwater system operability.
Description.
The licensee was not able to conclude that the thermally induced vibration was initiated by the starting of the non-essential auxiliary feedwater pump or if portions of the non-essential auxiliary feedwater train were involved. The inspectors were unable to support the licensees conclusion based on independent review of the facts. The inspectors observed that inaccurate information was used in the condition report/disposition request analysis, which could have impacted the results of the evaluation. Specifically, the evaluation stated that the auxiliary operator heard the loud banging associated with a hydraulic transient from both the 100 and 140 foot elevations of the turbine building, which helped the operator conclude that the source of the noise was in the vicinity of Heater Drain Tank B. Independent interviews conducted by the inspectors revealed that the auxiliary operator only heard the loud noise from the 140 foot elevation of the turbine building, from the direction of Heater Drain Tank B.
The inspectors noted that from where the operator was standing, the sounds were also in the same direction of portions of the non-essential auxiliary feedwater train.
Moreover, the auxiliary operator could not conclusively determine the source of the loud noises associated with the thermally induced vibration through direct observation.
Following a report of the loud noises, engineers performed a system walkdown and identified a pipe clamp that had rotated on a vertical section of auxiliary feedwater piping. This evidence of physical damage to the potentially affected piping was not used by the licensee to form the basis for the inconclusive evaluation. Based on the facts considered, the inspectors concluded that the thermally induced vibrations most likely occurred in portions of the non-essential auxiliary feedwater train.
As was determined by the AIT, the transient was not severe enough to cause an operability concern. The licensee has proposed corrective actions to develop procedural guidance for restoration of condensate and feedwater systems, after a reactor trip with a loss-of-offsite power, to minimize water hammer events. At the time of the inspection there was nothing to preclude a repetition of this event since corrective actions were not scheduled for implementation until the end of November. Although the inspectors determined the proposed corrective actions were appropriate, the prioritization and timeliness of these proposed actions were inadequate because of the human factors cross-cutting aspects that did not consider this event attributable to placement of the non-class auxiliary feedwater pump in service.
Analysis.
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment reliability. Using the Significance Determination Process Phase 1 worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function. The inspectors noted human performance cross-cutting aspects that contributed to this finding, in that, engineering personnel did not properly use all information and facts available to determine if a condition adverse to quality existed that warranted prompt corrective actions.
Enforcement.
Criterion XVI of Appendix B to 10 CFR Part 50, Corrective Action, requires, in part, that conditions adverse to quality are promptly identified and corrected.
Contrary to the above, the licensee failed to promptly implement corrective actions for a condition that could impact the reliability of the auxiliary feedwater system in a timely manner. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-04).
3. HUMAN PERFORMANCE AND PROCEDURAL ASPECTS OF THE EVENT
A number of unresolved items associated with human performance and procedures were revealed during and following the event. Each of these issues was inspected to assess the licensees effectiveness in determining the root and contributing causes, the extent of condition, and corrective actions.
3.1 (Closed) Unresolved Item 05000528; -529; -530/2004012-011: Auxiliary Feedwater System Operation
a. Inspection Scope
During the loss-of-offsite power event, control room operators manually initiated a main steam isolation signal actuation in accordance with Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, Step 6. In addition to closing the main steam isolation valves, this step also causes closure of drains associated with two critical steam traps required to maintain operability of the turbine-driven auxiliary feedwater pump. With the steam traps unavailable, condensate could accumulate in the steam lines and lead to an overspeed trip of the turbine during startup. Guidance established in Normal Operating Procedure 40OP-9SG01, Main Steam, Revision 37, provides the necessary instructions for manually draining those sections of piping necessary to maintain operability of the pump.
The AIT identified a unresolved item to review the root and contributing causes, extent of condition, and corrective actions associated with emergency operating procedure implementation, the availability of equipment to accomplish manual drains on the turbine-driven auxiliary feedwater system, and the decision-making process for implementing manual drain procedures.
The inspectors reviewed the licensees evaluation and associated corrective actions documented in Condition Report/Disposition Request 2719463; interviewed engineering and operations personnel; and performed walkdowns of the equipment to accomplish manual drains on the turbine-driven auxiliary feedwater system.
b. Observations and Findings
.1 Inadequate Emergency Operating Procedure
Introduction.
A noncited violation of very low safety significance was identified for an inadequate procedure in that emergency operating procedures failed to provide direction to maintain turbine-driven auxiliary feedwater pump operability following a main steam isolation signal.
Description.
The AIT noted that the emergency operating procedure did not caution the operators that a main steam isolation signal would potentially make the turbine-driven auxiliary feedwater pump inoperable. The emergency operating procedure also did not direct the operators to implement the applicable sections of Normal Operating Procedure 40OP-9SG01, Main Steam, Revision 37, which provide the necessary instructions for manually draining those sections of piping necessary to maintain operability of the pump. This procedure requires that the piping associated with the critical steam traps be blown down every 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> until a dry steam condition is reached and then every 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> thereafter. On the day of the event, operators did not commence actions to drain the associated piping until 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> after the reactors tripped.
Analysis.
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process Phase 1 worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function of a single auxiliary feedwater train for greater than the technical specification allowed outage time. The inspectors noted problem identification and resolution aspects that contributed to this finding associated with incomplete corrective actions to provide instructions for manually draining those sections of piping necessary to maintain pump operability. The licensee revised surveillance and operating procedures in response to turbine-driven auxiliary feedwater pump overspeed events in 1990, but overlooked the need to provide the necessary instructions in emergency operating procedures.
Enforcement.
Technical Specification 5.4, Procedures, requires that written procedures be established, implemented, and maintained covering the applicable procedures recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978. Regulatory Guide 1.33, Appendix A, Section 6c, recommends procedures for combating loss of electrical power events. Contrary to the above, Emergency Operating Procedure 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, was inadequate, in that, the emergency operating procedure did not caution the operators that a main steam isolation signal would potentially make the turbine-driven auxiliary feedwater pumps inoperable and did not provide the necessary instructions to maintain operability of the pump. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-05).
.2 Failure to Manage Risk Profile
Introduction.
A noncited violation of very low safety significance was identified for failing to perform a risk assessment to address the changed plant conditions following the June 14, 2004, loss-of-offsite power event, which affected the licensees decision making process for prioritization of equipment restoration.
Description.
Following the loss-of-offsite power event and stabilization of the units, the licensee did not immediately recognize that a risk evaluation needed to be performed. A management review team was held on the afternoon of June 14 to discuss current unit status and prioritize recovery efforts. The licensee failed to recognize that a risk management action level of RED existed on Unit 2 in accordance with Procedure 30DP-9MT03, Assessment and Management of Risk When Performing Maintenance in Modes 1 - 4, Revision 8. The RED risk management action level condition on Unit 2 was because of the combination of having both the turbine-driven auxiliary feedwater pump and Emergency Diesel Generator 2A inoperable.
Procedure 30DP-9MT03 delineates management controls for a RED risk management action level, which includes focusing with the highest priority, to restore out-of-service equipment rapidly. Management inappropriately decided to begin draining the Unit 1 turbine-driven auxiliary feedwater pump steam traps first, without consideration of the higher risk configuration of Unit 2.
The AIT noted that with Unit 2 having only one of two emergency diesel generators available, it was a more prudent decision to restore the Unit 2 turbine-driven auxiliary feedwater pump to service first. The inspectors observed during the followup inspection that this NRC identified issue was not addressed by the licensees corrective action program. The licensee initiated Condition Report/Disposition Request 2735952 to evaluate this issue following the inspectors identification of this oversight.
Analysis.
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process Phase 1 Worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone and did not result in the actual loss of a safety function of a single auxiliary feedwater train for greater than the technical specification allowed outage time. The inspectors noted human performance cross-cutting aspects associated with poor decision making that contributed to the cause of this finding. The inspectors also noted problem identification and resolution aspects associated with this finding in that this NRC identified issue was not addressed in the licensees corrective action program until prompted by the NRC.
Enforcement.
Requirements for monitoring the effectiveness of maintenance at nuclear power plants, 10 CFR Part 50.65, Section (a)(4), states, in part, that before performing maintenance activities, the licensee shall assess and manage the increase in risk that may result from the proposed maintenance activities. Contrary to the above, the licensee did not assess and manage the increase in risk associated with the prioritization of equipment restoration and unit recovery. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-06).
3.2 (Closed) Unresolved Item 05000529/2004012-012: Unit 2, Train E Positive Displacement Charging Pump Trip
a. Inspection Scope
The AIT identified a unresolved item associated with multiple operator errors that occurred during Unit 2 charging pump operations. The errors occurred while implementing Procedures 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, and 40EP-9EO10, Standard Appendices, Revision 33, in response to the June 14, 2004, loss-of-offsite power event. This issue was made unresolved to review the root and contributing causes, extent of condition, and corrective actions associated with the operator errors.
The inspectors reviewed the licensees evaluation and associated corrective actions documented in Condition Report/Disposition Requests 2716521 and 2716806.
b. Observations and Findings
Introduction.
A noncited violation of very low safety significance was identified for multiple examples of failing to follow emergency operating procedures.
Description.
The AIT reviewed the emergency operating procedures and the control room operator response to the loss-of-offsite power event with respect to the charging pumps to determine the effect on the response to the event. The team also interviewed plant personnel and reviewed Condition Report/Disposition Requests 2716521 and 2716806 regarding the activities surrounding the charging pump operation. The licensees evaluation of the condition was consistent with observations made by the AIT.
No additional observations were identified associated with the operators response following the loss-of-offsite power event.
The inspectors observed that the evaluations performed by the licensee only addressed operational issues with the auxiliary operator. The NRC identified issue associated with procedural adherence by the control room supervisor was not addressed by the licensees corrective action program. The licensee initiated Condition Report/Disposition Request 2736503 to evaluate this issue following the inspectors identification of this oversight. The licensees evaluation concluded that the control room supervisor did not implement step 11 of Procedure 40EP-9EO07 in accordance with expectations. Sub-step 11b should have been performed when step 11a was actually implemented. The licensee also concluded that the error was the result of the control room supervisors mind set of unavailability of motor operated valves without power.
Analysis.
The inspectors determined that the finding was greater than minor since it was associated with the equipment performance attribute of the mitigating systems cornerstone and affected the cornerstone objective of equipment availability. Using the Significance Determination Process Phase 1 worksheet, the finding is determined to have very low safety significance because it only affects the mitigating systems cornerstone, did not result in the actual loss of a safety function, and no significant delays occurred that adversely impacted operator response to the loss-of-offsite power event. The inspectors noted human performance cross-cutting aspects that contributed to this finding in that operators lacked attention to detail when implementing emergency operating procedures. The inspectors also noted problem identification and resolution aspects associated with this finding in that this NRC identified issue was not addressed in the licensees corrective action program until prompted by the NRC.
Enforcement.
Technical Specification 5.4, Procedures, requires that written procedures be established, implemented, and maintained covering the applicable procedures recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978.
Regulatory Guide 1.33, Appendix A, Section 6c, recommends procedures for combating loss of electrical power events. Contrary to the above, operations personnel failed to properly implement Emergency Operating Procedures 40EP-9EO07, Loss of Offsite Power/Loss of Forced Circulation, Revision 10, and 40EP-9EO10, Standard Appendices, Revision 33. Because the violation is of very low safety significance and has been entered into the corrective action program, this violation is being treated as a noncited violation consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-07).
3.3 (Closed) Unresolved Item 05000528; -529; -530/2004012-013: Entry Into Technical Specification Action Statements
a. Inspection Scope
The AIT identified an unresolved item to review how technical specifications are used during and following an event in which emergency operating procedures were used.
Specifically, the team observed that technical specification limiting conditions for operation were not started until the applicable step in the emergency operating procedure was reached to assess limiting conditions for operation. The inspectors evaluated control room log entries associated with the plant trips caused by the loss-of-offsite power. The inspectors also assessed the operator response as it related to the required entry into Technical Specification Action Statements. The inspection was accomplished through a review of documents and interviews with operators and engineering staff.
b. Observations and Findings
The team found that in each of the following examples, the time of entry into the limiting conditions for operation did not reflect the time of discovery of the inoperability of the affected components.
A review of the Unit 2 control room log entries disclosed that operators exited the emergency operating procedure at 5:10 a.m. MST on June 15, 2004. Co-incident with this log entry were the entries into Technical Specifications Limiting Conditions for Operation 3.7.5 for an inoperable turbine-driven auxiliary feedwater pump and Limiting Conditions for Operation 3.8.1 for an inoperable Train "A" emergency diesel generator.
The emergency diesel generator was not operable shortly after the reactor trip because a failed diode in the exciter prevented it from accepting loads from the load sequencer (Section 2.4). When the manual main steam isolation signal actuation occurred, the turbine-driven auxiliary feedwater system steam trap drains were isolated, which could cause the turbine-driven auxiliary feedwater pump to become inoperable without manual action to drain the associated piping within 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> (Sections 2.9 and 3.1). The manual action did not occur until approximately 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> after the main steam isolation signal actuation. Consequently, both components were considered inoperable prior to exiting the emergency operating procedure.
During the plant transient, the battery chargers to the Unit 2 A and C Vital 125 V batteries were not operable for approximately 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> when the Train "A" electrical bus was not powered by either offsite power or the emergency diesel generator. Technical Specification 3.8.4 requires that, within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, battery cell parameters be verified to meet Table 3.8.6-1, Category "A" limits when the required battery charger is inoperable.
The batteries were discharged for 110 minutes until offsite power was restored to the electrical bus and the battery charger. The entry into the required technical specification action was not documented in the control room log and the action to verify battery cell parameters was not taken until approximately 5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> after the battery charger became inoperable. Additionally, the batteries were declared operable solely on the restoration of offsite power to the bus and battery charger and without any surveillance to verify compliance with the technical specification.
The Unit 3 Loop 2A Safety Injection Check Valve SIE-V217, is a 14-inch swing check valve. At 10:12 a.m. on June 14, an alarm indicating back leakage through this check valve was received. Alarm Response Procedure 40AL-9RK2B, requires that, when indicated pressure is greater than 1850 psig, Low Pressure Safety Injection Train "B" be declared inoperable and Technical Specification 3.5.3 be entered. At 8:44 p.m. on June 14, 1850 psig was exceeded. Entry into Technical Specification 3.5.3 was logged as being the time that the loss-of-offsite power emergency operating procedure was exited, 12:40 a.m. on June 15, 2004, and not at the time that 1850 psig was exceeded.
The Normal Operating Procedure 40DP-9OP02, Conduct of Shift Operations, Revision 28, requires that when reliable plant indication identifies a condition that requires entry into a technical specification condition, the applicable condition shall be entered immediately. The logging of entry into the applicable limiting conditions for operation after the time of discovery created the potential for failing to meet technical specification requirements.
As a result, the 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> clock for the inoperable Unit 3 diesel-generator was not started until 22 hours2.546296e-4 days <br />0.00611 hours <br />3.637566e-5 weeks <br />8.371e-6 months <br /> after it was known to have become inoperable. Although the 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> clock was not violated and technical specifications were satisfied, this failure to enter the limiting conditions for operation resulted from inadequate guidance in your Emergency Operating Procedure Users Guide Procedure. We determined that this procedure did not adequately ensure compliance with technical specification requirements. If this erroneous practice were to be left uncorrected, the implementation of this procedure could result in exceeding allowed outage times. This represents an inadequate procedure and a failure to meet the requirements of 10 CFR Part 50, Appendix B, Criterion V. Because the technical specifications were not violated, this is a minor finding, which is not subject to enforcement action in accordance with Section IV or VI of the NRCs Enforcement Policy. However, human performance cross-cutting aspects contributed to this finding because operators lacked attention to detail when implementing technical specification requirements.
3.4 (Closed) Unresolved Item 05000528; -529; -530/2004012-014: Technical Support Center Emergency Diesel Generator Trip
a. Inspection Scope
The AIT identified that during the loss-of-offsite power event of June 14, 2004, the technical support center emergency diesel generator failed to supply power as designed to the technical support center and other plant loads. The inspectors reviewed the investigation report associated with Condition Report/Disposition Request 2715749, Emergency Response Organization Response to Reactor Trips on 06/14/04, dated September 3, 2004. The inspectors reviewed the corrective actions identified, and the status of completion of the corrective actions. The inspectors evaluated the completeness of those corrective actions against the standards of condition report/disposition request procedure and 10 CFR Part 50, Appendix E. The inspector also reviewed other documents as listed in the Appendix to this report.
Introduction.
The inspectors identified a noncited violation of 10 CFR 50.54(q) for failure to follow the emergency plan. Specifically, the finding involved the failure of the technical support center to be available to the Emergency Response Organization for performance of their emergency response functions during the ALERT declared on June 14, 2004.
Description.
On June 8, 2004, while conducting Work Order 2623863, Monthly Inspection of Technical Support Center DG Battery and Battery Charger, the maintenance technician identified a need to revise the work order due to excessive corrosion on the post and connector on the No.1 battery negative connection. Approval of the scope change, which involved removing the battery connector, and cleaning the negative terminal and connector, and starting the emergency diesel generator for a re-test, was approved by the electrical maintenance team leader. However, detailed retest procedure steps were not added to the work order to conduct a retest of the technical support center emergency diesel generator following the completion of the battery terminal maintenance.
Nuclear Administrative and Technical Manual Procedure 30DP-9MP01, Conduct of Maintenance, Revision 36, step 3.4.7, requires that Changes, amendments, additions of modifications to work order documents shall be accomplished in agreement with 30DP-9WP02, Work Document Development and Control. Revision 34, step 6.2.1 of 30DP-9WP02, requires that Changes to Work Orders (Including Pen and Ink changes)shall be handled as described in Appendix O - Work Order Change. Appendix O, step 2.2.1.2, requires that the scope change be approved by the team leader and operations.
A retest was required by the approved change to the work order, however, the retest procedure was not written in detail to ensure restoration of the technical support center diesel generator to a standby lineup. When the maintenance technician completed the retest of the emergency diesel generator, he failed to restore the idle/run switch to the run position, as would have been required as the last step in the retest procedure for starting the technical support center emergency diesel generator. The retest procedure would not meet the criteria for tool pouch work, as described in Appendix D to Procedure 30DP-9MP01 and, therefore, would require a formal procedure to be used and in-hand when conducting the maintenance.
Failure to restore the idle/run switch to the run position disabled the auto-load feature of the emergency diesel generator. Following a demand signal, the emergency diesel generator would come up to idle speed, but would not load onto the E-NGN-L50 bus and, therefore, the loads powered from that bus remained de-energized. Some of the loads on that bus include the technical support center computers, lighting and receptacles, the technical support center emergency diesel generator cooling fan, and some lighting at the primary access point. Without power to the technical support center emergency diesel generator cooling fan, the emergency diesel generator would approach an automatic shutdown condition within a few minutes of starting because of high temperature in the emergency diesel generator cooling water.
Following the loss-of-offsite power on June, 14, 2004, the technical support center emergency diesel generator started and ran at idle speed, but did not load onto the L50 bus. Within several minutes, the emergency diesel generator automatically shutdown because of high cooling water temperature. The combination of the loss-of-offsite power and the failure of the technical support center emergency diesel generator resulted in no power to the technical support center until offsite power was restored.
This failure resulted from human performance aspects that contributed to this finding.
The licensee took several corrective actions related to this issue. On June 16, 2004, Work Order 2715869 was completed to verify the operability of the technical support center emergency diesel generator. The event was reported in the Electrical Industry Events for operational experience benefit of other power plants, and maintenance technicians were provided specific coaching and positive discipline. The licensee identified 2 additional actions related to maintenance processes to be completed to prevent recurrence of this issue and one additional corrective action to provide an alarm at the technical support center annunciator panel when the technical support center emergency diesel generator is in other than an automatic start and load configuration.
All of the corrective actions were assigned completion dates prior to December 3, 2004, and an effectiveness review was assigned a completion date of August 31, 2005. The inspectors concluded the corrective actions taken were adequate to address the issue, and when completed, the identified corrective actions were adequate to prevent recurrence.
Analysis.
The failure of the technical support center emergency diesel generator to be maintained in an operable condition was a performance deficiency, in that, adequate facilities and equipment for emergency response are required to be maintained to allow emergency responders to perform their emergency functions and current licensee procedures were adequate to prevent the maintenance error that resulted in disabling the technical support center emergency diesel generator. The failure to maintain the technical support center emergency diesel generator in an operable condition also had human performance/cross cutting aspects that are noted in Section 4OA4 of this report.
The issue is more than minor because it is associated with the Emergency Preparedness Cornerstone attribute of Facilities and Equipment, and affected the cornerstone objective in that loss of an emergency facility and associated emergency equipment would affect the licensees ability to implement adequate measures to protect the health and safety of the public.
The finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem.
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 8 is considered a failure to comply with Emergency Planning Standard 8 during an actual event implementation. The event was a declared an Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609 Appendix B, §2.0. Therefore, the finding is of very low safety significance.
Enforcement.
10 CFR 50.54(q) provides, in part, that "[a] licensee authorized to possess and operate a nuclear power reactor shall follow . . . emergency plans which meet the standards in [section] 50.47(b). . . ." 10 CFR. 50.47(b) requires that the onsite emergency response plans for nuclear power reactors must meet each of 16 planning standards, of which, Standard 8 states, in part, the ". . . facilities and equipment required for emergency response shall be maintained. . . ."
Contrary to the above, on June 14, 2004, following the loss-of-offsite power and failure of the technical support center emergency diesel generator to supply power to the technical support center, the technical support center was not available to support the emergency response organization. Failure of the technical support center to be available resulted in additional challenges to the emergency response organization to coordinate site-wide efforts to respond to the event and provide accurate and timely notification to the state and local officials.
Because the failure to implement the emergency planning standard was of very low safety significance and has been entered into the licensees corrective action program as Condition Report/Disposition Request 2715726, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528,529,530/2004013-08).
3.5 (Closed) Unresolved Item 05000528; -529; -530/2004012-015: Emergency Response Organization Issues
a. Inspection Scope
The AIT identified three issues associated with this unresolved item. The first was the accuracy and timeliness of notification of state and local officials of the emergency events at the PVNGS. The second was the unavailability of radiological dose projection computers during the loss-of-offsite power event. The third was the delays in notification of the PVNGS emergency response organization and the subsequent failure to meet emergency facility staffing timeliness goals. The inspectors reviewed the investigation report associated with Condition Report/Disposition Request 2715749, Emergency Response Organization (ERO) Response to Reactor Trips on 06/14/04, dated September 3, 2004. The inspectors reviewed the corrective actions identified and the status of completion of the corrective actions. The inspectors evaluated the completeness of those corrective actions against the standards of the condition report/disposition request procedure and 10 CFR Part 50, Appendix E. The inspector also reviewed other documents as listed in the Appendix to this report.
b. Observations and Findings
.1 Timely and Accurate Notification of State and Local Officials
Introduction.
The inspectors identified a noncited violation of 10 CFR 50.54(q) for failure to follow the emergency plan. Specifically, the licensee failed to ensure that adequate command and control was established during the June 14, 2004, Alert event where multiple units were in a declared emergency condition. Failure to establish adequate command and control resulted in the notification irregularities to the state and local officials.
Description.
On June 14, 2004, at 7:41am local time, PVNGS, Units 1, 2 and 3 experienced a loss of all offsite power, resulting in automatic reactor trips at all three units.
Unit 1 plant equipment responded as expected. With a loss-of-offsite power continuing for greater than 15 minutes, Unit 1 met the conditions of Emergency Action Level 2-1 for declaration of a Notification of Unusual Event at approximately 7:56 a.m. local time.
Unit 2 plant equipment responded as expected with the exception of the Emergency Diesel Generator 2A, which failed to maintain output voltage and was manually tripped by the Unit 2 operators at 7:50 a.m. local time. With a loss-of-offsite power continuing for greater than 15 minutes, and the failure of one emergency diesel generator to supply power to its respective electrical bus, Unit 2 met the conditions of Emergency Action Level 2-3 to declare an Alert at 7:56 a.m. local time.
Unit 3 plant equipment responded as expected, with the exception of the steam bypass control system, which resulted in an automatic main steam isolation. With a loss-of-offsite power continuing for greater than 15 minutes, Unit 3 met the conditions of Emergency Action Level 2-1 for declaration of a Notification of Unusual Event at approximately 7:56 a.m. local time. The failure of the steam bypass control system did not affect the applicable emergency classification.
Each of the units respective shift managers initially assumed the role of the emergency coordinator. Unit 2 declared an ALERT at 7:54 a.m. local time, and Units 1 and 3 both declared a Notification of Unusual Event at approximately 7:58 a.m. local time. The emergency coordinator for Unit 2 initiated notifications to the state and locals of the ALERT at Unit 2 at 7:59 a.m. local time, but did not include status of the other units.
The emergency coordinator in Unit 3 initiated notifications to the state and locals of the Notification of Unusual Event at Unit 3 at 8:18 a.m. local time, but did not include status of the other units. The emergency coordinator in Unit 1, in consultation with the site shift manager and the Unit 2 operations director, decided that since Unit 2 was in a condition requiring the highest level of declaration of the three units, that the site shift manager should go to the Unit 2 control room and relieve the Unit 2 shift manager as the emergency coordinator. The site shift manager relieved the Unit 2 shift manager as the emergency coordinator at 8:03 a.m. local time. The Unit 2 operations director also stated that Unit 2 would assume the site emergency plan communication requirements.
Based on this communication, the Unit 1 shift manager did not initiate notification of the Notification of Unusual Event declaration at Unit 1, believing that the emergency coordinator in Unit 2 would accomplish the notification. The site manager, as emergency coordinator located in Unit 2, focused on Unit 2 events and did not initiate offsite communications for the other units. Consequently, the state and local officials were not notified of the Notification of Unusual Event at Unit 1. This resulted from a failure to recognize procedural requirements and is a human performance cross-cutting aspect that contributed to this finding.
The declaration of ALERT in Unit 2 required notification of the emergency response organization and activation of the emergency facilities at PVNGS. While the emergency facilities were being staffed, the emergency coordinator in Unit 2 downgraded the Unit 2 classification to an Notification of Unusual Event at 9:51 a.m. local time. When the emergency operation facility was staffed and ready to perform its emergency functions, the Emergency Operations Facility director relieved as the site emergency coordinator.
The site emergency coordinator then initiated the notification to the state and local authorities of the downgrade to an Notification of Unusual Event for Unit 2 at 10:05 a.m.,
but did not include status of the other units. This notification erroneously included Emergency Action Levels 2-3 and 8-2, none of which were met at the time of the downgrade. Emergency Action Level 2-3 is an ALERT based on ac power capability to essential busses reduced to a single power source for greater than 15 minutes and Emergency Action Level 8-2 is an Unusual Event based on other conditions existing, which in the judgement of the Emergency Director, warrant declaration of an Unusual Event. The site emergency coordinator terminated the event at 12:07 p.m. local time, and initiated the notification to state and local officials at 12:15 p.m. local time. This final notification also incorrectly included Emergency Action Levels 2-1 and 8-2 as being met at the time of the notification, when they were not.
The licensee took several corrective actions related to this issue. On July 1, 2004, an operations night order was issued to all three units, which summarized the human performance issues associated with the emergency plan activities of June 14, 2004, and also emphasized that the Emergency Plan Implementing Procedure 01, Satellite Technical Support Center Actions, states that for multiple unit events, the Unit 1 shift manager is responsible for initially classifying and declaring the emergency, and assuming the position of the on-shift emergency coordinator. An Operation News Flash issued on July 13, 2004, repeated the message from the July 1, 2004, operations night order.
The licensee identified six additional actions to be completed to prevent recurrence of this issue, and six additional corrective actions to improve communications reliability both between onsite emergency facilities and with the state and local authorities. All of the corrective actions were assigned completion dates prior to December 2004, and an effectiveness review was assigned a completion date of August 31, 2005. The inspectors concluded the corrective actions taken were adequate to address the issue, and when completed, the identified corrective actions were adequate to prevent recurrence.
Analysis.
Emergency planning standard 10 CFR 50.47(b)(1) states, in part, that responsibility for emergency response is assigned . . . . Section 6.2 of the PVNGS emergency plan requires that, for multiple unit events, the Unit 1 shift manager will assume the position of the onshift emergency coordinator. Emergency Plan Implementing Procedure-01, Section 3.1, states the same requirement as the emergency plan. The onshift emergency coordinator maintains that position until relieved by the emergency facility director in the technical support center or the emergency operations facility.
The issue is a performance deficiency, in that, the licensee failed to follow the emergency plan and Emergency Plan Implementing Procedure 01, in that, the Unit 1 shift manager did not assume the responsibility of the emergency coordinator, resulting in miscommunication between the three units and the offsite officials. The finding is more than minor because it is related to the emergency preparedness cornerstone attribute of response organization performance, and affects the cornerstone objective in that command and control challenges resulting in inaccurate communications to the offsite officials could potentially affect the ability to ensure that adequate measures would be taken to protect the public health and safety.
This finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem.
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 1 is considered a failure to comply with Planning Standard 1 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609, Appendix B, §2.0. Therefore, the finding is of very low safety significance (Green.)
Enforcement . 10 CFR 50.54(q) provides, in part, that "[a] licensee authorized to possess and operate a nuclear power reactor shall follow . . . emergency plans which meet the standards in [section] 50.47(b). . . ." 10 CFR. 50.47(b) requires that the onsite emergency response plans for nuclear power reactors must meet each of 16 planning standards, of which, standard
- (1) states, in part, the ". . . responsibility for emergency response is assigned." The licensees Emergency Plan, Section 6.2, states, For those situations involving more than one unit, the Unit 1 shift manager is responsible for initially classifying and declaring the emergency and assuming the position of the On shift emergency coordinator.
Contrary to the above, on June 14, 2004, following the loss-of-offsite power and other complications to each of the 3 PVNGS units, the Unit 1 shift manager did not assume the emergency coordinator position. Failure to implement the emergency plan requirement affected the ability to coordinate licensee response to the event and provide accurate and timely notification to the state and local officials.
Because the failure to implement the emergency plan requirement was of very low safety significance and has been entered into the licensees corrective action program as Condition Report/Disposition Request 2715749, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy:
(NCV 05000528,529,530/2004013-09).
No significant findings were identified.
Because of the loss-of-offsite power on June 14, 2004, power to the electrical receptacles in the satellite technical support centers of each unit was unavailable. The dose assessment computers in the satellite technical support centers were normally powered by these receptacles. Therefore, without operator action, the dose assessment computers in the satellite technical support centers were not available.
Additionally, the dose assessment computer in the technical support center was unavailable because of the failure of the technical support center emergency diesel generator to supply power to the technical support center. The licensee did identify, during the event, that power was available to the operations support building, and that a computer was available for dose assessment and could be accessed in a timely manner. During the Alert of June 14, 2004, there was no need for dose assessment and the ability to use the operations support building facility to support dose assessment was not demonstrated. The licensee also identified that power was available to certain electrical receptacles in the satellite technical support centers, and that simple operator action to unplug the dose assessment computers and plug them into the energized receptacles was all that was required to restore dose assessment capability in the satellite technical support centers. Dose assessment capability was always available from the emergency operations facility, where the dose assessment computers are equipped with uninterruptible power supplies.
The licensee took adequate corrective actions related to this issue. Uninterruptible power supplies were added to each of the satellite technical support center dose assessment computers. Procedure revisions have been identified to ensure identification of those dose assessment computers with uninterruptible power supplies and ensure quarterly testing of the computers and power supplies. The procedure revisions were assigned a completion date of September 30, 2004, and an effectiveness review was assigned a completion date of August 31, 2005.
.3 Emergency Response Organization Notification and Staffing Delays
Introduction.
The inspectors identified a noncited violation of 10 CFR 50.54(q) for failure to follow the emergency plan. Specifically, the licensee failed to meet minimum staffing goals of Table 1, Minimum Staffing Requirements for PVNGS for Nuclear Power Plant Emergencies, following the Alert declaration on June 14, 2004.
Description.
On June 14, 2004, at 7:41 a.m. local time, PVNGS Units 1, 2 and 3 experienced a loss of all offsite power, resulting in automatic reactor trips at all three units. With a loss-of-offsite power continuing for greater than 15 minutes, all three units met the conditions of Emergency Action Level 2-1 for declaration of a Notification of Unusual Event at approximately 7:56 a.m. local time. Unit 2 additionally lost voltage output on the Emergency Diesel Generator 2A, resulting in loss of power to its respective electrical bus. With a loss-of-offsite power continuing for greater than 15 minutes, and the failure of one emergency diesel generator to supply power to its respective electrical bus, Unit 2 met the conditions of Emergency Action Level 2-3 to declare an Alert at 7:56 a.m. local time.
The emergency coordinator, based on imminent conditions, declared an Alert at Unit 2 at 7:54 a.m. local time, and commenced notifications to the emergency response organization (these positions are identified in the Emergency Plan, Table 1,) and the state and local officials. The emergency coordinator attempted to initiate a WPAGER system notification to the essential Emergency Response Organization personnel as required by Emergency Plan Implementing Procedure-99, EPIP Standard Appendices, Appendix D, Notification. However, due to the loss-of-offsite power, the WPAGER computers in the control room were not powered. The emergency coordinator did not attempt to initiate the autodialer system, which was available and was the primary method to notify all Emergency Response Organization personnel of the event classification and the need to report to their respective emergency facilities as required by Emergency Plan Implementing Procedure-01 and Emergency Plan Implementing Procedure-99, Appendix D. After approximately 31 minutes from the Alert declaration, the WPAGER system notification was initiated from an operations support building computer, which had not lost power as a result of the loss-of-offsite power. The licensee indicated that based on the average amount of time required for the communicator to initiate an autodialer notification following an Alert declaration, the actual delay in initiating the Emergency Response Organization notification was approximately 16 minutes.
The licensee noted in the root-cause report that the emergency coordinator chose not to use the autodialer system because he believed he was following the guidance in Emergency Plan Implementing Procedure 01, which stated, During off-normal hours, if an Alert or higher is declared then direct activation fo the Autodialer IAW EPIP-99 Standard Appendix - H Autodialer Activation. The emergency coordinator failed to realize that Monday, June 14, 2004, was a work day off for non-operations shift personnel and, therefore, should have been considered as off-normal hours.
Additionally, the onshift Emergency Response Organization staff was not aware that the autodialer system also provided an automatic initiation of the WPAGER system.
As a result of the delay in notification to the essential Emergency Response Organization members, and no notification to the non-essential Emergency Response Organization members, emergency facility staffing goals for essential Emergency Response Organization members were not met, and many of the non-essential Emergency Response Organization members never responded to the Alert declaration, as required by the facility emergency plan.
The technical support center was staffed with all of its emergency plan required essential responders approximately 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> and 6 minutes after the Alert declaration.
Only five of the required six radiation protection technicians were identified as having responded to the operations support center. Other radiation protection technicians were identified as being onsite, but they did not report to and staff the operations support center and would have, therefore, not been immediately available to perform work functions. Many of the non-essential support positions for the designated essential responders did not staff their respective emergency facilities during the event.
The licensee took several corrective actions related to this issue. On July 1, 2004, and again on July 13, 2004, operations night orders were issued to all three units, which summarized the human performance issues associated with the emergency plan activities of June 14, 2004. Emergency Plan Implementing Procedures 01 and 03 were revised to remove the qualifier that the autodialer should be activated during off-normal hours, now requiring its activation for any Alert declaration or higher.
The licensee identified eight additional corrective actions including training on multiple unit events, clarification of the use of the autodialer as a backup to the WPAGER system, and addition of uninterruptible power supplies to select computers and the site pager tower. All of the corrective actions were assigned completion dates prior to December 13, 2004, and an effectiveness review was assigned a completion date of August 31, 2005. The inspectors concluded the corrective actions taken were adequate to address the issue, and when completed, the identified corrective actions were adequate to prevent recurrence.
Analysis.
Emergency planning standard 10 CFR 50.47(b)(2) states, in part, that process for timely augmentation of on-shift staff is established and maintained . . . .
Table 1 of the PVNGS emergency plan establishes the staffing requirements for augmenting emergency responders. For the technical support center electrical engineer position and the six operations support center radiation protection technicians, Table 1, requires a maximum response time of 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. The Emergency Plan, Section 6.2, also requires that the entire Emergency Response Organization be notified of the declared emergency and to respond to their designated emergency facilities.
The issue is a performance deficiency in that the licensee failed to follow the emergency plan and Emergency Plan Implementing Procedures 01 and 03, in that, the entire Emergency Response Organization was not notified and all essential positions were not staffed in the required times of Table 1. The finding is more than minor because it is related to the emergency preparedness cornerstone attribute of response organization performance, and affects the cornerstone objective in that failure to fully staff the emergency response facilities could potentially affect the ability to ensure that adequate measures would be taken to protect the public health and safety.
This finding was evaluated using Inspection Manual Chapter 0609, "Significance Determination Process," Appendix B, Sheet 2 - Actual Event Implementation Problem.
Failure to implement the requirements of the emergency plan associated with Emergency Planning Standard 2 is considered a failure to comply with Planning Standard 2 during an actual event implementation. The event was a declared Alert, but was not a failure to implement a risk significant planning standard, as defined in Inspection Manual Chapter 0609, Appendix B, §2.0. Therefore, the finding is of very low safety significance.
Enforcement . 10 CFR 50.54(q) provides in part that "[a] licensee authorized to possess and operate a nuclear power reactor shall follow . . . emergency plans which meet the standards in [section] 50.47(b). . . ." 10 CFR. 50.47(b) requires that the onsite emergency response plans for nuclear power reactors must meet each of 16 planning standards, of which, standard
- (2) states, in part, the ". . .process for timely augmentation of onshift staff is established and maintained . . . ." The licensees Emergency Plan, Table 1, establishes that the technical support center electrical engineer position and the six operations support center radiation protection technicians positions be filled within 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> of the declaration of an Alert or higher. The Emergency Plan, Section 6.3, also requires that the entire Emergency Response Organization be notified of the declared emergency and to respond to their designated emergency facilities.
Contrary to the above, on June 14, 2004, following the loss-of-offsite power and declaration of an Alert at Unit 2, the entire Emergency Response Organization was not notified of the declared event, and the technical support center electrical engineer, and one of the six required radiation protection technicians, did not report to their emergency facilities within 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> of the Alert declaration.
Because the failure to implement the emergency plan requirement was of very low safety significance and has been entered into the licensees corrective action program as Condition Report/Disposition Request 2715749, this violation is being treated as a noncited violation, consistent with Section VI.A of the NRC Enforcement Policy (NCV 05000528, 529, 530/2004013-10).
OTHER ACTIVITIES
4OA2 Problem Identification and Resolution
Section 2.2 (Unresolved Item 02) describes a finding where the licensee failed to adequately resolve reactor operator comments concerning the apparent malfunction of an atmospheric dump valve and consequently failed to identify needed operator training.
Section 2.4 (Unresolved Item 04) describes a finding where the licensee failed to thoroughly investigate the failure of safety related equipment and did not conduct troubleshooting nor capture the failure in the corrective action program.
Section 2.5 (Unresolved Item 05) describes a finding where licensee personnel became aware of other power transient scenarios where the steam bypass control system could fail to perform as expected and this information was not incorporated in the licensees corrective action program.
Section 2.5 (Unresolved Item 06) describes a finding where the licensees evaluation to delay repairs on a failed main generator excitation control circuit only addressed concerns with generator protection. The evaluation was limited in scope and failed to consider all possible impacts.
Section 2.6 (Unresolved Item 07) describes a finding where a previous trip of a reactor coolant pump lift oil pump motor was discounted without adequate basis. The licensee concluded that the problem no longer existed and declared the motor operable following a 30 minute run. This ineffective corrective action led to a subsequent failure. Since there was clear indication of some form of intermittent motor/pump malfunction from the earlier tests, the failure to pursue this issue and determine a root cause was not adequate.
Section 2.9 (Unresolved Item 10) describes a finding where prioritization and timeliness of proposed corrective actions were inadequate because the engineers did not consider the event attributable to the identified root cause, which was the effect of the placement of the non-class auxiliary feedwater pump in service.
Section 3.1 (Unresolved Item 11) describes a finding where the licensee revised surveillance and operating procedures in response to turbine-driven auxiliary feedwater pump overspeed events in 1990, but overlooked the need to provide the necessary instructions in emergency operating procedures.
Sections 3.1 (Unresolved Item 11) and 3.2 (Unresolved Item 12) describe findings where the NRC identified issues were not addressed in the licensees corrective action program until prompted by the NRC.
4OA3 Event Followup
(Closed) Licensee Event Report 05000528/2004-006-00: Loss-of-Offsite Power - Three Unit Trip.
The June 14, 2004, a loss-of-offsite power and subsequent reactor trip of all three PVNGS plants was the subject of this licensee event report. The licensee event report was reviewed by the inspectors and no findings of significance were identified. Because the event was the subject of an NRC AIT, and because no additional significant information was contained in the licensee event report, this licensee event report is closed.
4OA4 Human Performance
Section 2.3 (Unresolved Item 03) describes a finding where engineering personnel did not properly consider a loss of power to the letdown system when designing a temporary modification and, as a result, the system failed to isolate when needed.
Section 2.5 (Unresolved Item 06) describes a finding where required procedures for the dispositioning of a degraded condition for continued use were not utilized and, as a result, a rigorous evaluation was not performed to ensure that continued operation would not challenge plant systems or operators.
Section 2.8 (Unresolved Item 09) describes a finding where operators in two units failed to follow contingency actions in Emergency Operating Procedures.
Section 2.9 (Unresolved Item 10) describes a finding where engineering personnel did not properly use all information and facts available to identify a condition adverse to quality that warranted prompt corrective actions.
Section 3.1 (Unresolved Item 11) describes a finding where poor management decision making led to the failure to assess and manage the increase in risk associated with the prioritization of equipment restoration and unit recovery.
Section 3.2 (Unresolved Item 12) describes a finding where operators lacked attention to detail when implementing procedures that resulted in a failure to follow several emergency operating procedure steps during the response to the loss-of-offsite power event.
Section 3.3 (Unresolved Item 13) describes a finding where operators lacked attention to detail when implementing technical specification requirements.
Section 3.4 (Unresolved Item 14) describes a finding where detailed retest procedure steps were not added to a work order to conduct a re-strike test of the technical support center emergency diesel generator following the completion of the battery terminal maintenance. This error led to the failure of the technical support center emergency diesel generator to function when needed.
Section 3.5 (Unresolved Item 15) describes a finding where a failure to recognize procedural requirements for a single emergency coordinator led to notification irregularities to state and local officials.
4OA6 Exit Meeting
On September 24, 2004, the team presented the preliminary results from the ongoing AIT followup. On October 26, 2004, the AIT Followup Inspection Team Leader presented the results of the inspection in a meeting held at the PVNGS Nuclear Generating Station site to Mr. G. Overbeck and Mr. J. Levine, and other members of his staff. Mr. Overbeck acknowledged the inspection findings. Proprietary information reviewed by the team was returned to the facility.
ATTACHMENT 1
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee
Scott Bauer, Department Leader, Regulatory Affairs
David Crozier, Program Leader, Emergency Planning
Carl Churchman, Director, Steam Generator Replacement
Jim Levine, Executive Vice President, Generation
Gregg Overbeck, Senior Vice President, Nuclear
David Mauldin, Vice President, Nuclear Engineering & Support
Paul Mueller, Senior Electrical Engineer
John Holmes, Section Leader, Electrical Engineering
Steven Kestler, Electrical Engineer
Bajranga Aggarwal, Systems Engineer
John Hesser, Director, Emergency Services
Larry Leavitt, Senior Program Advisor, Emergency Services
Randy Sorensen, Department Leader, Chemistry
Terry Radtke, Director, Operations
Fred Riedel, Director, Nuclear Training
Brian Ramey, Section Leader, Reliability Engineering
Mark Van Dop, Department Leader, Design Engineering
Mike Hodge, Engineer
Jim McDowell, Investigator, Nuclear Assurance Department
Kevin Sweeney, Section Leader, System Engineering
Don Wheeler, Section Leader, Nuclear Assurance Department
Mike Winsor, Director, Engineering
Pete Borchert, Director, Work Management
Dave Smith, Plant Manager, Operations
Michael Sontag, Department Leader, Nuclear Assurance Department
John Scott, Department Leader, Nuclear Assurance Department
Steve Kesler, Section Leader, Design Engineering
Dan Marks, Section Leader, Regulatory Affairs
Craig Seaman, Director, Nuclear Fuel Management
Ken Manne, Senior Attorney
Michael Grigsby, Department Leader, Operations
Donald Vogt, Section Leader, Operations
Mark McGhee, Department Leader, Operations
Don Straka, Senior Consultant, Regulatory Affairs
Harvey Leake, Senior Consultant, Design Engineering
John Hesser, Director, Emergency Services
1-A Attachment 1
ITEMS CLOSED
05000528/2004012-01; URI Corrective actions to improve the reliability and
05000529/2004012-01; independence of offsite power (Section 2.1). 05000530/2004012-01
05000528/2004012-02 URI Unit 1 Atmospheric Dump Valve 185 Failure
(Section 2.2). 05000528/2004012-03 URI Unit 1 Letdown System Failure to Isolate
(Section 2.3). 05000529/2004012-04 URI Unit 2 Train "A" Emergency Diesel Generator
Failure (Section 2.4). 05000530/2004012-05 URI Unit 3 Main Turbine Bypass Valve Control System
Operation (Section 2.5). 05000530/2004012-06 URI Unit 3 Main Generator Excitation Controls and
Variable Overpower Trip on June 14, 2004
(Section 2.5). 05000530/2004012-07 URI Unit 3 Reactor Coolant Pump 2B Lift Oil Pump
Motor Breaker Thermal Overload Sizing
(Section 2.6). 05000528/2004012-09; URI Magna-Blast Circuit Breaker Reliability
05000529/2004012-09; (Section 2.8). 05000530/2004012-09
05000528/2004012-10; URI Auxiliary Feedwater System Operational Issues05000529/2004012-10; (Section 2.9). 05000530/2004012-10
05000528/2004012-11; URI Turbine-driven Auxiliary Feedwater System Drains,05000529/2004012-11; Design Control, and Procedures (Section 3.1). 05000530/2004012-11
05000529/2004012-12 URI Unit 2 Charging Pump Operations Errors
(Section 3.2). 05000528/2004012-13; URI Use of Plant Technical Specifications05000529/2004012-13; (Section 3.3). 05000530/2004012-13
05000528/2004012-14; URI Technical Support Center Emergency Diesel
05000529/2004012-14; Generator Failure (Section 3.4). 05000530/2004012-14
2-A Attachment 1
ITEMS OPENED and CLOSED
Opened and Closed
05000528/2004013-001 NCV Failure to address EDG circuit failure (Section 2.4). 05000529/2004013-001
05000530/2004013-001
05000528/2004013-002 NCV Failure to evaluate main generator excitation limiter
05000529/2004013-002 circuit problems (Section 2.5). 05000530/2004013-002
05000528/2004013-003 NCV Failure to follow and inadequate EOP (Section 2.8). 05000529/2004013-003
05000530/2004013-003
05000528/2004013-004 NCV Failure to implement corrective actions for AFW
05000529/2004013-004 (Section 2.9). 05000530/2004013-004
05000528/2004013-005 NCV Inadequate EOP for AFW operation (Section 3.1). 05000529/2004013-005
05000530/2004013-005
05000528/2004013-006 NCV Failure to manage station risk (Section 3.1). 05000529/2004013-006
05000530/2004013-006
05000528/2004013-007 NCV Failure to properly implement LOOP EOP (Section 3.2). 05000529/2004013-007
05000530/2004013-007
05000528/2004013-008 NCV Technical Support Center unavailable (Section 3.4). 05000529/2004013-008
05000530/2004013-008
05000528/2004013-009 NCV Failure properly implement emergency plan (Section 3.5). 05000529/2004013-009
05000530/2004013-009
05000528/2004013-010 NCV Untimely augmentation of emergency personnel
05000529/2004013-010 (Section 3.5). 05000530/2004013-010
3-A Attachment 1