05000254/LER-2013-001

LER-2013-001, Emergency Diesel Generator Cooling Water System Not Aligned

Quad Cities Nuclear Power Station Unit 1
Event date:	02-28-2013
Report date:	04-30-2013
Reporting criterion:	10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications
2542013001R00 - NRC Website

PLANT AND SYSTEM IDENTIFICATION

General Electric - Boiling Water Reactor, 2957 Megawatts Thermal Rated Core Power Energy Industry Identification System (EllS) codes are identified in the text as [XX].

EVENT IDENTIFICATION

Emergency Diesel Generator Cooling Water System Not Aligned Causes Condition Prohibited by Technical Specifications

A. CONDITION PRIOR TO EVENT

Unit: 1 Event Date: February 28, 2013 Event Time: 0539 hours0.00624 days <br />0.15 hours <br />8.912037e-4 weeks <br />2.050895e-4 months <br /> Reactor Mode: 1 Mode Name: Power Operation Power Level: 100%

B. DESCRIPTION OF EVENT

At approximately 0500 hours0.00579 days <br />0.139 hours <br />8.267196e-4 weeks <br />1.9025e-4 months <br /> on February 28, 2013, two Equipment Operators (EOs) were dispatched to perform actions in preparation for Unit 1 CS [BM] Division II Logic Testing. After barring the Unit One EDG [EK], EOs 1 and 2 prepared to perform actions to align the 1/2 EDGCWP [LB] (common to both units) to the Unit 1 ECCS [BJ,BM,BO] Room Coolers [CLR]. EO1 traversed to the area of the 1/2B Instrument Air [LD] Compressor (IAC) [CMP] to verify closed the isolation valve [ISV] from the 1/2 EDGCWP to the Unit 2 ECCS room coolers per the first step of procedure, 1/2 EDGCWP Cross Connect Alignment, E02 then traversed to the 1/2 EDG Room to start the 1/2 EDGCW P.

After verifying the 1/2 EDGCWP to Unit 2 ECCS Room Cooler isolation closed, EO1 waited at the flow indicator for the 1/2 EDGCWP to indicate flow as a result of E02 starting the pump. When flow from the 1/2 EDGCWP to the 1/2 EDG Cooling Water Heat Exchangers [HX] was verified satisfactory, EO1 returned to the isolation valve for the 1/2 EDGCWP to Unit 2 ECCS Room Coolers and started opening the valve to establish flow. Prior to manipulating the isolation valve, EO1 verified the train and component, but did not adequately verify the proper unit. EO1 had a preconceived belief that the actions to be taken were on the same valve that was previously verified. That preconceived belief carried on to the verification of Room Cooler flow on Unit 2 instead of Unit 1. As EO1 was establishing 1/2 EDGCWP flow, E02 returned to the 1/2B IAC area and watched EO1 as flow rate from the 1/2 EDGCWP to the Unit 2 ECCS Room Coolers was established. After establishing flow, E02 returned to the 1/2 EDG Room to secure the 1/2 EDGCWP as directed by the procedure.

This event timeline was initiated on February 28, 2013 at 0500 hours0.00579 days <br />0.139 hours <br />8.267196e-4 weeks <br />1.9025e-4 months <br /> when the Unit 1 CS Division II Logic Testing was initiated and TS 3.8.1, Condition B was entered. At 0539 hours0.00624 days <br />0.15 hours <br />8.912037e-4 weeks <br />2.050895e-4 months <br /> the 1/2 EDGCWP was reported as aligned to the Unit 1 ECCS room coolers. The 1/2 EDGCWP had actually inadvertently been aligned improperly at that time, since it was aligned to Unit 2 ECCS room coolers instead of the required Unit 1 ECCS room coolers. On March 1, 2013 at 1338 hours0.0155 days <br />0.372 hours <br />0.00221 weeks <br />5.09109e-4 months <br /> the Unit 1 EDG and Unit 1 EDGCWP were returned to operable status, and ECCS room cooling was returned to Operable status. At 1600 hours0.0185 days <br />0.444 hours <br />0.00265 weeks <br />6.088e-4 months <br /> it was discovered that the 1/2 EDGCWP had been incorrectly aligned; the Shift Manager was notified. At 1625 hours0.0188 days <br />0.451 hours <br />0.00269 weeks <br />6.183125e-4 months <br /> the 1/2 EDGCWP was aligned to its normal configuration.

During the time the 1/2 EDGCWP isolation valve to Unit 1 ECCS room cooling (1/2-3999-89) was improperly aligned, it was however, available to be opened via Operator manual actions had an ECCS initiation occurred.

During the time of improper alignment, TS 3.7.2, Diesel Generator Cooling Water (DGCW) System, Required Action B was not met since, neither (B.1) an operable DGCW subsystem was "aligned" to the ECCS room emergency coolers within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, nor (B.2) were associated ECCS declared inoperable within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. In addition, TS 3.0.3, was not entered when the 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> Actions of 3.7.2.6 were exceeded. This improper alignment condition had existed for 33 hours3.819444e-4 days <br />0.00917 hours <br />5.456349e-5 weeks <br />1.25565e-5 months <br />, before the condition was corrected by the return of the Unit 1 EDG and EDGCWP to operable status.

The safety significance of this event was minimal since the Unit 1 EDGCWP and the 1/2 EDGCWP were available and capable of supporting ECCS Room cooling for Unit 1. Given the impact of the misaligned 1/2 EDGCWP on the operability of Unit 1 ECCS Room cooling for RHR, CS, HPCI systems, this report is submitted in accordance with the requirements of 10 CFR 50.73(a)(2)(i)(B), which requires the reporting of any operation or condition which was prohibited by the plant's Technical Specifications.

C. CAUSE OF EVENT

This Root Cause investigation reviewed the actions surrounding this event and identified that the Operators in this event failed to rigorously apply all of their human performance tools and did not consistently follow the Exelon policies, procedures and expectations. It was determined that the actions of these Operators were based on overconfidence and complacency, and as a result, an inaccurate risk perception developed. Rigorous execution of proper Human Performance (HU) Tools, including Procedure Use and Adherence standards, by these Operations personnel declined to the extent that a behavioral shift occurred such that lower standards were exhibited. This event resulted from a slow but steady decline of sustained performance by some Operators that was seen as low level issues and not adequately addressed prior to this event.

In this event, both EOs lost focus and rigor when they operated with substandard use of proper HU tools. The actions taken by EO1 to ensure the proper component was being manipulated were done irrespective of the potential risk of being on the wrong unit. After the improper determination was made, EO1 continued to operate only on assumptions based on which flow gauges were associated with each pump/valve. When E02 returned to EO1 for flow rate verification, neither EO verified that the gauges where the flow rates were being taken from were correct compared to the controlling document.

A contributing cause identified in this event was an inadequate pre-job brief that was performed with E02 prior to EO1 and E02 lining up the 1/2 EDGCWP to the ECCS Room Coolers. The briefs did not adequately prepare the EOs for this task. In this event, the potential configuration control precursor of verifying all three Unit designations (01, 02, 1/2) during the task did not drive a consideration for additional forms of verification (e.g., first check, peer check) in the pre-job brief for the EOs.

D. SAFETY ANALYSIS

System Design The EDGCW System is designed to provide cooling water for the removal of heat from the emergency diesel generator (EDG) heat exchangers and the ECCS room coolers. Each unit EDGCW subsystem provides cooling water to its associated EDG and the unit ECCS room coolers. The 1/2 EDGCW subsystem may be manually aligned to provide cooling to either unit's ECCS room coolers.

The EDGCW pump autostarts upon receipt of a EDG start signal when power is available to the pump's electrical bus.

Cooling water is pumped from the suction header of the Residual Heat Removal Service Water (RHRSW) [BI] System by the EDGCW pump to the associated EDG heat exchangers. After removing heat from the heat exchangers, the water is discharged to the station discharge flume. The EDGCW subsystem associated with EDG 1 (EDG 2) is also normally aligned to provide cooling water to the unit ECCS room coolers. However, the EDGCW subsystem associated with EDG 1/2 can be aligned as an alternate source of cooling water to the Unit 1 or Unit 2 ECCS room coolers. The EDGCW subsystem associated with EDG 1 can be aligned as an alternate source of cooling water to the EDG 1/2 heat exchanger. Updated Final Safety Analysis Report (UFSAR) Section 9.5.5 provides additional details.

Safety Impact During restoration of the 1B Division II CS logic test, the lineup for the 1/2 EDGCWP was to be restored to normal. It was discovered that the 1/2 EDGCWP was lined up to Unit 2 instead of the required line up to Unit 1, since the 1/2 EDG was operable to Unit 1 and the Unit 1 EDGCWP was lined up to Unit 1 ECCS Room coolers. For Unit 2 the 1/2 EDGCWP was lined up to Unit 2 ECCS room coolers therefore Unit 2 ECCS room coolers would be supplied by either the Unit 2 or Unit 1/2 EDGCWP. At the time of discovery all required systems were operable. Prior to discovery, EDG and Unit 1 EDGCWP were available via Operator manual action, and the 1/2 EDGCWP was available and its isolation valve to Unit 1 ECCS room cooling (1/2-3999-89) was also available to be opened via Operator manual action had an ECCS initiation occurred.

QCOS 1400-16, Unit 1 Division II Core Spray Logic Functional Test, states: "This procedure will test 1B Core Spray System, Unit 1 Diesel start, System II Core Spray initiation, manual override of MO 1-1402-25B valve interlocks, loss of system logic power, and auxiliary trips.

During the functional test:

• The 1B CS loop and the Unit 1 EDG are both inoperable.

• The 1/2 EDGCWP is aligned to Unit 1 and will start automatically on a valid initiation signal. [Note, however, the 1/2 EDGCWP was not properly aligned per procedure requirements, and instead had its 1/2-3999-89 ECCS room cooler supply valve closed.]

• The Unit 1 EDGCWP is also aligned to Unit 1 and will not start automatically on a valid initiation signal.

• The Unit 1 EDGCWP can be started manually.

• There was no simultaneous defective equipment condition.

For the situation in which the 1/2 EDGCWP was not properly aligned per the procedure requirements, and had a valid ECCS initiation signal been received, the situation would have been readily observed by the Control Room soon after the initiation signal. At this point: 1) the Unit 1 EDG CWP would have been quickly manually started to supply the ECCS corner room coolers (note, the Unit 1 EDGCWP has no common ECCS room cooler isolation valve), and 2) the 1/2 EDGCWP would have auto-started, and its ECCS room cooler common supply valve (Y2-3999-89) would have been verified/manually realigned if required to its open position during Operator verifications, and during high ECCS pump room area temperature alarm annunciations, with resulting subsequent manual actions required by abnormal operating procedures. Operating procedures and Operator training exist to ensure proper actions are taken during a valid ECCS initiation signal.

According to ECCS pump room area temperature heat up calculations, the worst case room heatup rate scenario with no operator action is the CS pump room. The CS room will reach 150 degrees F in approximately six (6) hours, and the pump will reach its unavailable temperature of 185 degrees F in 21 hours2.430556e-4 days <br />0.00583 hours <br />3.472222e-5 weeks <br />7.9905e-6 months <br />. Operator awareness of abnormal room temperatures is provided via the annunciator response procedure QCAN 901-3 D-3 (Core Spray Pump Area High Temperature) when the QCAN alarm setpoint of 150 degrees F is reached. This QCAN results in requiring operator action to check alignment of the ECCS room cooler common supply valve (1/2-3999-.89). Since the 185 degrees F unacceptable room temperature causing unavailable equipment is not reached until approximately 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> after the QCAN alarm, operator actions would occur in reasonable time prior to impacting equipment unavailability.

Although the required 1/2 EDGCWP function to supply Unit 1 ECCS room cooling was not operable for a short period of time during this event, this did not create any actual plant or safety consequences since the Unit was not in an accident or transient condition requiring use of ECCS room cooling during this period of time.

Risk Insights Both the Unit 1 EDGCWP and the 1/2 EDGCWP were available, since operator response is to verify and ensure proper operation and alignment in the case of an initiation. Proceduralized operator actions can be credited in the plant Probabilistic Risk Assessment (PRA) to maintain availability of equipment, therefore utilizing the plant PRA, risk is unaffected since there was no resulting unavailable equipment (i.e., Unit 1 EDGCWP and 1/2 EDGCWP were maintained available). As a result, there is no increase in risk (change in Core Damage Frequency (CDF)) due to the procedure alignment error of the 1/2 EDGCWP during the performance of QCOS 1400-16.

In conclusion, the overall safety significance and impact on risk of this event is negligible.

E. CORRECTIVE ACTIONS

Immediate:

1. Additional oversight of briefs was provided to ensure verification techniques are adequately covered.

2. During the first 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> of Spring 2013 refuel outage, Q1R22, each EO and Nuclear Station Operator (NSO) were observed by an Operations Supervisor and the observation was documented.

Follow-up:

1. Implement the process of focused intervention directed at the crew level based on Employee Observation System (EOS) trending and analysis data. The process will utilize the trend analysis tool within the new EOS program that establishes alerts and notices for predetermined set points based on departmental requests.

2. Document performance of the HU OBEs to verify that the EOs can demonstrate acceptable levels of HU tool use.

3. The Shift Managers will perform and document a review of the requirements of HU-AA-101, Human Performance Tools and Verification Practices procedure, with their employees and reinforce the need for rigorous use of HU tools in the field.

4. The Shift Managers will perform and document a review of the requirements of HU-AA-1211, Pre-Job Briefings procedure, with employees to reinforce that all pre-job briefs need to include a discussion of all first check opportunities for every task.

5. Issue a Standing Order: For the remainder of 2nd Quarter of 2013 through the 3rd Quarter of 2013, a copy of the Human Performance Tools and Verifications Practices Procedure will be referenced at all pre-job briefs with employee's explaining what HU tools they will be using in the field and how they will be used. The Brief Leader will ask the brief participants to verbalize one (or more) of the HU tools during the brief (i.e., what it will look like in the field).

6. Issue a Standing Order: For the remainder of 2nd Quarter of 2013 through the 3rd Quarter of 2013, all pre-job briefs will include a discussion of all first check opportunities for every task.

7. Conduct 2nd thru 4th quarter HU OBE's "in the plant" for Operations First Line Supervisors, Reactor Operators, and EOs (i.e., verifications, peer checks, self-checks, and communication) or Verbal Assessments.

8. Implement changes to QCOP 6600-15, 1/2 Diesel Generator Cooling Water Pump Cross Connect Alignment procedure, to add verification steps needed for any equipment manipulations.

F. PREVIOUS OCCURRENCES

The station events database, LERs, and INPO Consolidated Event System ICES (EPIX) were reviewed for similar events at Quad Cities. Specifically, this event was primarily attributed to improper use of the Human Performance tools. Based on the causes of this event and associated corrective actions, the events listed below, although similar in topic, are not considered significant station experiences that would have directly contributed to preventing this event.

• Station Issue Report (IR) 1295770, Level Three Clearance and Tagging Error Unit One Refuel Bridge, Quad Cities, November 29, 2011 - A Clearance Order was hung for work on the Unit One Refuel Bridge. One of the clearance order danger cards was hung on a telephone cord that was mistaken for the main power cord to the bridge. Apparent Cause - E02 failed to remain an "independent" verifier of the Out of Service. The HU issues of this LER event did not result from lack of "independent" verification since it was not required. Therefore this tagging error event is not directly applicable and is not considered a significant station experience that would have directly contributed to preventing the event of this current LER. .

• Station Issue Report IR 1345302, Closure of a Switchyard Disconnect, Quad Cities Station, March 24, 2012 - An Operator closed a switchyard disconnect, which connected a 345 kV transmission line (live at the time) to a hard ground via attached grounding straps intended to form a zone of protection for work that had recently been completed on another disconnect. This caused a severe voltage depression resulting various trips and actuations. Root Cause -The brief to perform the boundary swap did not provide adequate guidance for safe execution of the evolution. The switchyard event was primarily caused by an inadequate pre-job brief, and lack of oversight, which resulted in corrective actions primarily focused on briefings and following organizational interfaces directed by the written procedure. The HU issues of this LER event did not result from inadequate pre- job briefs or lack of oversight, but resulted from the individual worker's lack of execution of proper HU tools.

Therefore this switchyard event is not directly applicable and is not considered a station experience that would have directly contributed to preventing the event of this current LER.

• LERs - A review of LERs at Quad Cities Nuclear Power Station did not identify any events that were associated with this type of event.

G. COMPONENT FAILURE DATA

There was no component failure.

This event has been reported to ICES.