05000323/LER-2009-003
| Docket Number | |
| Event date: | 10-23-2009 |
|---|---|
| Report date: | 12-23-2009 |
| Initial Reporting | |
| ENS 45461 | 10 CFR 50.72(b)(3)(ii)(B), Unanalyzed Condition |
| 3232009003R00 - NRC Website | |
I.�Plant Conditions Unit 2 was in Mode 6 (Refueling) with the reactor vessel defueled.
Unit 1 was in Mode 1 (Power Operation) at approximately 100 percent reactor power at nominal full power operating temperature and pressure.
Description of Problem
A.�Background The Diablo Canyon Power Plants (DCPP) Units 1 and 2 are Pressurized Water Reactors (PWR) with four Reactor Coolant Loops (RCL)[AB] to circulate reactor coolant to each of the four steam generators (SG)[SG].
The Emergency Core Cooling System (ECCS)[BQ] is depicted below.
Recirc Volume Control 8105 TankAccumulators
- RefuelingWaterRegen HX4-151—(51-- 8106 112B 2i01111 111= MIN 8107 8108 A
- s;cgr HCV-142 8808A 8808B 8808C�8808D Injection41--- 1120
- ,
- ,
- ,
- Seal�
- 1 1FCV-128Gx 8805A8801A cL,}7B8803A CCP 1 -11 CCP 2 ♦ 88058Cold 8801B�8803BLegs 9003A
- 8804ASpray 641AHCV-6 89808700A� 1
- 8726B 2 Loop 4Hot RV-8707 Leg 8809B�HCV-637 0:1�Recirc8700B641B To PRT Spray
- �
- 9003B
- 8982B ris 8982A;8703 DI8 8 0 4 B Containment AID Recirc SumpEl 4�I RCS m 8802B r yc:
- 88218 -13�VV Hot 8923B Legs SI Pumps ra 8835
- 8821A 1 V
- 8802A 8923A Emergency Core Cooling System (ECCS) Flow Paths The ECCS is designed to cool the reactor core as well as provide additional reactor shutdown capability by injection of borated water following initiation of accident conditions for:
4 LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) 1. A pipe break or spurious valve lifting in the Reactor Coolant System (RCS) that causes a discharge larger than that which can be made up by the normal makeup system, up to and including the circumferential rupture of the largest pipe in the RCS.
2., Rupture of a control rod drive mechanism (CRDM) causing a rod cluster control assembly (RCCA) ejection accident.
3. A pipe break or spurious valve lifting in the steam system, up to and including the instantaneous circumferential rupture of the largest pipe in the steam system.
4. A steam generator- tube rupture.
The residual heat removal (RHR)[BP] function, including the long term recirculation from the containment sump, was not adversely affected by this condition. Also, containment spray [BE], high head [BQ] and intermediate head [BP] safety injection from the refueling water storage tank (RWST) was not adversely affected by this condition.
However, in the event of a Loss of Coolant Accident, (LOCA) this condition would have prevented RHR flow from reaching containment spray, high head and intermediate head safety injection pumps following realignment to long term recirculation from the containment sump without additional operator action. The realignment to the long term recirculation is required to provide continued core cooling after the RWST is exhausted.
To accomplish this licensed plant operators at the remote manual control switches in the control room perform the flow path realignment in accordance with the Emergency Operating Procedures (EOP) as described in Final Safety Analysis Report (FSAR) Update, Section 6.3.1.4.4.2, "Changeover from Injection Mode to Recirculation After Loss of Primary Coolant," and Table 6.3-5, "Safety Injection to Recirculation Mode: Sequence and Timing of Manual Changeover." Technical Specification 3.5.2, "ECCS — Operating," Bases states, in part:
"During the recirculation phase of LOCA recovery, RHR pump suction is transferred to the containment recirculation'sump. The RHR pumps then supply the other ECCS pumps. Initially, recirculation discharge is through the same paths as the, injection phase to the cold legs.
Subsequently, recirculation provides injection to both the hot and cold legs." And "The ECCS suction is manually transferred to the LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) containment recirculation sump to place the system in the recirculation mode of operation to supply its flow to the RCS hot and cold legs.
During the recirculation operation, the RHR pumps provide suction to the charging and SI pumps.
The containment recirculation sump suction valves 8982A and 8982B, utilize a geared limit switch that has multiple rotors, contacts, and gear boxes. Those rotors are driven directly by gearing that drives the valve stem, and can be set to operate independently at any specific number of turns of the actuator gear train. The specific number of turns corresponds to a specific valve position. The limit switch contacts, which are actuated by the rotors, can be electrically connected to other valves so that the valves can be opened and closed depending on the specific position of the rotors.
The open limit function for SI-2-8982A and SI-2-8982B is controlled by Rotor 1. The interlock function for SI-2-8804A, SI-2-8804B, CS-2-9003A and CS-2-9003B is controlled by Rotor 3 in the 8982A and B limit switch.
For proper function of the interlock with 8804A and B and 9003A and B, Rotor 3 must be set to actuate with or before Rotor 1.
Maintenance Procedure (MP) E-53.10V1 allows switch adjustments without specifying rotor coordination because it relies on the skill of the craft to perform switch adjustment and rotor coordination. That was the result of a legacy issue from Revision 0 of MP E-53.10V1. MP E-53.10V1 and its predecessors had always contained this potential deficiency.
B.� Event Description During July 2005 DCPP received INPO Operating Event, 0E20893 (Catawba small break LOCA), for evaluation. DCPP Engineering Calculation STA-220 determined that the resulting Ap for SI-2-8982A and B could be much greater than specified in the original design. Therefore, Action Request (AR) A0643107, "Evaluate 0E20893, Possible Concerns with Delta-P in SBLOCA", was entered into the DCPP CAP.
On November 22, 2005, a preliminary Engineering Calculation V-07 for a revised gear train identified a stroke time of 24.3 seconds that increased the valve Ap design margin to 20 percent. The increased stroke time reduced design margin but remained within the 25 second limit. The 24.3 second stroke time met the design requirement of 35 seconds and was therefore considered an acceptable modification.
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) The valves were modified on Unit 2 in refueling outage 2R14 during February 2006. The modified valves were found to take greater than 25 seconds to stroke. The valve stroke length was subsequently adjusted to assure the 25 second stroke requirement was met. Adjustment of valve position switches was not performed following adjustment of the stroke length, leaving the system configured such that the permissive interlocks to stroke valves SI-2-8804A, SI-2-8804B, CS-2-9003A and CS-2-9003B would not function. No testing of the interlock functions was performed following the stroke length adjustment. As a result, operators had no ability to operate these valves from the control room between 2R14 and 2R15.
The same problem did not occur during modification in refueling outage 1R15 because the unit 1 valve position switches were adjusted following stroke length adjustment.
On February 8, 2006, AR A0659468, "Evaluate V-7B Test Optimization to Reduce Draindown Window," was initiated. This September 13, 2006, entry states, in part:
"PEP V-7B will now perform the RHR valve interlocks. This test will be sectionalized to allow partial interlock testing. This provides flexibility in this procedure in that the whole test need not be performed, this will help reduce the outage hours.
On July 26, 2006, the Safety Injection (SI) System Engineer requested a design change for new gear ratio in SI-2-8982A and B. AR A0674066, "SI-2-8982A/B motor operated valve (MOV) gear change to improve design margin," was initiated.
On November 9, 2006, SI System Engineer requested a design change for a new gear ratio in SI-1-8982A/B (same actuator design). AR A0682070, "SI-1-8982A/B MOV gear change to improve design margin (Unit 1)," was entered into the DCPP corrective action program (CAP).
On April 19, 2007, Plant Engineering Procedure (PEP) V-7B, Revision 0 was issued and used in 1R14 (04/30/07 —05/29/07).
On August 20, 2007, AT MM A0702739 Revision A, "Design Change to improve design margin of SI-2-8982A/B (U2)," concluded that SI-2-8982A and B stroke time would be less than 25 seconds.
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1)
- DOCKET NUMBER (2) LER NUMBER (6) On August 23, 2007, AT MM AR A0702739 was approved for SI-2-8982A and B gear change anticipating less than 25 second stroke time.
On August 23, 2007, AT MM AR A0702739, Action. Evaluation (AE) 15 specified Surveillance Test Procedure (STP) V-3L7 (Stroke Time Test) and Viper Static Test as the required Post ModifiCation Test (PMT).
On August 30, 2007, the maintenance planner initiated a PMT request to the PMT Coordinator.
On September 19, 2007, Valve Sizing Calculation V-07 for SI-1/2-8982A and B was approved.
On October 16, 2007, AR A0702739, AE 17 specified a Design Verification Test (DVT) consisting of the STP V-3 valve timing test for SI-2-8982A and B.
On October 18, 2007, work orders (W0s) CO214725 and CO214751 were updated to identify PEP V-7B as a contingency. PMT.
On November 29, 2007, AT MM AR A0707169, "Design Change to improve design margin of SI-1-8982A/B (U1)," was approved anticipating a less than 25 second valve stroke time.
On February 16, 2008, SI-2-8982A and B Viper PMT stroke time was found to be greater than 25 seconds (WOs CO214725 & CO214751).
On February 16, 2008, Engineering revised the open limit (Rotor 1) setting specification to eliminate travel beyond what is necessary to fully open seat port.
On February 16, 2008, MOV Engineer E-mail on 2R14 MOV Dayshift Turnover, which stated, in part, "... We do not have to do the STP V-7B interlock test...
On February 16, 2008, AR A0674066, AE 06, "Documentation of new open limit setpoint basis," was taken to "COMPLT.
On February 17, 2008, SI-2-8982B open limit switch (Rotor 1) was reset using MP E-53.10V1 (CO214751-02). The As-left Viper test and Instrument and Controls Evaluation (ICE)-12 evaluation was noted as satisfactory (CO214751-03).
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) On February 18, 2008, SI-2-8982A open limit switch (Rotor 1) was reset using MP E-53.10V1 (CO214725-02). The As-left Viper test and ICE-12 evaluation was noted as satisfactory (SAT per CO214725-03).
On February 19, 2008, an E-mail from the MOV Subject Matter Expert (SME) to the Test Team MOV Engineering Team states that the interlock limit switches have not been adjusted for SI-2-8982A and B.
On February 21, 2008, the Engineering PMT Coordinator rejects PEP V-7B for SI-2-8982A and B (PMT module for CO214725 & CO214751). Therefore, SI-2-8982A and B were inadvertently returned to service with the interlocks disabled., On March 6, 2008, operation of the interlock between the stem mounted position switch on SI-2-8982A and B and RHR Pumps 2-1 & 2-2 was verified satisfactory per STP P-RHR-ST.
On April 8, 2008, Unit 2 enters Mode 4 with 8982A and 8982B containment recirculation interlocks for 8804A, 8804B, 9003A and 9003B inoperable, a violation of TS 3.5.2, "ECCS — Operating.
On October 23, 2009, with Unit 2 in Mode 6 (Refueling) and the reactor core offloaded to the spent fuel pool, plant operators identified failures of containment sump recirculation valves to activate their open interlock.
On October 23, 2009, at 14:44 PDT licensed plant operators notified the NRC via the Emergency Notification System (ENS) of the condition in accordance with 10 CFR 50.72(b)(3)(ii)(B) as an unanalyzed condition.
Status of Inoperable Structures, Systems, or Components that Contributed to the Event Containment sump recirculation valves 8982A and 8982B were degraded in that the valve open position interlock switches would not activate their open interlock, and thus would have prevented RHR flow from reaching the containment spray rings, high head and intermediate head safety injection pumps following alignment to long term recirculation from the containment sump without additional operator action from outside the control room.
D.�Other Systems or Secondary Functions Affected No additional safety systems were adversely affected by this event.
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) PAGE (3) E. Method of Discovery During scheduled surveillance testing performed during 2R15 the failure of the 8982A and 8982B interlocks was identified.
F. Operator Actions Plant operators were informed of the discovery and the details of the surveillance test failure documented in the DCPP CAP.
G. Safety System Responses None required.
III.�Cause of the Problem A.�Immediate Cause MP E-53.10V1, "MOV Diagnostic Testing with the Viper System," was used to set the stroke of valves 8982A and 8982B that did not contain adequate guidance regarding coordination of MOV limit switches.
No additional testing was performed, other than MP E-53.10V1.
B. Cause
1. A legacy issue from 1991 resulted in MP E-53.10V1, not including adequate guidance for rotor coordination if a limit switch is reset.
2. Maintenance Procedure MP E-53.10V1 and MP E-53.10S did not identify that performance of specific steps requires implementation of MMD M-000073-1. That resulted in MMD M-000073-1 not being properly implemented.
3. Engineering Calculation V-07 did not provide adequate precautions and limitations regarding the potential uncertainty in the final value when calculating stroke time. That resulted in the design change development group relying on the results of the calculation and, not considering alternatives to meet the 25 seconds stroke time specified in the licensing basis.
C.�Contributory Cause LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) 1. Conversion of STP V-7B, "Test of Engineered Safeguards, Valve Interlocks and RHR Pump Trip from RWST Level Channels," to PEP V-7B, "Test of ECCS Valve Interlocks," facilitated an organizational decision to not perform V-7B in its entirety every outage in order to reduce critical path time and/or outage duration.
2. There was no rigorous method of evaluating, documenting, and communicating information regarding work performed that would be needed to make decisions about conditional PMT. Informal and incomplete communication between the MOV engineer and the PMT coordinator led to the rejection of PEP V-7B as a post maintenance test.
3. A legacy issue from 1998 in Calculation V-07, Appendix K, resulted in narrative information not being formatted for ready retrieval or use.
That led to a human error by the Engineer having to remember the need to coordinate Rotor 3 with Rotor 1.
IV.�Assessment of Safety Consequences There were safety consequences as a result of this event.
The Unit 1 reactor was maintained in Mode 1, with TS-required equipment operable, as confirmed by a review of the successful surveillance testing performed during the Unit 1 fourteenth and fifteenth refueling outages.
Pacific Gas and Electric Company and the Nuclear Steam Supply System (NSSS) vendor, Westinghouse, performed a limited scope best estimate analysis of the effect of the described condition. From those analyses, it was determined that for a large break LOCA; no significant adverse affects occur as the RCS pressure decreases rapidly, the accumulators inject, and SI system operates to keep the core in a coolable geometry with the RHR pumps alone providing the required SI flow necessary. Therefore, with the availability of the RHR pumps and flow path throughout Cycle 15, there is reasonable assurance that there were no significant adverse consequences resulting from a postulated large break LOCA.
For the small break LOCA conditions a range of breaks were investigated that concluded for five (5) inch and smaller diameter piping breaks the containment spray pumps were not automatically actuated during the injection phase of the accident. Therefore, there is sufficient time from the RWST low level alarm and RHR pump trip point to effectively manually transfer the containment recirculation flow path to the reactor core, with greater than one hour of total time available for LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) remote manual and local manual valve operation. A review of the anticipated Unit 2 Cycle 15 radiological conditions at the local manual valve locations found no significant radiological "turn back" condition that would preclude successful operator actions.
Additionally, from the analysis, it was confirmed that for breaks larger than approximately five (5) inch diameter that the containment spray could be actuated, drawing the RWST inventory more quickly. However, the RCS pressure dropped significantly allowing the RHR alone to satisfy the long term recirculation cooling.
Therefore, this event did not adversely affect the health and safety of the public.
PG&E considers this event a Safety System Functional Failure (SSFF). Since it does not result in crossing an NRC PI threshold DCPP will include it with the next routine quarterly PI data submittal.
V.CCorrective Actions A.CImmediate Corrective Actions 1. Engineering confirmed that appropriate testing was performed for all ECCS interlocks prior to Mode 4 entry for Unit 2 following 2R15.
2. Engineering confirmed that appropriate tests were performed for all ECCS interlocks for the Unit 1 fifteenth refueling outage.
3. An operations summary of the event was published and given to all crews. Included in the summary was a discussion of the importance of knowing the locations of valves identified in the EOPs that might be required to be manually operated and the actuation of MOVs using the contactors at the Motor Control Centers (MCC).
B.CCorrective Actions to Prevent Recurrence (CAPR) 1. MP E-53.10V1 has been revised (11/12/09) to include guidance for limit switch setting and rotor coordination.
2. Revise MOV maintenance procedures to identify that performance of specific steps requires implementation of MMD M-000073-1.
3. Revise Calculation V-07 to specify limitations for the use of stroke time calculation in design applications.
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) PAGE (3) C.�Administrative Actions 1. Revise PEP V-7B to:
- Have a full performance of the test of every refueling outage,
- Include in the "Discussion" section a statement that the test is not only performed as PMT but also as a test to verify the design function of the interlocks even when no maintenance was performed on the specific tested components.
- Reference this event and order 60020753, and
- Reference AR A0551483, "Evaluate Deferral of STP V-7B Scope.
2. Revise AD8.DC58, "Outage Scope Control," and AD8.DC60, "Outage Schedule Preparation," to require that deletions of non-TS driven STPs or PEPs that have been routinely performed during multiple outages be collectively evaluated and reviewed by Engineering, Maintenance, Operations, and Outage Management; and the basis documented prior to deletion from an outage schedule:
3. Revise AD13.ID4, "Post Maintenance Testing," to require that any proposed PMT changes due to work scope change be documented in SAP, evaluated and independently reviewed by individuals with adequate technical knowledge, and approved by a supervisor or higher prior to the PMT being rejected.
4. Revise AD13.ID4 to include PEP V-7B as PMT for ECCS interlock MOVs.
5. Revise Engineering Calculation V-07, Appendix K, to include, specific reference to rotor coordination and to make narrative information easily usable.
VI.�Additional Information
A. Failed Components
None, this condition involved individual switch settings that were capable of functioning but were not properly coordinated to ensure operability.
B. Previous Similar Events
None were identified at DCPP.
C. Industry Reports
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6)
- TEXT 390-061026-1, "Licensee Event Report 390-06009 — Containment Spray Valve Interlock," reported an industry condition similar to this event.
The Watts Bar event was similar in that a mis-coordinated rotor resulted in a required function being inoperable. In the Watts Bar event, the function was containment spray during recirculation. The corrective actions at Watts Bar are similar to those taken for this event.
PG&E was not previously aware of this industry event was not previously identified because Watts Bar, INPO and NRC did not publish an operating experience report (e.g. OE, SEN, IN) regarding the issue.