05000387/LER-2010-002
| Susquehanna Steam Electric Station Unit 1 | |
| Event date: | 04-22-2010 |
|---|---|
| Report date: | 02-01-2012 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(iv)(A), System Actuation |
| Initial Reporting | |
| ENS 45866 | 10 CFR 50.72(b)(2)(iv)(B), RPS System Actuation, 10 CFR 50.72(b)(3)(iv)(A), System Actuation |
| 3872010002R02 - NRC Website | |
CONDITION PRIOR TO THE EVENTS
Event 1: April 22, 2010, Unit 1 - Mode 1, 32 percent Rated Thermal Power Event 2: May 14, 2010, Unit 1 - Mode 1, 94 percent Rated Thermal Power
EVENT DESCRIPTIONS
Event 1 - On April 22, 2010, at 1051 hours0.0122 days <br />0.292 hours <br />0.00174 weeks <br />3.999055e-4 months <br />, Susquehanna Steam Electric Station (SSES) Unit 1 experienced an automatic reactor [EllS Code: AC] scram on low reactor water level (+ 13 inches). This scram occurred while implementing planned refueling outage. During the outage, upgrades to the feedwater level control system, the reactor feed pump turbine speed controls, and the reactor recirculation speed controls were made by the installation of ICS. During testing at low power conditions (i.e., 32% power), the second of three (A, B and C) reactor feed pumps (RFP) [EllS Code: SJ] was placed in automatic flow control mode for the first time with the goal of parallel automatic operation of two RFPs. A reactor water level transient occurred when the second RFP began feeding into the reactor. Reactor water level reached a maximum of approximately 12 inches above normal operating level. As a result, the control room operator placed the second (i.e., the oncoming) RFP speed controller in manual mode and reduced the rate of feed into the reactor vessel in accordance with procedures. Concurrent with this action, the ICS responded to the high level condition by lowering the other operating RFP's speed. The resulting concurrent flow reduction of both RFPs quickly lowered water level to the low water level scram setpoint. A control room operator was in the process of taking the mode switch to shutdown when an automatic reactor scram signal initiated on low reactor water level.
The Reactor Protection System (RPS) [EllS Code: JC] response to the automatic scram was as expected and all control rods fully inserted. Following the reactor scram, reactor water level dropped to approximately -30 inches wide range. The Reactor Core Isolation Cooling (RCIC) [EllS Code: BN] system automatically initiated as expected. The 'A' RFP and RCIC restored level to within normal limits. An Alternate Rod Insertion Div 2 signal was received during the transient and was subsequently reset. Primary containment integrity was maintained during the event. An ENS notification (# 45866) was made to the NRC in accordance with 10 CFR 50.72(b)(2)(iv)(B) for an event or condition that resulted in the actuation of the RPS when the reactor was critical, and 10 CFR 50.72(b)(3)(iv)(A) due to a valid actuation of the RPS and RCIC.
Event 2 - On May 14, 2010, at 2301 hours0.0266 days <br />0.639 hours <br />0.0038 weeks <br />8.755305e-4 months <br />, SSES Unit 1 automatically scrammed from 66% power due to a main turbine trip on high reactor water level (+ 54 inches). As expected, all three RFPs tripped on high reactor water level. The high water level condition occurred during additional planned testing of the new digital ICS which involved a planned trip of one of the four condensate pumps. This test was performed as required by an Extended Power Uprate (EPU) license condition, to verify that a loss of all feedwater would not occur due to low feedwater pump suction pressure. As designed, this resulted in an automatic reactor recirculation [El IS Code: AD] pump run-back and a subsequent reactor water level transient. A larger than expected steam flow/feed flow mismatch developed due to insufficient gain on the master feedwater level controller (MFWLC), which resulted in steam flow dropping off at a much greater rate than feedwater flow. As a result, an aggregate vessel level rate of increase of 30 inches per minute was experienced.
When the reactor water level exceeded the expected upper response range of the test, the operator placed the mode switch in shutdown. An automatic reactor scram signal initiated on a turbine trip due to high reactor water level occurring nearly simultaneously as the control room operators placed the mode switch in shutdown.
The RPS response to the automatic scram was as expected and all control rods fully inserted. Following the scram, reactor water level lowered to -30 inches wide range. RCIC was manually initiated at -24 inches and injected for level control. In accordance with plant procedures, the control room operator restored the 'B' reactor feed pump and RCIC was secured. Reactor water level was restored to within normal limits. Primary containment integrity was maintained during the event. An ENS notification (# 45930) was made to the NRC in accordance with 10 CFR 50.72(b)(2)(iv)(B) for an event or condition that resulted in the actuation of the RPS when the reactor was critical, and 10 CFR 50.72(b)(3)(iv)(A) due to a valid actuation of the RPS and RCIC.
In accordance with NUREG-1022, Rev. 2, Section 2.3, Reporting of Multiple Events, more than one failure or event may be reported in a single Licensee Event Report (LER) if; 1) the failures or events are related (i.e., they have the same general cause or consequences), and 2) they occurred during a single activity (e.g., a test program) over a reasonably short time (e.g., 60 days LER reporting). Since both reactor scrams occurred during implementation of the ICS testing program and the events are related, LER 2010-002-00 was submitted on June 21, 2010, for both events in accordance with 10 CFR 50.73(a)(2)(iv)(A) for an event that resulted in the manual or automatic actuation of the RPS and RCIC.
CAUSE(S) OF THE EVENT
PPL completed a root cause investigation shortly after the above events occurred. This root cause evaluation determined the following:
Event 1 — Performance characteristics of the mechanical equipment associated with the RFP turbine steam admission system were not well understood. As a result, the ICS system gains were inadequate for the low power and steam flow conditions that existed at the time of the test. In particular, there existed an unrecognized RFP turbine transition from low pressure (LP) to high pressure (HP) and HP to LP steam supply that decreases the response of the RFPs when operating at low power conditions with an increasing speed demand signal. It is believed that this non-responsiveness of the feewater pumps was due to the lack of significant change in steam flow (i.e., motive force) to the feedwater pump turbine while the governor control valve position was moving through this transition zone. Because of this, manual control of the RFP turbines during this water level transient was ineffective in preventing the low level condition. As such, the initial gain settings in ICS did not sufficiently account for system performance characteristics at low power conditions. While the transition zone still exists, the speed controller gain has been increased to make the speed control governor control valve open or close more (i.e., admit more steam or less steam) for a given demand within this transition zone.
In addition, the plant simulator, which was used to confirm these gain settings prior to performing the test, did not accurately model plant performance. As a result, the misunderstanding of the performance characteristics of the mechanical equipment was not identified prior to testing.
Event 2 — The ICS MFWLC was not originally configured with sufficient gain to handle a large transient such as the loss of a Condensate Pump. This conclusion is based on review of event data that showed that the MFWLC demand to the 'A', 'B' and 'C' RFP turbine speed controllers did not decrease sufficiently to terminate the reactor vessel level increase following the condensate pump trip.
The MFWLC demand did not decrease sufficiently to mitigate the transient because the original ICS gain parameters resulted in a relatively low flow gain. The original flow gain was selected during system tuning and implemented at lower power levels based on acceptable response for small flow upset events. The selection of these values did not anticipate the need for larger gains during transients that have larger flow and level impacts.
The plant simulator was used prior to the condensate pump trip test to train the operators on expected plant response.
The simulator did not react as the plant did due to an unknown software error in the ICS controller function. Therefore, pre-site acceptance testing on the simulator and training for the Operators did not reveal the gain inadequacy prior to the actual performance of the Unit 1 condensate pump trip test.
The root cause for both the April 22, 2010 and May 14, 2010 Unit 1 scram events was due to less than adequate engineering rigor being applied during the development and implementation of the ICS gains/tuning factors as evidenced by:
- Failure of the plant simulator to accurately model the ICS master feedwater level controller function;
- Failure to use alternative methods (e.g., control system vendor simulator or other tools/models) to validate simulator changes prior to its use to predict actual plant performance; and
- Failure to test the installed feedwater control systems with sufficient rigor (i.e., a less than adequate incremental approach to testing was employed).
Both events occurred as a result of improper gain settings for the ICS reactor feed pump turbine speed and main feedwater level control functions. The first event was related to the performance characteristics of the mechanical equipment associated with the RFP turbine steam admission system that was not well understood. The second event was related to the inadequate gain settings for the ICS MFWLC to respond to a large steam/feedwater flow mismatch introduced during a single condensate pump trip test performed at 94% power. This conclusion is based on review of event data and showed that the MFWLC demand to the `A, '13', and 'C' RFP turbine speed controllers did not decrease sufficiently to offset the reactor vessel level increase following the condensate pump trip.
PPL subsequently determined that the original investigation did not comprehensively address the organizational, programmatic, and safety culture contributors to the event and established a root cause investigation team to supplement the original root cause evaluation. The root causes identified by the supplemental root cause evaluation include:
The root cause analysis determined the causes for the inadequate gain settings were due to the following:
- The process used to set the ICS Control Settings did not adequately use risk considerations, independent oversight, analytical techniques, and operating experience and resources were not adequately managed.
- The station management decision (in 2007) to use the plant simulator to establish the gain settings, and not to procure and use FSIM as an analytical tool, was not risk-informed and prevented its use to validate and identify the gain settings.
In addition, the root cause analysis determined there was a missed opportunity to identify the inadequate gain settings associated with the MFWLC system after the first scram and preclude the second event. The tuning parameters for the main feedwater level control system were not evaluated and the readiness for restart was not adequately verified.
The cause of this missed opportunity was determined to be:
- The station's post-event analysis of the April scram did not result in an adequate causal analysis to determine the cause of the scram. Cause techniques were not implemented and the analysis did not adequately evaluate the extent of condition or extent of cause.
ANALYSIS/SAFETY SIGNIFICANCE
The ICS is a digital Distributed Control System (DCS) that includes three major control systems that are integrated into a single DCS architecture known as ICS. The ICS installation was deemed necessary to support plant operation at EPU power levels due to the need for additional condensate and feedwater flow and corresponding increased reliability of the overall feedwater system. At uprated conditions, the operators will be making more speed changes in order to maximize generation and address atmospheric impacts. The ICS system includes enhanced reactor recirculation system runbacks, feedwater margin (rundown) capability, eliminates single point failures, and provides the benefits of an integrated control system on a common platform. The ICS modifications included 1) the replacement of the obsolete RFP Turbine Speed Bailey Control system for all three RFP turbines, Feed Water Level Bailey Control system, and Reactor Recirculation Pump Speed Bailey Control system, and 2) the replacement of the manual control switches, indicators, and hard-wired controls on the control room panels with touch-screen digital controls and displays, for added system monitoring and stability, and enhanced user-interface and controls.
Actual Consequences Event 1 - Following the reactor scram all control rods fully inserted. RPS performance was as expected. Reactor water level lowered to -30 inches wide range following the scram signal due to void collapse in the core. RCIC automatically initiated and injected. RCIC and the 'A' RFP restored level to within normal limits. RCIC response to the automatic initiation was as expected.
The balance of plant response was not typical due to loss of one of the two 13.8 kV auxiliary buses caused by planned maintenance on one of the offsite feeder breakers. Auxiliary Bus 11A automatically transferred to an offsite source as expected. However, the offsite source to Auxiliary Bus 11B was unavailable and as a result this bus de-energized.
This resulted in the loss of several large loads, including the 'B' Condensate Pump, 'B' Reactor Recirculation Pump, 'B' Circulating Water Pump, and the 'B' Service Water Pump. Consequently, the 'C' and 'B' RFPs tripped due to low suction pressure. After the trip of the second RFP (i.e., 'B'), suction pressure recovered such that the 'A' RFP remained in operation.
Due to loss of Auxiliary Bus 11B, six non-essential load centers lost power and were subsequently manually cross-tied to restore power to various load and motor control centers. Power to Auxiliary Bus 11B was subsequently restored at 1217 hours0.0141 days <br />0.338 hours <br />0.00201 weeks <br />4.630685e-4 months <br />.
The main steam isolation valves (MSIVs) remained open during the transient and pressure was controlled via the main turbine bypass valves. No safety relief valve actuations occurred during this event. Primary containment integrity was maintained throughout the event. There were no diesel generator starts. The event was bounded by transients analyzed in Chapter 15 of the SSES Final Safety Analysis Report.
Event 2 - Following the reactor scram all control rods fully inserted. RPS performance was as expected. Reactor water level lowered to -30 inches wide range following the scram signal due to void collapse in the core. RCIC was manually initiated at -24 inches and injected for level control. The 'B' RFP was restored and RCIC was secured. Level was restored to within normal limits. The lowest reactor water level reached was approximately -30 inches on wide range.
The MSIVs remained open during the transient and pressure was controlled via the main turbine bypass valves.
There were no safety relief valve actuations during the event. Primary containment integrity was maintained throughout the event. There were no diesel generator starts. The event is bounded by transients analyzed in Chapter 15 of the SSES Final Safety Analysis Report.
Based upon the above discussion, the actual consequences of these events were minimal. There was no impact to the health and safety of the public.
Potential Consequences None.
- Immediate corrective actions were taken to adjust the ICS speed controller gain and master feedwater level controller gain to compensate for the RFP non-responsiveness at low power conditions. The appropriate procedures were revised to reflect these changes.
Event 2 -
- The ICS design incorporates Flow Control, Level control (e.g., feed forward / cascade control), adaptive tuning techniques (e.g., different gains at various power levels), and "gap" control (i.e., higher / lower gains within a defined band) as well as dynamic system features that are intended to maximize reactor water level control stability. The corrective actions taken as a result of this event provided increased system gains for large transients on the ICS Level Controller and the Flow Controller.
- Adjustments were made that maintained the existing gains for small transients while increasing the gain for large transients. These changes provide a more aggressive control action response during large transients and keeps the current "as tuned" control response for smaller transients (i.e., normal expected operational occurrences).
Subsequent testing of ICS for both small and large transients, including the successful Unit 1 condensate pump trip test on May 30, 2010, indicated that the corrective actions were appropriate.
Events 1 and 2 -
- Established guidance for developing, controlling, and revising control system tuning parameters.
- Performed an independent review of FSIM software to verify that it correctly modeled the ICS plant response.
- Established written policy for Risk Based Decision Making and identify the attributes of risk based decisions and the need to appropriately document the decisions. The policy includes a requirement for periodic reinforcement communications for sustainability.
- Revised applicable procedures to include risk-based decision making for restarts and continued operation after transients using the attributes from the new risk-based decision making policy.
- Issued a Post Event Analysis procedure that includes the principles of the problem identification and resolution process (i.e., approved analytical method).
- Established guidance for the use of the plant simulator for non-training purposes. This procedure considered the use of alternative methods (e.g., control system vendor simulator or other tools/models) to validate plant simulator changes prior to use of the simulator to predict actual plant response.