05000247/LER-2017-001

LER-2017-001, Manual Reactor Trip Due to Decreasing Steam Generator Levels Caused By Main Boiler Feedwater Pump Turbine Low Pressure Governor Valves Failed Closed

Indian Point 2
Event date:	2-6-2017
Report date:	08-22-2017
Reporting criterion:	10 CFR 50.73(a)(2)(iv)(B), System Actuation 10 CFR 50.73(a)(2)(iv)(A), System Actuation
2472017001R00 - NRC Website

LER 17-001-00 for Indian Point, Unit 2 Regarding Manual Reactor Trip Due to Decreasing Steam Generator Levels Caused By Main Boiler Feedwater Pump Turbine Low Pressure Governor Valves Failed Closed

ML17240A165
Person / Time
Site:	Indian Point
Issue date:	08/22/2017
From:	Vitale A J Entergy Nuclear Northeast
To:	Document Control Desk, Office of Nuclear Reactor Regulation
References
NL-17-104 LER 17-001-00
Download: ML17240A165 (8)

comments regarding burden estimate to the Information Services Branch (T-2 F43), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by e-mail to used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

Note: The Energy Industry Identification System Codes are identified within the brackets { }.

DESCRIPTION OF EVENT

On June 26, 2017, at 1000 hours0.0116 days <br />0.278 hours <br />0.00165 weeks <br />3.805e-4 months <br />, Operations commenced a planned downpower from 100 percent to approximately 93 percent reactor power to support performance of the Indian Point Unit 2 (IP2) Main Turbine Stop and Control Valve {SB, ISV, FCV} Test, which is required to be performed every 6 months. In preparation for this planned special evolution, just-in-time training had been conducted for the operating crew, an infrequently performed test and evolution brief had been performed, and continuous manager oversight was established in the Central Control Room (CCR).

At 1037 hours0.012 days <br />0.288 hours <br />0.00171 weeks <br />3.945785e-4 months <br />, with reactor power at approximately 94 percent, the 22 Main Boiler Feed Pump Turbine (MBFPT) {SB, TRB} speed control trouble alarm {ALM} annunciated coincident with 22 Main Boiler Feed Pump (MBFP) {SJ, P} speed swings of approximately 800 revolutions per minute (rpm). The operators ceased the downpower and placed 22 MBFP in Manual CCR speed control per the main feedwater system {SJ} operating procedure (SOP) in an attempt to control the rpm swings. This was unsuccessful, and the rpm swings continued. Field operators were dispatched to the MBFP local control panel in accordance with the MBFPT speed control trouble alarm response procedure (ARP), and Operations entered the loss of main feedwater abnormal operating procedure (AOP) at 1041 hours0.012 days <br />0.289 hours <br />0.00172 weeks <br />3.961005e-4 months <br />. The field operators reported that the trouble alarm was due to high delta pressure {PDA} on both (A and B) control oil orifice {SL, OR} flow paths, with both alarms lit at the local panel. The field operators were briefed and then dispatched with an off-watch shift manager to clean the standby B control oil orifice filter (fine mesh screen canister) {FLT} and place it in service per the MBFP lube oil SOP.

The B orifice flowpath filter was cleaned and placed in service at 1144 hours0.0132 days <br />0.318 hours <br />0.00189 weeks <br />4.35292e-4 months <br />. Debris was visible on the filter.

Attempts to isolate and clean the now standby A orifice {SL, OR} flowpath filter {FLT} were unsuccessful, as the solenoid valve {FSV} that switches between the A and B flow paths would not fully isolate from the B flow path.

This led to the belief that particulate in the oil system was causing the 22 MBFP rpm swings, and a concern that the 21 MBFP {SJ, P} would be similarly affected because the oil system is common to both MBFPs.

The operators and engineers involved recognized that the 22 MBFP needed to be removed from service. Senior management was notified, and they concurred with the recommendation to downpower the unit to approximately 65 percent reactor power and remove the pump from service. The 22 MBFPT low pressure (LP) governor valves {SB, FCV} were cycling from full-closed to full-open as control oil pressure was cycling from 20 to 29 pounds per square inch gauge (psig). The LP steam supply to the turbine is controlled by eight governor valves that open and close sequentially. The LP governor valves should go full open at 40 psig control oil pressure. Attempts to lower the speed of the 22 MBFP led to larger rpm swings, which were upwards of 1000 rpm. In addition, the 22 MBFP rpm swings were causing the 21 MBFP suction flow to cycle from about 7500 gallons per minute (gpm) to 8000 gpm with the speed control in Auto, and this resulted in a 21 MBFP bearing monitor alarm {TA} on thrust bearing rear facing metal temperature, which was an added concern. Speed control for the 21 MBFP was left in Auto to dampen swings in the pump discharge pressure, as there was a concern that the excessive rpm swings would cause elevated pump discharge pressures without this modulation.

During the ongoing attempts to stabilize the 22 MBFP rpm swings, steam generator (SG) {AB, SG} water levels remained relatively stable. The operating crew was briefed on the potential loss of an MBFP and the potential for a unit trip. At approximately 1330 hours0.0154 days <br />0.369 hours <br />0.0022 weeks <br />5.06065e-4 months <br />, the outage control center (OCC) was staffed to organize efforts to assist in the downpower and remove the 22 MBFP from service. The unit was stabilized at 93 percent reactor power at this time. In an attempt to stabilize the cyclic speed oscillations, Operations transferred the 22 MBFP to - 001 comments regarding burden estimate to the Information Services Branch (T-2 F43), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by e-mail to used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

startup/governor control in accordance with the loss of main feedwater AOP and MBFPT speed control trouble ARP. This action placed the startup control signal in service and removed the feedwater signal, and was only partially effective. The 22 MBFP LP governor valves continued to cycle and any further attempts to reduce reactor power made the rpm swings worse.

At approximately1500 hours, the decision was made to take local pneumatic control of the 22 MBFP in accordance with the MBFPT speed control trouble ARP and main feedwater system SOP in an effort to further stabilize pump speed. Taking local pneumatic control allows for direct input of a steady air signal to the control oil system speed changer valves to control the turbine steam supply via the LP governor valves. After establishing local pneumatic control, LP governor valve operation initially stabilized, but about two minutes later the governor valves went to full closed. An automatic main turbine {TA} runback signal should have been generated on a low speed signal (less than 3300 rpm) when the 22 MBFP tripped (governor valves closed). However, there was no indication that the main turbine runback feature actuated. The turbine runback feature is designed to reduce main turbine load to approximately 76.5 percent. This allows secondary steam flow and feed flow to reach a new equilibrium state and, thus, decrease the probability of a reactor trip.

With the rapid reduction in speed following the 22 MBFP trip, the pump was no longer delivering feedwater flow to the SGs. In response, the CCR operators commenced a manual runback to reduce main turbine load as directed by the loss of main feedwater AOP. The operators closely monitored SG levels and established a predetermined level threshold of 15 percent to manually trip the reactor (7 percent above the automatic reactor trip setpoint). At 1531 hours0.0177 days <br />0.425 hours <br />0.00253 weeks <br />5.825455e-4 months <br />, the decreasing SG levels reached the 15 percent threshold and the CCR operators initiated a manual reactor trip. All control rods {AA} fully inserted and all required safety systems functioned properly. The plant was stabilized in hot standby with decay heat being removed by the main condenser {SG}. There was no radiation release. The emergency diesel generators {EK, DG} did not start, as offsite power remained available. The Auxiliary Feedwater System (AFWS) {BA} automatically started as expected due to SG low level as a result of void fraction (shrink) effect. As required, on June 26, 2017, at 1839 hours0.0213 days <br />0.511 hours <br />0.00304 weeks <br />6.997395e-4 months <br />, a 4-hour non-emergency notification was made to the NRC for an actuation of the Reactor Protection System {JC} while critical, and included an 8-hour notification for a valid actuation of the AFWS under 10 CFR 50.72(b)(3)(iv)(A) (Event Log No. 52829). This event notification also satisfied the 10 CFR 50.72(b)(2)(xi) notification requirement for Entergy's planned press release/news release regarding the IP2 reactor trip event.

During the initial investigation, when the housing cover for the LP governor valve servomotor (LP servomotor) was removed, the shoulder screws that secure the fulcrum bar to the piston shaft link pin (linkage) were found detached and laying in the bottom of the servomotor housing. The male threads on the shoulder screws and the female threads on the fulcrum bar were inspected, and both the male and female threads were found to be in good condition with minimal wear to the threads. There was sufficient thread engagement to allow the screws to be engaged back into the fulcrum bar with no looseness. Wear was found on the fulcrum bar at the mating surface of the fulcrum bar and piston shaft link pin.

Siemens, the original equipment manufacturer, was contacted to analyze the failure of the LP servomotor and assist in the servomotor repairs. Siemens Field Services personnel are contracted by Entergy during outages to perform overhaul work on major plant components, including the main turbine generator and MBFPTs. Entergy provides oversight of the work and Siemens personnel. Siemens performed an overhaul of the 22 MBFP LP governor valve servomotor during the IP2 Spring 2016 (2R22) refueling outage, which consisted of a complete servomotor disassembly, inspection, replacement of worn components, and reassembly. During the investigation to determine why the LP servomotor failed, it was identified that the piston shaft appeared to be very slightly twisted and not in its correct alignment. With the piston shaft misaligned, uneven forces were placed on the fulcrum bar, link pin, and shoulder screws, and this was evident by the wear found on the fulcrum bar. In addition, based on a Siemens/Westinghouse proprietary drawing of the LP servomotor, the threads for the shoulder screws - 001 comments regarding burden estimate to the Information Services Branch (T-2 F43), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by e-mail to used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

e Nor Indian Point 2 05000-247 2017 -001 - 00 should have been staked to securely retain the screws. Contrary to the drawing, Siemens personnel had not staked the screws during the 2R22 refueling outage overhaul of the LP servomotor, or in any previous outage overhauls. If the shoulder screws become detached from the LP servomotor linkage fulcrum bar and piston shaft link pin, the MBFP LP governor valves fail in the closed direction. This was the direct cause of the rpm swings and eventual loss of the 22 MBFP, and subsequent manual reactor trip that occurred on June 26, 2017.

As part of the cause investigation, the 2R22 LP servomotor overhaul Work Order (WO) was reviewed to determine why the shaft and linkage were left slightly twisted. While reviewing the WO, it was observed that a step text document was used in place of the Entergy MBFPT inspection procedure. The step text reduced the complete overhaul of the LP servomotor (i.e., visual inspection, disassembly, cleaning, replacement of worn parts, reassembly) down to only two steps. Siemens personnel are not accustomed to using Entergy procedures, so the step text document was created to eliminate the confusion. However, many of the notes, caution statements, and required measurements were not transferred from the Entergy MBFPT inspection procedure to the 2R22 LP servomotor overhaul WO step text document. Consequently, vital information was not included in the 2R22 WO.

Specifically, the caution statement regarding the use of excessive force or the twisting of mechanical linkages during assembly potentially causing equipment damage or malfunction was not transferred over to the 2R22 WO.

The omitted caution statement may have contributed to the condition identified during the inspection of the LP servomotor where the piston shaft and linkage appeared to be slightly twisted.

An investigation was conducted to determine the cause of the apparent failure of the main turbine runback feature when the 22 MBFP was tripped. There are two independent main turbine runback circuits, with each circuit controlling one load limit valve (LLV1 or LLV2) {TG, FCV}. The LLVs restrict steam to the main turbine in order to runback (lower) turbine load. The turbine runback function is armed when the runback permissive bistable (PC- 412B-1 or PC-412A-1) {RLY} within an LLV circuit is actuated (contact closes) on a loss of the MBFP (speed drops below 3300 rpm) coincident with turbine power above 76.5 percent, and the Arm/Defeat switch must be in the Armed position. Actuation of either of the two bistables will initiate a turbine runback via its respective LLV circuit.

As part of the functional checks performed during the investigation, the PC-412B-1 bistable for LLV1 was tested and found to be not functioning, and the PC-412A-1 bistable for LLV2 was found with an out of tolerance setting and needed adjustment. Both circuit bistables were replaced and the other circuit components were tested and verified to be functioning properly. Considering these results, it was concluded that LLV1 would not have performed its turbine runback function. It was indeterminate whether LLV2 would have functioned properly with the PC-412A-1 bistable as-found out of tolerance setting; however, the PC-412A-1 bistable was replaced to eliminate the vulnerability. Proper function of the alarm, time delay relays, and the arming of the circuit via the control switch were all verified. The tachometer relays for both circuits were functionally tested and verified to be operating properly. Additionally, the drive motors for both LLVs were checked and the results were compared against the design for number of turns to stroke and stroke speed. The results were verified to be in agreement with the intended design. Thus, all of the turbine runback circuit components were functionally tested and verified to be operating as designed following the replacement of the bistables.

An extent of condition (EOC) investigation was conducted and it was determined that the conditions identified on the 22 MBFP involving the shoulder screws used to secure the fulcrum bar and piston shaft link and the LP servomotor component alignment issue extend to the 21, 31, and 32 MBFP LP servomotors. The LP servomotor for the 21 MBFP was disassembled, inspected, and reassembled during the forced outage for this reactor trip event. The shaft and linkage were found to be aligned correctly with no twists, and all but one of the shoulder screws were staked. The one shoulder screw not staked was inaccessible due to the LP servomotor linkage being already reassembled. Station management elected not to remove the LP servo motor housing covers on the 31 and 32 MBFPs to perform visual inspections of the shoulder screws, as Indian Point Unit 3 (IP3) was operating at 100 percent power at the time of the IP2 forced outage. The EOC corrective actions are: (1) to perform semi- annual preventive maintenance (PM) inspections of the 21, 22, 31, and 32 MBFP LP servomotor shoulder screws comments regarding burden estimate to the Information Services Branch (T-2 F43), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by e-mail to used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

Indian Point 2 05000-247 2017 -001 -00 to ensure no backing out is occurring and (2) to install and stake the 21, 22, 31, and 32 MBFP LP servomotor shoulder screws using Loctite 609 during their next scheduled overhaul, and all future outage overhauls (corrective action to revise 0-TUR-402-MFW).

CAUSE OF EVENT

The direct cause of the 800 rpm swings on the 22 MBFP, and erratic operation of the LP steam supply governor valves, was that the shoulder screws, which secure the LP servomotor fulcrum bar to the piston shaft link pin, backed out over time and detached from the assembly. This caused the 22 MBFP LP governor valves to fail closed.

The cause of the failure of the main turbine runback feature was indeterminate. The discovered failure of-the PC- 412B-1 bistable would have prevented LLV1 from performing the runback. However, it could not be established with certainty whether the as-found out of tolerance setting found on the PC-412A-1 bistable would have prevented LLV2 from performing the runback.

Causal factors which may have contributed to this event are:

1. The LP servomotor shoulder screws were not staked by the vendor as required by Siemens/Westinghouse proprietary design drawings.

2. The piston shaft and LP servomotor linkage were found very slightly twisted and worn during initial investigation, which placed slight uneven forces on the shoulder screws due to vendor workmanship issues that may have contributed to loosening.

3. A step text was created to perform the 2R22 LP servomotor PM inspection which omitted vital instructions, notes, and caution statements on how to reassemble the LP servomotor that were included in the MBFPT inspection procedure (0-TUR-402-MFW).

CORRECTIVE ACTIONS

The following corrective actions have been or will be performed under the Entergy Corrective Action Program to address the causes of this event.

• Repair 22 MBFP LP servomotor during forced outage. Reassemble 22 MBFP LP servomotor with shoulder screws staked as required by propriety drawings and secure screws with Loctite 609 to ensure no backing out occurs while operating.

• Verify proper function of the main turbine runback alarm, time delay relays, and arming circuit.

• Replace runback permissive bistables PC-412B-1 and PC-412A-1.

• Issue Engineering Change (EC) 72905 to allow Loctite 609 use on LP servomotor shoulder screws.

• Revise procedure 0-TUR-402-MFW for detail on LP Servomotor shoulder screw assembly. This is to include staking of the shoulder screw threads per Siemens drawings 634J5 and 634J4, and applying Loctite 609 to the threads per EC-72905.

• Perform review of LP servomotor failure with Siemens personnel who performed work in 2R22 to obtain their recollection of why requirement to stake shoulder screws was not performed, and perform a rework investigation per procedure EN-MA-123.

• For step texts that are elected to be used to perform work instead of site specific procedures, ensure all vital instructions, notes, and cautions statements are transferred from the procedure over to the step texts so all required information is captured and provided to those performing the work. This applies to all MBFP and main turbine generator (MTG) work for the IP2 2018 (2R23) and IP3 2019 (3R20) refueling outages.

comments regarding burden estimate to the Information Services Branch (T-2 F43), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by e-mail to used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

Indian Point 2 05000-247 2017

• Create a PM for IP2 and IP3 to perform semi-annual inspections and match marking of the MBFP LP servomotor shoulder screws to ensure no backing out of the screws is occurring while operating.

EVENT ANALYSIS

The event is reportable under 10 CFR 50.73(a)(2)(iv)(A). The licensee shall report any event or condition that resulted in manual or automatic actuation of any of the systems listed under 10 CFR 50.73(a)(2)(iv)(B). Systems to which the requirements of 10 CFR 50.73(a)(2)(iv)(A) apply for this event include the Reactor Protection System (RPS) {JC} including reactor trip and AFWS actuation'. This event meets the reporting criteria because the RPS was actuated by manual operator action at 1531 hours0.0177 days <br />0.425 hours <br />0.00253 weeks <br />5.825455e-4 months <br /> on June 26, 2017 in response to decreasing SG water levels, and the AFWS was automatically actuated on a valid low SG water level signal.

PAST SIMILAR EVENTS

A review was performed of the past three years for Licensee Event Reports (LERs) that involved a reactor trip caused by a MBFP speed control malfunction. One LER (IP3 LER 2015-007-01) was identfied that reported a miswired 31 MBFP Track and Hold board in the Lovejoy speed control system. This condition caused a minimum speed control signal to the 31 MBFP that resulted in lowering SG water levels, and a manual reactor trip was initiated at 15 percent SG level. The root cause was that the procurement for the MBFP Lovejoy Track and Hold boards was at an insufficient quality level commensurate with its criticality. As such, the corrective actions for this past similar event would not have prevented the event reported in this LER.

SAFETY SIGNIFICANCE

This event had no effect on the health and safety of the public. There were no actual safety consequences for the event because it was an uncomplicated manual reactor trip with no other transients or accidents, and the required primary safety systems performed as designed. The AFWS actuation was an expected reaction to low the SG water level caused by SG void fraction (shrink). This occurs after a reactor trip due to main steam {SB} back pressure that results from the rapid reduction of steam flow following turbine control valve closure. A reactor trip with the reduction in SG level and AFWS actuation are conditions for which the plant is analyzed. This event was bounded by the analyzed event described in IP2 Updated Final Safety Analysis Report (UFSAR) Section 14.1.9, Loss of Normal Feedwater. The AFWS has adequate redundancy to provide the minimum required flow assuming a single failure.

The UFSAR analysis demonstrates that the AFWS is capable of removing the stored and residual heat plus reactor coolant pump waste heat following a loss of normal feedwater event, thereby preventing over pressurization of the Reactor Coolant System (RCS) {AB} and preserving reactor coolant inventory.

An automatic turbine load runback signal is generated on a trip of one of the two MBFPs to initiate an automatic main turbine load runback. An automatic main turbine runback was not actuated during this event. The turbine runback is a design feature, and is not required for reactor protection. The UFSAR Section 14.1.4 analysis for rod cluster control assembly (RCCA) {AA, ROD} drop no longer credits turbine runback, nor is it credited in any other licensing basis analysis. For this event, rod control was in automatic and all rods inserted upon initiation of the manual reactor trip.

The AFWS actuated and provided required feedwater flow to the SGs. RCS pressure remained below the setpoint for pressurizer power operated relief valve (PORV) {AB, RV} or code safety valve {AB, RV} operation, and above the setpoint for automatic safety injection {BQ} actuation. Following the reactor trip, the plant was stabilized in hot standby with decay heat being removed by the main condenser.

- 001 - 00