ML19332F901

Responds to NRC 891113 Ltr Re Violations Noted in Insp Rept 50-285/89-27.Corrective Actions:Replacement of Failed Differential Transmitter DPT-1039 Will Be Completed by 900131 & Procedures Re Speed Control Loop Issued by 900210

ML19332F901
Person / Time
Site:	Fort Calhoun
Issue date:	12/13/1989
From:	Morris K OMAHA PUBLIC POWER DISTRICT
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
LIC-89-1142, NUDOCS 8912190276
Download: ML19332F901 (11)

Text

...

l W-

-1

'- l l

Omaha Public Power District 1623 Harney Omaha, Nebraska 68102 2247 402/536 4000 1

December 13, 1989 ,

LIC-89-1142 J L

, U. S. Nuclear Regulatory Commission ]'

Attn: Document Control Desk Mail-Station PI-137 Washington, DC 20555 l

References:

1. Docket No. 50-285

2. Letter from OPPD (K. J. Morris) to NRC (Document Control Desk) dated March 24, 1989 (LIC-89-202)

3. Letter from OPPD (K. J. Morris) to NRC (R. D. Martin) dated April 5, 1989 (LIC-89-335)

4. Letter from NRC (R. D. Martin) to OPPD (K. J.' Morris) dated ,

November 13, 1989

5. Letter from OPPD (K. J. Morris) to NRC (Document Control ,

Desk) dated December 1, 1989 (LIC-89-1036)

Gsntlemen:

SUBJECT:

Reply to a Notice of Violation (NRC Inspection Report 50-285/89-27)

Omaha = Public Power District (OPPD) received the subject Notice of Violation dated November 13, 1989 which identified one violation. The violation

-concerned OPPD's failure to establish operability of the turbine-driven auxiliary feedwater pump. Please find attached OPPD's response to the Notice-of Violation in accordance with 10 CFR Part 2.201. This violation, including a

~

review of the root causes and OPPD's corrective actions, was discussed during an enforcement conference held on July 28, 1989, at the Region IV office.

OPPD recognizes the severity level of this violation and is confident that the corrective actions taken will prevent future occurrences of this type.

8912190276 891213 g/

gDP ADOCK0500g5

/i opponunnu  !

as se.

unpiourneng;r

-j f

.- )

'.U. S. Nuclear Regulatory Commission  !

LIC-89-1142 Page 2--

]

l If you have any questions concerning this matter, please contact us.

Sincerely,  !

K. . Morris Division Manager Nuclear Operations ,

KJM/pjc Attachment c: LeBoeuf, Lamb, Leiby & MacRae R. D. Martin, NRC Regional Administrator, Region IV A. Bournia, NRC Project Manager ,

P. H. Harrell, NRC Senior Resident Inspector -

P w, -

ms-

. 1

,Page'l of 9 ATTACHMENT REPLY TO A NOTICE OF VIOLATION E A. During an NRC inspection conducted on June 22-23, 1989, a violation of NRC requirements was_ identified. In accordance with the " General Statement of Policy and Procedure for NRC Enforcement Actions," 10 CFR Part 2, Appendix C (1989) (Enforcement Policy), the violation is listed below:

Failure to Establish Goerability Becanig of Inadeauate Testina Fort Calhoun Station Technical Specification 3.3(1)a. requires inservice testing of ASME Code Class 1, 2 and 3 pumps in accordance with "ASME XI" Boiler & Pressure Vessel Code,' as required by 10 CFR 50, Section  ;

50.55a(g)." ASME XI, Article IWP-3000, requires that tests be conducted, periodically, and at a reference speed for variable speed pumps.

Fort Calhoun Station Technical Specification 3.9 requires that the operability of the turbine-driven auxiliary feed pump be demonstrated at least monthly by testing.

Contrary to the above, the operability of the Fort Calhoun Station turbine-driven auxiliary feedwater pump, an ASME Class 2 variable speed pump, was not demonstrated in that the monthly tests were not conducted at a reference speed, and therefore the failure of a speed controller in 1985 was not detected until 1989.

This is a Severity Level III violation. (Supplement I)

OPPD RESPONSE:

1. Reason for the Violation, if Admitted OPPD admits the violation occurred as stated. The primary cause for the violation was an inadequate surveillance test procedure, in that it did not satisfy the reference speed criteria of ASME,Section XI, Article IWP-3000. Contributing factors to the violation were: 1) lack of a permanent Inservice Inspection / Inservice Testing (ISI/IST) Coordinator to ensure that the surveillance test was in compliance with the ASME Code requirements, and 2) lack of a permanent Auxiliary Feedwater (AFW) System expert (System Engineer) with the detailed knowledge necessary to properly review and evaluate the surveillance test results.

The surveillance test procedure used to demonstrate adequate pump performance did not require that the test be conducted at the same reference speed each time as is required by ASME Section XI, Article IWP-3000. The performance of the test at a specified reference speed requires the injection of a test signal and/or manipulation of the control loop's setpoint controller to simulate steam generator pressure. This simulation allows for the increasing or decreasing of the AFW turbine speed thus determining the operability of the speed controller.

1

- -- , . - - . -- - . _ . - - - - - ~ . ,

,Page'2 of 9 Therefore, although the procedure inadequacy did not directly cause the mechanical failure of the pump's pneumatic speed control system, it was the primary contributor to the inability to detect the degradation and failure of the speed control system.

Contributing to the surveillance test deficiency was the fact that until February, 1989 there was not an ISI/IST Coordinator position. The -

principal responsibility of the ISI/IST Coordinator is to evaluate and verify that the Station's surveillance testing program is in compliance with the governing ASME Codes'and the Station's Ten-Year ISI Plan. This >

1ack of a permanent coordinator was a major contributor to the failure to previously detect that the surveillance test was not in compliance with the code.  :

A review of previous surveillance testing of FW-10 per ST-FW-1 was performed by OPPD's Special Services Engineering to determine if the failure of the speed control loop should have been evident from the data '

available. The surveillance test was found to contain sufficient information to determine that a problem existed with the speed control loop, however, the pattern 'that indicated the failure was discovered only after considerable manipulation of the available data. If the surveillance test reviewers had been thoroughly familiar with the design and operation of the turbine-driven pump, the degradation and failure of the speed control loop may have been detected earlier. The newly established System Engineer program is designed to provide the system specific technical expertise to address problems such as this.

2. Corrective Steos Which Have Been Taken and Results Achieved Immediately following the discovery of the failure of the speed control system, the pump was tested with instrument air isolated to the pneumatic speed control system to confirm that the pump would operate at the maximum speed permitted by the overspeed limiting governor, as stated in the USAR.

The pump discharge pressure at this maximum speed was observed to be within the range required for the pump to meet its intended safety function, yet not exceed the design pressure of the discharge piping. The instrument air supply to the pneumatic speed control system was then tagged out so that the degraded speed control system could no longer restrict the pump speed.

9

.Page'3 of 9 The pump is currently configured to run at the maximum constant speed '

permitted by the mechanical overspeed limiting governor and will- remain in this configuration until the pneumatic speed control system is returned to service. The pump will be subsequently tested to obtain baseline values for trended parameters at an established reference speed as required by the ASME Code, Section'XI, Article IWP-3000. The control loop will be returned to service in accordance with the timetable given in Section 3, page 8.

Following the discovery of the control loop failure, two significant activities were undertaken. First, a thorough review of the Inservice Testing Program for safety-related pumps was conducted to ensure compliance with the ASME Code,Section XI. No deviations from the code were found beyond those which were already identified through the ISI/IST Program Upgrade outlined in Reference 2 and OPPD's Safety Enhancement Program (Reference 3), which were both in progress before the June 13, 1989 event.

The other significant activity undertaken was the formclation of an action plan to address cause, consequences and corrective actions for the event.  !

The action plan is itemized in detail, along with the associated results and conclusions, in Licensee Event Report (LER)89-016, Revision 2 (Reference 5). In addition, Reference 5 describes the subject event in detail and also describes the individual components which comprise the speed control loop. The specific action plan items are listed below with a summary of the key results and conclusions,

a. Determine the direct cause of the derivative controller failure in the FW-10 speed control loop.

Special Procedure SP-FW-13 was performed to troubleshoot and repair the speed control loop on FW-10. The pneumatic controller vendor participated in the effort. The failure of the derivative unit was caused by excessive clearances which developed between close tolerance parts within the unit during its installed life. These increased clearances between the diaphragms and nozzles caused erratic operation and eventual complete failure of the unit. The body bolts which hold the stacked body parts together were found to be loose. Similar problems of increased clearances and loose body bolts were found with the two-mode controller upstream of the derivative unit. In addition, the zero setpoint adjustment screw on the two-mode controller was found turned to the extreme clockwise position further prohibiting the normal functioning of the unit. When and why this adjustment was made could not be determined. Both the setpoint controller and derivative units have been replaced with new, identical, calibrated units. The requirement for verification of proper torque on the body stud nuts for both units will be included in new preventive maintenance (PM) procedures. These PM procedures will be issued, in accordance with the timetable given in Section 3, page 8.

During the troubleshooting process, the differential pressure transmitter was also found to have failed. The bellows leaked internally, resulting in the failure to respond to differential pressures applied at the inputs. A replacement transmitter has been ordered. For this reason, the speed control loop of FW-10 remains out

7

~

.yage4~of9 of service with its air supply valve tagged shut. Disassembly and inspection of the transmitter to determine its failure mechanism will be conducted after it is replaced. The new transmitter is expected to be installed in accordance with the timetable given in Section 3, page 8.

b. Determine the root cause of the failure of FW-10 to respond to manually injected air signals during the conduct of SP-FW-12.

A root cause investigation was conducted by members of the Nuclear Safety Review Group (NSRG). The root cause was determined to be failure to include the speed control loop for FW-10 in the preventive maintenance / calibration program at initial plant startup. Subsequent investigation by the NSRG determined the following contributing factors: the control loop components were never classified as Critical Quality Element (CQE) and an inadequate surveillance test did not verify the proper operation of the speed control loop. Preventive maintenance and calibration procedures will be in place in accordance with the timetable given in Section 3, page 8.

The control loop components are now classified as CQE components and are documented in the CQE list. The surveillance test will be upgraded to incorporate the baseline data determined from testing the pump at an established reference speed following the replacement of the transmitter, in accordance with the timetable given in Section 3, page 8.

c. Review and evaluate available information to ascertain, if possible, when failure /misadjustments of the speed control components occurred.

Immediately following the discovery of the failure of FW-10 to respond l to pneumatic test control signals, a review of past surveillance test data (ST-FW-1) was initiated. Several different test result parameter combinations were graphed to find a trend or pattern that would indicate when the failure might have occurred. A graph depicting the

trends of turbine steam inlet pressure and the pressure differential l between the pump discharge and steam inlet was constructed.

The data indicates that, since July,1985, the differential pressure between pump discharge and steam inlet pressures varied inversely with the steam inlet pressure changes. If the pump speed control loop had been operating properly, this differential pressure trend would be a relatively straight line. The inverse relationship was caused by the 1: pump operating at a constant speed, and therefore constant discharge pressure, due to the control loop being unresponsive. That is, no matter what the air input ~ signal from the differential pressure p ' transmitter DPT-1039, the turbine throttle linkage positioner output air signal remained at approximately 12 psig output, causing the pump to operate at a relatively constant speed. This speed was not sufficient for developing the head required for the pump to fulfill its safety-related function during a Design Basis Accident (DBA).

Therefore, it is concluded that since July,1985, the pump speed control loop was inoperable, causing FW-10 to be inoperable.

.hage'5'of.9 Prior to July, 1985, the test data points for the pump discharge pressure / steam inlet pressure differential were widely variable, ,

indicating that the control loop was not functioning properly.

l However, the pump did operate at sufficient speeds to develop the discharge pressure necessary for injection of Jater into the steam generators under DBA conditions. Therefore, although the controller may not have been fully operable prior to July,1985, it did not restrict the speed of the pump enough to cause FW-10 to be inoperable,

d. Review previous testing of FW-10 to determine if the failure of the speed control loop should have been evident from the data available.

Evaluate the effectiveness of the existing surveillance tests on FW-10 '

used to demonstrate operability.

As evidenced by the resolution of item c. above, the surveillance test I did contain sufficient information to determine that a problem existed with the speed control loop. However, the pattern that indicated a failure was discovered only after manipulation of the available data.

The fact that the failure occurred prior to the impl3 mentation of the System Engineer program, and a formal surveillance trending program, make it highly improbable that the failure of the speed control loop would have been recognized by the previous test reviewers.

An evaluation was performed to determine the existing surveillance test's effectiveness in demonstrating the operability of FW-10. This evaluation revealed several deficiencies, summarized as follows: a) turbine steam bowl pressures were not recorded or trended; b) pump  :

suction pressure was not directly measured; c) the pump speed was not varied from the steady state speed; and d) the full flow capability of the pump was not periodically tested. Based on these deficiencies, OPPD concluded that ST-FW-1 was not adequate in demonstrating the operability of FW-10 prior to April, 1989.

Items a) and b) have been incorporated into the current revision of ST-FW-1. Item c) will be incorporated by testing FW-10 at an established reference speed following replacement of the differential pressure transmitter. Item d) has been addressed by a special full flow test, performed in April,1989, during a scheduled outage, to verify the full flow capabilities of beth AFW pumps. A new full flow test line will be installed by July 30, 1990 after which a periodic test will be developed. This will allow the test to be performed periodically at normal power operations. Until the full flow test line is installed; existing procedure will be used under which a full i flow test will be performed during the 1990 refueling outage.  ;

I l

i i

+

,Page,6 of.9

e. Investigate why the speed control loop components have had no equipment identification numbers, calibration procedures, or periodic maintenance. Evaluate whether speed control loop components are correctly classified with respect to Electrical Equipment Qualification (EEQ), safety class, and procurement class (CQE, etc.). l Investigation revealed that two speed control loop components did have assigned equipment numbers: the Differential Pressure Transmitter is PT-1039, and the Two Mode Nu11matic Controller is PC-1039. The remaining control loop components, the Derivative Nullmatic Unit, the Fisher Positioner and Linkage Actuator were not uniquely identified.

The following factors appear to have contributed to the fact that these instruments were not included in the preventive maintenance / l calibration programs during and after initial plant startup: the i instruments were supplied skid-mounted with FW-10; the instruments I were never identified as CQE (or safety-related); the instruments were not identified on the startup instrument punch lists; and the instruments were always considered to be part of the Main Steam System (steam supply to FW-10) instead of the Auxiliary Feedwater System.

i 0 PPD has reviewed other ISI pumps in an attempt to identify skid- I mounted equipment similar to that involved in this incident and none was identified. Unique AFW component identification numbers have been j assigned to each device in the control loop.- The loop diagram and CQE  !

manual have been updated to incorporate the new component identification numbers. Calibration and preventive maintenance procedures for the speed control loop will be in place, in accordance ,

with the timetable given in Section 3, page 8. a

f. Evaluate the ' design basis of and the need for the differential pressure controller on FW-10.

The FW-10 speed control loop is designed to limit pump discharge  !

pressure to a setpoint sufficiently above steam generator pressure to permit injection at the required flow rate under various operational and design basis event conditions. The primary advantage of the existing speed control loop design is that it allows the pump to run at optimum speeds, sufficient to meet system head requirements while minimizing pump and valve wear. This efficiency feature is particularly advantageous for injection into a steam generator with decaying pressure conditions (i.e., long term heat removal).

Various alternate control methodologies were evaluated by OPPD. It was concluded that the current / original design configuration best met the overall system design requirements. The differential pressure transmitter must be replaced prior to returning the control loop to service, currently scheduled for completion in accordance with the ,

timetable given in Section 3, page 8.

g. Evaluate a loss of instrument air event to determine, if possible, the length of time between the loss of air and the instrument air pressure dropping low enough to cause FW-10 to operate on the speed limiter.

Include an evaluation of the impact of this time delay on events l

involving a demand for FW-10.

l

i...

l l

'.Page 7 of 9 j

The pump is considered operable with the instrument air supply )

isolated. A loss of instrument air has no effect on pump speed-in this configuration. For the period prior to June, 1989, during which the speed control loop was inoperable, a loss of instrument air pressure to below approximately 20 psi, the setpoint of the in-line air regulator, would have initiated an ,

increase in FW-10 speed. FW-10 speed would gradually increase as -

air pressure decayed, up to the maximum speed limited by the governor. However, the length of time for air pressure to drop below 20 psi would depend on several variables, including the type of initiating event and the instrument air system usage rate. For events involving no breach of the instrument air system and a loss of offsite power, the rate of instrument air depressurization would likely be low, it is impractical to accurately determine the interval between loss of instrument air and FW-10 reaching maximum speed for all events involving a loss of instrument air. Also, the anticipated operator response to most events would include restarting of an air compressor, using emergency power, long before air pressure degraded to 20 psi. '

For these reasons, it can be conservatively concluded that FW-10 would have been unable to mitigate certain accident conditions, since credit cannot be taken for immediate and continuing loss of instrument air pressure,

h. Evaluate the as-found condition of FW-10 for a loss of main feedwater design basis event concurrent with failure or unavailability of FW-6.

A review of the Updated Safety Analysis Report (USAR) Chapter 14 indicated that for design basis events demanding auxiliary feedwater flow for mitigation, the most limiting event is the small break LOCA concurrent with a loss of offsite power and single failure or unavailability of FW-6. The loss of offsite power would cause loss of main feedwater and loss of instrument air and thus bounds a loss of main feedwater event.

In this scenario, steam generator pressure is assumed to be 1000 psia. As discussed in item g. above, no credit was taken for full operability of FW-10, since it is assumed Operations would power an instrument air compressor from an emergency power bus, limiting FW-10 discharge pressure to the as-found value of approximately 996 psig (1010.7 psia). If piping frictional losses and head differential are taken into account, FW-10 would not have been able to provide auxiliary feedwater to the steam generators. In this situation, Procedure E0P-20, "RCS and Core Heat Removal Success Path HR-4," directs the operators to initiate once-through-cooling utilizing safety injection pumps.

. ~ . - . ._ __ - _ . . . __ _

?

Page '8 of 9 In addition to the-action plan described in items a, through h. above, the following corrective actions have been completed:

(1) An interim revision to surveillance test ST-FW-1 has.been made to reflect the operation of FW-10 at a constant speed as limited by the governor. The test revision also. includes the direct measurement of '

pump suction pressure and turbine bowl pressure to more accurately trend pump performance parameters for determining pump degradation.

(2) An ISI/IST Coordinator position has been eC.ablished. Included within the Coordinator responsibilities is the review of completed and revised surveillance tests involving equipment governed by ASME Code,Section XI, fer compliance with the Code.

(3) A System Engineering organization has been implemented to establish a cognizant engineer for each system in the plant. The System Engineer has responsibility for reviewing surveillance test results for the individual assigned system, including trends of the measured parameters, to verify that any performance degradation is within Code acceptable limits.

3. Corrective Steos Which Will Be Taken to Avoid Further Violationt The following corrective actions will be taken:

Activity Comoletion Date s Replacement of failed differential pressure January 31, 1990 transmitter DPT-1039, subject to receipt of ordered parts, a Issue preventive maintenance and calibration February 10, 1990 procedures for the speed control loop.

m Conduct special test to obtain baseline data at February 10, 1990 established reference speed and return control loop to service e

a Revise surveillance test ST-FW-1 to incorporate March 16, 1990 l baseline data at reference speed l

l m Conduct revised surveillance test Two months after L following return of speed control loop to service completion of 1990 Refueling Outage l

l l

q.

'~d; I i,

'Page 9 of 91 4.: The Date When Full Comoliance Will Be Achieved OPPD is currently in compliance with the requirement for reference speed testing based on the current pump configuration and the interim revision to

- surveillance test ST-FW-1 under which the pump operates at the maximum ,

. constant speed permitted by the mechanical overspeed limiting governor.

g.

i i

F i

. _ - - - . . -