IR 05000315/1990005
| ML17334B350 | |
| Person / Time | |
|---|---|
| Site: | Cook  |
| Issue date: | 02/20/1990 |
| From: | Burgess B NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| To: | |
| Shared Package | |
| ML17334B349 | List: |
| References | |
| 50-315-90-05, 50-315-90-5, 50-316-90-05, 50-316-90-5, EA-90-037, EA-90-37, NUDOCS 9003080318 | |
| Download: ML17334B350 (18) | |
Text
U.S.
NUCLEAR REGULATORY COMMISSION
REGION III
Report Nos. 50-315/90005(DRP)/EA 90-037; 50-316/90005(DRP)/EA 90-037 (
,<<Qwg'k+
Docket Nos.
50-315; 50-316 Licensee:
American Electric Power Service Corporation Indiana Michigan Power Company 1 Riverside Plaza Columbus, OH 43216 License Nos, DPR-58; DPR-74 Inspectors:
B.
L. Jorgensen J.
M. Jacobson Facility Name:
Donald C.
Cook Nuclear Power Plant, Units 1 and
Inspection At:
Donald C.
Cook Site, Bridgman, Michigan Inspection Conducted:
January 8 through February 2, 1990 D.
G.
Passehl
~,~fC~"~
Approved By:..
urgessg Chief Projects Sect>on 2A Ins ection Summar DA E Ins ection on Januar 8 throu h Februar
1990 (Re ort Nos.
50-315/90005 RP '-316 90005 DRP di p
i by id d
gi inspectors of Unit 2 main steam stop valve stroke timing events, including cause assessment and corrective actions, and Unit 1 steam stop valve assessments.
Results:
Potentially significant issues were identified involving: failure to assure satisfaction of performance and test requirements; failure to determine root cause for test failures, and; failure to take adequate corrective action for repetitive cases of degraded main steam stop valve performance noted during testing.
9003080318 900220 PDR ADQC)( 0500033
.".)
f'Ibad
DETAILS 1.
Persons Contacted
~A. Blind, Plant Manager
"J. Rutkowski, Assistant Plant Manager, Technical Support K. Baker, Assistant Plant Manager, Production
"B. Svenssop, Executive Staff Assistant J.
Sampson, Operations Superintendent T. Beilman, Maintenance Superintendent
"J. Droste, Technical Superintendent, Engineering
"T. Postlewait, Design Changes, Superintendent The inspector also contacted a number of other licensee and contract employees and informally interviewed operations, maintenance, and technical personnel.
~Denotes some of the personnel attending the Management Interview on February 9,
1990.
2.
Unit 2 Main Steam Sto Valve Events (61726 62703 93702)
a 0 Back round
On January 8-10, 1990, with D.
C.
Cook Unit 2 in MODE 3 for lower ice condenser door surveillance, a scheduled test of the main steam stop valves (HSSVs)
was conducted.
The test consisted of stroking each valve from the open to the closed position and verifying the valve closed within prescribed time limits.
One time limit was based on previous test history (Section XI inservice testing)
and another was based on a Technical Specification upper limit of 5 seconds.
Normal valve performance would give closure times in the range of 2-3 seconds.
All four Unit 2 MSSVs stroked abnormally slowly, with 3 of the 4 valves having stroke times in excess of the 5 second limit of Technical Specifications.
The slow strokes were accompanied by flashing and splashing of condensate at the outlet of the "dump" valves which are used to operate the HSSV closed.
A series of tests showed that the condensate was removed by one operating cycle, but that it reaccumulated over a period of about 12-15 hours, The valves could be maintained
"OPERABLE" in MODE 3 by frequent stroking (as the licensee did during the test series)
but this would not be possible during power operations.
The licensee therefore shut down to MODE 5 to disassemble, inspect, and repair or modify the MSSVs as necessary.
The operating principle of the D.
C.
Cook HSSVs involves use of main steam line pressure to operate a piston to which the valve disc is attached.
The piston is designed to move vertically within a
cylindrical operating chamber.
In the normal, open position, the piston is near the bottom of the cylinder and the disc is below the main steam flowpath.
The steam pressure both above and below the piston is nearly equal by virtue of a steam pressure equalization port drilled through the piston.
A separate drainage port is intended to accommodate the drainage of any condensate,formed above the;piston in the operating cylinder or associated piping.
This port feeds:~a drainage tube which extends several inches below the piston.
e
,*
The MSSVs are'~closed by denting off the steam above 'the p)s'ton~v'ia either of two independently operated dump valves'hese 2-inch valves are capable of venting pressure off much more rapidly than it can be equalized by the much smaller steam port.
The unbalanced pressure below the piston drives it upward, moving the disc to the closed position across the steam flow path.
Depressurization of the upper cylinder is retarded by the presence of condensate which, because it is at saturation conditions, begins to flash to steam instantly as pressure begins to drop.
The effect is to create a potentially large steam source at near full pressure.
A non-proprietary drawing of a stop valve assembly is attached to this report.
b.
Corrective Actions/Findin s
During the MODE 5 outage from January 11-19, the MSSVs were modified to increase the diameter of drain and equalization holes in the operating piston.
The French had performed a similar modification a few years ago (see Paragraph S.c below) and had reportedly "cured" their valves of slow stroking.
The modification was reviewed by NRC, as discussed in Paragraph 4.
below.
When the unit was returned to NODE 3 and post-modification testing was performed late on January 22, 1990, one of the modified valves (2-MRV-210) again exhibited slow stroke time, accompanied by flashing and splashing outboard of the operating dump valve.
The other three stroked "normally" in less than three seconds.
The only obvious difference between the valve which "failed" and those which
"passed" was that there was a substantial dump valve leak on the former.
This was repaired, and the previously slow valve stroked normally in less than three seconds.
Based on the above, it was concluded that slow valve operation was due to excess condensation in the chamber above the operating piston which was being caused by "excess" leakage of steam past one or both dump valves.
What constituted
"excess" leakage was not known.
Some leakage could apparently be tolerated without a significant adverse effect on stroke tim erabilit of Unit 2 Valves At the time of the MSSV stroke timing problems on January 8-10,,
1990, there were three "open"'ob Orders (JO) already written to address leaks on dump valves associated with two of the four MSSVs (2 dump valves per MSSV) ~
The inspector performed a review to ascertain whether these known leaks had existed for long periods and to assess their implications for MSSV operability prior to discovery of the problem.
One Job Order (No.
B17495)
was dated November 18, 1989, and addressed known leakby on dump valve 2-MRV-242 associated with MSSV 2-MRV-240.
Prior to that time, on June ll, 1989, the MSSY was tested via both of its dump valves and stroked in 1.8 seconds each time.
Thus, 2-MRV-240 was OPERABLE in June, 1989, had a leaking dump valve by November, and failed a stroke test in January, 1990.
It is probable that 2-MRV-240 was inoperable for some time prior to discovery of the problem, but it could not be determined just how long this inoperability may have existed.
MSSV 2-MRV-210 had open Job Orders on both its associated dump valves; JO No.
B12012 concerned leakby on 2-MRV-211 and JO No.
B12013 concerned a packing leak on 2-MRV-212.
Both Job Orders were initiated on March 22, 1989.
In the interim, on June ll, 1989, MSSV 2-MRV-210 experienced a slow stroke event, with the initial test on Train B
(2-MRV-212) taking 5.5 seconds.
The MSSV was declared inoperable, but immediate retesting on Train "B" and verification testing on Train "A" (apparently - before condensate could reaccumulate)
showed stroke times of 2.5 and 2.7 seconds, respectively.
The MSSV was declared OPERABLE with no physical corrective action.
Subsequently (reference NRC Reports 50-315/89021;50-316/89021 and 50-315/89029;50-316/89029)
the inspector questioned the licensee's assessment of this matter but, lacking evidence that the valve was inoperable prior to the June 11 test, the matter had been tabled pending an internal inspection of the valve.
A Job Order was written in September 1989, to inspect the MSSV internals during the "next (MODE 5) outage of sufficient duration."
It is now apparent that leakby associated with the 2-MRV-210 dump valves was sufficient in June 1989, to affect MSSV performance.
Since the condition was not corrected, it is further apparent that the same leakage was still present and caused the slow stroke problem of early January 1990.
It thus appears that 2-MRV-210 could not have met its required Technical Specification stroke time of 5-seconds during the period from June 1989, until January 1990, and should not have been considered OPERABLE.
Two of the MSSVs (2-MRV-220 and 2-MRV-230) had no open Job Orders concerning dump valve leaks before the events of early January, 1990.
Subsequent investigation involving pipe temperature measurements, however, showed these MSSVs also had leaking dump valves.
Such leaks presumably predated the discovery of slow stroke
times, but it cannot be determined how long these conditions may have been present.
In summary, all four Unit 2 stop valves were significantly degraded; three could not meet the stroke time requirements of the Technical Specifications and were not OPERABLE for some time prior to.the discovery of the condition in early January 1990.
In the case of valve 2-MRV-210, it was likely not OPERABLE since at least the preyi.ous failed test on June ll, 1989.
As discussed in Paragraph 5 below, this has been a recurrent and previously uncorrected phenomenon, because root cause was not identified and corrected.
The Unit 2 stop valves were all OPERABLE, based on proper testing, prior to unit startup on January 24, 1990.
The continuing operability of Unit 2 stop valves is being addressed in the ongoing monitoring of the valves in both units discussed in Paragraph
below.
3.
Unit 1 Main Steam Sto Valves (61726 62703 71707)
On January 23, 1990, upon learning that MSSV dump valve leakby could degrade performance of the MSSV, the inspector asked the licensee to investigate Unit 1 dump valve conditions, then immediately inspected the conditions himself.
Two dump valves, each associated with different MSSVs, were found with existing Job Order tags (dated November 1989)
concerning dump valve leakage.
The inspector's judgement, based on noise and temperature around the enclosed valve and valve outlet areas, was that an additional four dump valves also had leaks - making six of eight in all on Unit 1.
Within hours, the licensee s investigation using electronic temperature measurements confirmed six of the eight valves were probably leaking.
Four additional Job Orders were initiated for repair of the dump valves, however these valves were classified safety-related and no repair parts of the proper classification were available.
The dump valves are 2 inch angle valves of 3/4 inch stoke manufactured by Fisher Controls.
The manufacturer had ceased being a nuclear safety-grade supplier.
The valves are air-operated to close, and "fail" open.
Based on preliminary assessment of the relative indications of leakage, comparing Unit 1 to Unit 2, the licensee concluded that Unit 1 dump valve leakage was less severe than that on the one Unit 2 valve which had passed its stroke test before it was modified with larger drain ports.
Therefore, it was determined acceptable (on January 23) to continue Unit 1 operation.
A written justification for the continued operation of Unit 1 was provided to NRC Region III and to the Office of Nuclear Regulation by facsimile the following da Telephone conference calls were conducted among licensee plant and corporate personnel, and NRC Region III and headquarters personnel (Office of Nuclear Reactor Regulation NRR) on January 24 and 25, to discuss short-term assurance that neither Unit 1 nor Unit 2 dump valve leakage would be allowed to significantly degrade.
Long term assessments of the best means to prevent future dump valve leakage, including consideration of replacement with a different valve, were also disc'Ussed.
Pursuant to these discussions, the licensee committed, in letters dated February,.l,and 2, l990 (AEP: NRC:1121-,and ll21A),- ~to implement-an;ongoing qualitat)ve and quantitative dump va1ve'leakage monitoring 'program on both units, to assess the continuing operability of the main steam stop valves.
The program consists of acoustic monitoring to detect qualitative changes at the valves, supplemented with pressure-drop testing'on any valve upon which predetermined criteria for acoustic changes are met.
The details of this program will be subject to independent technical review by NRC.,
The licensee also committed to assess the application of.the current model dump valves and to make a
determination, by the completion of the 1990 refueling outages, on the option of replacing the valves with a different type.
Sto Valve Modification 37828 As noted elsewhere in this report, the licensee initially approached the Unit 2 MSSV performance problems as a potential design deficiency.
Information was acquired concerning the implementation of a design modification by the French, and a similar approach was applied.
A Region III NRC inspector made a site visit to review the modification and post-modification testing.
The modification'(Number 02-MM-079) was processed as a "Minor Modification" under the licensee's design change control process.
Choosing this approach appeared appropriate.
The modification consisted of enlarging the drain and equalizing orifices in the valve operating piston, to permit more efficient draining of condensate from the valve upper volume.
Buildup of condensate in the upper volume was considered to be the proximate cause of excessive MSSV closure times.
The Minor Modification was considered safety related because the main steam stop valves are Seismic Class 1 components which affect accident mitigation.
The inspector reviewed licensee technical documentation associated with the modification, including calculations performed to demonstrate a
substantial improvement in upper volume drainage.
No problems were identified.
The licensee had consulted Hopkinson, Ltd. of Great Britain (the valve designer)
and retained the U. S.
agent (Atwood-Morri11) to support the implementation of the modification by supplying a vendor technical representative.
These consultants had concurred in the technical aspects of the modification.
The inspector also reviewed the safety evaluation information contained in the minor modification package.
No deficiencies were note Post-modification testing was observed to be successful for both Train A and Train B testing of MSSV 2-MRV-220.
5.
Eval uati on 61726 62703)
As noted in Paragraph 2 above, three Unit 2 MSSVs, were found inoperable (and the fourth was just at the limit) in early January'990.
'When the cause was found to be condensate accumulation in the operating steam chamber,,primarily due to small,. pressure loss'~via dumppvalve..s'team~-.'<;."'-,.
leakage, st became evident th'at MSSV inoperability'as'a pre"-eXisting condition of indeterminate duration.
Duration was likely at least seven months in one case - valve 2-MRV-210.
Further NRC assessment of the issues raised by these findings involved review of previous MSSV test and maintenance histories.
a ~
~Teetin Test results from testing the stroke times of MSSVs in both units were reviewed to the earliest data extant, which were prior to 1980.
The test results were evaluated against two criteria: the criterion of ASME Section XI (IWV-3410) for changing test frequency if stroke time increases by 50-percent or more in consecutive tests; and the five second overall stroke time limit of the Technical Specifications.
For Unit 1, the following were found.
(1)
IWV-3410 was not in effect prior to April 13, 1983.
(2)
Valve 1-MRV-210 experienced no fai lures against either criterion in 28 tests, although one pair of consecutive tests in 1981/1982 (prior to IWV-3410 being applied)
showed a
stroke-time increase of over 100-percent.
(3)
Valve 1-MRV-220 experienced two failures against IWV-3410 for the pairs of tests conducted in September 1982 through May 1983 and in November 1988 through July 1989.
The data base was
tests.
(4)
Valve 1-MRV-230 experienced five failures against IWV-3410 for the pairs of tests conducted in September 1982 through May 1983, July 1984 through January 1985, December 1985 through April 1987, January 1988 through September 1988, and November 1988 through July 1989.
The test of January 1985 exceed 5.0 seconds (5. 1)
and the test of April 1987 was at 5.0 seconds.
The data base was 33 tests.
(5)
Valve 1-MRV-240 experienced one failure in September 1987 against both criteria, when a stroke time of 5.4 seconds was recorded after the April result of 3. 5 seconds.
The data base was 28 test Summarizing Unit 1: of 118 tests recorded, nine involved consecutive results failing the criterion of IWV-3410 (one predated application of the requirement);
of those nine, three were at or in excess of the 5.0 second limit of Technical Specifications, and; the majority of the problems occurred within the last three years, since early 1987.
For Unit 2 the following were found:
8 l'SQ (1)
'IWV-3410 was not in effect as currently applied prior to April 13, 1983.
(2)
Valve 2-MRV-210 experienced two instances (one predating the current requirement) of exceeding the IWV-3410 criterion in the pairs of tests conducted in June and August 1978 and in October 1987 through March 1989.
Further, two different cases failed the 5.0 second limit of Technical Specifications in June 1989 (5.5 seconds)
and January 1990 (5.9 seconds).
The data base was
tests.
(3)
Valve 2-MRV-220 experienced two failures (one predating current requirements)
of the IWV-3410 criterion in the pairs of tests conducted in October 1980 through May 1981 and in June 1989 through January 1990.
The latter also failed Technical Specifications at 5.6 seconds.
The data base was 28 tests.
(4)
Valve 2-MRV-230 experienced three failures (two predating cur rent requirements)
against IWV-3410 for pairs of tests conducted in June and August 1978, July and October 1980, and March and June 1989.
Subsequently, in January 1990, the Technical Specification criterion was matched with a stroke time of 5.0 seconds.
The data base was 30 tests.
(5)
Valve 2-MRV-240 experienced two failures of the IWV-3410 criterion for pairs of tests conducted in March and July 1987, and in June 1989 - January 1990.
The January 1990 test also failed Technical Specifications with a stroke time of 5. 2 seconds.
The data base was 29 tests.
Summarizing Unit 2: of 114 tests recorded, nine involved consecutive results failing the criterion of IWV-3410 (four predated current application of the requirement);
of those nine, two were at or in excess of the 5.0 second limit of Technical Specifications three additional tests were also at or over the 5.0 second limit, and; 'the majority of the problems occurred in the last three years.
Failing either criterion (IWV-3410 or Technical Specifications) is evidence of potentially significant valve degradation.
The IWV specifies increasing the frequency of testing ".
.
. until corrective action is taken."
The Technical Specifications provide that a
stroke time of less than 5 seconds is required to be demonstrated to consider a
The MSSVs are not capable of being stroke tested with the unit at any significant power level.
The effect of
this has been that stroke timing tests have not been performed at increased frequency as envisioned by IWV-3410 except in rare instances when the unit happened to be in a compatible MODE in a timely manner.
Further, with the exception of a January 1985 test on valve 1-MRV-230, (repairs and retest within about two weeks)
there was no apparent performance of "corrective action" until after the events of January 1990.
Maintenance
The maintenance histories on the main steam stop valves and associated dump valves were reviewed by the inspector.
Since early 1987, a total of six Job Orders were found for repair of leaking dump valves on Unit l.
At the time of the inspection, two of these repairs had yet to be performed.
As noted above, four additional Job Orders were written for Unit 1 dump valves as a
consequence of investigations prompted by the Unit 2 events of early January 1990.
The average time for completion of Unit 1 dump valve repairs in the past three years was in excess of six months after initiation of a Job Order.
During one interval, after a leak was identified on dump valve 1-HRV-221 and Job Order JO 735836 was written on November 3, 1988, the associated stop valve 1-HRV-220 experienced a "slow stroke" event in June 1989, before the repairs were completed in July.
Job Order JO 730231 likewise was open to repair 1-MRV-231 when MSSV 1-MRV-230 was "slow" (3. 1 seconds)
in June 1989.
Two Unit 1 Job Orders were opened shortly after "slow" stroke events and may indicate a dump valve leak was present and caused the problem before a Job Order was initiated.
A total of twelve Unit 2 Job Orders address dump valve leaks since early 1987, of which three had yet to be performed at the time of the inspection.
At least one additional Job Order was initiated as.
a consequence of the findings and followup associated with the Unit 2 events of early January, 1990.
The average time for completion of Unit 2 Job Orders during this period was between three and four months.
Unit 2 Job Order JO 27269 concerned repair of 2-HRV-242 in late 1987.
Shortly before, MSSV 2-MRV-240 was "slow" (3. 1 and 3.6 second strokes)
when timed on October 9-10, 1987.
As noted previously, both HSSV 2-MRV-210 and 2-MRV-240 had Job Orders
"open" an their respective dump valves (including both dump valves on 2-MRV-210) at the time both MSSVs required over 5 seconds to stroke in early January, 1990.
The above shows that the dump valve maintenance history correlates to stop valve performance, and that such maintenance was frequently not performed in a timely fashion.
In nine of the thirteen examples of "slow" stop valve stroke events since early 1987, there is documented evidence suggesting one or both associated dump valves were leaking at the time.
In two more cases (the two Unit 2 Valves not having "open" Job Orders in January 1990) followup investigation after a slow stroke event found one or both dump valves leaking.
Thus, there was evidence in the licensee's own experience to connect
dump valve leakage to stop valve performance in a cause-effect relationship.
The licensee had not recognized this implication of his own experience.
Industr Ex erience/Information The D.
C.
Cook and Salem nuclear plants (two units each)
are the only domestic nuclear plants with main steam stop valves designed by Hopkinson: Ltd. of.Great. Britain. Two French plants,;at>gessynha$
m and at Bugey, are also equipped with MSSVs of this type.
In November 1984, the French Commissariat a 1'Energie Atomique (the Atomic,.Energy Commission of France)
completed a report covering
..
oper'ational evaluations of the (Hopkinson)
steam isolation valves at Fessenheim and Bugey.
A copy of this proprietary report was provided to the D.
C.
Cook licensee corporate office by the NRC Office of International Programs via a letter (K.
D. Burke to P.
A. Barrett)
dated July 5, 1985.
The corporate office in turn transmitted a copy of the French report to the plant site on July 12, 1985.
Because the report is proprietary, no technical details will be addressed herein. It was noted by the inspector in review of the report, however, that the French had experienced slow stroke events and that among the hypotheses considered as potential causes of these events was the condensation of water (condensate accumulation)
within the steam valves.
A marginal notation by the Plant Manager of the time suggested this hypothesis might apply at the Cook plant; more so-than the other possibilities considered.
The French also favored closures observed were
condensate in the upper identify any underlying or maintenance effects.
weakness and the French condensate by enlarging the French valves.
this hypothesis, concluding that the slow ikely attributable to the presence of steam cylinder.
The French report did not dynamic cause(s)
such as operational, test Rather, the problem was treated as a design valves were modified to improve removal of the drainage port. This apparently
"cured" The details of the French report were not followed up by the licensee at the time the report was received.
There was no plan to modify the D.
C.
Cook valves in the manner of the French.
The report was apparently filed in a proprietary valve file. Thereafter, there was no specific regard taken concerning the French experience until after June 11, 1989, when two Unit 2 MSSVs were "slow" at Cook plant.
Excess accum<~lation of condensate was identified as a
potential root cause of the June problems, and a Job Order was written on September 28, 1989, to disassemble and inspect valve 2-MRV-210 (focusing on the condition of the drainage port) at the next "outage of sufficient duration."
When this Job Order was written, the Unit was already back in service and remained so until the events of early January 1990 overtook this plan.
i p
In summary, prior industry information, in the form of an operational assessment report from the French Atomic Energy Commission, which was provided to the licensee by the NRC, was not thoroughly investigated and properly applied to the D.
C.
Cook plant in a timely manner.
6.
Re ulator Issues
,
Title",10;.~Code of Federal>Regulations,'art'.
50, Ap'pendix'B,~ Criterion/XI,
"Test Control"'equires that test results be evaluated to assure that test requirements have been satisfied.
The licensee s repeated failure to adequately evaluate MSSV test results as described above, appears to involve a violation of these requirements.
Title 10, Code of Federal Regulations, Part 50, Appendix B, Criterion XVI,
"Corrective Action" requires that, when significant conditions adverse to quality have been identified, that corrective actions be taken to ensure that the cause of the condition is determined and that corrective actions be taken to preclude repetition.
The licensee's failure to determine the root cause of MSSV test failures as described above, and to take corrective
, actions to preclude repetition, appears to involve a violation of these requirements.
7.
Mana ement Interview (30703)
The inspectors met with licensee representatives (denoted in Paragraph 1)
on February 9, 1990, to discuss the scope and findings of the inspection.
In addition, the inspector also discussed the likely informational content of the inspection report with regard to documents or processes reviewed by the inspector during the inspection.
The licensee did not identify any such documents/processes as proprietary.
Inspection Report 50-315/90005(DRP):50-3]6/QAAAR(DPP)
ATTACHMENT Hydraulic Unit to a tmosph ere Valve Stem
)1 Dump (
Valves~
Upper Operating Volume Piston Drain Tube S.team Equal iza tion Port
. 3-Way
.Isolation Valve Slide Gate MAIN SYE M FLOW'~
VALVE MODIFICATIONS Drain Port Original: 0.125" Modified: 0. 197" (5mm)
Steam Port Original:
0 ~ 125" Modified: 0;394" (10mm)
isc D
C COOV.
MAIN STEAM STOP VALVE SCHEMATIC (TYPICAL)