ML20154S230
| ML20154S230 | |
| Person / Time | |
|---|---|
| Site: | Waterford  |
| Issue date: | 05/27/1988 |
| From: | Chamberlain D, Will Smith, Staker T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
| To: | |
| Shared Package | |
| ML20154S229 | List: |
| References | |
| 50-382-88-16, GL-87-12, NUDOCS 8806090004 | |
| Download: ML20154S230 (9) | |
See also: IR 05000382/1988016
Text
- - _ - _ _ - _ _ _ - _
.
.
l
, . ,
,
U. S. NUCLEAR REGUI ATORY COMMISSION
REGION IV
NRC Inspection Report: 50-382/88-16
License:
HPF-38
. Docket: '50-382
Licensee: Louisiana Power & Light Company (LP&L)
142 Delaronde Street
New Orleans, Louisiana 70174
Facility Name: Waterford Steam Electric Station, Unit 3
Inspection At: Taft, Louisiana
Inspection Conducted: May 12-20, 1988
Inspectors:
.
7/88
W. FV Smith, Senior 3esident In#pector
Date'
-
27
5
T.R.vStaker, Resident'Inspectnr/
Date
s
k22/N
Approved:
,
D. D. Chamberlain, Chief'
Date
Reactor Project Section A
hbNADO
O
82
-Q
. . . . . . .
.
_ - _ _ _ _
,
,
.,
.
.
.
>
,
,
2
. Inspection Summary .
Inspection Conducted May '2-20, 1988
1
(Report 50-382/88-16)
Area' Inspected: The events which occurred and actions taken by LP&L personnel
prior to, during and subsequent to the incident on May 12, 1988, when problems
with refueling water level instruments twice resulted in a possible loss of
Results:
A potential violation involving two examples of failure to implement
procedures was identified. Also a potential deviation. involving two examples
oof failure to implement commitments made in the licensee's response to Generic Letter 87-12.
I
'
,
_
-_- _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - - _ _ _ .
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ _
_ _ _ _ _ _ _ -
_ _ - _ _ _ _ - _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _
.
.
.
.
,
.
3
DETAILS
1.-
Persons Contacted
Principal Licensee Employees
- R..P. Barkhurst, Vice President, Nuclear Operations
- N. S. Carns, Plant Manager, Nuclear
'P. V. Prasankumar, Assistant Plant Manager, Technical Support
- D. F. Packer, Assistant Plant Manager, Operations and Maintenance
- T. P. Brennan, Nuclear Operations-Construction Manager
- J. R. McGaha, Manager of Nuclear Operations Engineering
- D. E. Baker, Manager of Events Analysis Reporting & Responses
- R. S. Starkey, Operations Superintendent
- L. W. Laughlin, Site Licensing Support Supervisor
- Present at exit interview.
In addition to the above personnel, the NRC inspectors held discussions
with various operations, engineering, technical support, maintenance, and
administrative members of the licensee's staff.
2.
Follow-up on Near Loss of Shutdown Cooling Event (93702)
Overview
On May 12,1988, Waterford-3 was in Operational Mode 6, with the reactor
shut down, depressurized, and at approximately 89 F.
The second refueling
had just been completed and the reactor Fead had been placed on the reactor
vessel.
Reactor Coolant Systrin (RCS) draining operations were underway to
ic.?r RCS leve1~ in support of planned work on the steam generators and
reactor coolant pump seals.
At about 6:15 a.m. and again shortly after 9:35 a.m. , the in-service
shutdown cooling pump, Low Pressure Safety Injectici "A" (LPSI-A),
exhibited signs of cavitation due to possible air intrusion or impending
loss of suction.
On both occasions, the operators apparently placed
LPSI-B in service before the signs of cavitation on LPSI-A had degraded to
the extent that a complete loss of shutdown cooling was experienced.
LPSI-B showed no signs of covitation.
During this plant condition,
Technical Specification 3.9.8.2 required both trains of shutdown cooling
to be operable with one tra1n in operation.
As the course of events progressed, the operators finally determined that
the water level in the RCS was below the center line of the hot leg, about
four feet lower than they thought. Therefore, the apparent cause of
cavitntion on LPSI-A was air vortexing where the shutdown cooling piping
ties in to the RCS hot leg. Failure to be in control of RCS level was
apparently caused by a series of personnel errors, training deficiencies
and procedure deficiencies which are described in detail below.
,
-
9
m
v
-
_ _ _ _ _ _ _ _ _-
.
.
.
-
.
i
.
4
l
'
Of particular significance is the fact that Waterford-3 had previously
experienced a total loss of shutdown cooling due to RCS level instrument
problems.
On July 14, 1986, shutdown cooling wa, lost while the plant was
in a similar condition for reactor coolant pump seal work. A series of
level indication problems, personnel errors and procedure deficiencies
l
resulted in a loss of control of RCS level.
The actual level was low
I
enough to cause air and/or steam binding of both LPSI pumps. By the time
LPSI-B was filled, vented, and restored to service, RCS hot leg
temperature had increased from 138 to 232
F.
The details of this
incident were reported in Licensee Event Report (LER)86-013, dated
August 13, 1986.
By September 30, 1986, corrective actions committed in
LER 86-015 were completed.
The causes of loss of shutdown cooling at that
time were identified as simultaneous draining of the RCS from two points
and erroneous level indication due to insufficient nitrogen pressure in
the pressurizer (the tygon tubing had collapsed).
Procedures were revised
at that time.
Details
The resident inspectors conducted a series of interviews, reviewed logs,
procedures, and other data related to the above incident to gain a
complete understanding of what occurred, why, and what actions were taken
by licensee management.
The inspectors also reviewed the licensee's
investigative results as reported to senior management.
Additionally, Generic Letter 87-12, dated July 19, 1987, cited numerous
cases where shutdown cooling had been lost on other plants, and it
directed that all affected licensees take appropriate steps to preclude
future occurrences. On September 21, 1987, LP&L responded to the generic
letter by describing the equipment and controls used to ensure maintenance
i
of reactor water level control and shutdown cooling at Waterford-3. Among
these, significant to the May 12, IN8, incident, was assurance that the
tygon tubing length of the Refueling L? vel Indication System (RLIS) would
>
be maintained to a minimum.
LP&L also committed to change the RCS
draindown procedure to visually inspect, prior to each drain down, RLIS
hose and tubing for anomolies which could affect the accuracy of level
indication. The response also connitted to install a permanent Refueling
Water Level Indication System (RWLIS) by the end of the second refueling
outage.
This system would not have the problems attendant to the soft,
temporary hoses used in the existing RLIS.
The RWLIS would have a wide
range of +12 to +48 feet mean sea level (MSL) and a narrow range of +12 to
+15 feet MSL indication in the control room with an annunciator.
By May 7, 1988, the new RWLIS was installed and flushed but not cor.nected
to the RCS. With the refueling canal flooded (and no need for the RLIS)
at about +30 feet MSL, the RLIS point of connection to the RCS at drain
valve RC-105 was disconnected so that the RWLIS could be permanently
connected to the RCS.
RC-105 was tagged shut in accordance with the
licensee's clearance procedure.
The PWLIS was then installed at RC-105.
The new system has a connection point to accommodate the RLIS hostc, should
it be desirable to use the RLIS as a backup. Tnis connection is several
.
.
.-
.
'.
,
,
5
g
feet away from RC-105, thus when the RLIS was reconnected, between 20 and
30 feet of excess length of RLIS hose remained. . Although the licensee's
response to GL 87-12 assured minimum length, the excess hose was not
removed nor did any of the licensee's procedures require it. This is the
first part of an apparent Deviation from the. licensee's response to
GL 67-12 which stated that RLIS tubing length is maintained to a minimum
(382/8816-01).
About four hours after RC-105 was tagged shut, the clearance was released.
The Shift Supervisor was informed that the RWLIS ?luid boundaries were
intact and the RLIS was reconnected. A Nuclear Auxiliary Operator (NA0)
walked down the new system but did not recognize the excess hose on the
RLIS as a condition with the potential of causing erroneous level
indication. The clearance was removed and RC-105 was reopened. At this
time, there was no procedure in the control room which addressed the new
RWLIS system operating requirements. The shift supervisor apparently did
not recognize that the RLIS was now connected through a system that could
be full .of air and not aligned for operation. Technical Specification 6.8.1.a requires written procedures to be established,
implemented, and maintained covering refueling equipment operation, and
draining and filling of the reactor vessel, as recommended in Appendix A
of Regulatory Guide 1.33, Revision ?, February 1978.
Placing the new
RWLIS in service in combination with the RLIS without a controlling
procedure is therefore an apparent violation of NRC regulations. This
carries significance in light of previous problems with RCS level control
(LER 86-015) and the theme of the licensee's response to GL 87-12 which
assured the NRC of careful and controlled handling of the RCS level
indicating systems (382/8816-02).
It was not until May 9, 1988, that the procedure controlling RWLIS and its
interface with the RLIS hoses appeared in the control room.
The setup and
operation of the RCS level indicating systems were implemented by
Revision 6 to Operating Procedure OP-1-003, "Reactor Coolant System Drain
Down." Attachment 8.4 of the new revision requires a backfill of the
tubing in the RWLIS to ensure air is removed from the system prior to
placing the RWLIS in service and appears to be a prerequisite to placing
the RLIS in service. As a side issue, the new revision was approved by
the Plant Manager on May 7,1988, and thus could probably have been made
available to the control room if it had been pursued.
On the morning of
May 8, 1988, the engineer responsible for the station modification which
installed the new RWLIS inspected the completed work.
He noticed the
excess RLIS tubing hanging down to the -11 feet MSL level in the
containment. He moved it to a grating at -4 feet MSL in a large loop to
accommodate the excess tubing. He apparently did not recognize the
'
potential of such a condition to cause erroneous level indications,
because no action was initiated to shorten the hose to minimum length.
Between the time the RWLIS and RLIS systems were placed in service on the
night of May 7, until May 12, 1988, the refueling canal was flooded to at
least +30 feet MSL. This means the level was about 10 feet above the
reactor vessel head flange.
.
.
.
.
,
,
'
6
On the morning of May 12, 1988, shortly af ter midnight, the operators
drained the refueling cavity to 19 feet 10 inches MSL in accordance with
Refueling Procedure RF-6-001, "Refueling Reactor Vessel Head and Internal
Installation" to accommodate installing the head. During this cvolution,
the RCS level could have been monitored by observation of the actual level
in the refueling canal. The reactor vessel head was then installed.
At 3:09 a.m., the operators comenced an RCS draindown from 19 feet
10 inches MSL to 13 feet 6 inches MSL to facilitate work on the steam
generators and reactor coolant pump seals. This was to be controlled
by Operating Procedure OP-1-003, Revision 6.
The operators apparently
presumed the RWLIS and RLIS were in service and failed to verify among
other things that'all of the filling and venting procedures for the RWLTS
were completed, as required by Section 6.4.1 of OP-1-003.
Section 6.4.1
also leads to an inspection of the RLIS tubing; however, if previously
done, the procedure does not specifically require another physical
inspection of the tubing.
LP&L's response to GL 87-12 committed to revise
OP-1-003 to require a visual inspection of the RLIS hose prior to each
draindown
.There was some controversy between the inspectors and licensee
personnel as to whether or not Section 6.4.1 of OP-1-003 specifically
requires such'al inspection prior to each draindown.
The inspectors
considered the commitment to be inadequately implemented.
This is
the second part of an apparent Deviation from the licensee's response to
GL 87-12 whicn stated that the above revision would be made (382/8816-01).
Failure to comply with the initial requirement of Section 6.4.1 of
OP-1-003 to verify filling and venting of the RWLIS and then proceeding
with the draindown is the second part of the apparent violation to
implement procedures covering draining and filling of the reactor vessel
as required by Technical Specification 6.8.1.a (382/8816-02).
At 4:55 a.m., the operators secured RCS draining due to disagreement
between the RWLIS, which was indicating 13.92 feet MSL and the RLIS was
indicating 18 feet 2 inches MSL.
The licensee stated that three
individuals walkej down the RLIS tubing and that they had even momentarily
disconnected the tubing from the pressurizer to ensure there was no vacuum
in the hose.
No problems were identified with the RLIS as a result of
those actions.
After contacting the engineer responsible for the RWLIS and determining
that the RWLIS reference leg should be dry, the Shif t Supervisor directed
the RWLIS reference leg to be drained.
The reference leg is the dry, low
pressure side of the detector and should be vented to the pressurizer.
The variable leg senses the head of water in the RCS.
Upon opening the
reference leg drain, RWLIS indicated 19.05 feet MSL while RLIS was
indicating 17 feet 11 inches MSL. With the knowledge that the RWLIS had
not been fully tested, the close agreement with the RLIS restored
confidence in the RLIS level indication. At 5:14 a.m., RCS draining was
resumed.
,.
,
_
_
_
_
-_
_.
- -_-_
_ _ - _
- _ _ _ _ _
.
.
.
.
.
.,
'7
reactor operator noted a slight
At about 6:00 a.m., an on-coming (LPSI-A) flow. By 6:13 a.m., LPSI-A
oscillation in shutdown cooling
appeared to start cavitating, so draining was again secured. An NA0 was
dispatched to the pump room to vent the pump. The NA0 reported the
presence of significant amounts of air in the pump. At 6:19 a.m., HPSI-A
was started to raise RCS level, per Off-Normal Operating Procedure OP-901-
046, "Loss of Shutdown Cooling." By 6:33, LPSI-B was placed in service
and then LPSI-A was secured.
No cavitation was experienced by LPSI-8, and
shutdown cooling flow was apparently not interrupted.
There was a sight
increase in RCS temperature from 89 to 92
T.
This could have been
caused in combination by the flow purtebations during LPSI-A cavitation
and by the fact that shutdown cooling flow had been reduced at about
3:50 a.m.
At 6:38 a.m. with shutdown cooling flow stable, RCS level was
at 17 feet 2 inches MSL by RLIS.
The HPSI pump was secured at that time.
Over the next two hours, the RWLIS and RLIS systems were again inspected,
and LPSI-A was recirculated and vented.
No sources of air intrusion were
found. The licensee stated that the operators were fully confident that
RLIS was indicating the correct RCS level.
At 8:56 a.m., LPSI-A was placed back in service for shutdown cooling.
LPSI-B was not fully secured.
It was left running in a recirculating mode
in anticipation of more problems with LPSI-A, because the source of air
intrusion was not yet resolved.
At 9:13 a.m. , RCS draining was recomenced. At 9:32 a.m., the draindown
was again secured because even though the RLIS had been considered
correct, it was not decreasing whereas RWLIS was. Within two minutes, the
NA0 stationed at LPSI-A reported cavitation again.
The operators shifted
promptly to LPSI-B and again raised RCS level with HPSI-A for about four
minutes. At this time, the engineer responsible for the RWLIS walked down
the RWLIS and RLIS again and found that the excess RLIS hose he had
previously placed on the grating at -4 feet MSL was now hanging over the
grating and down to the -11 feet MSL level. When he pulled the hose up
and started shaking it to see if any air was trapped, the RWLIS indicators
in the control room became erratic, and bubbles showed up in the RLIS
indicator.
Level dropped from +17 feet MSL to just above +13 feet MSL.
This was reported to the control room. The operators immediately started
HPSI-A until the RLIS indicated 13 feet 8 inches MSL, which agreed with
the RWLIS.
No further problems occurred.
During the second draining which commenced at 9:13 p.m., the Operations
Superintendent monitored operations in the control room due to the
uncertainties revealed when LPSI-A first cavitated.
This management
attention was provided to help ensure shutdown cooling was maintained.
Although licensee management claimed to have been sensitized to the
significance of reactor vessel level problems in view of LER 86-015 and
GL 87-12, implementation of corrective actions comitted in both the LER
and the response to GL 87-12 did not appear to be effective.
b
- .
.
.
-
.
,
8
Root Causes
The -licensee attributed the loss of RCS level indication to a
combination.of causes. The NRC inspectors-arrived at similar conclusions:
a.
Due.to a programatic breakdown in the process of turning over plant
modifications to operations, the new RWLIS was not properly placed in
operation thus effecting the attached RLIS.
This left the operators
with no reliable RCS level indication while draining,
b.
There was insufficient training of personnel expected to inspect and
evaluate functionability of the RLIS.
Consequently, obvious
deficiencies were not identified and corrected prior to draining
operations.
c.
Procedure OP-1-003, "Reactor Coolant System Drain Down" failed to
clearly implement all of the commitments in the licensee's response
to GL 87-12.
This is a reflection on the review conducted by the
Plant Operations Review Committee when the procedure was changed to
incorporate those commitments, as well as the performance of those
who initiated the change.
Corrective Actions
The licensee is in the process of determining what corrective actions must
be taken to. prevent RCS level problems and/or loss of shutdown cooling in
the future. The resident inspectors discussed tentative plans with
licensee management and determined that the items listed below are under
consideration. These actions would take place prior to the next draindown
of the RCS. As of the end of this inspection period, the plant was in
Mode 5, filled and vented, and pressurized to about 180 PSIA.
No further
draindowns were contemplated during this outage. Again, the items listed
below are under consideration by the licensee and are not to be construed
as a commitment at this time.
a.
Nuclear Operations Support Assessment (N0SA) will perform an
assessment of the Stucion Modification Program werall work flow
process.
Particular attention will be focussed on modifications that
could result in partial completion turnover to Operations.
b.
The Plant Operations Review Committee (PORC) will evaluate processing
and distribution of procedure revisions / changes specifically for
station modification implementation.
i
This event will be incorporat"d into requalification training and
c.
initial training for licensea and non-licensed operators.
j
d.
This event will be reviewed with the engineers responsible for
implementing station mdifications or design changes.
l
!
,
b
.
.
.
.
-
.
,
9
,
e.
Training for engineering personnel will reemphasize the importance of
operational impacts during the design and implementation'of a station
modification or design change,
f.
Procedure OP-1-003 will be revised to specifically address minimizing
the length of tubing of the RLIS during installation and subsequent
inspections,
g.
Procedure OP-1-003 will be revised to include RCS volumes for drain
down from refueling. This should include intact steam generators
and/or steam generators with nozzle dams installed.
h.
Operations Quality Assurance will perform an audit of SMP-138 to
verify adequate installation to date.
i.
Nuclear Operations Engineering will review the post modification
testing process.
J.
Modification Pre-implementation will be strengthened by placing
additional emphasis on attendance and content of the meetings.
k.
The licensee will review the response to GL 87-12 to ensure that all
commitments are implemented.
3.
Exit Interview
The inspection scope and findings were summarized on May 23, 1988, with
those persons irdicated in paragraph 1 above.
The licensee acknowledged
the NRC inspectors' findings. The licensee did not identify as
proprietary any of the material provided to or reviewed by the NRC
inspectors during this inspection.
,