ML20205P914
Jump to navigation
Jump to search
| ML20205P914 | |
| Person / Time | |
|---|---|
| Site: | Pilgrim |
| Issue date: | 05/16/1986 |
| From: | Kister H, Strosnider J NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML20205P911 | List: |
| References | |
| 50-293-86-17, CAL-86-10, NUDOCS 8605280047 | |
| Download: ML20205P914 (44) | |
See also: IR 05000293/1986017
Text
{{#Wiki_filter:- -- _ _
- .
.
.i .
U. S. NUCLEAR REGULATORY COMMISSION
AL'GMENTED INCIDENT RESPONSE TEAM
Report No. 50-293/86-17
Docket No'. 50-293
Licensee: Boston Edison Company M/C Nuclear
ATTN: Mr. William D. Harrington
Senior Vice President, Nuclear
800 Boylston Street ,
Boston, Massachusetts 02199
4
Facility Name: Pilgrim Nuclear Power Station
Inspection At: Plymouth, MA
Inspection Conducted: April 12, 1986 through April 25, 1986
Team Leader: J. Strosnider, Chief,
Section 18, DRP, RI
Team Members: L. Doerflein, Martin McBride, Senior
Project Engineer,RI Resident Inspector, Pilgrim
K. Murphy, R. Fuhrmeister
Technical Assistant, DRS,RI Reactor Engineer, RI
M. Chiramal, Section Chief, AE00 S. Pullani
Fire Protection Engineer,
- DRS, RI
Reviewed By .
/J/ Strosnider, Chief
LProj cts Section 1B, DRP
'
Approved By:
!
fi. Kistdfl Chief
Protects Branch No. 1, DRP
l r ,
8605280047 860516
PDR ADOCK 05000293
G PDR _
(J
-
.
.-
2
TABLE OF CONTENTS
Page
1.0 Introduction ............................................. 4
2.0 Summary of Events ........................................
-
5
2.1 Ap ri l 4, 1986 Reactor Scram. . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 April 12, 1986 Reactor Scram......................... 6
3.0 Evaluation of Inadvertent Closure of the MSIVs ........... 7
3.1 Background .......................................... 7
3.2 PCIS Trip Logic Circuit and MSIV Control Circuit
Designs ............................................. 8
3.3 Investigation ....................................... 9
3.4 Root Cause and Safety Significance ................. 12
3.5 Conclusions and Recommendations .................... 12
4.0 Evaluation of MSIV Problems ............................. 14
4.1 Chronology of Events ............................... 14
4.2 Valve Design and Operation ......................... 15
4.3 : Investigation ...................................... 16
4.4 Root Cause and Safety Significance ................. 18
4.5 Conclusions and Recommendations..................... 19
5.0 Evaluation of LPCI Injection Valve Leakage .............. 20
5.1 Chronology of Events .............. ................ 20
5 . 2. RHR Isolation Valve Descriptions ................... 21
5.3 Past System Leakage Experience ...... .............. 22
i. ! r
O ~
3
5.4 History of RHR Valve Refurbishment and Leak Testing... 22
5.5 As-Found RHR Walkdown and Valve Leakage Measurements.. 24
5.6 Root Cause and Safety Significance ................... 25
5.7 Conclusions and Recommendations ...................... 25
6.0 Overall Summary and Conclusions ....................... ... 27
Figures / Pictures
Attachments
_-
.
~
4
1. INTRODUCTION
On April 4 and 12, 1986, the Pilgrim reactor scrammed from low power
during routine reactor shutdowns. Both scrams were caused by unexpected
group I primary containment isolations. In both cases, the isolation
signal was promptly reset, but the four outboard main steam line
isolation valves (MSIVs) could not be promptly reopened. As a result,
the main condenser was not available as a heat sink during a portion of
- ~~
the reactor cooldown. The shutdown.on April lith was initiated because
the residual heat removal (RHR) system had been pressurized by leakage of
reactor coolant past a check valve and two closed injection valves in
the "B" RHR loop. An Unusual Event was declared because of the RHR
valve leakage.
NRC management discussed concerns about the recurring isolation and RHR
valve leakage problems with senior licensee management and issued Con-
firmatory Action Letter (CAL) No. 86-10 on April 12, 1986. This letter
required that all affected equipment be maintained in its as-found condi-
tion (except as necessary to maintain the plant in a safe shutdown con-
dition) until an NRC Augmented Inspection Team (AIT) was onsite to inspect
and reconstruct the events. The letter also required that the licensee
provide a written evaluation to the NRC cf 1) intersystem leakage through
RHR injection valves in the RHR system, 2) the spurious primary containment
isolation that occurred on April 12, and 3) the failure of the outboard
MSIVs to reopen after the isolation. The licensee agreed to seek authori-
zation for restart of the reactor from the Regional Administrator of NRC
Region I. The CAL is included in this report as Attachment 1. An AIT
was dispatched to the site on. April 12, 1986.
. ~
5 I
2.0 SUMMARY OF EVENTS
2.1 April 4, 1986 Reactor Scram
-At 1:00 p.m. on April 4, 1986, a reactor shutdown was initiated after
oil leakage was detected in the main turbine control oil system. The
low pressure coolant injection (LPCI) system was considered inoper-
able at that time due to an unrelated problem, water leakage past a
block valve, MD-1001-36A, in the residual heat removal system torus
cooling line.
At 8:15 p.m. on April 4, a group I primary containment isolation
(resulting in a reactor scram) occurred as reactor pressure decreased
to 898 psig in the shutdown sequence. The two low main steam line
pressure alarms (set to approximately 880 psig) were received at the
time of the isolation. The reactor mode switch had been moved from
the "run" to the "startup" position 45 minutes prior to the isolation.
The low steam line pressure containment isolation function is active
in the run mode but is bypassed when the mode switch is placed in the
startup mode.
The containment isolation signal was promptly reset following the
scram, however, the outboard MSIV's could not be reopened for
approximately one and a half hours. The inboard MSIV's were opened
during that time period. As a result of the closed MSIVs, most of
.the subsequent reactor cooldown was controlled by directing reactor
steam to the high pressure coolant injection (HPCI) turbine. The
HPCI system was operated in the test mode and dio not inject water
into the reactor.
During the review of this event the licensee concluded that all the
contacts in the reactor mode switch did not close properly when the
switch was transferred from the run to the startup mode during the
shutdown. As a result, the low pressure containment isolation func-
tion was still active when steam line pressure dropped below the
trip setpoint (about 880 psig). The licensee determined that proper
positioning of the mode switch required removing the Key from the
switch each time it was moved to a different mode. Training for all
control room operators on proper mode switch operation was conducted
prior to the subsequent reactor startup. Additional details of the
licensee's evaluation of the inadvertent closure of the MSIVS are
discussed in Section 3.0 of this report.
The licensee also concluded that an air leak in the "A" outboard
MSIV, A0-203-2A, (coupled with repeated attempts to open the valves)
probably lowered air pressure to the four outboard valves, preventing
them from fully opening. The air leak was attributed to foreign
materials in the MSIV pneumatic control valve. Additional details
of the licensee's evaluation of the problem with the MSIVs failing to
open upon demand and corrective actions are discussed in Section 4.0
of this report.
. ~
6
The evaluations of the Mode Switch and MSIV problems were reviewed
by the Operational Review Committee (0RC) on April 8, 1986. The
reactor was restarted at 2:46 a.m. on April 10, 1986.
2.2 April 12,1986 Reactor Scram
Periodic RHR system high pressure alarms (400 psig) were received on
April 10 and 11, indicating that the RHR system was being pressurized
by reactor coolant leakage. The RHR piping in the "B" loop was warm,
indicating the leakage was coming through the normally closed injec-
tion valve, M0-1001-298, and an inline check valve, 1001-688. At
2:16 p.m. on April 11, a second "B" loop injection valve, MO-1001-28B,
was closed in the RHR system in an attempt to stop the leakage. The
low pressure coolant injection (LPCI) subsystem of the RHR system
was declared inoperable at that time. However, leakage continued
into the RHR system causing a high pressure alarm two and a half
hours later. At 4:53 p.m. on April 11, 1986, a reactor shutdown
was initiated from about 92*s power and an unusual event was
declared due to the leaking valves.
At 1:56 a.m. on April 12, a group-one primary containment isolation
(with an associated reactor scram) occurred during the shutdown se-
quence. Reactor pressure was 908 psig at the time of the isolation.
The mode switch had been mcved from the "run" to the "startup" posi-
tion and the key removed from the mode switch twenty minutes earlier,
at 1:36 a.m. The isolation and scram occurred about 30 seconds after
the two main steam line low pressure alarms annunciated in the con-
trol room.
As before, the outboard MSIVs could not be opened for approximately
one and a half hours after the isolation signal was reset and the
HPCI system (in the test mode) was used to cool the reactor. The
reactor was placed in cold shutdown and the unusual event terminated <
at 9:08 a.m. on April 12, 1986.
- . .- -. . - - -- - . .- -
,
-
i
~
- 7
i
i
*
3.0 Evaluation of Inadvertent Closure of the Main Steam Isolation Valves
:
.
'
Following the scram on April 12, 1986, the licensee promptly organized a
team consisting of approximately 14 technical and support personnel to
investigate potential failures of the reactor mode switch (RMS), other
<
potential problems in the PCIS circuitry, and operator errors which could
have contributed to this event. The scope of the investigation included
a thorough analysis of previous events and included trouble shooting
'
plans, procedures and special tests. Members of the NRC Augmented Inspec-
tion Team (AIT) monitored the activities of the licensee team and assessed
the operational anomalies that occurred in the PCIS circuitry.
~~~ --~~
3.1 Background ,
<
On April 4 and April 12, 1986, while shutting down the reactor, the ,
- Pilgrim unit experienced a reactor trip due to inadvertent closure
of all eight main steam isolation valves (MSIVs). On both occasions -
the reactor mode switch was in the "Startup/ Hot Standby" position and
the inadvertent closure of the MSIVs occurred after the operators
'
received alarms indicating main steam line pressure was less than 880
psig. During the April 4th event, the reactor scram due to MSIV
closure occurred almost immediately following the alarm; while on
April 12, the scram apparently occurred 30 to 40 seconds after the
alarms came in.
' 1 j Following investigation and analysis of the April 4th event, the
- licensee had concluded that the cause of inadvertent closure of the
!
MSIVs and subsequent reactor scram was due to failure of some
l contacts of the reactor mode switch. The contacts in question are , in the primary containment isolation system (PCIS) logic channel
circuits and are designed to inhibit the actuation of the trip
i circuits on a low steam line pressure condition. That is, i the mode switch contacts, when the mode switch is in any position
other than "Run", bypass the low steam line pressure trip of the
PCIS.
4 l Based on testing of a spare mode switch the licensee also determined
- that, as a means of assuring that the mode switch contacts function
properly, the operators should remove the key from the mode switch
'
- handle after the switch is operated. The mode switch key can be re-
moved from the handle only if the switch is aligned fully in one of
the four required positions, i.e., the key cannot be removed if the
switch is in an intermediate position. All operators were trained on
proper mode switch operation, using the spare mode switch, prior to
the reactor startup on April 10, 1986.
'
On April 12, 1986, while shutting down, the mode switch was moved
j from the "Run" position to the "Startup/ Hot Standby" position and
the key was removed from the handle. However, 30 to 40 seconds
- following the expected alarms indicating steam line low pressure,
i
_ _ _ . __ _______ . _ - - -
_ _ _ _ _ _ _ _ _ _- . . _ - . _ .
.
-
3
the reactor scrammed due to the unexpected closure of the MSIVs.
Once again the reactor mode switch contacts in the PCIS trip logic
channel circuits associated with the MSIVs were suspected to have
caused the inadvertent closure of the MSIVs.
3.2 PCIS Trip Logic Circuit and MSIV Control Circuit Designs
3.2.1 PCIS Trip Logic Circuit
_
-~~~
The PCIS trip logic scheme consists of four trip logic
channels (designated A1, A2, B1, and B2) arranged in a
one-out-of-two taken twice logic (i.e., Al or A2 and 81 or
B2) to cause a trip. Figure 3.1 is an elementary diagram
showing the trip logic channel Al of the PCIS for the MSIVs,
main steam line drain valves and reactor water sample
valves. When the reactor mode switch is in the "Run" mode,
the following conditions will cause the actuation of the
PCIS trip logic channels (i.e., deenergization of relay
16A-K7A, B, C, and D):
(1) Main steam line low pressure (<880 psig)
(2) Low low reactor water level
(3) Main steam line high radiation
(4) Main steam line high flow
(5) Main steam tunnel high temperature
These are referred to as isolation conditions 1, 2, 3, 4,
or 5 in the discussion that follows.
.
When the mode switch is in other than the "Run" mode (i.e.,
shutdown, refuel or Startup/ Hot Standby), a main steam
line low pressure condition will not cause the actuation of
the PCIS trip logic channels. This feature enables the
MSIVs to remain open while the reactor pressure is less
than 880 psig during a normal reactor startup. However, a
high reactor water level condition during these modes
(i.e., other than "Run") will cause the actuation of-the
PCIS trip logic channels.
As stated before, the actuation of the PCIS trip logic
means deenergization of relays 16A-K7A, B, C and D.
Contacts of these relays, arranged in a one-out-of-two
taken twice logic, actuate the MSIV control circuits
discussed below and close the MSIVs.
_
. .
~
9
3.2.2 MSIV Control Circuit
Each MSIV is controlled by two, three-way, direct acting,
solenoid valves; one powered by 120 VAC and the other'120
VDC. The MSIV pilot system is arranged so that when one or
both solenoid valves are energized, normal air supply pro-
vides pneumatic pressure to an air operated pilot valve
which in turn directs air pressure to the MSIV valve opera-
tor so that the MSIV can be opened against the action of
the spring. When both the solenoids are deenergized by a 3
PCIS trip logic actuation or a manual closure signal, the
air pressure is directed to the opposite side of the valve
operator piston which along with action of the spring
closes the MSIV.
3.3 Investigation
Investigation revealed that the reactor scrams which occurred on
April 4 and 12, 1986 were initiated by the actuation of the Reactor
Protection System (RPS) due to closure of the Main Steam Isolation
Valves (MSIVs). The closure of the MSIVs was initiated by the PCIS
trip logic circuitry discussed in Section 3.2.1.
On both the April 4 and 12, 1986 events, it was initially determined
that the only PCIS signal present at the time of the isolation was
main steam line low pressure. On both occasions, the reactor mode
switch was in the "Startup/ Hot Standby" position and the reactor
pressure was being reduced below 880 psig during the controlled cool
down of the reactor. The PCIS trip signal from the four main steam
line low pressure switches (261-30A through D) should have been
inhibited by the previously performed operator action of transferring
the mode switch from the "Run" position to the "Startup/ Hot Standby"
position.
The reactor mode switch is a pistol grip, key locked, four position
control switch. The four positions are: " Shutdown", " Refuel",
"Startup/ Hot Standby", and "Run" (see Figure 3.2). The switch is
made up of four banks of General Electric Model SB-1 rotary control
switches (see Figure 3.3), having 8 stages per bank (i.e.,16 sets
of cam operated contacts per bank). The banks are coupled together
by gears. The pistol grip handle is attached to the second bank
from the left hand side of the switch.
Reactor mode switch malfunctions causing problems of a similar nature
have occurred at several other nuclear plants and were the subject of
IE Information Notice 83-43. Pilgrim had experienced a problem with
the mode switch in 1983 (Reference ORC Meeting Minutes 84-104 and
Failure & Malfunction Report 83-133). General Electric Information
Letters (SIL) Number 155 and its supplements 1 & 2; and SIL 397 dis-
cuss instances of failure of "SB" model switches and recommend
actions to be taken.
i 4
. ~
10
Previously, in accordance with SIL 397, during Refuel Outage VI, an
SB-9 model mode switch was bought and tested. The SB-9 mode switch
was a used unit rebuilt by GE. Testing was also performed on a S.8-1
model switch. Both switches functioned properly and each had a
definite ' feel' during a transfer operation. The SB-1 required a
specific technique be used to ensure proper align. ment of contacts
while the SB-9 operated in a stiff and hard manner.
Following this testing, Operations personnel visited the test site
and familiarized themselves with the feel and technique used to
properly transfer the existing SB-1 switch. This familiarization
~~~--
reduced the concern for the proper operation of the SB-1 Mode Switch.
This experience coupled with the knowledge that the new SB-9 switch
operated in a stiff and hard manner and the extensive time required
to change out and post-work test the replacement switch contributed
to a subsequent licensee decision to continue operation with the
existing SB-1 Mode Switch.
3.3.1 Analysis and Evaluation of the April 4, 1986 Event
During the shutdown on April 4th, the mcde switch was trans-
ferred by an operator-in-training under direct supervision
of the Nuclear Watch Engineer. The watch engineer " wiggled" -
the mode switch to " feel" that it was in the right position.
The mode switch key was not removed from the switen handle
following the transfer.
The operator who had transferred the mode switch in the
control room prior to tne April 4, 1986 scram, had not :
been trained on the SB-1 Mcdel Switch at Pilgrim and had
no previous experience with it. Even though tne watch '
engineer checked the position of the mode switch, it is
possible that the switch was not actually in the correct .'
position because the key was not removed (as a positive
verification of proper positioning) after this trantfer. -
In retrospect, inadequate training of the operatcr could :
have contributed to the event. -
Following this event and in accordance with the recommenda-
tions in SIL 155, an inspe: tion of the Reactor Mode Switch
was conducted at Pilgrim on April 5, 1986. No ir.dication
of cracking or broken contacts or of any other adverse con- ;
dition was observed. Examination did indicate that proper '
preloading of the switch contacts existed.
In summary, the licensee concluded that the most probable !
root cause of this event was that at laast two of the tode ,
switch contacts (10, 26, 42 & 58) did not close or remain +
closed after the mode switch was transferred from "Run" to
"Startup/ Hot Standby". Corrective actions taken as a result
.
l
I
i
>
,e aw-~.wewe,e . e w e -
-.
. .
11
4
.
of this event included development of a prescribed technique i
for transferring the modo switch and training of Operations i
personnel in its application. ;
3.3.2 Analysis _andEvaluationoftheApril 1_2, 1986 Event
As discussed earlier in this report, the containment isola- ,
tion and reactor scram on April 12, 1986 were similar to
the April 4th event. However, on April 12, the tcram oc-
curred 30 to 40 seconds after the main steam line low
~~ -"~~
pressure alarms cane in. Also, on April 12, the transfer
was performed by an experienced operator and the mede '
switch key was removed.
The licensee investigation team's cyaluation of possible
means by which the MSIVs could close was comprehensive. It
considered loss of instrument air, failure of the MSIV's AC !
and DC solenoid valves, loss of AC and DC control power, ,
simultaneous actuation of MSIV test switches or associated +
circuits, operation of MSIV hand switches, failure of
relays associated with the MSIV close circuit, and the PCIS
logic circuits. The team analyzed available event data,
interviewed plant operators, reviewed past history for
similar events, performed adaitional functional tests and i
calibration tests, conducted special tests and conducted
walk-downs of the associated systems.
The NRC inspectors reviewed test documents to assess their i
technical adequacy, evaluated the safety consequences of l
these actlyities, and analyzed the test results to ascer- i
tain that the components functioned as intended. No signi- -
ficant problems were identified. Attachment 3 lists the ;
- . tests reviewed and performed as of April 26, 1986.
a . 4' ; Ouring the performance of one test, surveillance test
- 8.M.1-19, an unanticipated closure of the MSIVs occurred. !
After the initial round of tests and analyses the licensee
decided that the inadvertent closure of the MSIVs was due '
*
'
to actuation of the PCIS trip logic circuits. Based on the
results of the tests conducted, it was further concluded '
p that testing of the reactor mode switch was necessary. On i April 19, 1986, a special test of the switch was conducted. l The test involved monitoring of the mode switch contacts in
the suspect PCIS trip logic circuits and multiple operations ,
of the mode switch in its various positions. To consider
t the human factors aspect of the mode switch operation, [ several operators were used in the manipulation of the f switch. The switch was moved from the "Run" to "Startup" -
position approximately thirty tinies during this test.
, During this testing the contacts in the node switch were ,
instrumented in order to determine if they were opening
j. and closing properly. , 4-
e
+nw.we n
. - __
.
.
12
Review of the mode switch special test data showed that the -
moc:e switch contacts ia the PCIS trip circuits functioned
consistently as designed. It could also be concluded that,
discounting random failures, the mede switch was not the
root cause of the events of April 4 and April 12, 1986.
Following the mcJe switch test, the 11censee's team con-
centrated in identifying and testing fcr other potential
failu*es affectir.g at least two channels of the PCIS trip .
logic circuits. Possible causes such as icose wires and
tarminations, voltage surges on c7rcuit neutrals, ground
circuit anomalies, and wiring errors during the recent
replacement cf RPS and PCIS relays vere assessed through
te stir.g and ir.spection. Inis testing and inspectior did
not confl.vm the cause of the unanticipated containment
isolations.
3.4 hot Cause and Saf 1 tyjhnjfjcance
The licersee and its special teens are continuing their investigation
into the root cause of the inadverter.t closures of the MSIVs that
occurred on Aprfl 4 and April 17, 1985. No root cause for the un-
expected containment isolations riad been identified at the conclusion
of this inspection, although a . mode switch failure was suspected,
Until a root cause is established, the possibility that these inad-
vertent closures c0uld cccur 1.n any mode cf reactor operation cannot
be ruled out. The safety functicn of the main steam isolation vaives
is to close when needed to isolate the reactor primary systeA.
Although inadvertent cicsure of the MSIVs aligns the valves in their
safe configuration, such closures are of concern for the following
reasons:
1. Inadvertent closure can lead to a reactor trip, a turbine trip, *
and a loss of the normal heat sink and normal pressure control
of the reactor.
,
.
2. Closure could cause challenges to safety related systems such
as the main steam line safety and relief valves, the RPS, HPCI,
and RCIC. ,
3. Closure could result in increasing the stress level of the opera-
tors, as a result of the potential transients identified in
items 1 and 2 above.
3.5 Conclusions and Recommendations
The licensee has worked hard to determine the root cause of inadver-
tent closures of the MSIVs. However, the root cause or causes of
the problem have not been established as yet. Due to the concerns
raised by the inadvertent closures of MSIVs, the root cause should
I
- - -. .
.
'
13
be determined prior to restart of the unit or prior to operation
under conditions where an unanticipated containment isolation could
significantly challenge reactor safety systems or operators.
In addition, considering the important safety functions the mode
switch performt, its operation should not be subject to an operator's
" feel", or a prescribed technique for its transfer operation. The
licensee should continue to work on resolving these noted problems.
Licensee activities in this area will be evaluated in future
,_ . _ _ _ . -
inspections (86-17-01).
o i
.
14
4.0 EVALUATION OF MAIN STEAM ISOLATION VALVE (MSIV) PROBLEMS
This section discusses the failure of the outboard Primary Containment
Main Steam Isolation Valves (MSIVs) to open upon demand following the
reactor trips on April 4 and 12, 1986. For reference, a simplified
drawing of an MSIV and an enlarged drawing of the valve pilot poppet
assembly are included as figures 4.1 and 4.2 respectively. A list of
procedures and other documents reviewed is included in attachment 4.
4.1 Chronology of Events
On April 4, 1986 at 8:15 p.m., during a planned reactor shutdown,
the reactor tripped due to all eight MSIVs closing (Group I
Isolation). Following the trip, the operators reset the Grcup I
Isolation signal and attempted to open the outboard MSIVs. The MSIV
control switches were left in the open position for approximately one
minute. During this time, the operators observed both red (open) and
green (closed) position indication on the outboard MSIVs, however,
the valves did not go full open. When the operators placed the con-
trol switches to the closed position, they observed the valve indica-
tion went green (full closed) in less than one second. The inboard
MSIVs were then successfully cycled open and closed The High Pres-
sure Coolant Injection (HPCI) system was used in the full flow test
lineup to control reactor pressure which is the normal method of
pressure control if the MSIVs can't be opened. Approximately one and
a half hours after the MSIV isolation was received the outboard MSIVs
opened upon demand.
The licensee considered four possible causes for the failure of the
outboard MSIV's to reopen: 1) simultaneous mechanical binding of the
four outboard MSIV's, 2) excessive differential pressure across the
valves, 3) low instrument air pressure, and 4) loss of electrical
control power. During the followup investigation, the licensee dis-
covered a large air leak on the control system for the "A" outboard
MSIV which continuously ported the under piston area of the MSIV air
cylinder. During the repair of the air leak, debris (paper and
yellow plastic) was found lodged in the pneumatic four way valve.
Some of the pieces of paper were folded, indicating that they were
manually placed in the controller rather than blown in from the in-
strument air system. The entire air distribution manifold on the
"A" outboard MSIV (last disassembled during the 1984 outage) was
removed for cleaning. Inspections for debris were also performed on
the air distribution manifolds of the "A", "B" and "C" outboard MSIVs
as well as the "A", "C" and "D" inboard MSIVs with negative results.
The "0" outboard and "B" inboard MSIVs were not inspected as they had
recently been worked on. The licensee's evaluation of the source of
the debris was ongoing during the AIT and will be examined during a
future it'spection (86-17-02).
- - .. .. -. _ - - - --. .
r
.
*
15
Following the inspections of the MSIt air system, testing was perform-
ed to determine if reduced air pressure would preclude the MSIVs
from achieving full stroke. The test results indicated that approxi-
mately 40 psig supply pressure would open the MSIV one half inch,
resulting in both red and green valve position indication and that
full valve stroke could not be achieved when normal supply pressure
was introduced slowly to the air cylinder. As no other problems were
identified duri :g the followup investigation, the licensee concluded
that the failure of the outboard MSIVs to open upon demand was most
likely caused by a lowered cylinder air supply pressure due to the
leak on the "A" outboard MSIV. The reactor was restarted on April 10,
1986.
On April 12, 1986 at 1:56 am, during another planned shutdown, the
reactor tripped due to all MSIVs closing. Approximately four and a
half minutes after the MSIV closure, the operators reset the Group I
Isolation signal and attempted to open the outboard MSIVs. As during
the previous event, the MSIV control switches were left in the open
position for approximately one minute, operators observed both red
and green valve position indication, and the MSIVs failed to open.
The control switches for the outboard MSIVs were placed in the closed
position. Then with personnel stationed in the steam tunnel to ob-
serve MSIV stem movement, operators made several attempts to open
only the "A" and "C" outboard MSIVs. In one case the MSIV control
switch was left in the open position for approximately five minutes.
personnel in the steam tunnel reported that, during the attempts to
open the "A" and "C" outboard MSIVs, the valve stem would travel ap-
proximately one half inch and then stop. There was no sound of steam
flow when the MSIVs stroked the one half inch. It was also observed
that MSIV air cylinder supply pressure was normal.
Again, as during the April 4, 1986 event, the operators were able to
open the inboard MSIVs (which were left open) and HPCI was used to
control reactor pressure. Approximately one and a half hours after
the Group I Isolation, the outboard MSIVs opened upon demand. The
operators noted that the differential pressure across the outboard
MSIVs was 30 psi when the valves were opened. Reactor pressure at
that time was approximately 310 psig.
4.2 Valve Design and Operation
Valve Design
The Main Steam Isolation valves, manufactured by Atwood and
Morrill Company Inc., are 20 inch globe valves having a "Y"
pattern body. The valves have a cylindrical main disc (poppet)
moving in a centerline 45 degrees upward from the axis of the
horizontal main steam inlet line. An air cylinder is utilized
to operate the isolation valve. Air for the outboard valves
and air or nitrogen for the inboard valves is used to open the
. -
16
valve while springs and/or air (nitrogen) close the valve. The
air cylinder is capable of opening the MSIV with the design
differential pressure of 200 psi across the main poppet. The
MSIV also contains an internal pilot valve whose seat is in the
middle of the main poppet. The pilot valve provides a means of
balancing the pressure across the main poppet, just before the
main poppet is lifted and while it is off its seat. The first
three quarters of an inch stem travel only opens the pilot
poppet after which the main poppet is lifted of." its seat. The
total MSIV stem travel from full close to full open is nine and
one half inches.
Due to a history of problems with leak tightness and two valve
stem failures in 1978 and 1982, the licensee modified all eight
MSIVs during the sixth refueling outage, which ended in December
1983. These modifications included: new main poppets with an
elongated poppet nose to position the poppet in a proper seating
position; increasing stem diameter and fillet radius on the
backseat surface; addition of main poppet anti-rotation devices;
and addition of self-aligning pilot poppets.
Valve Operation
Opening MSIVs with the reactor pressurized, such as following a
Group I Isolation, is described by procedure. Basically the
sequence requires that all the , outboards MSIVs be opened first
to allow trapped condensation to drain. The outboard valves
should open after the pilot poppet reduces the differential
pressure across the main poppet to within 200 psi. The steam
line drain valves (numbers MOV 220-1, MOV 220-2 and MOV 220-3)
are then opened to equalize pressure across the inboard MSIVs.
When the differential pressure across the inboard MSIVs is
within 50 psi (administrative limit), as measured between
reactor pressure and main steam pressure upstream of the turbine
stop valves, the inboard MSIVs are opened and the drain valves
are shut.
4.3 Investigation
Following the MSIV isolation and reactor trip on April 12, 1986, the
licensee formed a multi-disciplined team to investigate and determine
the cause of the outboard MSIV failure to open upon demand. Activi-
ties of the team were observed by the NRC inspectors who found that
the evaluation team performed a detailed review and analysis of the
MSIV problem. Actions taken by the team included: bringing a valve
vendor representative onsite to review valve characteristics; operator
interviews; review of surveillance test data; review of all previous
trip reports for similar events; system walkdowns; identification and
discussions of potential causes; and contact with the Institute of
Nuclear Power Operations and other utilities to identify similar
occurrences at other facilities.
. _ .
.
*
17
,
The evaluation team identified the following seven possible causes
for the failure of the MSIVs to reopen: electrical failure; air
supply problems; all pilot poppets broken off; insufficient time
allowed by operator for area above main poppet to bleed off; inboard
MSIVs leaking so that the differential pressure across outboard
'
.
M3IVs could not be reduced to less than 200 psi; mechanical binding
of main poppet; and mechanical binding of pilot poppet. Based on
system walkdowns, functional tests, etc., the team concluded that ,
the most probable cause of the outboard MSIVs failure to open upon
'
- _
demand was the pilot poppet becoming detached from the valve stem.
Prior to the sixth refueling outage, during which the MSIVs were
modified, the pilot valve was an integral part of the MSIV stem. No
cases were found, prior to this outage, where the MSIVs could not be
opened following an isolation with the reactor pressurized. Follow-
ing the modifications and plant;startup in December 1983, only three
MSIV isolations occurred with the reactor pressurized. Two of the
three were the events of April 4 and 12, 1986 during which the out-
board MSIVs would not open upon demand. .The third event occurred
during a planned shutdown on June 15, 1985. However, in this case
- no attempt was made to reopen the MSIVs.
The modification to the MSIV pilot valve involved installation of a
" floating" pilot poppet. The design was intended to provide a
laterally floating pilot poppet to improve leakage characteristics
and reduce MSIV stem bending stresses. As seen in Figure 4.2, the
pilot poppet is attached to the stem by tnreading tae poppet onto the
pilot poppet nut which is held on the stem by the split retaining
ring installed in the stem groove. A set screw is installed and
>
staked into the pilot poppet to prevent the poppet from unscrewing
itself from the pilot poppet nut.
The evaluation team developed a test to verify their conclusion that
the pilot poppet had become disconnected from the stem. The test
'
consisted of pressurizing the volume between a pair of MSIVs to 23
psig and then slowly increasing the air supply pressure to the out-
board MSIV air cylinder to slowly open the valve. Expected results
would be that within the first three quarters of an inch stem travel
the pilot poppet should lift and depressurize the volume between the
MSIVs. After three quarters of an inch stem travel (the limit of
pilot poppet travel) the main poppet would open to depressurize the
volume between the MSIVs. The inspector reviewed the test procedure
'
to verify it was technically adequate and approved by the Operations
Review Committee. In addition, the inspector observed the test per-
formed on the "A" outboard MSIV. The results of this test clearly
indicated that the~ pilot poppet was not attached to the valve stem.
Similar tests were run on the remaining outboard MSIVs and, although
the results were not as definitive, there were indications the pilot
poppets were not opening as soon as expected.
I
. . . . _ _ . --.. , -_ _ __ . _ _ - . _ _ _ . _ _ . _ ,
_ .- .-.- ..
'
.
*
18
Based on the test results, the licensee disassembled all eight MSIVs
for inspection. The results of these inspections were: on two MSIVs
("A" outboard and "C" inboard) the pilot poppets were detached from
the valve stem; on the "D" outboard MSIV the pilot poppet became de-
tached during MSIV disassembly; three other pilot poppets ("0"
inboard, "B" and "C" outboard) had started to unscrew themselves from
the pilot poppet nut and exhibited 3/8 to 5/16 of an inch axial play;
-and the remaining two MSIVs ("A" and "B" inboards) had the pilot
poppet fully engaged to the pilot poppet nut. In those cases where
the pilot poppet had started to unscrew itself, the threads on the
-
poppet and nut were damaged.
Prior to disassembly the licensee also performed Local Leak Rate
Testing (LLRT) of all MSIVs. The results of the LLRT are included
in the following table. Leakage rates are in standard liters per
minute (sim).
MSIV Leakage
"A" inboard (IA) 44.5 slm
"A" outboard (2A) 5.5 slm
"B' inboard (IB) 23.2 slm
"B" outboard (28) 2.8 slm
"C" inboard (1C) 4.03 slm
"C" outboard (2C) 0.47 slm -
"D" inboard (1D) 33.5 slm
"D" outboard (20) 8.5 slm
The inspector noted that the Technical Specification limit for valve
leakage is 5.43 sim. However, the inspector also noted that the
measured leak rates were significantly lower than those from the two
previous LLRTs.
4.4 Root Cause and Safety Significance
The cause of the outboard MSIV failure to open upon demand was the
pilot poppets b u ming detached from the valve stem or inhibited
from fully opening so that the differential pressure across the main
'
poppet would prevent the MSIV air cylinder from opening the valve.
~
At the end of the AIT inspection the cause for the pilot poppets
becoming unscrewed and/or detached from the pilot poppet nut was
still under analysis by the licensee to determine if it was due to
> an installation error or design error. However, it was clear that ,
the set screw did not prevent tne pilot poppets from unscrewing from
the pilot poppet nut.
Subsequent to the AIT inspection the licensee concluded that the
lack of positive set screw engagement was due to an inadequate
,
installation procedure coupled with the absence of a torque
^
requirement between the pilot poppet and poppet nut allowing imposed
rotational / vibrational forces to unscrew these assemblies.
- - - .
_ - - - , - _ -- .__ _ _. _ _ _ _ _ _
_
I%
.
I
v '
'
'
.
1_9
.
Analysis by the lidensee is on going' to ensure that, with the problems
- identified, the MSIVs met the safety design basis as stated in the
Final Safety Analysis Report. However, the safety objectives of the
MSIVs are to close to limit the loss of reactor coolant and limit
the release of radioactive materials. The design of the valve is
such thatJapparently even a detached pilot poppet cannot become dis-
lodged and prevent the MSIV from fulfilling the safety cbjective.
This was reinforced by the LLRT results. Nonetheless, failure of
the valves to reopen did result in using a safety system to control
reactor pressure and temperature and presented a.dditional challenges
to the reactor operators. I'n, addition, based on this event and on
,
reports from other facilities, there may be generic safety implica-
.
tions with regard to the use of set screws.
4.5 Conclusions and Recommendations
-The MSIV evaluation team did a thorough job in identifying the cause
. sor the MSIVs failing to open on demand. Based on the observations
r
and testing performed during the,first event of April 4, 2986, the
inspector could not fault the licensee for not identifying the prob-
lem then. ,Also, based on the LLRT results, it appears that the MSIV
modifications have significantly reduced the valve leakage problems
roted/prvviously.
,
The licensee, sbould continue the root cause analysis, to identify why
the set screws did.not prevent the MSIV pilot poppets from unscrewing
off the-poppet nut.in order that a perma.nent fix can be implemented.
The corrective actions including proposed design changes will be
evaluated when they'are available (86-17-03).
:
,
I
e
1
a s , , , - , - .:,, . ~ - - -
- . - - - . .
.
*
20
5.0 EVALUATION OF LPCI INJECTION VALVE LEAKAGE
-5.1 Chronology of Events
Tables 5.1, 5.2, and 5.3 summarize the chronology of events signifi-
cant to the RHR valve leakage question. The events begin with the
pulling of control rods on April 10, 1986 and end with the securing
from the Unusual Event (declared as a result of RHR valve leakage) on
April 12, 1986.
~
At about 10:00 a.m. on April 10 after reactor pressure was increased
'-
to about 500 psig, the "B" RHR Rosemount flow transmitter indicated
an increase in pressure was occurring in the RHR system. Normal
system pressure is 105 psig controlled by the keepfill system via a
connection from the condensate transfer system. Though no pressure
indicator exists for the segment of piping in question, the flow
transmitter was determined to be pressure sensitive; as pressure in-
creases the "B" RHR flow indicator is driven negative. This anomaly
was substantiated by the inspector by observation, discussion with
control room operators, and by a contact made with the meter manufac-
turer, Rosemount, Inc. The flow transmitter reading is charted in
the control room and thus a permanent record exists that records all
of the pressurization events that have occurred. At approximately
11:00 a.m. the "RHR Discharge or Shutdown Cooling Suction High
Pressure" alarm (hereafter referred to as RHR Hi alarms) sounded,
indicating pressure of about 400 psig. This alarm had been frequent-
ly received in the past. The alarm response procedure requires the
operator to diagnose the source of the leakage and to depressurize
the system by opening valves that lead to the tcrus. A number of
closely spaced RHR Hi alarms were subsequently received, approximately
once every fifteen minutes. In the afternoon the alarms continued
to be received, approximately once every half hour. During this time
the unit was placed on the line as the normal startup continued.
During the afternoon and into the night maintenance personnel at-
tempted to control the RHR leakage by increasing the closing torque
on Valve 298, this valve was diagnosed as the leaky valve because the
outboard piping was hot to the touch. The valve torque was increased
three or four times, each time the second MOV (288) was closed and
-29B opened then torqued closed, after which 28B was reopened. Ad-
justing-the closure torque to its highest allowable design value had
no affect on valve leakage.
At 2:15 p.m. on Friday, April 11, a decision was made by the operating
staff to close valve 28B in an attempt to stop the leakage. This
required that the plant enter a seven day LCO. Several hours after
the closure of 28B the RHR Hi alarm again sounded indicating that the
leakage continued. As both the MOV's appeared to leak the possibility
of violating containment integrity forced the operating staff to de-
clare an Unusual Event and start a slow, controlled shutdown of the
plant. The shutdown was subsequently speeded up after discussions
with NRC and by the next morning the reactor was in cold shutdown.
. __ ., _ __
.
'
21
5.2 RHR Isolation Valve Descriptions
Three valves form the isolation barrier between the high pressure
reactor coolant and the low pressure piping of each of the two RHR
loops. Figure 5.1 shows a schematic of the RHR loop B isolation
valves. The three isolation valves are:
-
Valve 688
A Rockwell, 900 lb., 18" testable, tilting disc, 316ss, check
valve. The valve has been modified by the removal of the post-
tion indication and air cylinder that provided the testability
feature. Thus, the valve is now a simple check valve. The
valve was once required to undergo containment leak testing but
was taken off the Appendix J list via an exemption granted by
NRC in 1977.
-
Valve 29B
A 600 lb.,18" x 14", 316ss, gate valve rated for 1250 psig at
586 degree F. The valve is operated by a limitorque motor opera-
tor controlled from the control room. The valve will automati-
cally open in the event of a LOCA in combination with a reactor
pressure less than 400 psig. The valve is interlocked with
valve 288 preventing the inadvertent opening of both valves when
reactor pressure is greater than 400 psig. The valve is required
to undergo local leak rate testing as part of the containment
integrity program.
-
Valve 28B
A 600 lb., 18", 316ss, globe valve with a plug type disc,
rated for 1250 psig at 586 degrees F. This valve is operated
by a limitorque motor operator controlled from the control
room. As with 29B,_it is sent an automatic open signal in the
event of a LOCA in combination with a reactor pressure of less
than 400 psig. The valve can be throttled by manual control
room operation for flow control purposes. The valve is required
to undergo local leak rate testing as part of the containment
integrity program (it has replaced valve 68B as the second iso-
lation valve for the purpose of containment integrity).
Design requires one of the two motor operated valves be closed
during normal standby. Valves 28A and 28B of the two loops were the
original valves to be kept closed. In early 1986, leakage past 28B
began causing RHR Hi alarms. At that time, the valve was judged to
,
have remained within LLRT leakage limits by trending of past leak
tests, but to eliminate the RHR Hi alarms it was decided to operate
with 29B as the closed valve. This change took place on February
26, 1986. The A loop valves were left as is with 28A being the
closed MOV in that loop.
. - - .-
..
*
22
5.3 Past System Leakage Experience
The RHR Hi' alarm indicative of RHR isolation valve leakage has been
received in the past. The most recent period in which RHR Hi alarms
were received prior to April 10 began on January 10, 1986 and con-
tinued through February 26, 1986 (the day that Valve 29B replaced
valve 288 as the normally closed valve). During this period the
average time between RHR Hi alarms was about 10 hours. From
February 27, 1986 through March 8, 1986 no alarms were received.
The reactor was then shutdown until April 10 as a result of leaks
found in the Head Spray and Reactor Level Instrumentation systems.
The inspector made a bounding calculation in attempt to estimate a
conservative value of the amount of leakage from the valves.by ob-
taining the time and quantity of torus water that had been trans-
ferred to the rad waste tanks for processing. Between January 10
and February 27 a total of 81,500 gallons of torus water had been
pumped. Since the reactor was at pressure for 47 days during this
period the estimated leak rate of between one and two gpm is calcu-
lated. As there are other sources of water draining to the torus,
e.g., HPCI turbine pump exhaust, this estimate should be considered
as a high bound for the leak rate. Also this calculation assumes
that torus level at the beginning and end of the period was the
same. Though there is a chance of error in this calculaticn, it
providet some evidence that the isolation valve leak rate was not
substantial during the January 10 - February 27 time period.
5.4 History of RHR Vaive Refurbishment and Leak Testing
5.4.1 Check Valve 68B
Valve 68B was disassembled and rebuilt in May-June 1984.
The inspector reviewed the rebuilding documentation and
interviewed the engineer and technicians that worked on
the valve. In addition, the valve design was reviewed to
judge if the internal components provided reliable support
of the disc. The visual inspections upon disassembly in-
dicated that the valve internals were acceptable, other
than a light lapping of the valve seating surfaces no
component degradation was noted. The cover nuts and bolts
were found not to be acceptable and were replaced. As the
valve is no longer on the Appendix J valve list no leak
test was performed. Only a visual bluing technique was
used to assure that the seating surfaces were in full
contact. The fact that the valves as-found condition was-
generally acceptable and that the design of the valve disc
and hinge is substantial, with little chance of misposition-
ing of the disc, provided evidence to the inspector the
- Valve 68B would reliably perform its closure function. As
will be discussed in Section 5.5, a significant pressure
.
differential, which forces the seating surfaces together,
. ..
.
23
is required to tightly seat the valve. However, without a
pressure differential, as is the normal case with one of
the MOVs closed, the check valve is likely to provide no
additional resistance to leakage flow.
5.4.2 Globe' Valve 288
In early 1986, this valve was diagnosed as leaking. At
that time it was predicted that the valve leak rate was
~-~ - ~ -
still within the 7.89 standard liters per minute (sla)
Appendix J limit for a single penetration. The local leak
rate testing of Valve 28B is as follows:
1980 - As found 0.1 sim, as left 0.2 slm
1982 - As found 0.5 sim, as left 0.5 slm
1984 - As found 1.9 sim, as left 1.9 slm
Utility staff calculated that (assuming an exponential
trend) a valve leak of 4.8 slm would be predicted for early
1986. It was then concluded that the valve, though leaking,
was still acceptable. No other mechanical problem with the
valve was identified with one exception; electrical main-
tenance personnel had found that the closing amperages of
the valve were not initially repeatable. This led to the
disassembly and inspection of the motor operator, no
abnormalities were found.
,
5.4.3 Gate Valve 298
This valve has a history of failing the local leak rate
testing (LLRT). The valve failed its LLRT on January 7,
1984 and again on October 11, 1984. Based on an interview
with the valve maintenance contractor representative, the
valve has a design deficiency that makes it difficult to
. '
maintain low leakage over a long period of use. The
distance between the bottom of the valve wedge and the
bottom of the valve housing is only 3/16 inch. As the
seating surfaces wear the' wedge must drop lower and with
enough wear will bottom out on the housing. During last
November the wedge was removed and the seating surfaces
built-up and ground smooth. The post maintenance LLRT
done on November 29, 1984 resulted in zero leakage, so at
that time the valve was leaktight. As indicated
previously this valve replaced valve 28B as the normally
closed valve on February 26, 1986. The valve has been
considered for replacement, but no hard schedule exists.
i [
- , --.
.- .
24
5.5 As-Found RHR Walkdown and Valve Leak Measurements
During the period between April 13 and April 19 extensive RHR system
walkdowns and isolation valve leak measurements were conducted. NRC
inspectors observed these activities. The following summarize the
findings of these efforts.
5.5.1 System Walkdowns
An as-found visual inspection of the RHR "B" loop system
piping, components, and structural supports was planned.
The inspection was to determine if any adverse effects had
resulted from the isolation valve leakage or possible
water hammer events associated with depressurizing the
piping after the RHR Hi alarm annunciated or as a result
of recent events involving water hammer events of the head
spray piping. Evidence of overheating, overpressurization,
or piping / component movement was of most interest. The
utility staff's planning and conduct of the walkdown was
careful and detailed. Drywell, "B" RHR quadrant, and torus
room entries were made and potential defects for each pipe
segment, component, and support was documented. No defects
were identified which could be associated with thermal,
overpressurization, or component movement, thus it was
determined that no visual evidence existed suggesting any
adverse conditions as a result of the isolation valve
leakage.
5.5.2 Water Leak Measurements
A special water leak rate test was designed that would
simulate reactor water pressure on the reactor side of
valve 688. The test was conducted to determine the amount
of water leakage associated with the as-found Valve condi-
tion. Thus the leak rate of the check valve 688, the
closed gate valve 29B, and the closed globe valve 28B, in
series with one another, was to be determined. In addi-
tion, the test continued until pressurization of the RHR
piping was achieved and the RHR Hi alarm sounded in the
control room. Each segment of piping between the isolation
valves and between valve 28B and the RHR pump check valve
was monitored for pressure.
The test was conducted on April 17, 1986. The normally
locked open manual valve, 338, near the reactor was closed
and water pumped betweei it and valve 688. Table 5.4 sum-
marizes the test results as recorded by the NRC inspector.
The pressure between valves 33B and 68B was increased to
300 psig. The technician found it difficult to hold pres-
sure constant. At one point the pumping was stopped com-
pletely for several minutes and then varied between 10 and
_ . :-
-.-
25
20 strokes / minute. An air operated positive displacement
water pump was used. It became apparent to the inspector
that as the technician increased pressure, valve 688 became
an effective barrier until such time as the pressure dif-
ference across the valve equalized, after which the valve
had no affect. The pressure was increased to 600 psig and
then to 950 psig. It was held at 950 psig for 95 minutes
at which time the RHR Hi alarm sounded in the control room.
During_this period the pump flow rate, on average, was
_
about 1/2 gpm.
5.5.3 Appendix J Measurements
After the water test was concluded, the RHR piping was
drained and the air testing was conducted on April 18, 1986
for each of the three valves. Even though an LLRT is not
required for 68B an informational test was conducted. The
following are the LLRT results of the as-found valves:
688 - 76 standard liters per minute
298 - 1.0 standard liters per minute
288 - 1.5 standard liters per minute
5.6 Root Cause and Safety Significan e
The root cause of the RHR Hi pressure alarms was an approximate 1/2
gpm water leak past the loop "B" RHR isolation valves in conjunction
with apparently relatively leak tight RHR pump discharge check
valves. This condition caused a build up in pressure in the inter-
vening pipe segments to about 390 psig resulting in the alarm. There
was no indication that the potential for sudden failure of all three
isolation valves and resultant sudden overpressurization of the RHR
piping existed.
Prior to the decision to-shutdown, the operating staff could not know
the extent of isolation valve leakage and their decision to shutdown
was a good one. Though the low leak rate of the isolation valves
poses no safety problem, the inability of the operating staff to
determine significance due to instrument and procedural inadequacies
should be addressed. In addition greater utility attention should
be focused on the isolation function of valves that protect low
pressure ECCS systems from the high pressure reactor coolant.
5.7 Conclusions and Recommendations
The licensee did a thorough job in evaluating the LPCI injection valve
leakage and recurring RHR pressurization events. The low leak rates
which were measured do not pose a safety problem. However, continued
power operation with the recurring pressurization of the RHR piping
and the resultant RHR High Pressure Alarm is unsatisfactory because
1) the operator's attention is frequently drawn to an alarm that has
--
.: '
26
uncertain and undefined operational / safety significance, and 2) the
excessive cycling of the two safety related isolation MOVs (valves
348 and 368) used to vent the pressure to the suppression pool contri-
butes to premature wearout of these valves. The licensee should
eliminate the cause of the recurring pressurization of the RHR piping.
In addition, several areas were identified where improvements are
needed to. ensure the significance of similar events in the future can
be determined and/or minimized. These include: periodically verify-
ing that the LPCI injection check valve properly seats with a dif-
ferential pressure across the valve; installation of pressure monitor-
ing equipment on the RHR piping; and development of a method to quan-
titatively measure the LPCI injection valve _ leakage during reactor
operation. Licensee activities in this area will be reviewed in
future inspections (86-17-04).
<
.
.
27
6.0 OVERALL SUMMARY AND CONCLUSIONS
The AIT reviewed three recent operational problems at Pilgrim: 1) the
spurious group-one primary containment isolation on April 4 and 12,1986,
2) the failure of the main steam line isolation valves to open after the.
1solations, and 3) recurring pressurization events in the residual heat
removal (RHR) system.
The team noted that the licensee's problem solving approaches were
carefully structured and appeared thorough. In addition, the team drew
-~~
the following conclusions for the three areas of concern:
--
No root causes for the spurious primary containment isolations on
April 4 and 12, 1986 were identified during the. inspection period,
despite considerable licensee effort. The team did not identify any
weaknesses in the licensee's problem solving approach.
--
The failure of the outboard main steam line isolation valves (MSIV)
to re-open following the containment isolations on April 4 and 12
was caused by partial or complete mechanical separation of the valve
pilot poppets from the MSIV valve stem assemblies. Pilot poppet set
screws did not prevent the poppets from unscrewing from the stem
assemblies.
--
The RHR pressurization events reflect slow leakage (about 0.5 gpm)
past a check valve and two motor operated injection valves in the
"B" RHR loop. Lack of RHR pressure instrumentation and the lack of
periodic tests of the RHR injection check valves inhibit a more
thorough diagnosis. No apparent RHR valve failure mechanism has
been identified as the reason for this leakage.
--
The licensee's conduct of the reactor shutdown on April 11 and 12,
1986, was prudent in light of the recurring RHR pressurization
events.
The licensee's root cause evaluations were not completed and corrective
actions were not finalized during the AIT inspection. NRC review of
these actions should be conducted prior to startup from this outage.
Based on the AIT review, the first four items in CAL No. 86-10 have been
completed. The fifth and final item will be closed when the licensee
submits a written report on the three areas of concern to the Regional
Administrator and the Administrator authorizes reactor restart.
. . -
23
TABLE 5.1 - EVENTS OF THURSDAY APRIL 10, 1986
Plant RHR System
Time Conditions Conditions Comments
0246 Started pulling RHR in standby with Reactor
rods cross connect open startup
between A & B loops. begins
Pressure at-105 psig
provided by keepfull
system.
0345 Critical
0700 300 psig
11% steam flow
1000 500 psig RHR flow chart in-
12% steam flow dication showing
pressure rise in RHR
piping.
1100 660 psig RHR Hi alarm; 6 alarms First alarm
12% steam flow once every 15 mins. indicated on RHR
flow chart, no log
entry.
1200 >900 psig Reactor at
12% steam flow pressure.
1300 Turbine Rolling RHR Hi alarm; 4 alarms,
once-every 30 mins.
1336 Unit on Line
1500 STA log -
indicates look-
ing into valve
'29B leakage
1600 NWE Log (1600 to
2400) -
maintenance is
torquing up
valve 29B
1800 RHR Hi alarm; 4 alarms,
once every 30 mins.
_. _
.
*
29
Plant RHR System
Time Conditions Conditions Comments
2200 No RHR Hi alarms
between 2200 and
0200
2400 Reactor near
100's steam flow
_
. -_- .- ._ - _.
. . _ - . . - - . . - - .
. _. . _ . -_ ._.__. _ .. . . _ _ __.. .._ _ .__ . __
l
l
30
TABLE 5.2 - EVENTS OF FRIDAY APRIL 11, 1986
Plant RHR System
Time Conditions Conditions Comments
0200 RHR flow test Checks opera-
bility of all
four RHR pumps
0219' RHR Hi alarm First notation
of RHR Hi alarm
found in control
room log
0315 RHR Hi alarm
0336 "B" RHR loop in torus Pressurization
cooling mode of RHR pre-
vented when loop-
open to torus
1115' RHR secured from
torus cooling
1158 RHR Hi alarm
1415 RHR Hi alarm; valve Declared LPCI
288 closed,'both loop "B" inoperative
MOVs (28B & 24B)
no closed
1653 RHR Hi alarm
1710 Initiated a con- Declared an
trolled shutdown Unusual Event,
steam flow decrease notified NRC
rate of 5% per hour
2000 960 psig, steam
flow decrease rate
increased to 30%
per hour
2200. 930 psig, 33%
steam flow
2215 RHR Hi alarm Notified NRC
.
*
31
TABLE 5.3 - EVENTS OF SATURDAY, April 12, 1986
Plant RHR System
Time Conditions Conditions Comments
0030 Turbine off line
_ _ _ __.
0136 Out of run mode
0200 HPCI in recir- Initiate torus cooling
culation mode mode of RHR
for reactor
pressure control
0215 Significant
pressure
reduction begins
0400 <100 psig
0645 Out of torus cooling
RHR loop A placed in
shutdown cooling mode
0908 Reactor Secured from
<212 degrees F Unusual Event
O
~
32
Table 5.4
Summary of Water Leak Test Data Recorded By Inspector
April 17, 1986
Approximate
Pump Strokes Pressure Between Valves, PSIG
Per Minute
~--
- Time 338/68B 688/298 298/28B 28B/ Pumps Comments
~ 1500 0 22 25 65 104
~ 1510 0-20 300-500 290 100 104 5 min after
reaching 300
~ 1520 0-20 600-700 575 330 145 10 min after
reaching 600
1540 0-20 950 - - -
1606 4-8 950 950 700 185
1715 4.75 975 975 725 375 to 380 RHR Hi Alarm
received
Note: 4.75 pump strokes per minute is equivalent to ~ gpm.
-
33
FIGURE 3.1
-
PCIS INITIATION LOGIC FOR CHANNEL A-1
(Typical of Channels A-2, B-1 and B-2)
~
h
l -
i
<
d . 16 A - K4 A (.OPEN ON .-
5 A-S1 ( REACTOF. MODE
V
M SL Lo Priss,4980 psi swi7cp : ray eAss
$ STM, LINT LO. PR TRIP;
* O PE W IN"RUN"
$ MODE ONLY)
'
W
, I 6 A - K I A ( OPErd ON \ 16 A-k !9 A ( O PFN ON M'
$ LO LO RX. WTR. LEVEL) Rx. W ATER
LE\E L #\
I ,
IGA- R44 A ( OPEN ON } ZZ
[
t
MSL Nl RAD.) J
,
16 A - k A COPEN Orl )
t'
; HSL Hi T E M P.) b '
>
o . 16 A - K 3 A (OPEN Or]
9 MSL H! FLOW)
16 A - x '7 A ( 6 ROUP 1 PCI.5
INITI ATioN CHANNEL A.i RELAY
-
,
C -
:
.
blO TE : RELAYS ARE NOR>1 ALLY C!)ERGl~Ely i BUT SHCw A) It)
'DE E!, ER G l2 ED (SIDEL F) Co t3 DI T ION "
. _ _ _
-
34
~
FIGURE 3.2
REACTOR MODE SWITCH
! . - -
REACTOR MODE I
--
-
.-
'
.
X REFUEL
STA R T* {
?
$44
0
g
l HO T S TBY
.
m
9
\ /
l
'
.
*
ca -RUN
l 'L , ,
.-
.
i
.
I
i .
o
._._.-._. _ -- . _ _ _ .-. . . _ . _ . _ - _ - _ - _ _ _ _ _ _ _ _ _ _ _ _
. 35
i
FIGURE 3.3 !
l .
GENERAL ELECTRIC SB-1 MODEL CONTROL SWITCH l
l
l
l
l l
'
l I I
l
l
i
.
vuiitacts numcerec for easy
*
Silver-to-silver contacts.
Circuit designation plate is
marked for easy identification -
of switch functions.
4' i
!
.
l
.
,
I Good selection of functional f l
handles. ,
I
f
l l l
Front support spaces body of
f switch % inch from rear of the
panel, allowing ample rocm for
inserting leads into the switch.
Escutcheon plates of per.
manent-finish molded material Protective cover (not shown) f
'
are neat in appearance and completely covers all live
uniform in size. parts, meets NEMA 1 require-
ments for panel mounted
80407s3 switches. i
Type SD-1 Switch With Cover Removed
! L
. 36
FIGURE 4.1
,
MAIN STEAM ISOLATION VALVE
.
I 1
I T
l *
(?
o
d"
,E
-
-
M[ a;
AIR CYLINDER
N .. s
hp}? V HYORAULIC
N
'
, DASH POT
~
HEllCAL $PRINC1
5PmhG Gu1DE
'
! SPEED COPITROL VALVF
ACTUATOR SUPPORT AND
h[ k;, ; $PRING CUl0E $ HAFT
SPRING SEAT WEW9ER
-
-
_
/ ST EW PACKING
GL
l l
'Jh LEAK OFF CONNECTION
BONNET BOLT 5
CLE ARA NCE BONNET
b ,
PILOT SPRING ~
j~Q R p(gg
_ "_
x t=)- /
Ni ; 1. -
\ : *
)
%
t ? /I? / POPPI t(PLUG WAIN 0isk) .
/ ~
>
uAiN vatvt 5f at
'
,
/
\ Pit 0i 1( At
Pot ot
'
/
,
L
-
.
37
.
FIGURE 4.2
. MSIV pit.0T POPPET ASSEMBLY ,
l
.
m- ]
.
*
i
I
. ) {,
.
- ,
.s
F7 >
$
h jl }-l .
-
}
-
l 'o :s-
g% k x. '
l
TI
,
A
: N
:
~
n h . \'N . '
!@
~ l
l
!AMl,}yn[,T
NW
'; k I
w-
i
"-
n:^4
t)k.g -
..,
_ ,,/,
g' g - - . . . . . .
'
,
,
, . ;,;,
,.
,
.
< ,
j
,. 3' n,.s s
,.
.. 9l _.. lg'/d
j
.< i ,
i
i : t
,,
.e .. w o- ,
~,
k
3': .
l'1
: Gr l -{1' ! (.. ,
,
, :
! e-~ .
4
&
! %
ty
f'f5
:
'l
:
er I
J9
L
*
i-
1
g
! [1
G I
.O
I
s
g= - STEM
-
M .
l
.
efy JB <
k's'
"
- '
a. i [Q %:
#
sE+s
'
. .K.( g' ~3
I , N, _
,__.
j.',s PILOT POPPET
s
gsgj L/ 'g'W # .
,
' NUT
SCREW ; gw ._ p%(% .
~
' s' ,
=
'
I 4' '
t ._-..
n
ig'2sa$f9....tti4 '4 '1 8$$$.$
i
.
gMMkW
SPLIT RING MAIN POPPET
PILOT P0PPET
< 1
38
.
FIGURE 5.1
. SIMPLIFIED DIAGRAM 0F RHR LOOP B
.
.,
'
To
t*Yuna_s ^
2EE(To,? SfRAT
9tEnde ro oreNS ' ~N0
#
sg u el 5'Y# 788
$ oNc6
-nb
l
.ns #8
'A N
*/013
RECstC
vr
Cross ~ Tor ,
To
'# #
N Loo ? 'h'
$(-nB
Mots: M L L. J4Lur NuessR S
ART P RFCE 2, cc thy " loof ** [#M MNfatR
A$'
*tS O '191 &
.. kua
7e R w ', -(,n E- /:9
292 2%R
fumP i Pump
y .o- ) 'B
Stok
sum,
gg MS /G C001143%
SutTrov
-
- -
- 7B
L
,. ' **..,' UNITCl3 S T AT cs
** ',<
[ '* ,
( NUCt.l AH REGUI.ATORY COMMISSION ATTACHMENT 1
-
, j' ncGION I
hJ1 F ARK AVLNUI
_, 5; p
*s f KING 08 Puusst A. PENNSYL VANIA 194(4
....+
April 12,1986
CAL No.: 86-10
Docket Nu@ce: 50-293
Bosten Edison Company M/C Nuclear
ATTN: Mr. William D. Harrington
Senior Vice President, Nuclear
800 Boylston Street
Boston, Massachusetts 02199
Gent.lcmen:
Subject: Confirmation of Actions to be Taken with Regard to the Pilgrim
Plant Events Which Occurred on April 11-12, 1986
Pursuant to our telephone conversation on April 12, 1986 with Mr. Oxsen it is
our understanding that you have taken or will take the following actions:
1. Maintain all af fected equipment related to the events which occurred
on April 11-12, 1926 in its as-found condition-(except as
nu essary to maintain the plant in a r,afe >liutdown t.undition) In
order to preserve any evidence which would be needed to inspect
or reconstruct the events.
2. Deveinp troubleshooting plans and procedures and provide those to
the NRC Augmented Inspection Team (Ali) for their review and
comment prior to initiating any troubleshooting of the affected
equipment.
3. Advise the AIT leader prior to the conduct of any troubleshooting
ar,tiv ities. *
4. Make available to the NRC AII relevant written material related to
previous problems with the affected equipment.
5. Provide a written report to the ftegional Administrator prior to
restart that contains your evaluatlon of the following:
a. Intersystem leakage through the motor-operated injection
valves (including the check valve) of the residual heat
removal system;
'
b. The primary containment isolation which occurred daring
shutdown af ter the reactor mode switch was repositioned
from the run mode to the startup mode;
f _
r I r1 ~d Il
h b y ~/ ' I
i
*
2
.
..
l
C. The failure of the outboard main steam isolation valves' to
reopen after resetting the primary containment isolation
signal.
This report should include the underlying causes for the above
noted events, an assessment of their relationship to previous events
including the events of April 4, 1985, corrective actions taken and
your basis for restart, including the criteria used and your analyses
associated with these criteria.
Further we understand that restart will not occur until you receive authoriza-
tion from the Regional Administrator.
If your understanding of the actions to be taken are different than those
described above, please contact this of fice within 24 hours of the receipt of
this letter.
Thank you for your cooperation.
Sincerely,
.
Thomas E. Hurley
Regional Administrator
cc: L. Oxsen, Vice President, Nuclear Operations
C. J. Mathis, Station Manager
Joanne Shotwell, Assistant Attorney General
Paul Levy, Chairman, Department of Public Utilities
Plymouth Board of Selectmen
Plymouth Civil Defense Director
Senator Edward P. Kirby
Public Document Room (PDR)
local Public Document Room (LPDR)
Nuclear Safety Information Center (MSIC)
NRC Resident Inspector
Commonwealth of Massachusetts (2)
'
.
A , o
ATTACHMENT 2
PERSONS CONTACTED
The following is a partial listing of the licensee personnel that were
contacted during the inspection.
W. Harrington, Senior Vice President, Nuclear
L. Oxsen, Vice President, Nuclear Operations (Senior Licensee Manager Present
at the Exit Interview)
C. Mathis, Nuclear Operations Manager
P. Mastrangelo, Chief Operating Engineer
K. Roberts, Director Outage Management
N. Brosee, Maintenance Section Head
T. Sowdon, Radiological Section Head
J. Seery, Technical Section Head
E. Ziemianski Management Services Section Head
S. Wollman, On-Site Safety and Performance Group Leader
R. Sherry, Chief Maintenance Engineer
E. Graham, Compliance and Administrative Group Leader
P. Smith, Chief Technical Engineer
W. Clancy, Nuclear Engineer, FS and MC Group Leader
T. McLoughlin, Nuclear Operations Sr. Electrical Engineer
A. Morisi, Operations Assistant to Director of Outage Management
.
,
o
ATTACHMENT 3
Tests / Checks Performed During Mode Switch /PCIS Investigation
The licensee performed the following tests / checks of tne PCIS components,
including the reactor mode switch. The mode switch testing was performed
in all four mode positions under various human factor scenarios i.e.,
with and without key removed, pulling up or pushing down while turning
_ _
the mode switch, etc.
-
Surveillance Test Procedure 8.M.2-1.5.3.1, 2, 3, and 4 Primary Con-
tainment Isolation Logic Channel Test - Channels A-1, A-2, A-3, A-4,
respectively Revision 6; performed on April 14, 1986.
-
Inspection of contacts of the PCIS relays in Channels A-1, A-2, B-1
and B-2, in accordance with Procedure 3.M.3-8, Inspection / Trouble
Shooting - Electrical Circuits, Revision 6, performed on April 14,
1986, along with the above 4 PCIS Logic Tests.
-
Surveillance Test Procedure 8.M.1-19, Reactor Water Level (RPS/PCIS),
Revision 13; performed on April 15, 1986. (While performing this
test, an inadvertent closure of the MSIVs and steam line drain valve
M0-220-2 occurred)
-
Trouble Shooting Procedure for.the investigation of inadvertent
closure of MSIVs and M0-220-2 during performance of the above Sur-
veillance Test Procedure (8.M.1-19) on April 15, 1986; performed in
accordance with procedure 3.M.3-8 on April 15, 1986.
-
Surveillance Test Procedure 8.M.2-1.4.4, Main Steam Line Low Pressure,
Revision 5, performed on April 16, 1986.
-
Trouble Shooting Procedure to check out the AC and DC solenoid circuits
of the MSIVs, performed on April 17, 1986.
-
Temporary Procedure TP86-59, Mode Switch Test for Steam Line Low
Pressure Bypass, Revision 0; performed on April 19, 1986.
-
Trouble shooting procedure 3.M.3-8 to check out the effect of vibra-
tion on reactor vessel level Yarway level indicating switches;
performed on April 21, 1986.
-
Trouble shooting procedure 3.M.3-8 to confirm the vibration effect
observed during the above test; performed on April 21, 1986.
-
Trouble shooting procedure 8.M.1-19 to investigate the cross charnel
interaction of relays suspected during the performance of the above
two' tests; performed on April 21, 1986.
O e 0
-
Trouble shooting procedure 3.M.3-8 to investigate the vibration / cross
channel interaction observed as the April 21, 1986 testing; performed
on April 23, 1986.
-
Trouble shooting procedure 3.M.3-8 to check out the contact resis-
tances of the relays in the PCIS trip circuitry, performed on
April 23, 1986.
-
Surveillance test procedure 8.M.2-1.4.3, Main Steam Line High Flow,
Revision 1; performed on April 24, 1986.
-
Surveillance Test Procedure 8.M.1-12, Main Steam Line High Radiation,
Revision 11; performed on April 24, 1986.
-
Temporary Procedure TP 86-68, Mode Switch Resistance, Revision 0;
performed on April 24, 1986.
-
Trouble shooting procedure 3.M.3-8 to check out loose wire in the
PCIS circuitry and the RPS grounding connection; performed on
April 24, 1986.
A
o-
t
O
ATTACHMENT 4
DOCUMENTS REVIEWED
Plant Design Change Request No. 83-48, "MSIV Refurbishment", dated
October 5, 1983
Atwood and Morrill Co. Inc., " Instruction Manual for 20" Main Steam
Isolation Valves".
- ~'
Procedure No. TP 86-61, "MSIV Plot disassociation Test", Revision 0,
dated April 17, 1986
Procedure No. 2.2.92, " Main Steam Line Isolation and Turbine
Bypass Valves", Revision 15, dated May 8, 1985
Procedure No. 8.7.4.4, "MSIV Trip", Revision 12, dated January 30, 1986
i
}}