ML17313A560
| ML17313A560 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 08/28/1998 |
| From: | Howell A NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
| To: | James M. Levine ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR |
| Shared Package | |
| ML17313A561 | List: |
| References | |
| 50-528-98-14, 50-529-98-14, 50-530-98-14, NUDOCS 9809090298 | |
| Download: ML17313A560 (77) | |
See also: IR 05000528/1998014
Text
UNITED STATES
NUCLEAR REGULATORY COMMISSION
~
REGION IV
6) ) RYAN PLAZADRIVE SUITE >00
ARLINGTON TEXAS 760)) 6064
August 28,
1998
EA 98-382
James
M. Levine, Senior Vice
President, Nuclear
Arizona Public Service Company
P.O. Box 53999
Phoenix, Arizona 85072-3999
SUBJECT: NRC INSPECTION REPORT 50-528/98-14; 50-529/98-14; 50-530/98-14
Dear Mr. Levine:
From May 26 through July 21, 1998, a special inspection was conducted at your Palo Verde
Nuclear Generating Station, Units 1, 2, and 3, reactor facilities in response to your identification
of degraded performance capabilities of the high-pressure safety injection system.
Additional
in-.office review was conducted following the onsite portion of the inspection and an exit meeting
was conducted on July 21, 1998. The enclosed report presents the scope and results of that
inspection.
Based on the results of the inspection, four apparent violations were identified and are being
considered for escalated enforcement action in accordance with the "General Statement of
Policy and Procedure for NRC Enforcement Actions" (Enforcement Policy), NUREG-1600,
Revision 1. These apparent violations involve: 1) one apparent violation with two examples of
Technical Specification 3.5.2 for operating Units 1 and 2 with an'inoperable high-pressure
safety injection system flowpath for greater than the allowed outage, time; 2) one apparent
violation with three examples of Technical Specification 3.0.3 for having both trains of the
high-pressure safety injection system (Units 1 and 2) inoperable without initiating actions to shut
down the plant; 3) one apparent violation with five examples of 10 CFR Part 50, Appendix B,
Criterion XVI,"Corrective Action," for not promptly identifying and not promptly
correcting'roblems
with the high-pressure safety injection pump discharge check valves; and 4) one
apparent violation with two examples of Technical Specification 6.8.1 for not making required
control room log entries.
Accordingly, no Notice of Violation is presently being issued for these
inspection findings. In addition, please be advised that the number and characterization of
apparent violations described in the enclosed report may change as a result of further NRC
review.
An open predecisional enforcement conference to discuss these apparent violations has been
scheduled for September
14, 1998, at 9 a.m. (CDT) in the Region IV office. The decision to
hold a predecisional enforcement conference does not mean that the NRC has determined that
violations have occurred or that enforcement action will be taken.
This conference is being held
ADQCK 05000528
1
J'.
Arizona Public Service Company
-2-
to obtain additional information to enable the NRC to make an enforcement decision, such as a
common understanding of the facts, root causes,
missed opportunities to identify the apparent
violations sooner, corrective actions, significance of the issues, and the need for lasting and
effective corrective action.
The predecisional enforcement conference willprovide an opportunity for you to point out any
errors in our inspection report and for you to provide any information concerning your
perspectives on 1) the severity of the violations, 2) the application of the factors that the NRC
considers when it determines the amount of a civil penalty that may be assessed
in accordance
with Section VI.B.2 of the Enforcement Policy, and 3) any other application of the Enforcement
Policy to this case, including the exercise of discretion in accordance with Section Vll.
In addition, during the predecisional enforcement conference, we would like you to address:
1) the adequac'y of your operating experience assessment
program to identify the applicability
of operating experience and to properly assess
industry or in-house information on the
operation of your facilities; 2) the adequacy of your inservice testing program to determine
check valve operability; 3) efforts to improve operator logkeeping practices; 4) the evaluation of
the as-found degraded condition of the high-pressure safety injection system with respect to
conformance to the emergency core cooling system acceptance
criteria of 10 CFR 50.46; and
5) the justification that a failure of an emergency diesel generator to continue running at some
point during its mission time should not be considered the most limiting single failure
consideration.
With respect to item 4), to the extent that this evaluation used assumptions and
inputs that were different from those that were in effect prior to the discovery of the condition,
we request that you specifically identify these differences and discuss the bases for them.
You willbe advised by separate correspondence
of the results of our deliberations on this
matter.
No response
regarding these apparent violations is required at this time.
In accordance with 10 CFR 2.790 of the NRC's "Rules of Practice," a copy of this letter and its
enclosure willbe placed in the NRC Public Document Room (PDR).
Should you have any questions concerning this inspection, we willbe pleased to discuss them
with you.
Since rel,
Arthur T. Ho
II III Director
Division of Reactor Safety
Docket Nos.:
50-528; 50-529; 50-530
License Nos.: NPF-41; NPF-51; NPF-74
Arizona Public Service Company
~3-
Enclosure:
NRC Inspection Report
50-528/98-'I4; 50-529/98-14; 50-530/98-14
cc w/enclosure:
Mr. Steve Olea
Arizona Corporation Commission
1200 W. Washington Street
Phoenix, Arizona 85007
Douglas K. Porter, Senior Counsel
Southern California Edison Company
Law Department, Generation Resources
P.O. Box 800
Rosemead,
California 91770
Chairman
Maricopa County Board of Supervisors
301 W. Jefferson, 10th Floor
Phoenix, Arizona 85003
Aubrey V. Godwin, Director
Arizona Radiation Regulatory Agency
4814 South 40 Street
Phoenix, Arizona 85040
Angela K. Krainik, Manager
Nuclear Licensing
Arizona Public Service Company
P.O. Box 52034
Phoenix, Arizona 85072-2034
John C. Horne, Vice President
Power Supply
El Paso Electric Company
2025 N. Third Street, Suite 220
Phoenix, Arizona 85004
Terry Bassham,
Esq.
General Counsel
EI Paso Electric Company
123 W. Mills
El Paso, Texas
79901
Arizona Public Service Company
Mr. Robert Burt
Los Angeles Department of Water & Power
Southern California Public Power Authority
111 North Hope Street, Room 1255-B
Los Angeles, California 90051
Mr. David Summers
Public Service Company of New Mexico
Albuquerque, New Mexico 87102
Mr. Brian Katz
Southern California Edison Company
14300 Mesa Road, Drop D41-SONGS
San Clemente, California 92672
Mr. Robert Henry
Salt River Project
6504 East Thomas Road
Scottsdale, Arizona 85251
Arizona Public Service Company
-5-
E-Mail report to T. Frye (TJF)
E-Mail report to D. Lange (DJL)
E-Mail report to NRR Event Tracking System (IPAS)
E-Mail report to Document Control Desk (DOCDESK)
bcc to DCD (IE01)
bcc distrib. by RIV:
Regional Administrator
DRP Director
Branch Chief (DRP/D)
Senior Project Inspector (DRP/D)
Branch Chief (DRP/TSS)
W.L. Brown, RC
OE:EA File, MS: 7-H5
Resident Inspector
DRS-PSB
MIS System
RIV File
G.F. Sanborn, EO
J. Lieberman, OE, MS: 7-H5
DOCUMENT NAME: r:QpApv814rp.rib
To receive co
of document, Indicate Inbox: "C ~Co
without enclosures 'E" =Co
with enclosures
N = N
co
RIV:Rl:EB E Rl'PBD
E.
RLB
r/Imb
NSal ad
AC:EB
TStetk
6Sa
orn
C:PBD
PHarrel
D:DR
AT
ell III
08/ /9
08/"/9
08
/98
08/
98
0
98
OFFICIALRECORD COPY
Arizona Public Service Company
-5-
E-Mail report to T. Frye (TJF)
E-Mail report to D. Lange (DJL)
E-Mail report to NRR Event Tracking System (IPAS)
E-Mail report to Document Control Desk (DOCDESK)
bcc to DCD (IE01)
bcc distrib. by RIV:
Regional Administrator
DRP Director
Branch Chief (DRP/D)
Senior Project Inspector (DRP/D)
Branch Chief (DRP/TSS)
W.L. Brown, RC
OE:EA File; MS: 7-H5
Resident Inspector
DRS-PSB
MIS System
RIV File
G.F. Sanborn, EO
J. Lieberman, OE, MS: 7-H5
~ ti V<-<.
~
DOCUMENT NAME: rA pv'tpv814rp.rib
To receive co
of document, Indicate In box: "C = Co
without enclosures
E" = Co
with enclosures
N = No co
RIV:Rl:EB
E
RLB
r/Imb
08/ /9
b
Rl:PBD
E.
NSal ad
08/~/98
AC:EB
TStetk
08
/98
GSa
orn
0
98
C:PBD
PHarrel
08/
98
OFFICIAL RECORD, COPY
D:DR
AT
ell III
ENCLOSURE
U.S. NUCLEAR REGULATORYCOMMISSION
REGION IV
Docket Nos.:
License Nos.:
Report No.:
Licensee:
Facility:
Location:
Dates:
Inspectors:
Approved By:
50-528; 50-529; 50-530
I
50-528/98-14; 50-529/98-14; 50-530/98-14
Arizona Public Service Company
Palo Verde Nuclear Generating Station, Units 1, 2, and 3
5951 S. Wintersburg Road
Tonopah, Arizona
May 26 through July 21, 1998
R. Bywater, Reactor Inspector, Engineering Branch
N. Saigado, Resident Inspector
Thomas F. Stetka, Acting Chief, Engineering Branch
Division of Reactor Safety
ATIACHMENTS:
Attachment 1:
Supplemental Information
Attachment 2:
Simplified HPSI Flow Diagram
Attachment 3:
Degraded HPSI Flow Profile
-2-
EXECUTIVE SUMMARY
Palo Verde Nuclear Generating Station, Units 1, 2, and 3
NRC Inspection Report 50-528/98-14; 50-529/98-14; 50-530/98-14
This special inspection was conducted to review the safety and regulatory implications of
excessive high-pressure safety injection pump discharge check valve reverse leakage for
Units 1 and 2. The licensee identified and reported these conditions to the NRC in accordance
with 10 CFR Part 50.72 on May 14 and 15, 1998. The onsite portion of the inspection was
conducted during the weeks of May 25 and June 8, 1998. Additional in-office inspection was
conducted through July 21, 1998. The inspection also assessed
the licensee's evaluation of
the degraded conditions.
~oerations
Two examples of an apparent violation of Technical Specification 3.5.2 were identified
for inoperability of the Unit 1 Train "B" high-pressure safety injection flow path for
approximately 6 years and the Unit 2 Train "A"high-pressure safety injection flowpath
for approximately 5 years (Sections E1.1.b.2 and E1.1.b.4).
Two examples of an app'arent violation of Technical Specification 6.8.1 were identified
for inadequate logkeeping practices.
Abnormal conditions were not recorded in the Unit
2 control room logs when an unexpected safety injection tank level decrease
occurred
on October 10 and 28, 1997 (Section E1.1.b.1).
Two examples of an apparent violation of 10 CFR Part 50, Appendix B, Criterion XVI,
were identified for not identifying and correcting excessive reverse flowthrough
Valve 2PSIB-V405 following two Unit 2 safety injection tank level decrease
events on
October 10 and 28, 1997 (Section E1.1.b.1).
Units'
and 2 log entries on May 13 and 14, respectively, did not address operability
of the opposite-train high-pressure safety injection flow path when operability of
Valves 1PSIA-V404 and 2PSIB-V405 was in question (Section 01.1).
One example of an apparent violation of Technical Specification 3.0.3 was identified for
not initiating actions within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, to place Unit 1 in a mode in which Technical Specification 3.5.2 did not apply when the Train "A"high-pressure safety injection
system was declared inoperable because of a problem with Valve 1PSIA-V404
(Section 01.1).
A second and third example of an apparent violation of Technical Specification 3.0.3
were identified for performing online maintenance
on the Unit 1 Train "A"and Unit 2
Train "B" high-pressure safety injection systems without isolation from the opposite train,
in excess of 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />, while the associated
high-pressure safety injection pump
discharge check valves were inoperable (Section E1.2).
-3-
~
The licensee's investigation report was objective and provided a candid self-assessment
of its performance; however, it did not evaluate inspector-identified issues in the areas
of operations or online maintenance (Section E8.1).
Maintenance
A third example of an apparent violation of 10 CFR Part 50, Appendix B, Criterion XVI,
was identified. On April 9, 1998, the licensee missed an opportunity to correct the
inoperable condition of Valve 1PSIA-V404 when a personnel error was made during
maintenance,
resulting in the valve being reassembled
incorrectly and the excessive
reverse leakage not being corrected (Section E1.1.b.2).
~En ineerin
~
A fourth and fifth example of an apparent violation of 10 CFR Part 50, Appendix B,
Criterion XVI,were identified for inadequate corrective actions in the development of
maintenance and testing procedures following operating experience assessments
of
NRC Information Notices 88-70 and 89-62 (Section E1.4).
Re ort Details
Event S no sis
This event involved excessive reverse flow through high-pressure safety injection (HPSI) pump
discharge check valves in Units 1 and 2 caused by the incorrect assembly of the check valves.
The incorrect assembly of the check valves prevented the valves from properly seating during
reverse flowtest conditions. Therefore, the reverse leakage caused by an improperly seating
valve, could have caused reduced HPSI injection flowduring an accident
A simplified diagram of the HPSI system is depicted in Attachment 2. The HPSI system for
each unit is comprised of two trains, "A"and "B," and each train has a pump, which takes
suction from either the refueling water tank (RWT) or the containment recirculation sump. The
discharge of each pump passes through a check valve, designated xPSIA-V404 for the
Train "A"valve (where 'x's the unit designator) and xPSIB-V405 for the Train "B"valve. The
discharge from each pump is then separated
into four cold-leg injection lines, one for each of
the cold legs of the reactor coolant system (RCS), and a hot-leg injection line. Each cold-leg
injection line contains a motor-operated valve and the flowpath is combined with the opposite
HPSI train to form a single flowpath. Upon a safety injection actuation signal, the HPSI trains
are cross connected via the cold-leg injection lines.
If a HPSI pump is not operating and the
motor-operated valves for both trains are open, the HPSI pump discharge check valve for the
idle pump provides isolation from the operating train. This prevents diversion of HPSI flowfrom
the RCS to ensure that an adequate amount of flow is provided for emergency core cooling
~
during a loss-of-coolant accident (LOCA).
A check valve that was improperly assembled
could affect the leakage through the valve during
reverse flow conditions.
For example, referring to the simplified flowdiagram in Attachment 2, if
the "B" HPSI pump discharge check valve SIB-V405 were to fail to close completely under
reverse flow conditions, and the "B" HPSI pump was not running with the injection valves open,
part of the injection flowfrom the "A"HPSI pump would be directed back through HPSI Valves
SIA-UV617, -627, -637, -647 and SIB-UV616, -626, -636, -646. The flowwould then be
directed through the "B" HPSI pump mini-flowline Valves SIB-UV667 and SIB-659, and back to
the RWT. As stated previously, upon a safety injection actuation signal, the cold-leg injection
valves would open and remain open.
The HPSI pump discharge check valves are 4-inch, 1500-pound, bonnet-hung, pressure-seal
swing check valves manufactured by Borg-Warner.
On April 9, 1998, during a Unit 1 refueling
outage surveillance test, Valve 1PSIA-V404, the Unit 1, Train "A"HPSI pump discharge check
valve, failed to meet its acceptance
criterion for reverse flowduring a flow test. The cause of
the condition was determined to have been vertical misalignment of the valve disc within the
valve body, which caused the disc to become cocked and not seat properly. On May 7, 1998,
the licensee concluded that the amount of leakage resulted in a condition outside the design
basis of the facility.
Valve 1PSIA-V404 was repaired during the Unit 1 refueling outage and its post-maintenance
test was completed satisfactorily.
However, on May 13, 1998, with Unit 1 operating, the check
-5-
valve system engineer determined that the valve had been repaired incorrectly. Additional
testing confirmed that the valve was inoperable because of excessive reverse leakage.
The
licensee repaired the valve and returned the Unit 1 HPSI system to an operable condition on
May 15, 1998.
During its review of the applicability of the vertical misalignment issue to other HPSI pump
discharge check valves, the licensee performed non-intrusive measurements
of valve
dimensions and reviewed previous surveillance test data.
On May 14, 1998, the licensee
suspected that Valve 2PSIB-V405, the Unit 2, Train "B" HPSI pump discharge check valve was
also misaligned and testing confirmed that the valve had excessive reverse flow. The licensee
reworked the valve and returned the Unit 2 HPSI system to an operable condition on May 16,
1998. On June 5, 1998, they reported the condition to the NRC in Licensee Event Report
(LER) 50-528/-529/-530/98-006.
The reverse leakage conditions were masked prior to April 9, 1998, due to an inadequate
surveillance test.
Specifically, the surveillance test in use only measured the injection flow rate
to the reactor vessel under limited conditions and did not quantify the reverse flow leakage
through the check valves..
The NRC initiated this inspection to review the event, determine if the licensee had returned the
HPSI systems to an operable condition, and assess
the licensee's performance.
I 0 erations
01
Conduct Of Operations
01.1
Unit 1 and Unit 2- Technical S ecification TS 3.0.3 Late Ent
The inspectors reviewed events and circumstances associated with the initial logging of
a late entry into TS 3.0.3 for Unit 1 and Unit 2 when HPSI pump discharge check valves
were inoperable because of excessive reverse flow leakage.
Observations and Findin s
Unit 1
The Limiting Condition for Operation (LCO) for TS 3.5.2 requires that two emergency
core cooling system (ECCS) subsystems
shall be operable for each unit with each
subsystem comprised of an operable HPSI pump, an operable low-pressure safety
injection (LPSI) pump, and an independent operable flowpath.
If a HPSI pump
discharge check valve allows excessive reverse flow, then the opposite-train ECCS
subsystem does not contain an independent operable flowpath. When the HPSI
systems are interconnected via the cold-leg injection valves, the check valve is the
flowpath boundary.
An action requirement associated
-6-
subsystem to be inoperable for 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> prior to requiring the initiation of a plant
shutdown.
Technical Specification 3.0.3 requires if an LCO is not met, except as allowed by the
associated
aqtion requirement, that the licensee shall initiate action within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> to
place the unit in a mode in which the LCO is not applicable.
As documented
in the Unit 1 log for May 13, 1998, at 3 a.m. the licensee began a
planned Train "A"equipment maintenance outage and declared Train "A"equipment,
including the Train "A" HPSI system inoperable.
On May 13, 1998, at 1 p.m., the licensee conducted a meeting to discuss operability of
Valve 1PSIA-V404, the Unit 1 Train "A"HPSI pump discharge check valve. The
licensee had questioned whether the valve had been assembled correctly following
maintenance during the April 1998 refueling outage.
(For additional details, see Section
E1.1.b.3).
The licensee's reportability determination, dated May 21, 1998, documented
that at this meeting, operations personnel concluded that sufficient evidence existed to
suggest that the valve might not perform its intended function, and that the valve was
declared inoperable, pending testing.
J
Following the
1 p.m. meeting, the Unit 1 operators made an entry in the Unit 1 log at
2:32 p.m. that HPSI Train "A"was inoperable due to the potential that the internals for
Valve 1PSIA-V404 were misaligned (HPSI Train "A"was already inoperable.
This entry
explicitly identified a new cause for the inoperable condition.) The entry also stated that
operations would establish conditions necessary to test the valve for reverse flowand
was proceeding with isolating the Train "A"HPSI system from the Train "B" HPSI
system.
The inspectors noted that the log entry did not address operability of the HPSI
Train "B" flowpath.
The operators completed the isolation of the HPSI trains at 3:45 p.m.,
1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> and 13
minutes after the 2:32 p.m. Iog entry. The licensee informed the inspectors that the
isolation of the HPSI trains, by closing and removing power from the Train "A"HPSI loop
injection valves, was performed to provide equipment isolation in preparation for testing
and maintenance on Valve 1PSIA-V404. The licensee further stated that the isolation
was not initiated to separate
the HPSI trains to prevent flow diversion from the Train "B"
HPSI system through the Train "A"HPSI pump discharge check valve during an
accident.
Since, as discussed
in Section E1.2 of this report, the reverse flow leakage
through the check valve reduced the HPSI flow required to mitigate an accident
condition, the inspectors considered that Train "B"did not have an independent operable
flowpath. The Train "A"HPSI system was already inoperable for maintenance.
Consequently, this condition exceeded the LCO for TS 3.5.2, and therefore, if operators
had known that the check valve would not perform its intended function, an entry into
TS 3.0.3 was required.
Control room operators did not record the TS 3.0.3 entry in the Unit 1 log. The LER for
this event, dated June 5, 1998, stated that TS 3.0.3 was unknowingly entered when
Valve 1PSIA-V404 was declared inoperable.
During later discussions,
the licensee
-7-
informed the inspectors that operators recognized the flowdiversion potential and
impact on system operability, but considered the condition to be a degraded condition
until testing confirmed otherwise.
The inspectors believed that this contradicted the
documentation of the conclusions reached at the May 13 meeting as documented in the
reportability determination.
The inspectors concluded that TS 3.0.3 should have been
entered at 2:32 p.m. on May 13, when HPSI Train "A"was declared inoperable due to
Valve 1PSIA-V404.
TS 3.0.3 required that within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, action shall be initiated to place the unit in a mode
in which TS 3.5.2 did not apply. The basis for T.S. 3.0.3 stated that the purpose of the
1
hour was to allow for the preparation for an orderly shutdown before initiating a change
in plant operation and that this time permits the operator to coordinate the reduction in
electrical generation with the load dispatcher to ensure the stability and availability of the
electrical grid. Even though the control room operators recognized that the inoperable
check valve had an impact on system operability, they did not initiate action to place the
unit in a mode in which the HPSI system was not required.
When the inspectors
discussed this issue with department management
representatives,
the inspectors
determined that these managers were unfamiliar with the 1-hour requirements of
TS 3.0.3. The managers informed the inspectors that they considered the
1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />
specified in TS 3.0.3 as an additional hour that was available to correct the inoperable
condition instead of the time allowed to prepare for an orderly plant shutdown.
At 3:45 p.m. on May 13, 1998, operators completed the isolation of all four Train "A"
HPSI RCS injection valves (Valves SIA-HV-617, 627, 637, 647). Therefore, Train "B"of
the HPSI system was made operable at that time. On May 16, 1998, operators
concluded that both trains of the HPSI system were inoperable on May 13, 1998, and
documented a late entry into TS 3.0.3. The operators initiated CRDR 180317 to have
,the TS 3.0.3 issue evaluated.
The inspectors verified that a late entry was made into
the Unit 1 log. The inspectors considered that TS 3.0.3 was declared from 2:32 p.m. on
May 13, 1998, when Valve 1PSIA-V404 was inoperable, until the Train "B" HPSI system
flowpath was made operable at 3:45 p.m. by isolating the two HPSI trains. Subsequent
to the onsite portion of the inspection, the licensee changed its position with respect to
the late entry into TS 3.0.3 and revised the logs accordingly.
The licensee's failure to comply with the requirements of TS 3.0.3 was determined to be
an example of an apparent violation (50-528/-529/-530/9814-01).
Unit 2
On May 14, 1998, at 9:55 p.m., operations declared the Train "B" HPSI pump inoperable
after receiving a memorandum from engineering (see Section E1.1.b.4) recommending
that Valve 2PSIB-V405, the Train "B" HPSI pump discharge check valve, be declared
The memorandum did not differentiate between the operability of the safety
function of the valve to close versus the safety function of the valve to open. The
inspectors'eview of the log entries indicated that operators did not recognize that when
the Train "B" HPSI pump discharge check valve was inoperable, it affected HPSI system
operability of both trains, and required entry into TS 3.5.2 for an inoperable Train "A"
flowpath and possibly an entry into TS 3.0.3 if there was a concern about the operability
of the safety function of the valve to open.
Operators did not declare the Train "A"HPSI
flowpath inoperable.
In a similar manner to the Unit 1 actions, operators isolated the
HPSI trains in anticipation of performing testing and maintenance on Valve 2PSIB-V405.
For this case, however, isolation of the Train "B" HPSI injection valves was completed
within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, at 10:35 p.m. On May 16, 1998, the Unit 2 log had a late entry
documenting that at 9:55 p.m. on May 14, 1998, an entry into TS 3.0.3 was required due
to the Train "A"HPSI system being inoperable due to Valve 2PSIB-V405 being
suspected of having excessive back leakage.
Another late entry on May 16
documented that at 10:35 p.m. on May 14, TS 3.0.3 was exited when the Train "B" HPSI
injection valves were isolated.
Subsequent
to the onsite portion of the inspection, the
licensee changed its position with respect to the late entry into TS 3.0.3 and revised the
logs accordingly.
Conclusions
03
Operations personnel did not demonstrate
an understanding of TS requirements nor an
understanding of the impact of the misaligned HPSI check valve on system operability.
A'n example of an apparent violation was identified for failure to implement the
requirements of TS 3.0.3 for Unit 1
~
Operations Procedures and Documentation
03.1
Res
onse to Inade
uate HPSI Flow
Ins ection Sco
e
The inspectors reviewed existing procedures that the licensee would have used during
LOCA scenarios involving inadequate
HPSI flow. The inspectors also interviewed
reactor operators and observed two crews in the simulators respond to a LOCA with
degraded HPSI flow.
Observations and Findin s
A senior reactor operator walked the inspectors through existing procedures that control
room operators would be expected to use during a LOCA with inadequate safety
injection flow. Emergency Procedures 40EP-9EO03, "Loss of Coolant Accident,"
Revision 5, and 40EP-9EO09, "Functional Recovery," Revision 6, addressed
the
degraded HPSI flow condition and the required actions to recover the inventory control
safety function. The inspectors verified that the instrumentation available to the
operators in the control room would allow for the diagnosis of a degraded HPSI flow
condition.
If HPSI flowwas unacceptable
and unrestorable, the emergency procedures
provided a method of alternate response to recovery.
The inspectors also observed two
crews successfully respond to a LOCA with degraded HPSI flow scenario on the plant
simulators.
The circumstances
of this simulator scenario, "LOCAwith Degraded HPSI
Flow," July 14, 1998, were a LOCA and a failure of the Train "B" HPSI pump with its
associated
discharge check valve stuck at 15 perceht open.
Operator performance was
I'q
0
-9-
considered acceptable
if the crew took action to depressurize
injection flow prior to the reactor vessel outlet plenum level decreasing to less than 23
percent.
The performance of both crews was good.
Conclusions
The inspectors concluded that existing procedures were adequate and available for
responding to a degraded HPSI flowcondition.
II Maintenance
M1
Conduct of Maintenance
M1.1
Valve Maintenance
The inspectors reviewed the maintenance history of the HPSI pump discharge check
valves, interviewed maintenance and engineering personnel, reviewed maintenance
procedures, and examined a spare valve in the maintenance shop.
b.
Observations and Findin s
Evolution of Maintenance Procedure
The subject HPSI pump discharge check valves were 4-inch, 1500-pound, bonnet-hung,
pressure-seal
swing check valves manufactured by Borg-Warner. The licensee's
maintenance procedures for these valves had evolved since plant construction.
The
original maintenance procedure applicable to the subject valves, 08 M Manual 1024,
Revision A, did not identify specific instructions regarding how far the bonnet retaining
ring should be threaded into the body of the valve. The licensee informed the inspectors
that during plant construction, valves were disassembled
prior to being welded in place.
During reassembly,
the bonnet retaining ring was threaded into the valve body until it
bottomed after the internals were installed.
As described to the industry in NRC Information Notice (IN) 89-62, "Malfunction of
. Borg-Warner Pressure
Seal Bonnet Check Valves Caused By Vertical Misalignment of
Disc," dated August 31, 1989, the assembly instructions for the subject valves were
missing an essential assembly step, which, if not implemented, would result in the disc
assembly being suspended
too low inside the body of the, valve.
If the disc assembly
was suspended
too low, the valve may initiallyseat acceptably but still become jammed
after forward flow exercised the valve, thus preventing proper valve seating and
subsequently allowing reverse flow leakage.
The original factory assembly process
included a step that unthreaded the bonnet retaining ring after it bottomed in the valve
~ body until the correct disc height, as viewed through the end of the uninstalled valve,
was obtained.
Backing out the retaining ring to achieve the correct vertical dimension
4C'
I
I
I
I
I
-10-
was not included as a required step in the licensee's original procedure
(i.e. vendor
instruction manual).
The licensee issued Procedure 31 MT-9ZZ17, "Disassembly and Reassembly of
Borg-Warne~ Check Valves," Revision 0, on November.30, 1992, after receiving.vendor:
information developed in response to IN 89-62. As discussed
in Section E1.4, the
licensee had initiallydetermined that no actions were necessary
in response to the IN.
This contributed to the 3-year delay from the time that IN 89-62 was issued to the time
that Procedure 31MT-9ZZ17 was issued.
As recommended by the vendor, the
procedure included a step of measuring the distance from the top of the valve body to
the top of the bonnet retaining ring (called the "A"dimension) before valve disassembly
and after reassembly.
However, this step did not ensure that the valve disc-to-body
vertical alignment was correct.
If the valve had been previously disassembled,
measuring the "A"dimension prior to maintenance and returning the valve to that "A"
dimension during reassembly would have merely returned the valve to its previous
configuration, which may have been incorrect.
The licensee issued Revision
1 of Procedure 31MT-9ZZ17 on November 17, 1994.
This revision included measurement
of the "B"dimension, the distance from the top of
the valve bonnet to the top of the valve body. The procedure contained instructions to
determine the correct vertical disc-to-body configuration, as determined by the
disc-to-body measurements,
and place the valve in the correct alignment.
However,
performing these steps was only required if inspection of the valve identified improper
seating.
An enhancement
to the dimensional measurement
process was implemented
in Procedure 31MT-9ZZ17, Revision 4, on January 24, 1997, to simplifythe
measurement
process.
The licensee did not adjust valve alignment when this procedure
revision was made.
The 1994 revision of Procedure 31MT-9ZZ17 appeared adequate
to ensure that the correct valve alignment was established.
Maintenance Histo
With respect to Valve 1PSIA-V404, the Unit 1, Train "A"HPSI pump discharge check
valve, when maintenance was performed on April 10, 1998, Procedure 31MT-9ZZ17
.
was not followed because of personnel error while taking measurements
to establish the
correct "A"and "B"dimensions.
This resulted in the erroneous conclusion that the
valve disc had been approximately 0.5 inches too high inside the body of the valve.
To correct this condition, a 0.5-inch spacer ring was installed.
Later, as discuhsed in
Section E1.1.b.3, the licensee recognized this error and corrected the condition by
removing the spacer on May 15, 1998.
Th'e previous time that Valve 1PSIA-V404 was
disassembled
was May 1, 1992. The steps to measure the "A"or "B" dimensions and
verify that the disc-to-body alignment was correct were not included in the maintenance
procedure at that time. Therefore, the valve disc was susceptible to cocking open since
May 1, 1992.
On May 15, 1998, the licensee discovered that the disc in Valve 2PSIB-V405, the
Unit 2, Train "B" HPSI pump discharge check valve was vertically misaligned.
Valve 2PSIB-V405 was previously disassembled
on April 14, 1993. The Unit 2 valve
4
-11-
was last reassembled
prior to the revision of Procedure 31MT-9ZZ17 that would have
established the correct vertical alignment. Therefore, the valve disc was susceptible to
cocking open since April 14, 1993.
'For the remaining HPSI pump. discharge check valves on all three units, the licensee,'.-,
'valuated
the maintenance and testing history, performed as-found external
measurements
of the "A"and "B"dimensions, and performed reverse flowtests.
Each
of the remaining four valves had as-tound reverse flowtest results of 0 gpm.
Conclusions
The licensee's maintenance procedure for Borg-Warner bonnet-hung, pressure-seal
check valves did not include adequate instructions for ensuring correct vertical disc
alignment until November 1994. The inspectors concluded that the licensee missed an
opportunity to identify and correct the vertical alignment issue at that time. Once the
problem was recognized in 1998, the licensee developed and implemented an
acceptable plan to identify and correct the adverse condition on all of the HPSI pump
discharge check valves.
Valve Testin
The inspectors reviewed the inservice testing (IST) history of the HPSI discharge check
valves, interviewed maintenance and engineering personnel, and reviewed testing
'rocedures.
Observations and Findin s
The licensee identified in its OEA review of NRC IN 88-70, "Check Valve Inservice
Testing Program Deficiencies," that its IST program did not require reverse-flow testing
of the HPSI pump discharge check valves.
As discussed
in Section E1.4, the licensee
originally concluded that because they had an NRC-approved IST program, any
changes to the program were considered enhancements.
Therefore, the need for
procedure revisions was not considered a priority. The licensee added reverse-flow
testing requirements for these valves to its IST program on July 26, 1992. The test
methodology involved operating one HPSI pump and verifying adequate flowwas
delivered to the RCS when both trains of the HPSI system were cross-connected.
This
method was intended to demonstrate that the HPSI pump discharge check valve in the
idle train had closed as evidenced by maintaining sufficient flow to the RCS with
inconsequential flow diversion through the check valve. No explicit acceptance
criterion
was specified for reverse flow through the check valve. The test was performed for
each valve during refueling outages to satisfy the IST program surveillance
requirements of TS 4.0.5.
Due to the absence
of reverse flow measurements
through
these check valves, these surveillance tests did not identify that Valves 1PSIA-V404 and
2PSIB-V405 were inoperable since 1992 and 1993, respectively.
I
I
I
(
-12-
This forward-flow testing methodology continued until the 1998 Unit 1 refueling outage.
As discussed
in Section E1.1, Procedure 73ST-9XI33, "HPSI Pump and Check Valve
Full Flow Test," Revision 9, was issued on March 12, 1998, to perform a forward-flow
type test and included an allowable 20 gpm variance in the measured flow in the
acceptance
criterion. The development of an explicit acceptance
criterion for check,
valve reverse flowwas a corrective action from Condition Report/Disposition Request
(CRDR) 2-7-0420 that was written following the October 28, 1997, Unit 2, safety
injection tank (SIT) level decrease
event described in Section E1.1.b.1. After
Valve 1PSIA-V404 failed this test on April 9, 1998, and was repaired, plant conditions
had changed and precluded the forward-flow type test from being performed.
Therefore, the licensee issued Revision 10 of Procedure 73ST-9XI33 on April 11, 1998,
which included a new testing methodology consisting of pressurizing the downstream
side of the check valve and monitoring reverse flowthrough the valve. The new
methodology was subsequently described in a new procedure (Procedure 73ST-9XI35)
issued on May 13, 1998.
After the test failure of Valve 1PSIA-V404 during the Unit 1 refueling outage,
engineering personnel continued investigation of the performance requirements for
HPSI discharge check valve reverse flow. Based on engineering analysis of ECCS
performance requirements and actual HPSI system performance (Section E1.2), the
licensee determined that a reverse flowacceptance
criterion of 10 gpm at a test
differential pressure of 50 - 125 psid would be valid for all six valves in the three units.'-
. Procedure 73ST-9XI35, "HPSI Pump Discharge Check Valve Closed Exercise Test,"
was revised to include the 10 gpm acceptance
criterion in Revision 6 on May 26, 1998.
Technical Specification 4.0.5 states, in part, that, "... Inservice testing of ASME Code
Class 1, 2, and 3 pumps and valves shall be performed in accordance with Section XI of
the ASME Boiler and Pressure Vessel Code and applicable Addenda...."
Prior to January 15, 1998, the applicable Edition of the ASME Code was the 1980
Edition through Winter 1981 Addenda.
Subsection IWV, "Inservice Testing of Valves in
Nuclear Power Plants," of Section XI of the ASME Code, Article IWV-2000, defines
Category A valves as those for which seat leakage is limited to a specific maximum
amount in the closed position for fulfillmentof their function and defines Category C
valves as those which are self-actuating in response to some system characteristic,
such as check valves. Article IWV-2000 also states that valves within the scope of this
section shall be placed in one or more categories and that when more than one
distinguishing category characteristic is applicable, all requirements of each of the
individual categories are applicable.
Effective January 15, 1998, the applicable Edition of the ASME Code was the 1989
Edition. Subsection IWV, "Inservice Testing of Valves in Nuclear Power Plants,"
required that valve testing be performed in accordance with the requirements stated in
ASME/ANSI Inservice Testing of Valves in Light-Water Reactor Power Plants OM-10,
OMa-1988 Addenda to the OM-1987 Edition. The definitions and requirements
identified above for the 1980 ASME Code Edition are equivalent to the 1989 ASME
Code Edition.
jl
1
II
I
I'I
l
I
f
f
j
I
-13-
The HPSI pump discharge check valves have a safety-related function to close to
prevent diversion of flow between trains of a system.
As identified in NUREG-1482,
"Guidelin'es for Inservice Testing at Nuclear Power Plants," the ASME Code does not
specifically require that these valves be Category A; although, there may be a leakage
limitbased qn the total system requirements.
The licensee conducts their IST program
in accordance
with Procedure 73DP-9XI01, "Pump and Valve Inservice Testing Program
- Component Tables," Revision 5. Procedure 73DP-9XI01 identified that the HPSI pump
discharge check valves were classified as Category C valves. This classification was
acceptable; however, NUREG-1482 also identified that for valves of this type that the
licensee should evaluate the consequences
of reverse flow. This evaluation should
consider: 1) the loss of water from the system and connecting systems; 2) the effect that
the leakage might have on components and piping downstream of the valve; and 3) any
increase in radiological exposure resulting from the leakage.
The licensee had not
performed evaluations of this type until the current problems with the HPSI pump
discharge check valves were identified. These evaluations are discussed
in Sections
E1
~ 1 and E1.2. At the conclusion of the onsite portion of the inspection, the licensee
was evaluating whether the HPSI pump discharge check valves should be re-classified
as Category A in the IST program.
Conclusions
The IST program was ineffective at demonstrating operability of the HPSI pump
discharge check valves.
Conduct of Engineering
HPSI S stem De raded Condition Review
Ins ection Sco
e
To review the HPSI system degraded condition, the inspectors toured relevant areas
of the facilityincluding the auxiliary building, control room, and maintenance shop.
The inspectors also conducted interviews with licensee personnel, reviewed
selected procedures, calculations, maintenance packages,
OEAs, and corrective
action documents, and reviewed the licensee's investigation report and
LER 50-528/-529/-530/98-006.
Observations and Findin s
1997 Unit 2 Outa
es
On October 28, 1997, during Unit 2 restoration prior to startup from a forced outage,
operators were performing Procedure 40OP-9SI02, "Recovery from Shutdown Cooling
to Normal Operating Lineup," Revision 14, Section 7.0, "Boration Of Cold Leg Injection
Lines," using the Train "A" HPSI pump.
During the evolution, pressure and water level
in SIT 1A decreased
unexpectedly when Valve SIB-UV638, the SIT 1A check valve
1
I
-14-
leakage line isolation valve, was opened to establish a recirculation path for the 1A
injection line. The reactor operator closed Valve SIB-UV638, which terminated the loss
of inventory, and restored the SIT pressure.
The onshift crew contacted engineering
personnel, who determined the cause of the loss of inventory in SIT 1A was leakage
past Valve 2PSIB-V405, the Train "B" HPSI pump discharge check valve, to the.RWT;
.
The engineering personnel reviewed the results of the previously performed reverse
exercise test of Valve 2PSIB-V405, conducted during the previous refueling outage in
accordance with Procedure 73ST-9XI33, and identified that the check valve was found
to have a leakrate of approximately 30 gpm. This test had been performed to satisfy the
IST program requirements.
As stated previously in Section M1,2, the refueling outage
test was accepted because adequate forward flow to the RCS was obtained when the
HPSI trains were cross connected but there was no explicit acceptance
criterion for
reverse flowthrough the check valve. Therefore, the operators concluded that this
amount of reverse flow did not prevent the HPSI system from performing its design
function and plant startup continued.
The onshift crew initiated CRDR 2-7-0420 to
evaluate a procedure change to alert operators of the potential to affect SIT level when
performing the cold-leg boration evolution.
The check valve leakage caused a depressurization
of the loop injection piping during
the cold-leg boration to approximately 600 psig and the SIT began to discharge as
designed.
The licensee documented in the CRDR that the condition could have been
identified before ifoperations had questioned why safety injection header pressure did
not respond as expected during the cold-leg recirculation line-up prior to opening
SIB-UV638. The safety injection header pressure should have not decreased
below SIT
pressure and the SIT should not have discharged.
The CRDR also documented that,
"Operations noted the response to Pl-339 [safety injection header pressure] was not
normal, however, they were unable to explain the condition so the procedure was
continued."
The CRDR also contained a reference to a similar event, which occurred on October 10,
1997, during plant restoration prior to startup from the previous refueling outage.
Again,
SIT level decreased
during the performance of Procedure 40OP-9SI02, Section 7.0 with
the Train "A"HPSI pump.
In that particular instance, the decrease
in level was
terminated by the reactor operator closing the Train "B" injection valve and isolating the
trains. This similar example was highlighted in the CRDR because the same crew was
on duty when each SIT transient occurred during the cold-leg boration evolution. The
crew did not request engineering assistance
to evaluate the SIT level transient and the
crew did not conclude that Valve 2PSIB-V405 was leaking. Reactor startup activities
continued and the reactor was made critical later that day. With respect to the October
10, 1997, event, operations personnel had concluded that the unexpected loss of SIT
inventory was caused by an "eductor effect," which caused a localized low pressure
condition at the SIT outlet during HPSI cold-leg recirculation, and that the condition
should have been expected.
The inspectors did not consider this to be a plausible
explanation because of the high localized pressure drop that would be required to
decrease
pressure below SIT pressure.
J
I
0
-15-
The inspectors reviewed control room and unit logs for October 10 and 28, 1997, and
identified that no log entries were made regarding the SIT level decreases.
The
licensee's Procedure 40DP-9OP22, "Operations Logkeeping," Revision 9, Step 3.3.5
stated, in part, "The information entered in the Control Room Log shall include...
abnormal occurrences,
unless previously logged and identified by an active corrective,. -.,
action document." The licensee did not agree with the inspectors'haracterization
that
the SIT level transients were abnormal occurrences.
The inspectors determined that the
licensee's procedures did not exempt the conditions experienced on October 10 and 28
from being logged in accordance with the logkeeping procedure.
Therefore, the
inspectors concluded that the failure to log the abnormal SIT level transients that
occurred on October 10 and 28, 1997, were two examples of an apparent violation of
TS 6.8.1, which required adherence to Procedure 40DP-9OP22
(50-528/-529/-530/9814-02).
While the licensee identified a check valve reverse leakage problem as documented
in
CRDR 2-7-0420 on October 28, 1997, they failed to take prompt corrective actions to
resolve the problem.
10 CFR Part 50, Appendix B, Criterion XVI,"Corrective Action,"
requires that measures
shall be established to assure that conditions adverse to quality
are promptly identified and corrected and in the case of significant conditions adverse to
quality that the measures shall assure that the cause of the condition is determined and
corrective action taken to preclude repetition. The inspectors considered the October 10
and 28, 1997, Unit 2 SIT drain events to have been occurrences where a significant
condition adverse to quality (excessive reverse flowthrough Valve 2PSIB-V405) was not
promptly identified and corrected.
These were considered the first two examples of an
apparent violation of 10 CFR Part 50, Appendix B, Criterion XVI
(50-528/-529/-530/9814-03).
During the evaluation and resolution of CRDR 2-7-0420, the licensee concluded
that a 20 gpm check valve reverse leak rate acceptance
criterion should be sufficient
to prevent unacceptable
HPSI flow diversion. This conclusion was not based upon a
rigorous calculation of the impact of flow diversion; rather, it was based on a review
of leakage acceptance
criteria for some check valves in the LPSI system and the
methodology employed to select their leakage acceptance
criteria.
Procedure 73ST-9XI33 was changed in Revision 9 to include the explicit acceptance
criteria of either: 1) less than 20 gpm difference between the sum of hot and cold-leg
injection flow rates before and after the HPSI trains were cross connected; or 2) hot-leg
injection flow greater than or equal to 525 gpm, cold-leg injection flow greater than or
equal to 525 gpm, and total HPSI flow less than or equal to1200 gpm. The
forward-flow acceptance
criteria were consistent with the surveillance requirements in
Conclusions
The inspectors concluded that the licensee had an opportunity to detect the flow
diversion through Valve 2PSIB-V405 on October 10, 1997; however, the operators failed
to identify the cause of unexpected decrease
in SIT level. On October 28, the licensee
identified the condition but did not take effective corrective action because of an
r
i
-16-
incorrect assessment
of the operability implications. Two examples of an apparent
violation were identified for the failure to make log entries for the SIT level decreases,
which occurred on two separate occasions while performing the cold-leg boration
process.
Two examples of an apparent violation were identified for inadequate
corrective aqtion to identify and correct a significant condition adverse to quality..
b.2
1998 Unit 1 Refuelin
Outa
e and Subse uent0
eration
On April 9, 1998, the licensee had indication of excessive reverse flowthrough
Valve 1PSIA-V404 during performance of Procedure 73ST-9XI33. The control room
received an unexpected equipment drain tank (EDT) high level alarm and the test was
suspended.
The licensee concluded that reverse-direction flowthrough the check valve
pressurized the Train "A"ECCS suction piping and the Train "A"containment spray (CS)
pump discharge piping to the 650 psig setpoint of Valve 1JSIA-PSV194, the Train "A"
shutdown cooling heat exchanger (SDCHX) outlet relief valve. The leak rate through
Valve 1PSIA-V404 was initiallyestimated to be 85 gpm. On April 10, 1998, the licensee
initiated CRDR 1-8-0238 to document the event and classified the CRDR as "significant"
with action requests to perform an equipment root cause of failure evaluation and
reportability determination.
During a subsequent
engineering evaluation, the licensee
estimated that the event resulted in relief valve blowdown to the EDT at a flow rate of-
approximately 120 gpm and a total leak rate through Valve 1PSIA-V404 of
approximately 214 gpm. The licensee also concluded that the valve body and disc had
been in a misaligned condition for the entire previous operating cycle and likely had
been misaligned since the last time maintenance was performed on the valve on May 1,
1992. At that time, the licensee had not yet incorporated additional guidance for
addressing vertical'misalignment concerns for Borg-Warner check valves in the
maintenance procedure.
The licensee's review and implementation of in-house and
industry operating experience is discussed
in Section E1.4 of this report.
The licensee performed Operability Determination (OD) 203 to evaluate the operability
impact of the inadvertent pressurization.
The OD evaluated the pressure rating and
ASME Code allowables for stresses
of piping and system components in the HPSI and
CS suction piping, CS discharge piping, and SDCHX. The OD concluded that the piping
and components remained operable on the basis of not exceeding ASME Code
allowable stresses for the suction piping, not exceeding the design pressure for the CS
discharge piping, and not exceeding the pressure rating of valves or flanges in the
suction piping. The licensee also performed a system walkdown with the HPSI and CS
pumps operating to examine performance of the pump seals and pipe flanges.
No leaks
were identified and no pipe supports exhibited signs of water hammer.
The inspectors
reviewed the OD a'nd concluded that the licensee's evaluation of system operability was
acceptable.
Maintenance technicians disassembled
and inspected Valve 1PSIA-V404 via Work
Order (WO) 836600 and Procedure 31MT-9ZZ17 on April 10, 1998. With engineering
support, the technicians determined that a vertical offset existed in the alignment of the '
4T
t
-17-
valve disc with respect to the valve body. The licensee concluded that the valve disc
had been located too high in the body of the valve and that the disc needed to be
lowered approximately 0.5 inch. Personnel reverified the calculation for the desired
vertical dimensions, but did not independently verify the measurement of the
dimensions., The technicians made a measurement
error that resulted in an. incorrect.
determination of the magnitude of the vertical offset. Maintenance and engineering
personnel concluded that a spacer ring would have to be installed to adjust the height of
the disc. Therefore, WO 836600 was amended to Deficiency Work Order
(DFWO) 836600 to install a carbon steel spacer ring between the silver plated pressure
seal ring and the threaded retaining ring to compensate for the vertical offset. Longer
bonnet studs were also required to accommodate the spacer.
Installation of the spacer
ring resulted in the valve disc being located too low within the body of the valve to seat
properly. The failure to correctly assemble the check valve to correct the reverse
leakage condition was considered to be the third example of an apparent violation of
10 CFR Part 50, Appendix B, Criterion XVI (50-528/-529/-530/9814-03).
A DFWO was the licensee's method for which degraded and nonconforming conditions
requiring engineering direction were dispositioned and corrected.
The licensee
considered the DFWO a design change and completed a 10 CFR 50.59 safety
evaluation to document that the change did not involve an unreviewed safety question.
The inspectors reviewed the DFWO and concluded that it included an adequate
'evaluation of the acceptability of these subcomponents
in the valve. The inspectors
asked if any as-built drawings were revised to reflect the implementation of the DFWO
and were informed that the design documents were not changed nor were they planned
to be changed.
The inspectors were concerned that installation of the spacer and
replacement of the bonnet studs without updating any design documentation was
representative of inadequate design control. The licensee informed the inspectors that
when a maintenance planner reviews the maintenance history of a component as part of
work package preparation, the maintenance history would reveal that subcomponents
had been installed or replaced such as the spacer and studs for Valve 1PSIA-V404.
The licensee, upon further evaluation, initiated CRDR 9-8-0893 to address this issue.
NRC review of CRDR 9-8-0893 is considered to be an unresolved item (URI)
(50-528/-529/-530/9814-04).
Procedure 73ST-9XI33 required full HPSI flowwith the reactor vessel head removed.
On April 11, 1998, after completion of the DFWO, the reactor vessel head had been
reinstalled.
Therefore, plant conditions did not allow the normal, forward flowtest to
be performed.
An alternate test method was developed in Revision 10 of
Procedure 73ST-9XI33 using demineralized water to pressurize the down stream side
of the check valve and opening a drain valve on the upstream side of the valve to
determine reverse flow. On completion of the test, Valve 1PSIA-V404 was declared
operable with a measured leak rate of 18 gpm at a test pressure of 96 psig. The leak
rate exceeded
an administrative limit of 10 gpm, which in accordance with the test
procedure, required the initiation of a work request.
The inspectors verified that Work Request 941341 was initiated to repair the valve during the next outage.
The licensee
informed the inspectors that it had accepted these test results as satisfactory because
they assumed that reverse-flow leakage would decrease
when HPSI pump discharge
i
pk
jt
I
-18-
pressure from the opposite train improved the valve disc-to-body seat contact during
system operation.
On May 7, 1998, the licensee determined during its investigation of CRDR 1-8-0238 that
the as-foun4 condition of Valve 1PSIA-V404 on April9, 1998, represented a condition.:-
that would have prevented it from performing its intended function during an accident.
The licensee concluded that this event was reportable to the NRC per 10 CFR 50.73
and submitted LER 50-528/98-006 on June 5, 1998. The results and consequences
of
this assessment
are discussed
in Section E1.2.
Technical Specification 3.5.2 requires in Modes 1, 2, and 3, that two independent ECCS
subsystems
shall be operable with each subsystem comprised of one operable HPSI
pump, one operable LPSI pump, and an independent operable flowpath. Action
statement a. associated with TS 3.5.2 required that with one ECCS subsystem
inoperable, restore the inoperable subsystem to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or be in
at least hot standby within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in hot shutdown within the following
6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.
As discussed earlier, Valve 1PSIA-V404 was installed incorrectly and did not
have correct vertical disc alignment from May 1, 1992, until May 15, 1998. Therefore,
Train "B"of the Unit 1 ECCS did not have an independent operable flowpath from
May 1, 1992, until May 15, 1998, and the licensee did not comply with Action a. of
TS 3.5.2. This was considered to be the first example of an apparent violation of
TS 3.5.2 (50-528/-529/-530/9814-05).
c.2
Conclusions
The April 1998, Unit 1 refueling outage test results demonstrated that Valve
1PSIA-V404 was in a significantly degraded condition. The maintenance history for the
subject valve indicated that it had been in that condition since 1992. An example of an
apparent violation of 10 CFR Part 50, Appendix B, Criterion XVIwas identified for failure
to correct the problem with Valve 1PSIA-V404 during maintenance.
An incorrect
interpretation of the 18 gpm April 11, 1998, post-'maintenance
testing results,
contributed to continued operation with an inoperable valve. An example of an apparent
violation of TS 3.5.2 was identified for having an inoperable independent HPSI flowpath
for approximately 6 years.
b.3
Online Corrective Maintenance of Valve 1PSIA-V404
On May 13, 1998, as part of the CRDR 1-8-0238 review, the check valve engineer
reviewed measurements
of a spare Borg-Warner check valve and discussed the
maintenance
procedure and spacer installation with maintenance technicians.
The
engineer determined that a measurement
error may have occurred during the April 1998
maintenance on Valve 1PSIA-V404, resulting in a vertical disc-to-body misalignment.
The test methodology that was used in Procedure 73ST-9XI33, Revision 10, was
extracted and placed in Procedure 73ST-9XI35, Revision 0. Since the Unit 1 refueling
outage, engineering personnel had determined that a new acceptance
criterion of
30 gpm reverse flow through Valve 1PSIA-V404 would still maintain acceptable
)I
I
-19-
forward flowfor system operability. This was based on an evaluation of the
operating performance of the HPSI pumps and system flowcharacteristics.
Procedure 73ST-9XI35, Revision 0 contained the 30 gpm acceptance
criterion. On
May 14, 1998, the licensee performed the test on Valve 1PSIA-V404 and it failed with an
as-found result of 33 gpm at a differential pressure of 95 psid. The licensee made a
- .
one-hour 10 CFR 50.72 notification to the NRC to report that this was a condition
outside the design basis of the facility.
After the failed test, the licensee disassembled
the valve, removed the spacer, ring in
accordance with WO 840712, and verified that the disc was misaligned in the valve
body. The maintenance technicians performed an inspection and exercised the valve
internals per WO 840826 and Procedure 73ST-9ZZ25. The technicians then attached a
stainless steel wire to the swing arm of the valve, installed the internals with the wire
extending outside the valve body (without the silver seal gasket installed), and set the
bonnet height to the desired vertical dimension.
When this was done, the technicians
stroked the valve by pulling on the wire until a metallic noise was heard, indicating that
the disc had contacted the backstop.
This provided an additional indication that the
valve was free to stroke. The technicians then removed the wire and reassembled
the
valve with the silver seal to the correct vertical dimensions.
The inspectors noted that WO 840712 was written to disassemble,
rework, and
reassemble
Valve 1PSIA-V404. There were no instructions included to remove the
spacer ring: The inspectors asked whether a special WO or WO amendment was
required prior to the removal of the spacer ring and whether removing the spacer ring
was a design change, as was the case previously for the installation of the spacer ring..
The licensee stated that they considered the removal of the spacer to be a rework
activity and not a design change.
Rework was defined as the process by which a.
degraded or nonconforming item is made to conform to a prior specified design
requirement by completion, machining, reassembly,
replacing, or other corrective
measures.
The inspectors noted that the materials list for Valve 1PSIA-V404 that
was attached to WO 840712 had not been updated to reflect the installation of the
new bonnet studs or spacer ring. This issue will be reviewed as part of
URI 50-528/-529/-530/9814-04 to determine whether the licensee implemented
adequate design control.
After reassembly of Valve 1PSIA-V404 without the spacer ring, operations performed
Procedure 73ST-9XI35, Revision
1 on May 15, 1998, and the valve passed its reverse
flow test with a result of 0 gpm at 142 psid.
c3.
Conclusions
The questioning attitude of the check valve engineer was instrumental to the
identification of the misalignment of Valve 1PSIA-V404. Corrective actions were taken
to restore the valve to an operable condition.
t~
t
~r
l
f
-20-
b.4
Ino erabili
of Valve 2PSIB-V405
After the April 1998 test failure of the Unit 1 Valve 1PSIA-V404, the licensee initiated
CRDR 2-8-0128 and OD No. 204 to evaluate the operability impact of this condition for
Units 2 and g.
- Included in this evaluation was a review of past HPSI fullflowtest results
conducted during the previous refueling outages and measurement of external valve
dimensions.
On May 14, 1998, as a result of this review and the results of the Unit 1
testing performed the previous day, engineering and maintenance personnel determined
that Valve 2PSIB-V405 was misaligned.
Engineering recommended to operations in
Memorandum 469-00170-BJR that the valve be declared inoperable, that reverse flow
testing be performed, and that corrective maintenance and post-maintenance
testing be
performed.
The inspectors noted that the memorandum only addressed
operability of
Valve 2PSIB-V405 and did not address operability of the entire HPSI system.
The operators declared the Unit 2 Train "B" HPSI pump inoperable based on the
recommendation of the memorandum from engineering.
The inspectors noted that this
was not consistent with the memorandum, which recommended that the Train "B"valve,
not the Train "B"pump, be declared inoperable.
The impact of reverse flowthrough
Valve 2PSIB-V405 on operability of the Train "A"HPSI system flowpath was not
addressed
in the Unit 2 log. The licensee informed the inspectors that the" Unit 2 log
entry only addressed
the inoperability of Train "B"because the operators were preparing
to perform testing and maintenance on the Train "B"valve, which required removing the
Train "B" HPSI system from service.
Based on the recommendation from Engineering
that the valve should be declared inoperable, this inoperable check valve required that
the opposite HPSI train flowpath be declared inoperable, ifthe trains were not isolated,
because of the safety function of the valve to close and prevent diversion of flowfrom
the RCS.
The licensee performed Procedure 73ST-9XI35 to measure the as-found condition of
Valve 2PSIB-V405 on May 15, 1998. The test results indicated a reverse flow rate
through the valve of 37.5 gpm at a differential pressure of 6 psid. As discussed
in
Section E1.2, this amount of reverse flowwould prevent the HPSI system from meeting
the minimum-required ECCS performance assumed
in the safety analysis during a
LOCA. The licensee issued WO 836005 to disassemble
the valve and verified that the
vertical alignment of the disc within the body of the valve was too low and had caused
the disc to hang in a cocked-open position. The valve alignment was corrected and the
valve was retested in a manner similar to that discussed above for Valve 1PSIA-V404.
The retest of Valve 2PSIB-V405 passed with a leakage rate of 0 gpm at 150 psid and
the system was declared operable on May 16, 1998. As discussed
earlier,
Valve 2PSIB-V405 was installed incorrectly and did not have correct vertical disc
alignment from April 14, 1993, until May 16, 1998. Therefore, Train "A"of the Unit 2
ECCS did not have an independent operable flow path from April 14, 1993, until May 16,
1998, and the licensee did not comply with Action a. of TS 3.5.2.
This was considered
to be a second example of an apparent violation of TS 3.5.2
(50-528/-529/-530/9814-05).
c.4
Conclusions
Cg
0
-21-
Licensee personnel successfully evaluated and confirmed that Valve 2PSIB-V405 was
misaligned and corrective actions were taken to restore the valve to an operable
condition. Testing results demonstrated that the valve was in a significantly degraded
condition and the maintenance history for the subject valve indicated that it had been in
that condition since 1993. The inspectors considered the communication between..;
.
engineering and operations a weakness,
as demonstrated
by the May 14, 1998,
memorandum from engineering, which did not provide a recommendation
regarding
HPSI system operability. An'example of an apparent violation of TS 3.5.2 was
identified.
b.5
Testin
and Vertical Dimension 0 timization of Remainin
Valves
The licensee performed reverse flowtests of the four remaining HPSI pump discharge
check valves. Allof the valves passed the 73ST-9XI35 test with 0 gpm reverse flow.
Based on external dimension measurements
and a review of previous forward-flow
surveillance tests performed during refueling outages, the licensee concluded that two
of the four valves, Valves 3PSIB-V405 and 1PSIB-V405, should be reworked for
"vertical dimension optimization."
Valve 3PSIB-V405 was disassembled,
inspected, reassembled
to a new desired
vertical dimension, and retested with 0 gpm reverse flowon May 18, 1998. When
Valve 1PSIB-V405 was inspected and reassembled
to the desired dimension on-
May 26, 1998, maintenance technicians discovered that the disc was not seating
properly. The technicians discovered that the seat angle of the valve was different than
expected.
Instead of an expected seat angle of 12 degrees, the licensee discovered
that this valve had a seat angle of 5 degrees.
The licensee contacted the vendor and
was informed that all 4-inch Borg-Warner hung-bonnet pressure-seal
valves were
supplied with a 12-degree seat angle. The inspectors questioned the licensee regarding
the procurement process and quality assurance
receipt inspection for the subject valve.
After further investigation, the licensee determined that the vendor implemented a
design change in 1980 that changed the seat angle from 5 to 12 degrees.
Valve 1PSIB-V405 was procured in 1979 and was the only HPSI pump discharge check
valve that was built prior to the vendor design change.
Therefore, its seat angle was
correct.
The licensee documented this condition in DFWO 842362.
The inspectors will
review the issue of updating design documents regarding this condition as part of the
URI regarding design control (50-528/-529/-530/9814-04).
The licensee calculated a
revised vertical dimension to account for the different seat angle, reassembled
the valve
and successfully retested the valve with no reverse flow on May 27, 1998.
c.5
Conclusions
Licensee personnel successfully confirmed that the remaining four HPSI pump
discharge check valves were operable by performing reverse-flow tests.
Testing
performed after maintenance confirmed that the valves remained operable.
The
licensee adequately demonstrated
that one of the valves (Valve 1PSIB-V405) that had
an unexpected seat angle, was acceptable for its application and was also operable.
-22-
Assessment of As-found Conditions
lns ection Sco
e
The inspectprs reviewed calculations and test results and interviewed engineering,.
~: *,
personnel to determine the safety and regulatory implications of the as-found degraded
condition of the Unit 1 and Unit 2 HPSI systems.
Observations and Findin s
Check Valve Leaka
e and De raded HPSI S stem Performance
The inspectors reviewed Calculation 13-MA-SI-982, "Evaluation of Allowable Leak Rate
Criteria for 1,2,3PSIA-V404 and 1,2,3PSIB-V405/Assessment
of As-Found Leakage for
2PSIB-V405/1PSIA-V404," Revision 0 and Calculation 13-MC-SI-215, "Revised Single
HPSI Pump Delivery Curve for Cold Leg Injection and Flow Rate Requirements for
Technical Specification 4.5.2.h," Revision 1. In Calculation 13-MA-SI-982, the licensee
performed a best-estimate
evaluation of the current as-built HPSI system delivery profile
for each HPSI pump based on historical system performance data and compared the
resultant delivery profiles to the minimum-required HPSI system delivery profile
identified in Calculation 13MC-Sl-215 that was used in the LOCA safety analyses.
The
margin between the as-built HPSI system delivery capability and the minimum-required
HPSI system delivery used in the LOCA analyses was used to estimate a maximum
allowable leakage profile'for the opposite-train HPSI pump, discharge check valve. The
results indicated that the most limitingsystem was Train "B"in Unit 3.. A maximum
leakage rate of 10.2 gpm at 40 psid was determined for Valve 3PSIA-V404, the
opposite-train HPSI pump discharge check valve, to ensure that the Train "B" HPSI
system would meet its minimum performance requirements.
The licensee also evaluated the April 9, 1998, as-found condition of Valve 1PSIA-V404
and the May 15, 1998, as-found condition of Valve 2PSIB-V405 in
Calculation 13-MA-SI-982. This evaluation concluded that the actual reverse flow rate
through Valve 1PSIA-V404 during the performance of the Procedure 73ST-9XI33,
forward-flow HPSI refueling outage test, was approximately 214 gpm. Because this test
had not been intended to obtain direct differential pressure measurements,
it did not
provide sufficient data to quantify a HPSI performance curve. However, the licensee
had enough information to conclude that the results were qualitatively similar to the
results obtained for the Unit 2 valve as described below.
Valve 2PSIB-V405 had an as-found reverse leakage test result of 37 gpm at 6 psid.
The licensee calculated an as-found loss coefficient for the subject valve and developed
a HPSI delivery curve that accounted for leakage through Valve 2PSIB-V405. The
as-found degraded HPSI delivery curve and the minimum-required ECCS delivery curve
are depicted in Attachment 3. As shown, the estimated as-found HPSI delivery
fi
I'
-23-
capability was significantly degraded compared to the minimum-required HPSI delivery
performance that was assumed
in the LOCA analyses.
For example, at an RCS
pressure of 1200 psig, the estimated HPSI flow rate was approximately 350 gpm less
than the flowrate assumed in the LOCA analyses..The
licensee concluded that the
results for Unit 1 were comparable.
Sin le Failure Consideration
The licensee identified in Calculation 13-MA-Sl-982 that the initial evaluation of the
April9, 1998, Unit 1 refueling outage event included assessments
of ECCS suction
piping overpressurization,
degraded HPSI performance, dose consequences,
and
impact on containment sump level following a LOCA. The licensee determined that
degraded HPSI performance (due to failure of a HPSI pump to start) was the most
limitingfactor with which to assess
the safety significance of the condition.
The inspectors questioned the basis for the licensee's conclusion. The inspectors
considered the failure of an emergency diesel generator (EDG) during a LOCA
coincident with a loss-of-offsite power to be the most limiting single failure for
consideration since none of the ECCS pumps in the affected train would be operating.
(The licensee documented
in the calculation that ifthe LPSI and CS pumps associated
with an idle HPSI pump were running, the ECCS suction piping would not be
overpressurized
as occurred on April 9, 1998, when the SDCHX relief valve lifted).
The licensee noted that failure of an EDG to start would not be a concern because the
RCS loop injection valves associated with the failed EDG would not open and cross
connect the HPSI trains,
However; the inspectors noted that under a condition where an
EDG failed to continue running after the RCS loop injection valves had opened due to a
safety injection actuation signal, a flow path would be established to pressurize the
ECCS suction piping and containment spray pump discharge piping to the SDCHX relief
valve setpoint as occurred during the April 9, 1998, Unit 1 test. The inspectors also
noted that this event created a release path via the vented RWT through the HPSI pump
minimum flow line and a loss of water inventory to the auxiliary building via the SDCHX
relief valve and should be evaluated. The licensee stated that it considered this possible
event scenario but determined that it was not necessary to consider the failure of a
component to continue running as an assumed single failure.
Section 6.3.1, "ECCS Design Bases," of the UFSAR states that, "Adequate physical
separation shall be maintained between the redundant piping paths and containment
penetrations of the SIS (safety injection system) such that the SIS willmeet its functional
requirements even with the failure of a single active component during the injection
mode, or with a single active failure or a limited leakage passive failure during the
recirculation mode."
The bases for TS 3/4.5.2 and 3/4.5.3, "ECCS Subsystems," states, "The operability of
two separate and independent ECCS subsystems
with the indicated RCS pressure
greater than or equal to 1837 psia, or with the indicated RCS cold-leg temperature
greater than or equal to 485
F ensures that sufficient emergency core cooling capability
j
l,
1
I
I
-24-
willbe available in the event of a LOCA assuming the loss of one s'ubsystem through
any single failure consideration."
The inspectors reviewed ANS-51.7/N658-1976, "Single Failure Criteria for PWR Fluid
Systems," ANSI/ANS-58.9-1981, "Single Failure Criteria for Light Water-Reactor -",.'.
-
Safety-Related
Fluid Systems," and NRC SECY 77-439, "Single Failure Criterion." The
inspectors concluded that guidance existed for application of the single failure criterion
that required the proposition that single failures can occur at any time. Therefore, the
inspectors concluded that the licensee should have demonstrated that its assessment
of
consequences
had assumed the most limiting single failure.
Online Maintenance
The inspectors asked the licensee if they had performed online maintenance of the
HPSI system involving system configurations consisting of an inoperable HPSI pump,
the associated discharge isolation valve open, and the associated
RCS injection valves
operable (i.e., would open on a safety-injection actuation signal). The inspectors also
asked if the licensee had considered this configuration as a system vulnerability in its-
event investigation. The licensee reviewed its maintenance
rule database
and provided
the inspectors a list dating back to 1994, that identified several occasions when online
HPSI maintenance was performed that matched this system configuration., The
inspectors were concerned that during these online maintenance periods, a single
failure was not necessary for degraded HPSI performance during a postulated accident.
The inspectors concluded that during these periods of maintenance on the Unit 1 Train
"A"HPSI system and Unit 2 Train "B" HPSI system (the trains with the reverse leakage
check valves) the licensee was in a condition prohibited by the TSs and that TS 3.0.3
required a unit shutdown.
The licensee informed the inspectors that it had not evaluated
this system configuration as a vulnerability in its event investigation.
The inspectors reviewed the out-of-service data provided by the licensee and noted that
the maintenance activities generally consisted of minor preventive maintenance
including oil changes, and valve and breaker maintenance.
With respect to
maintenance on the Unit 1 Train "A" HPSI system (associated with Valve 1PSIA-V404)
and Unit 2 Train "B" HPSI system (associated with Valve 2PSIB-V405) the length of time
for each occurrence ranged from as short as 0.92 hours0.00106 days <br />0.0256 hours <br />1.521164e-4 weeks <br />3.5006e-5 months <br /> to as long as 19.58 hours6.712963e-4 days <br />0.0161 hours <br />9.589947e-5 weeks <br />2.2069e-5 months <br />.
The out-of-service data indicated that the Unit 1 Train "A" HPSI pump was unavailable
when the unit was operating in Mode
1 on September 4, 1996, for 19.58 hours6.712963e-4 days <br />0.0161 hours <br />9.589947e-5 weeks <br />2.2069e-5 months <br /> arid that
no documentation was available to demonstrate that Train "A"wa's isolated from
Train "B." Therefore, the inspectors determined that Unit 1 did not have any
independent ECCS subsystem operable when Train "A"of the HPSI system was out of
service without isolation from Train "B" of the HPSI system, and Valve 1PSIA-V404 was
Action was not initiated to place the unit in hot standby within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> as
required by TS 3.0.3. This was considered to be the second example of an apparent
violation of TS 3.0.3 (50-528/-529/-530/9814-01).
~ lp
I
-25-
The out-of-service data also indicated that the Unit 2 Train "B" HPSI pump was
unavailable when the unit was operating in Mode 1 on October 21, 1994, for 19.8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />.
In this case, a clearance order for WO 6773826 identified that a clearance had been in
effect during this period of time that closed the Train "B" HPSI pump discharge isolation
valve. The exact length of time that the isolation valve was closed was indeterminate;=
The inspectors were able to conclude that the isolation valve was closed from between
2.5 to 6.25 hours2.893519e-4 days <br />0.00694 hours <br />4.133598e-5 weeks <br />9.5125e-6 months <br />. When the clearance order was cleared, the Train "B" HPSI pump
- discharge isolation valve was opened and the Train "B" HPSI pump was still unavailable
due to other maintenance.
The inspectors determined that Train "B"of the HPSI system
was unavailable during the maintenance activity and was not isolated from Train "A"for
a period of 13.5 to 17.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />. Therefore, Unit 2 did not have any independent ECCS
subsystem operable for a period in excess of 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> when Train "B"of the HPSI
system was out of service without isolation from Train "A"of the HPSI system, and
Valve 2PSIB-V405 was inoperable.
Action was not initiated to place the unit in Hot
Standby within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> as required by TS 3.0.3. This was considered to be the third
example of an apparent violation of TS 3.0.3 (50-528/-529/-530/9814-01).
Allother examples of online maintenance of the Unit 1 Train "A"HPSI system and the
Unit 2 Train."B" HPSI system were less than 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> in duration. Therefore, while the
completion of a unit mode change was not required, the licensee failed to recognize the
applicability of TS 3.0.3.
C.
Conclusions
The misalignment of the Unit 1 and Unit 2 HPSI pump discharge check valves placed
the units in a significantly degraded and vulnerable condition with respect to the
capability to mitigate the consequences
of a LOCA. Two examples of an apparent
violation of TS 3.0.3 were identified.
E1.3
Assessment
of Generic Im lications
Ins ection Sco
e
The inspectors reviewed licensee investigation reports, surveillance procedures, and
maintenance procedures to determine whether the licensee had adequately addressed
the issues of check valve misalignment and inadequate testing practices with respect to
other safety-related valves.
Observations and Findin s
The inspectors questioned whether the licensee had evaluated the generic aspects of
the misaligned HPSI pump discharge check valves relative to other safety-related
valves.
The licensee had performed a study of other check valves. There were
27 Borg-Warner pressure-seal,
bonnet-hung check valves installed in each of the three
units. None of the valves in this population except for the two HPSI pump discharge
check valves per unit had the welded-neck design that appeared most susceptible to the
disc-cocking phenomenon.
Valves with a forged-neck design did not have vertical
c
t'
-26-
alignment variability similar to welded-neck models. The licensee performed a review of
the closed-direction exercise tests for the remaining 25 valves per unit and concluded
that the testing performed on each valve adequately demonstrated operability. The
licensee also examined a 3-inch valve in the maintenance shop and was unable to
reproduce tfIe disc-cocking phenomenon, even with the bonnet retaining ring,threaded;:
fullyinto the valve body. During the inspection, the licensee also initiated a generic
review of check valve testing practices to evaluate the adequacy of testing.
The inspectors selected a sample of valves to verify that the surveillance tests were
adequate to demonstrate valve closure.
The valves chosen for this review were the
LPSI cold-leg injection valves (xSIE-V114, xSIE-V124, xSIE-V134, and xSIE-V144,
where 'x's the unit designator) and their associated
test, Procedure 73ST-9SI05, "Leak
Test of HPSI/LPSI Containment Isolation Check Valves," Revision 4. The test
methodology involved pressurizing the downstream side of the valves with a HPSI pump
or a hydro test pump and measuring the leak rate on the upstream side. The inspectors
determined that the surveillance test was adequate to verify check valve closure.
Conclusions
The licensee adequately addressed
the generic implications of the misalignment issue
to other pressure-seal,
bonnet-hung check valves to conclude that no additional
operability concer'ns existed.
In-house and Indust
OEAs
Ins ection Sco
e
The inspectors reviewed licensee in-house and industry OEAs pertinent to the HPSI
pump discharge check valves.
Observations and Findin s
NRC IN 88-70 "Check Valve Inservice Testin
Pro ram Deficiencies"
In its January 12, 1989, evaluation of NRC IN 88-70, the licensee identified that reverse
flow testing of several check valves, including HPSI pump discharge check Valves
1PSIA-V404 and 2PSIB-V405, was not performed.
However, the licensee concluded
that because they had an NRC-approved IST program, any changes the program were
considered enhancements.
The licensee did not give this item a high priority and a due
date to add these tests to the program was established for 1996. The licensee's quality
assurance
organization conducted an audit of the check valve programs in 1992,
concluded that this schedule for action was untimely, and identified this as a finding. As
corrective action, the licensee initiated action to revise the IST program to include
reverse flow testing of the subject valves.
Surveillance Procedure 73ST-xXI29, "Section
XI Check Valve Operability Verification - Mode 6 - Full Stroke Testing of Safety Injection
s t
-27-
Check Valves," was approved on July 26, 1992, to conduct the test. The test
methodology consisted of measuring forward HPSI flowto the RCS via the hot-leg
injection flow path. The HPSI pump discharge check valves were considered operable if
an acceptable forward flowwas measured to the RCS with one HPSI pump operating
while cross connected to the opposite train, but no e'xplicit acceptance
criterion was
.
included to determine check valve reverse flow.
10 CFR Part 50, Appendix B, Criterion XVI,"Corrective Action," required that measures
shall be established to assure that conditions adverse to quality are promptly identified
and corrected and in the case of significant conditions adverse to quality that the
measures shall assure that the cause of the condition is determined and corrective
action taken to preclude repetition. The test procedures developed in 1992, as a result
of the January 12, 1989, evaluation were not adequate to identify excessive reverse
flow, a significant condition adverse to quality (excessive reverse flowthrough HPSI
pump discharge check valves). This was considered to be the fourth example of an
apparent violation of 10 CFR Part 50, Appendix B, Criterion XVI
{50-528/-529/-530/9814-03).
NRC IN 89-62 "Malfunction of Bor -Warner Pressure Seal Bonnet Check Valves
Caused B Vertical Misali nment of Disc"
The licensee performed a review of IN 89-62 and concluded that no action was required.
Engineering Action Request (EAR) 89-1931, completed January 26, 1990, concluded
that the vendor manuals were adequate and had the necessary steps to 'ensure original
factory-established seat/disc alignment. The licensee's initial screening, documented in
'an October 9, 1989, memorandum correctly identified that vertical disc/seat
misalignment due to incorrect retainer rin'g position resulted in the problem identified in
the IN and that the vendor manual did not include procedure steps for adjusting retainer
ring position to achieve the correct alignment.
The licensee identified that a review
would be performed of check valve maintenance procedures to determine whether a
similar problem existed.
The licensee documented'its evaluation conclusions in the
EAR and in a memorandum dated February 15, 1990. The licensee identified that their
procedures included instructions for match-marking the valve body and bonnet to
ensure alignment of the disc and seat.
Therefore, the licensee concluded that
procedures contained adequate instructions. This conclusion addressed
horizontal
alignment concerns but did not consider the vertical misalignment issue and the need to
ensure correct retainer ring position.
In 1992, the licensee re-evaluated applicability of IN 89-62 and documented
in
CRDR 9-2-0412 that its original conclusions were incorrect. The vendor had revised its
technical manual in October 1990, which included instructions for measurement
of the
vertical "A"dimension (top of retainer ring to top of valve body) before and after
maintenance.
Borg-Warner issued a clarification to its Technical Alert 8909-77-001
(which addressed
the vertical seat alignment issue) in a letter to the licensee dated July
8, 1992.
The letter stated that the vertical alignment problem was limited to
~
~
~
I
I
-28-
welded-neck check valves (such as Valve 1PSIA-V404) and that the welding of the neck
to the body had caused the neck to shrink down, thus causing the bonnet disc assembly
to sit lower in the valve body and possibly not seat properly on the valve seat.
The
licensee issued Procedure 31MT-9ZZ17 on November 30, 1992, which included the
Borg-Warnel technical alert instructions regarding permanent match marks for,
horizontal orientation and measurement
of the "A"dimension.
The licensee's February
15, 1990, evaluation failed to correct an inadequate maintenance procedure for
establishing the correct vertical alignment of the valve disc within the valve body of
Borg-Warner check valves. The resultant excessive'reverse
flowthrough HPSI pump
discharge check valves was considered to be a significant condition adverse to quality.
This was considered to be the fifth example of an apparent violation of 10 CFR Part 50,
Appendix B, Criterion XVI (50-528/-529/-530/9814-03).
C.
Conclusion
The inspectors considered the licensee's OEAs and corrective actions regarding
Borg-Warner check valves to have been inadequate
in preventing or detecting, in a
timely manner, significant conditions adverse to quality relative to the HPSI system.
Two examples of an apparent violation of 10 CFR Part 50, Appendix B, Criterion XVI,
were identified.
e
'I
Miscellaneous Engineering Issues
E8.1
Review of Licensee's Followu
and Si nificant Investi ation Re ort
Ins ection Sco
e
The inspectors reviewed the licensee's activities to assess,
evaluate, and correct the
degraded HPSI system and any potential impact on other safety-related systems.
Included in the review was Significant Investigation Report CRDR 1-8-0238, "Excessive
Leakage Through HPSI Pump Discharge Check Valve 1PSIA-V404," Revision 0,
June 10, 1998. The licensee completed Revision
1 of the report on July 10, 1998, but
the inspectors did not perform a detailed review of the report because it was completed
following the onsite portion of the inspection.
b.
Observations and Findin s
CRDR 1-8-0238 was initiated to evaluate the April9, 1998, failure of Valve 1PSIA-V404
to satisfy its reverse-flow acceptance
criteria. On April 10, 1998, the CRDR review
committee classified the subject CRDR as significant. An investigation team was
assembled
to evaluate the event. The inspectors reviewed the licensee's significant
investigation report dated June 10, 1998. The licensee performed a thorough historical
review of past testing and maintenance
practices.
The report identified the root cause
of the failure of the check valve to be attributed to a common-cause
error in assembling
the valves stemming from inadequate maintenance instructions.
Periodic surveillance
testing was ineffective in identifying excessive reverse flow. The root cause of the
inadequate surveillance test was an ineffective testing configuration developed by
'I
i
r
~
0
-29-
engineering.
Missed opportunities to identify and correct problems with the check
valves also existed in the form of in-house and industry operating experience reviews.
Revision 0 of the report identified the need for 20 corrective actions associated with this
.
event. As of,June 12, 1998, three of the corrective actions had been completed.
A
schedule for the remaining corrective actions had been developed with planned
completion dates ranging from June 21, 1998, through the Unit 3 1998, and Unit 2 1999,
refueling outages.
The inspectors noted that the investigation did not identify any problems regarding
operator performance with respect to the October 1997, Unit 2 outages or the May
1998, entries into TS 3.0.3. Also, the report did not evaluate the safety consequences
of performing routine online maintenance of the Unit 1 and Unit 2 HPSI systems when
the HPSI trains were not isolated from each other.
The licensee informed the inspectors that additional assessment
of the safety
significance of the degraded HPSI condition would be performed and the results of the
assessment
would be documented in the LER supplement.
C.
Conclusions
The licensee's investigation report was objective and provided a candid self-assessment
of its performance; however, it did not evaluate inspector-identified issues in the areas
of operations or online maintenance.
V Mana ement Meetin s
X1
Exit Meeting Summary
The inspectors met with licensee representatives
on June 11, 1998, to conduct a
technical debrief prior to leaving the site.
Following additional in-office inspection, and
telephonic discussions of findings, the inspectors conducted an exit meeting with
licensee representatives
on July 21, 1998. These representatives
acknowledged the
findings presented,
but disagreed with the May 13, 1998, apparent violation of TS 3.0.3.
The licensee stated that operations recognized that Valve 1PSIA-V404 was in a
degraded condition but information existed to conclude that the valve was operable until
testing could determine otherwise.
With respect to Unit 2, the licensee stated that
operations also recognized that Valve 2PSIB-V405 was in a degraded condition but
information existed to conclude that the valve was operable until testing could determine
otherwise.
The inspectors asked the licensee representatives
whether any materials examined
during the inspection should be considered proprietary.
No proprietary information was
identified.
l
!
I
l
i
ATlACHMENT1
SUPPLEMENTAL INFORMATION
PARTIALLIST OF PERSONS CONTACTED
Licensee
G. Andrews, Section Leader, STNOperations
B. Blackmore, Engineer
P. Borchert, Site Manager, Operations
J. Brown, Engineer
R. Buzzard, Senior Consultant, Regulatory Affairs
D. Garnes, Unit 1 Department Leader, Operations
D. Fan, Acting Department Leader, System Engineering
R. Fullmer, Director, Nuclear Assurance
R. Hazelwood, Senior Engineer, Nuclear Regulatory Affairs
W. Ide, Vice President, Nuclear Engineering
A. Krainik, Department Leader, Regulatory Affairs
J. Levine, Senior Vice President, Nuclear
D. Marks, Section Leader, Nuclear Regulatory Affairs
D. Mauldin, Director, Maintenance
D. Oakes, Section Leader, Specialty Engineering
K. Parrish, Section Leader, Transient Analysis
G. Shanker, Department Leader, Specialty Engineering
D. Smith, Director, Operations
N. Spooner, Engineer
P. Wiley, Unit 2 Department Leader, Operations
NRC
M. Fields, Project Manager, Office of Nuclear. Reactor Regulation
J. Shackelford, Senior Reactor Analyst, Region IV
INSPECTION PROCEDURES USED
93702 Prompt Onsite. Response
to Events at Operating Power Reactors
'r
11
-2-
ITEMS OPENED CLOSED and DISCUSSED
~Oened
50-528/-529/-530/$ 814-01
APV
Three examples of an apparent violation of TS3.0.3
involving: 1) failure to initiate actions within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> to place
Unit 1 in a mode in which TS 3.5.2 did not apply on May
13, 1998 (Section 01.1); failure to initiate actions to place
the unit in hot standby within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> when Unit1 did not
have an independent ECCS subsystem operable on
September 4, 1996 (Section E1.2); and 3) failure to initiate
actions to place the unit in hot standby within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />
when Unit 2 did not have an independent ECCS
subsystem operable on October 21, 1994 (Section E1.2).
50-528/-529/-530/9814-02
APV
Two examples of an apparent violation of TS 6.8.1
involving: 1) failure to record in the Unit 2 log an
unexpected decrease
in SIT level on October 10, 1997
(Section E1.1.b.1); and 2) failure to record in the Unit 2 log
an unexpected decrease
in SIT level on October 28, 1997
(Section E1.1.b.1).
50-528/-529/-530/9814-03
APV
Five examples of an apparent violation of 10 CFR Part 50,
Appendix B, Criterion XVI involving: 1) failure to identify
and promptly correct excessive HPSI pump discharge
check valve excessive reverse flowleakage following an
unexpected decrease
in SIT level on October 10, 1997
(Section E1.1.b.1); 2) failure to identify and promptly
correct excessive HPSI pump discharge check valve
excessive reverse flow leakage following an unexpected
decrease
in SIT level on October 28, 1997 (Section
E1.1.b.1); 3) failure to correctly assemble the Unit 1 Train
"A"HPSI pump discharge check valve to correct a reverse
flow leakage condition on April 10, 1998 (Section
E1.1.b.2); 4) failure to develop adequate test procedures
to identify and correct HPSI pump discharge check valve
excessive reverse flowfollowing evaluation of IN 88-70
(Section E1.4); and 5) failure to correct inadequate
maintenance procedures for establishing the correct
assembly of Borg-Warner check valves following
evaluation of IN 89-62 (Section E1.4) ~
0
50-528/-529/-530/9814-04
Check valve design control issues (Sections E1.1.b.2,
E1.1.b.3, and E1.1.b.5).
II
I
1
I
l
l
-3-
50-528/-529/-530/9814-05
APV
Two examples of an apparent violation of TS 3.5.2
involving: 1) failure to have an independent operable flow
path for the Unit 1 Train "B" HPSI system from May 1,
1992 through May 15, 1998 (Section E1.1.b.2); and 2)
failure to have an independent operable flow path for the
Unit 2 Train "A"HPSI system from April 14, 1993 through
May 16, 1998 (Section E1.1.b.4).
Closed
None
LIST OF ACRONYMS USED
APV
'SME
CFR
CRDR
EAR
EDT
gpm
IN
LCO
LER
NRC
OEA
psia
psld
pslg
SDCHX
TS
apparent violation
American Society of Mechanical Engineers
Code of Federal Regulations
Condition Report/Disposition Request
Engineering Action Request
equipment drain tank
emergency operating procedure
gallons per minute
high-pressure safety injection
Information Notice
lnservice Testing
Limiting Condition for Operation
licensee event report
loss-of-coolant accident
low-pressure safety injection
U.S. Nuclear Regulatory Commission
operating experience assessment
pounds force per square inch absolute
pounds force per square inch differential
pounds force per square inch gage
refueling water tank
shutdown cooling heat exchanger
safety injection system
safety injection tank
Technical Specification
Updated Final Safety Analysis Report
Unresolved Item
l
0
-4-
LIST OF DOCUMENTS REVIEWED
Procedures
NUMBER
73DP-9XI02
DESCRIPTION
Pump and Valve Inservice
Testing Program-
Component Tables
Pump and Valve Inservice
Testing Program-
Administrative Requirements
REVISION
Revision 5
Revision 5
73ST-9XI33
HPSI Pump and Check Valve
Revision 9
Full Flow Test
HPSI Pump and Check Valve
Revision 10
Full Flow Test
HPSI Pump and Check Valve
Revision 11
Full,Flow Test
e
31MT-9ZZ17
31 MT-9ZZ17
73 ST-9ZZ25
40OP-9SI02
HPSI Pump Discharge
Check Valve Closed
Exercise Test
Disassembly and Assembly
of Borg-Warner Check
Valves
Disassembly and Assembly
of Borg-Warner Check
Valves
Disassembly and Assembly
of Borg-Warner Check
Valves
Check Valve Disassembly,
Inspection, and Manual
Exercise
ECCS Flow Balance Test
Recovery, from Shutdown
Cooling to Normal Operating
Lineup
Revision 6
Revision
1
Revision 4
Revision 5
Revision 2
Revision 7
Revision 14
ll
h
e +
t
I
f
-5-
73ST-9SI05
40EP-9EO09
Leak Test of HPSI/LPSI
Containment Isolation Check
Valves
Leak Test of HPSI/LPSI
Containment Isolation Check
Valves
Loss of Coolant Accident
Functional Recovery
Revision 4
Revision 3
Revision 5
Revision 6
Calculations
NUMBER
13-MA-Sl-982
DESCRIPTION
REVISION
Evaluation of Allowable Leak
Revision 0
Rate Criteria for
1,2,3PSIA-V404 and
1,2,3PSIB-V405/Assessment
of As-Found Leakage for
2PSIB-V405/1 PSIA-V404
13-MC-SI-215
Revised Single HPSI Pump
Delivery Curve. for Cold Leg
Injection and Flow Rate
Requirements for Technical Specification 4.5.2.h
Revision
1
Drawincrs
NUMBER
01-M-SIP-001
01-M-SIP-002
79120
DESCRIPTION
Safety Injection and
Shutdown Cooling System
Safety Injection and
Shutdown Cooling System
Valve Assembly- 4 inch,
1500 LB, swing check valve
REVISION
Revision 22
Revision 21
Revision C
-6-
Condition Re ort/Dis osition Re uests
NUMBER
1-5-0131
'
DESCRIPTION
REVISION
Valve 1PSIEV113 failed seat
May 22, 1995
leakage test
9-2-0412
9-8-0893
2-7-0420
1-'8-0238
2-8-0128
Incorrect conclusion in
review of IN 89-62
Design configuration
documents not updated "
SIT level dropped during
cold-leg boration
Valve 1PSIA-V404 failed
reverse-flow leakage test
Evaluate excessive check
valve back leakage
July 6, 1992
June 5, 1998
October 28, 1997
April 10, 1998
April 16, 1998
0 erabilit
Determinations
NUMBER
203
204
DESCRIPTION
Evaluation of piping
operability after
pressurization
Evaluation of HPSI
operability
REVISION
April 12, 1998
May 22, 1998
Work Orders
Deficiency Work Order 836600 and Work Orders 840712 and 840826 associated
with the
assembly and disassembly of 1PSIA-V404.
Other Documents
NUMBER
DESCRIPTION
REVISION
.7
-7-
Significant Investigation
Report CRDR 1-8-0238
I
Significant Investigation
Report CRDR 1-8-0238.
Reportability Determination
316-42-WEW/EDF
109-374-RAK/6LI/TNW
167-460-ECS/RRR
Excessive Leakage Through
HPSI Pump Discharge
Revision 0
Excessive Leakage Through
HPSI Pump Discharge
Revision
1
HPSI Discharge Check
Valves
Memorandum Regarding IN 88-70 Evaluation
Memorandum Regarding IN 88-70 Evaluation
Memorandum Regarding IN 89-62 Evaluation
June 10, 1998
July 10, 1998
May 21, 1998
August 31, 1990
January 12, 1989
October 9, 1989
161-2874-ACR/RAB/DAF
Memorandum Regarding IN
.
February 15, 1990
89-62 Evaluation
162-8368-KCP/PMC
281-2103-MAR/JAB
Simulator Training Scenario
Clearance Order for
Individual Plant Examination
Memorandum Regarding
HPSI Safety Assessment
Memorandum Regarding
Check Valve Leakage
Criteria
LOCAwith Degraded HPSI
Flow
- May 20, 1998
May 14, 1998
July 14, 1998
October 21, 1994
April 7, 1992
I
I
I
f
I
ATTACHMENT2
SIMPLIFIEDHPSI FLOW DIAGRAM
V
~ r
FrOnl-Linc Systems
+IA,YMA QIA nein
a) ~ y)et
Cl)1 Ily))e
Slk yle)
SIA Vyite
Srf>YU>>S
~r))rf
T))>$
Clil 1DI
N
te
SIO Iiyill
SOjii
)
line
Sle fsylee
>el
i~ ~
SIATni i)AWite
)
1)e vl)l Ile ytll lie T)N
ren
sl ~ well tons
~e
) l"
N
slA wits
cere )l)r>>r)e
t
se>rte
) ~
SIAwill'g Ile Ire
SIDVI)$
$)e Te>I
nrl)
S le leI
Sle Y~i
Sle Tile
S)ei)Tete
)OA) SIA I)Tell
)T~
SIA welt tant
Sa vlu
ly $)AW)li Sleet)t
$)% Sl)A
SHIA Crit lre
~vs
~VS)I
)IAliven
$)e v)ee
'gle"~~I
SIS WNI SIAYat lon
ne yni
Sill Yell
1)A IIY)et
fSy eee
IOll 1IAWI S
Ives
1)S Wai
SO)le
tee)
su welt tant
as
T Is)
SIS.T)$ ~
'N'
- L"-'::--::
l
'.~WIN RDTSS)
Sit selt
IrrtIe a>N)re
SLY)il SILTilt
MAWill
saw)el
$)e VIM tate
tea
Sls WIN lol)e
)IAwe)e
su weee
SIA Vi)l IOII
I
su weee )IA Ye) ~ tol)
stye)e
)u vi)e su.vat su )IT)ee
loll SIA Will
Su WI)l lotlt
SIA vile
Sile)es
lsuwai 1st Yt)i
$)S'5IIC
L>rt IAC)VSee
~TSAR t)iv)n
r>>J)r
Stte WMI steyM)l
~) >>
$%se)1
I
5)A Ilvil) SIA Ytll
IIA.vi)e
SSSI A~
nA tel
litt)A~
SIAlel
loll s)A weo
SIA nycti
SIA ye) ~
SIA Yi) r
IOII
L)O)ea>Mire
$)e.vile
i~
45 Sutsvne
"I%ASt>A'Pfe,.;t, , >><
l iy r
p
Figurc 5.2 l
Higlt/LowPrcssure Safety Injection System
pc1. 0
4/7/92
5.2.1 Front-l.Inc Systems
5-166
I
t
f
ATTACHMENT3
DEGRADED HPSI FLOW PROFILE
l
f
I
<<g>>
r,
4
1800
1600
1400
1200
CL
us
1000
D
th
800
O.
CAOK
600
400
200
0
0
ESTIMATED UNIT 2 TRAINA
PERFORMANCE WITH LEAKAGE
PAST 2PSIB-VA05
v t/,m+t
Pg't )"
- .}..'lk
r
%@i~+.~5
200
~ gears
FINALDEGRADED'Ht
l'FL
'"<<%'"""%.~Y
Qss '.CP <<.0g
,rg$XPqj;~pe+~~
~ <<s
<<v
g~)
'"
<<
wr
<<I
) t<<
\\ p
<<(Y,'r',<<)
4<<
s
~
4e
~
<<4
Ql
1%,-
M CH,
<<<<
s
r <<'rO}
'< ~ c rv ~'7V
t
.,w.:P.+.c
- r'r*
)f
.
.40
02QZ
0
XI
0
m
mZ
C
ZmO
Z l
<<
mC.
0
Cl
0
~
II
gg
-j8
)
Im
O fll
3) (
mr
~ C
mo
OZ
Q
f)l p
br
fil f
~O
C~
grfll
Oy
t<<l g)
ffl fll
pm
O +
O
O
Z.0
D
'.V)
e~l
)g
~g
O
A
0z
g