IR 05000369/1988029
| ML20155E733 | |
| Person / Time | |
|---|---|
| Site: | McGuire, Mcguire  |
| Issue date: | 09/23/1988 |
| From: | Croteau R, Nelson W, William Orders, Peebles T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML20155E730 | List: |
| References | |
| 50-369-88-29, 50-370-88-29, NUDOCS 8810120392 | |
| Download: ML20155E733 (11) | |
Text
."
'
.
p* Etcp 3"
t UNITED STATES j
.'f REGION 11 j
NUCLEAR REGULATORY COMMISSION o,
-% *
e 101 MARIETTA ST., N.W.
j e,,,e ATLANTA, GEORGIA 30323 Rep' ort Nos. 50-369/88-29 and 50-370/88-29 Licensee:
Duke Power Company 422 South Church Street Charlotte, NC 28242 Docket Nos.:
50-369 and 50-370 License Nos.:
NPF-9 and NPF-17 Facility Name:
McGuire Nuclear Station 1 and 2
'
Inspection Conducted, Au ust 29 - September 9, 1988
}ffft $/
9 Okf Inspectors-
"W. Ord rs Je ior Re dent Inspector
-
X)a't'e/Si gn'ed
~
/
.056 l
V f
"D. 'fie f of Reside Clnspector
/Dat(Signed klhMG/flAl /
V e2 / W
/R. Croteau, Residen ' Inspector
/Dat(Signed
~
Approved by:
f
.1
ff T. A/ Pe~ebles,'SFetTon Chief Ohte Signed Division of Reactor Projects SUMMARY Scope:
This special inspection was performed to evaluate the inoparability of safety systems due to inadequate post modification testing.
Results:
In the areas inspected, three violations were identified 8810120392 880927 PDR ADOCK 05000369 G
PNV
.
.
REPORT DETAILS 1.
Persons Contacted Licensee Employees
- B. Hamilton, Superintendent of Technical Services T. McConnell, Plant Manager
- E. McCraw, Project Services Engineer
- R. Pierce, Instrumentation and Electrical Engineer M. Rains, Project Services Engineer
- M. Sample, Superintendent of Maintenance
- R. Sharp, Compliance Engineer
- B. Travis, Superintendent of Operations Other licensee employees contacted included construction craftsmen, technicians, operators, mechanics, security force members, and office personnel.
- Attended exit interview 2.
Unresolved Items An unresolved item (UNR) is a matter about which more information is required to determine whether it is acceptable or may involve a violation or deviation.
There were no unresolved items identified in this report.
3.
Executive Summary Three situations were recently identified at M:Guire involving unknown inoperability of safety systems for extended periods of time.
The inoperabilities were created during modifications of valve motor operators and control circuitry.
In all three cases, inadequate review to assure proper installation, proper return to service, and adequate post modification testing allowed modification errors to go undetected.
In the context of this report, the term "testing" refers to all verifications, made at the completion of maintenance or modifications.
.
intended to ensure that the maintenance or modification was completed properly and that plant equipment affected i: operable.
The licensee's testing program divices testing into two areas: "functional verification"
,
and "retesting".
Functional verification is 6 check to ensure that requested maintenance was performed and that the subject equipment performs all of its intended functions and/or has been properly returned
,
to service. Functional verification is conducted by the craf t performing the work; e.g., af ter tightening valve packing, ensure packing leakage is
-, - _,
- -.
- - - -.
,.
,,..,. _. - _ _..
,_...-,c__...-
--_
.
.
.
acceptable; af ter modifying circuitry, perform an electrical continuity check.
Retesting is the formal dynamic performance of all or portions of Preoperational or Periodic Tests to verify the component or system meets applicable acceptance criteria and/or Technical Specification require-ments.
Retesting is conducted by the Performance organization; e.g.,
after valve motor operator replacement. conduct a valve stroke timing test.
The licensee considers that this "overlapping" testing scheme consisting of functional verification and/or retesting provides a high level of confidence that equipment will operate as designed.
Overlap testing frequently replaces dynamic "full function" testing particularly for safety related equipment for which a full function test would be impractical.
4.
Inoperability of 1NV142B a.
Event Description
'
On August 29, 1988, licensee personnel were conducting a random inspection of control circuit cabinets to ascertain the existence of any sliding links that may be incorrectly open. A sliding link is an electrical disconnect in a circuit that can be temporarily opened to de-energize portions of a circuit to permit maintenance or testing.
A link designated as I-1 in cabinet 1ATC4A in Unit I w&s discovered to be incorrectly open.
This link is included in the circuitry for automatic closure of Chemical and Volume Control (NV) System valve INV1428, B Train Volume Control Tank (VCT) outlet valve. NV1428 is intended to automatically close in the event of an Engineered Safety Features (ESF) actuation or in the event of Low-Low level in the VCT.
With link 1-1 open, 1NV142B will not function if called upon to do so automatically.
The normal open and close portions of the circuit were not af fected by the open link.
Therefore, the open and close functions controlled from the control board were unaffected. 1NV1428 was declared inoperable following the discovery of the open link.
Unit I was in mode 1 at the time with NV142B in its normally open position.
In normal operation, the valve cannot be shut without securing the Chemical and Volume control system.
The link was subsequently closed and electrical checks performed to insure no further problems existed in the circuitry.
No additional problems were found.
INV142B was then declared operable.
The licensee determined that the link had been open since the inst;'lation of Nuclear Station Modification (NSM) % 12066/00, Installation of Electric Anti-Hammer Circuit on 1NV142B, completed October 7, 1987 during the last refueling outage.
The licensee stated at the time that personnel error was responsible for leaving the sliding link open following installation of the NSM and considered this to be an isolated case.
However, reasonable assurance of the proper return to service of a component required to
-.
-
, -... _ - -
. _ _ _
.
- _ _ _ _ _
_
,
_ _ -
_
_
.
.
.
.
perform a safety function should have required a dual verification.
Additionally, personnel error wa< responsible for review and planning an inadequate post modificativ functional test.
The testing specified to be conducted for this NSM consisted only of pre-installation checks of new components and a simple valve stroke timing test (VST).
This VST requires remote operation of the valve f rom the control room which utili:es the normal OPEN and CLOSE portions of the valve's control circuitry.
The NSM, however, modified the automatic portion of the control circuit which is not uti'.i;ed during normal, manual valve cycling.
This portion of the circuit contained the open sliding link. Therefore, the valve stroke timing test alone was ir _Jfficient as a post modification test. Had complete testing been performed, i.e.,
testing to include the automatic circuit; the open sliding link would have been detected.
The licensee concluded, therefore, that valve INV1428 had been inoperable from the time of completion of the NSM on October 7, 1987, until August 29, 1988.
b.
System Description The NV system is composed of 3 main subsystems:
1) charging, letdown and Reactor Coolant Pump seal water; 2) chemical control, purification, and makeup; and 3) standby makeup.
The charging and letdown functions of the system are employed to maintain a programmed water level in the reactor coolant system pressurizer, thus maintaining proper reactor coolant inventory during plant operation.
The 3000 gallon VCT functions as a surge volume in the low pressure portion of the NV system. Hydrogen overpressure (approximately 30 psig) is maintained on the VCT to provide for oxygen scavenging and to ensure net positive suction head to the charging pumps.
In the event of an ESF actuation, the centrifugal charging pumps
',
function as a subsystem of the Emergency Core Cooling System (ECCS),
specifically as the high head safety injection pumps. During an ESF actuation, the low pressure portina of NV automatically realigns to shift charging pump suction fr_
ne VCT to the Refueling Water Storage Tank (FWST). NV142B is one of two motor operated valves in series that automatically shut to isolate the VCT from the charging pumps. The second valve, NV141A is the A train counterpart for the B Train NV142B, Two valves in parallel, NV221A and NV222B, A train and B train respectively, automatically open to provide cnarging pump suction from the FWST.
The FWST is intended to supply all of the
,
ECCS water supply for the first phase of a loss of coolant accident.
!
The PWST water contains 2000 ppm minimum boron concentration which is intended to mitigate insufficient shutdown margin that maybe caused l
by the postulated accident.
The VCT boron concentration, during steady state operations, is the same as the prevailing reactor coolant system boron concentration, which is much lower than that of the FWST.
.
- "
'
."
.,
!
c.
Safety Significance As stated above, NV141A and NV142B shut in the event of an ESF actuation. With these valves shut, the water and gas in the VCT is isolated from the flow path for the highly borated ECCS water from the FWST to the charging pumps.
If both valves fail to shut, the charging pumps will continue to take suction from the VCT, i.e., flow
'
to the pumps will be supplied by both the VCT and PdST. Initially, most flow would come from the VCT due to the hydrogen overpressure providing greater suction head to the pumps. The worst case accident scenario for this situation would be a Small Break Loss of Coolant Accident (SBLOCA) in which both charging pumps are injecting and reactor coolant pressure is still above the injection pressure of the
,
Intermediate Head Safety Injection (NI) pumps.
In this case,
'
,
licensee analysis indicates that the contents of the VCT would be i
depleted and the charging pumps rendered inoperable due to gas binding in approximately 18 1/4 minutes. This analysis was based on typical initial conditions of the VCT at 50% level and 30 psig gas pressure.
With the unknown inoperability of NV142B, any period of inoperability of NV141A would render both valves inoperable and thus create the possibility of both valves remaining open during an ESF actuation.
On numerous occasions while NV1428 was inoperable NV141A's emergency
!
'
i power supply or ESF actut*,1on logic was inoperable due to maintenance or testing.
Greater significance occurs, however, in consideration of a single failure analysis with NV142B unknowingly inoperable and a failure of NV141A to shut during an ESF actuation.
The Safety Injection Emergency Procedure, EP/1/A/5000/01, requires the operators to check the ESF Monitor Light Panel for mis-aligned valves and to establish correct alignment. Misalignment of NV141A and NV1428 would be indicated.
In this case, the licensee has stated that 18 1/4 minutes is sufficient time for operators to detect that the VCT outlet valves were open and to take action to shut NV1428 by using the control board switch. Manual operation of NV142B from the board was still available to the operator during the period of automatic inoperability.
The licensee conducted simulator testing of this
-
scenario and operators properly diagnosed the condition in approximately 45 seconds.
Therefore, gas binding of the charging
pumps would be prevented although a delay in injection of highly borated water from the FWST would still exist.
d.
Violation Technical Specification (TS) 3.5.2. ECCS Subsystems - T avg? 350 F, requires that two independent ECCS Subsystems shall be OPERABLE comprised of, in part, an OPERABLE flow path capable of taking suction from the refueling water storage tank on a safety injection signal. This is required in Modes 1, 2, and 3.
One subsystem may be
.
.
-
.
inoperable for up to 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.
Inoperability in excess of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> requires unit shutdown and progression to hot shutdown. The extended inoperability of INV142B from November 10, 1987, when Unit i entered
.
Mode 3, until August 29, 1988, is an apparent violation of TS 3.5.2.
l (Violation 369/88-29-01)
5.
Inoperability of INS 18 and 1NS1 l
a.
Event Description During the analysis of the 1NV1428 event discussed above, it became clear to the licensee that other cases of inadequate post
modification testing of safety related motor operated valves may have taken place.
Specifically, NSM MG-11930, Rotork Motor Operator -
Torque Switch Bypass Modification, modified the automatic portion of valve circuitry and, as in the case of 1NV142B post modification testing did not require an operability verification of the automatic function of the affected valves.
This modification had been
'
performed on numerous safety related valves in both units.
The
licensee took action to determine if any of these additional valves were inoperable. Also, the licensee stated that they had suspected for some time that valve interlocks on this and other modifications may not have been properly tested and began checks to determine if inoperabilities existed. The inspector determined that the licensee had suspected the interlock testing proolem at least since mid 1987.
This was based on interviews with licensee personnel and review of completed work requests.
,
On September 2,1988, during checks on Containment Spray (NS) System valves INS 16 and INS 1, A and B train containment sump to NS pump supply, it was determined that an interlock on each valve was improperly installed.
This resulted in a condition where neither valve could be opened from the control room when required during the containment sump recirculation phase of a loss of coolant accident i
(LOCA).
Thus, neither train of NS was operable. Unit I was in mode 1 at the timn. These normally shut valves do not automatically open on an ESF actuation, but must be opened if containment spray is required to continue when the PWST volume has been depl?ted during a LOCA.
The Shif t Supervisor on duty in the control rocm determined that the valves could still be opened by use of the handwheels or by shorting out contacts in the valve motors' motor control center. The
,
operators cn duty were instructed that these were the methods to be used if the valves were required to be opened prior to corrective action on the interlock being completed. Steps were taken to disable the interlock thus permitting operation from the control room switch.
,
Training was provided to all shifts specifying the chaage in the
'
function of the interlock.
1
_ _ _ _ _. _ _ _ _ _ _ _ __
_
_ _ _ _ _ _ _ _. _. _. _ _ _.
~. _ _. -
..
,
As in the case of NV142B, the licensee concluded that this
- tuation had existed since installation of the modification during
.e last Unit 1 outage in the fall of 1987.
Again, the review to assure proper installation and proper testing for the NSM was insufficient
- to reveal an error in installation.
No other valve inoperabilities were immediately identified during the licensee's investigation.
b.
System Description The NS System 11ong with the Ice Condenser is designed to prevent containment pressure from exceeding design values during a LOCA. NS consists of two independent trains of piping, valves, a heat exchanger, and a pump that are designed to transfer water from the FWST to the spray headers located high in the containment in the event of a LOCA that results in high containment pressure. The spray condenses the steam that would be present, thus lowering th:
pressure.
Utilizing A and B train convention as follows: A (B);
Flow from the FWST is provided for each train via a normally open valve, NS20 (NS3), to the duction of the NS pump. Upon depletion of the water in the FWST NS is required to be realigned to take suction from the con *.ainment sump if continued NS operation is needed. NS18 (NS1) is the valve that must be opened to supply flow from the sump.
Flow from the sump is provided to NS18 (NS1) via Safety Injection (NI) valve NI185A (NI1848).
(NI185A and NI1848 also provide sump flow to the Low Head Safety Injection (ND) pumps.) An interlock is designed to prevent opening NS18 (NS1) unless NS20 (NS3) is sSt and NI185A (NI184B) is open.
This interlock prevents aligning t te NS pumps to the PdST via the suction of the ND pumps, thereby degrading the ND flow. The interlock was discovered to be installed such that NS18 (NS1) could be opened only if NS20 (NS3) and NI185A (NI184B)
were shut. During an accident, however, NI18SA (NI184B) is opened to provide sump flow to ND prior to NS being aligned to the sump.
Therefore, since NI185A (NI1848) is open when the operator shuts NS20 (NS3) and then attempts to open NS18 (NS1), the incorrect interlock prevents NS18 (NSI) from opening, c.
Safety Significance The interlock problem existed in both trains of NS for an extenced period of time, rendering both trains inoperable.
The licensee stated that had an accident occurred requiring NS realignment to the sump, the valves could have been opened using the handwheels or at the motor control center.
Further, the licensee stated, the inability to open the valves from the control room in no way affected NS operation prior to realignment to the sump. However, since NS is secured just prior to sump realignment, then restarted following
'
i
, - - - -,
-
,
, -,, _ - - - - -
, - -, - _. - - _ - _ _ - -,,.,,. - _ _ _ _ _ _.
-__,
.yy g
_
~
-
.,
realignment, no containment spray would take place for the period of time required for operators to diagnose the problem and take manual action to open the valves. Neither an estimate of the increased time nor an analysis of the consequences was available from the licensee at the end of this inspection.
d.
Violation TS 3.6.2, Containment Spray System, requires that two independent Containment Spray Systems shall be operable with each spray system capable of taking suction from the Refueling Water Storage Tank and transferring suction to tne containment sump.
This is required in Modes 1, 2, 3, and 4.
With both independent systems inoperable, action must be initiated within one hour to shutdown the unit and proceed to cold shutdown. The extended inoperability of both trains of N5 from November S, 1987, when Unit 1 entered Mode 4, until September 3,1988, is an apparent violation of TS 3.6.2 (Violation 369/88-29-02).
6.
Inoperability of 2ND4B a.
Event Description On September 9, 1983, the licensee provided information to the inspectors concerning another case of inadequate review to assure proper installation and proper post modification testing that allowed an installation error to go undetected.
On June 27, 1988, during installation of an unrelated NSM, the licensee discovered that an interlock problem existed on Residual Heat Removal (ND) System valve 2ND4B in Unit 2.
Safety Injection (NI) System valve 2NI184B, B train ECCS supply from the Containment Sump, was wired incorrectly. This resulted in 2ND4B unable to automatically close as designed during the swap-over of ECCS water supply from the FWST to the containment sump in the postulated LOCA. A conductor had been connected to the wrong terminal during installation of NSM 20700 Relocate Torque Switch Bypass to Add-on Pack, in June 1937 during the previous Unit 2 refueling outage.
The miswiring was identified during the Unit 2 1983 refueling outage. This resulted in ND train B being inoperable for the entire cycle 4 run, although manual control board operation was still available.
At the time of discovery of the problem, Problem Investigation Report (PIR) 2-M88-0142 was initiated. Design Engineering evaluated past operability, as requested on the PIR, and incorrectly concluded that 2ND4B and thus ND B train remained operable during the period that the valve interlock problem existed since manual operation was not affected. TS 3.5.2, ECCS Subsystems -
T avg 350'F, specifically identifies automatic swap-over as being required for operebility.
Instrumentation and Electrical (IAE)
..
.'
t Engineers stated on the PIR's Proposed Resolution of Problem that the miswiring was an isolated incident, but identified a potential weakness in functional verification of motor operated valves with interlocks (this weakness had been previously identified).
No further corrective action was initiated to determine if other interlocks on other valves were affected.
Several NRC concerns arise:
-
'
'
Compliance personnel should have been able to identify that the valve had been inoperable, but solicited Design Engineering evaluation.
The Design Engineering evaluation of the valve's operability was incorrect.
Having known of the interlock problem (see paragraph 5), and again identifying a programmatic weakness in functional verification, the licensee took no action to further evaluate the extent of the problem until the 1NV142B t rent surfaced on August 29.
b.
System Description
.
The ND system functions as the Low Head Safety Injection Subsystem of ECCS during a LOCA. The system consists of two independent trains of piping, valves, a heat exchanger, and a pump.
On an ESF actuation, the ND system is designed to start and deliver water from the FWST to the reactor coolant system once pressure has decreased to below that j
of the discharge of the ND pumps. Upon imminent depletion of the
FWST volume, the system is designed to automatically realign to take suction from the containment sump. Af ter this realignment is made, ND can be configured to supply suction to the Intermediate and High Head Safety injection pumps.
ND4B is the normally open valve in 8 train supplying flow from the FWST that is required to shut during the swap-over. NI1848 is the valve that opens during the realignment to begin flow from the containment sump.
The interl.9 is designed to prevent ND4B from closing until N!1848 is open, thereby insuring
,
continunus flow to ECCS.
'
c.
Safety Significance Since ND4B would remain open due to the interlock error described above, B train ND and ECCS would be aligned to both the FWST and to the sump.
Left in this configuration, the volume in the FWST would be depleted and air binding of the ND oump could occur.
Upon automatic swap-over, the control room operators are required by procedure to verify that the proper system align ents are in place.
..
,
As in the case of the NV1428 problem discussed in paragraph 4, a misalignment of ND4B would be evident to the operators and manual action could be taken to shut the valve. An analysis by the licensee of the time required for operators to diagnose the problem and take corrective action and of the time for PWST depletion was unavailable at the close of this inspection period. During the period of B train inoperability there were numerous occasions of planned A train
'
!
inoperability for various reasons, thus rendering both trains of ND and ECCS inoperable.
d.
Violation TS 3.5.2, ECCS Subsystems-T avg? 350*F, requires that two independent ECCS Subsystems shall be OPERABLE comprised of, in part, an OPERABLE flow path capable of taking suction from the refueling water storage tank or, a Safety Injection signal and automatically transferring suction to the containmerit sump during tha recirculation phase of operation.
This is required in Modes 1, 2, and 3.
One subsystem may be inoperable for up to 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.
Inoperability in excess of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> requires unit snutdown and progression to hot shutdown.
The extended inoperability of 2ND4B from July 1, 1937, when Unit 2 entered Mode 3, until Unit 2 shutdown on May 27, 1988 is an apparent violation of TS 3.5.2 (Violation 370/83-29-03).
7.
Post Maintenance / Modification Testing A review of events in the recent past revealed several instances of inadequate testing, although not specifically post modification testing:
In February 1987, post maintenance testing on INM26B was inadequate
'
in that whatever testing was performed did not reveal that terminals had shorted, causing the valve to cycle open on an ESF signal instead of close.
This is apparently another example of the automatic portion of a valve's circuitry being affected by maintenance with the post maintenance test only verifying the operability of the normal open/close circuits.
(Inspection Report 87-03)
On March 28, 1983, it was determined that 1RN21 underwent maintenance without a subsequent retest.
(Inspection Report 88-09)
In August, 1987, no retest was performed on the flos instrument
controlling ND pump 1B recirculation valve, IND67c, resulting in a violation.
(Inspection Report 87-41)
On November 19, 1937, while reviewing an IAE work request for final signoff, an IAE staff person discovered that additional testing was needed on 1RN2358, 1B NS heat exchanger cooling water inlet (LER 369/87-27)
These instances illustrate that post maintenance testing has been a
problem at McGuire.
l l
-
-
'
.
. ' '
The licensee noted that some of the above examples, as well as the examples detailed in this report were caused by personnel error in that the existing testing progran was not followed. The fiRC recognizes that personnel errors are inevitable, and that is one of the major premises for establishing a post maintenance / modification verification and testing program - to revea! personnel errors committed during the maintenance.
The licensee's overlapping testing program allowing some functional testing to consist of static work verification (e.g., circuit continuity checks) by the same personnel who performed the work, introduces the possibility of personnel error into the testing. The result is that the testing program predicts that the equipment should work instead of verifying that it does work.
Retesting ESF equipment without utilizing the ESF circuitry results in, as can be seen in this report, a valve that will shut in tha TS required amount of time, but given an ESF signal, will not shut at all, thus limiting the valve's performance to the amount of time necessary for the operator to take action.
The licensee has i.cknowledged that a programmatic problem exists in its functional verification program, and stated that steps are being taken to alleviate the problem.
A revision to Maintenance Management Procedure (MMP) 1.6, Maintenance Activities Associated with Functional Verification, is to take ef fect October 1, 1988.
According to the licensee, this revision redefines verification methods, and should prevent the types of problems discussed in this report.
Additionally, the licensee has initiated a study of all safety related motor operated valves to identify potential candidates for further testing.
8.
Exit Meeting The inspection findings identified below were summarized on September 9, 1988, with those persons indicated in paragraph 1 above.
The following items were discussed in detail:
(OPEN) Violation 369/88-29-01, One train of ECCS flow path inoperable in excess of TS Limiting Condition for Operation (LCO) time limits.
(OPEN) Violation 369/83-29-02, Both trains of Containment Spray System inoperable.
(OPEti) Violation 370/88-29-03, One train of ECCS flow path inoperable in excass of TS LCO time limits.
The licensee representatives present of fered no dissenting comments, nor did they identify as proprietary any of the information reviewed by the inspectors during the course of their inspection.