IR 05000528/1993048
| ML20059K714 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 01/04/1994 |
| From: | Wong H NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| Shared Package | |
| ML20059K671 | List: |
| References | |
| 50-528-93-48, 50-529-93-48, 50-530-93-48, NUDOCS 9402020223 | |
| Download: ML20059K714 (23) | |
Text
__
_
'
.
.
'
.
,
'
U. S. NUCLEAR REGULATORY COMMISSION
~
REGION V
v Report Nos.
~
50-528/93-48, 50-529/93-48, and 50-530/93-48 Docket Nos.
50-528, 50-529, and 50-530
,
.
License Nos.
NPF-41, NPF-51, and NPF-74 Licensee:
Arizona Public Service Company P. O. Box 53999, Station 9082 Phoenix, AZ 85072-3999
,
-i Facility Name:
Palo Verde Nuclear Generating Station Units 1, 2, and 3
Inspection Conducted:
November 2 through December 6, 1993 Inspection Location:
Maricopa County, AZ
!
s Inspectors:
J. Sloan Senior Resident Inspector H. Freeman Resident Inspector
'
J. Kramer Resident Inspector A. MacDougall Resident Inspector
,
T. Alley Department of Energy i
P< h W
/h/19 Approved By:
E Wong, Ch'ief
Date Signed Reactor ProjectsSection II Summary:
Areas Inspected:
Rot"ine, announced, resident inspection of:
the review of plant activities
surveillance testing - Units 2 and 3
_
e plant maintenance - Units _1, 2, and 3
lifting of main steam safety valve - Unit 1
main steam safety valve operability determination - Unit 1
unmonitored reactor coolant system draining - Unit 1
missed surveillance - Unit 2
'
mid-loop operations - Unit 3
control element assembly subgroup slip and reactor trip - Unit 3
followup on previously_ identified items - Units 1,: 2, and 3
review of licensee event reports - Unit 3
During this inspection the following inspection procedures were utilized:
i 37700, 4.0500, 61726, 62703, 71707, 90712, 92700, 92701, 92702, and 93702.
j Safety Issues Management System (SIMS) Items: None.
9402020223 940113
"
PDR ADOCK 05000528 G
pyg
_j
__, _ __
-
,
,
_
~
l
--
._
_
_
__
_
_
..
-
,
Results:
General Conclusions and Specific Findinos:
,
Strengths:
- Operators and management demonstrated good command and control of activities during mid-loop operations. in Unit 3.
An inquisitive auxiliary operator identified disconnected hoses for the refueling water level indicating system prior to mid-loop operations (Paragraph 9).
'
The licensee performed a thorough and self-critical evaluation of the
>
cause of the lifting of the Unit 1 upper guide structure during refueling i
activities with a control element assembly unlatched (Paragraph 11.b).
,
Engineering response to the lifting of main steam safety valves in Unit 1
was sound (Paragraph 5).
QC inspection during emergency diesel generator work identified that a l
wrong type of splice was about to be used and the QC inspector flagged this to the workers attention (Paragraph 4).
Weaknesses:
i Operators failed to monitor for approximately eight minutes reactor
coolant water level.during a drain down operation (Paragraph 7).
Operators failed to perform a surveillance test when required by-
Technical Specifications (Paragraph 8).
Operators failed to independently verify the position of a valve after
repositioning the valve as required by procedures (Paragraph 2.d.(4)).
Operators failed to restore the chemical and volume control system to its
normal automatic configuration following a dilution ' operation, and failed
'
to correct the condition when identified during turnover (Paragraph
2.d.(4)).
'
The licensee missed at least two significant opportunities to determine'
that the core operating limit supervisory system databases contained data from the previous operating cycle (Paragraph 11.c).
-
'
Maintenance procedural weaknesses and communications errors resulted in
the refueling water level indicating system (RWLIS) not being properly connected prior to mid-loop operations in Unit 3 (Paragraph 9)
,
,
Sionificant Safety Matters: None.
.
Summary of Violations:
Of the 11 areas inspected, two cited violations were i denti fied. One violation in Unit 1 involved operators not monitoring a reactor coolant system draindown evolution for eight minutes, and one i
,
i
,-
..
~. - -
...
-.
.
.
-
-.
-
-
.
.
l
- --
-
'
violation i.n Unit 2 involved the failure to return a valve controller to
!
automatic after a dilution operation.
.
Three non-cited violations were identified ~ involving: (1) the failure to
)
. independently verify the~ position of a valve after its position was changed,
_(2) the failure to verify the operability of alternate power sources after an
'l emergency diesel generator was taken out of service, and (3) the failure to
verify a control element assembly was latched prior to lifting the upper guide'
structure from the reactor vessel.
i
f Summarv of Deviations: None.
i Unresolved-Items: None.
a i
.
l
!
i
!
i I
i
!
!
t l
{
!
i t
!
i
.f i
!
..!
I
?
I
[
!
!
E
.
- _., -..
.
. _
.
..
.
- -.
-_
.
.
_
_
_
_
_.. ~
.-
_
,
,
-
.
.
e
=
DETAILS
-
1.
Persons Contacted The below listed technical and supervisory personnel were among those
!
-
contacted:
Arizona Public Service Company (APS)
l R. Adney, Plant Manager, Unit 3
J. Bailey, Assistant Vice-President,. Nuclear Engineering &-
Projects
R. Bouquot, Supervisor, Quality Audits and Monitoring
L. Clyde, Manager, Operations, Unit 3
'
J. Dennis, Manager, Operations Standards / Plant Support
R. Flood, Plant Manager, Unit 2
'
R. Fountain, Supervisor, Quality Audits and Monitoring
-
R. Fullmer, Manager, Quality Audits and Monitoring
D. Gouge, Director, Plant Support
!
'!
B. Grabo, Supervisor, Nuclear Regulatory Affairs
W. Ide, Plant Manager, Unit 1
J. Levine, Vice President, Nuclear Production
,
D. Mauldin, Director, Site Maintenance and Modifications
'
G. Overbeck, Director, Site Technical Support F. Riedel, Manager, Operations, Unit 1 C. Russo, Manager, Quality Control-J. Scott, Assistant Plant Manager, Unit 3
C. Seaman, Director, Quality Assurance and. Control M. Shea,-
General Manager, Radiation Protection
'
R. Stevens, Director, Regulatory & Industry Affairs
C. Thiele, Supervisor, Reactor Engineering S. Troisi, Manager,OperationsComputerSystems(Acting)
P. Wiley, Manager, Operations, Unit 2
Others
.
J. Draper, Site Representative,-Southern California Edison
F. Gowers, Site Representative, El Paso Electric
R. Henry, Site Representative, Salt River Project
~
Denotes personnel in attendance at the Exit meeting held with the
NRC resident inspectors on December 9,'1993.
2.
Review of Plant Activities - Units 1. 2. and 3 (71707)
a.
Unit 1 Unit:1 began the inspection period near the end' of refueling outage IR4 in Mode 5.
On November 3, 1993, operators inadvertently allowed indicated reactor coolant system level to enter the reduced inven-
.
. tory range during a routine draining evolution (see Paragraph 7).
The unit entered Mode 4 on November 18, and Mode 3 on November 19.
.
.
-
.
,
.
While performing testing of the atmospheric dumps valves on November 20, one main steam safety valve lifted prematurely (see paragraph 5). The Unit entered Mode 2 on November 24, and the main generator breakers were closed on November 26. The unit ended the inspection period performing full power core physics testing at 98% power, with two main steam safety valves gagged closed and the intention to operate at 85% power after completion of core physics testing.
b.
Unit 2 Unit 2 entered this inspection period in Mode 3, following the November 1, 1993, reactor trip caused by the loss of power to bus NBN-S01 (described in NRC Inspection Report 50-528,529,530/93-43).
Following repairs to the bus NBN-S01 potential transformer drawer secondary contacts and the steam supply valve to auxiliary feedwater pump "A",
the licensee restarted the reactor, achieving criticality at 4:30 p.m. (MST) on November 2.
A normal power ascension to 85%
followed. The unit operated at essentially 85% for the remainder of the inspection period.
c.
Unit 3 Unit 3 began the inspection period at 85% power. While down-powering the unit to repair a steam leak on November 3,1993, control element assembly (CEA) subgroup 5 (4 CEAs) of regulating group 4 slipped into the core. Control room operators manually tripped the reactor and stabilized the reactor in Mode 3 (see paragraph 10).
The reactor was restarted and returned to 85% power on November 7.
On November 12, emergency diesel generator (EDG)
"B" was declared inoperable due to a failed relay. The relay was replaced and the EDG returned to operable status the same day.
At 2:25 a.m. on November 29, Unit 3 entered Mode 3 and commenced the mid-cycle outage to inspect the steam generator U-tubes. The licensee had committed to shutdown Unit 3 and conduct the inspections based on a review of the previous refueling outage data (see NRC Inspection Report 50-528,529,530/93-43). With NRC concurrence, the licensee had revised the date by which it would shut down Unit 3 to December 4.
The licensee it.tends to inspect the seven tubes that had potential axial indications and to conduct inspections on approximately 3800 tubes in the analytical arc identified during the Unit 2 outage and 20% of the hot-leg tubesheet locations identified during the Unit 1 outage, d.
Plant Tour The following plant areas at Units 1, 2, and 3 were toured by the inspector during the inspection:
Auxiliary Building
Control Building
Diesel Generator Building
E
.
.
.
_.
.
.
I d
Fuel Building
,
Main Steam Support Structure
Radwaste Building
,
Turbine Building ~
Yard Area and Perimeter
.
Containment Building (Units 1 and 3)
The following areas were observed during the tours:
. ;
(1)
Operatina loos and Records - Records were reviewed against Technical Specifications and administrative control procedure
'
requirements.
i
'
(2) Monitorina Instrumentation - Process instruments were observed for correlation between channels and for conformance with i
Technical Specifications requirements.
(3)
Shift Staffina - Control room and shift staffing were observed for conformance with 10 CFR Part 50.54.(k), Technical
'
Specifications, and administrative procedures.
,
!
(4)
Eauipment Lineups - Various valves and electrical breakers were verified to be in. the position or condition required by i
Technical Specifications and administrative procedures for the applicable plant mode.
Failure to Independently Verify Valve Position On November 23, 1993,.the NRC inspector noted in Unit 2 that valve CTA-V018 (Condensate Transfer Pump "A" Discharge to Spent Fuel Pool isolation valve) had been positior.ed on November 18,
,
1993, but not independently verified, and that the licensee was
'
incorrectly tracking the valve. position in 40DP-90P10, " Valves Locked for Operational Convenience," and not in 40DP-90P19,
" Locked Valve, Breaker, and Component Tracking." The operators subsequently verified the position of the valve and documented their actions in the correct procedure prior to the end of the shift. The licensee failed to follow procedures 40AC-0ZZ06 and 40DP-90P19 which require the position of valve CTA-V018 to be--
independently verified following position changes and Technical Specification (TS) 6.8.1 which requires that procedures be implemented. This violation is not being cited because the criteria specified in Section VII.B of the Enforcement Policy
,
were satisfied (NCV 50-528/93-48-01).
Failure to Return Controller to Automatic i
In Unit 2, on November 23, 1993, the NRC inspector noted shortly after shift turnover that the reactor makeup water to
,
the volume control tank (VCT) controller, CHN-210X, was in manual.
The primary operator subsequently placed the
I w
w r,
..
-
--
.
..
-.-
.
-
,
.
,
w controller in auto-and stated that the controller had been in manual during shift turnover.
The oncoming primary operator had assumed the controller was in manual because the off-going shift had been _ performing dilution operations. The oncoming primary operator had not questioned the offgoing primary opera.sr on the configuration of the controller. This did not mee*. management's expectations for conducting shift turnover.
The licensee acknowledged the failures and has counseled the
operators involved. The safety impact with the valve controller in manual was that the reactor water makeup flowrate to the VCT would be set by the manual controller (lever) rather than the thumbwheel controller as intended by the procedure.
Procedure 420P-2CH01, "CVCS Normal Operations," Step 7.3.4, directs the operator to. return the controller to automatic operation once the dilution process is complete. Technical Specification (TS) 6.8.1 requires that procedures be implemented. The inspector concluded that the failure of the
,
operator to follow this operational procedure 'is a violation of TS 6.8.1 (Violation 50-529/93-48-02).
,
(5)
Eauipment Taqaina - Selected equipment, for which tagging t
requests had been initiated, was observed to verify. that tags were in place and the equipment was in. the condition specified.
(6)
General Plant Eauipment Conditions - Plant equipment was observed for indicati_ons of system leakage, improper i
lubrication, or other conditions that could prevent the systems i
from fulfilling their functional requirements.
,
(7) Fire Protection - Fire fighting equipment and controls were observed for conformance with Technical Specifications and
t administrative procedures.
On December 8,1993, the inspector accompanied a security guard performing a fire watch tour in the Unit 3 auxiliary building.
'
The security guard conducted the tour in two parts. The first portion of the tour started at the_51'-6" level. This is where -
j
!
the guard normally completes the previous hour's security patrol / roving fire watch. The guard demonstrated knowledge of
'
roving fire watch duties. The first portion of the tour ended at the 140' level near the "RP island." This portion of the
tour took approximately seven minutes to perform. The guard normally exited the RCA at this point. The guard then
demonstrated the second half of the fire watch tour that
started and ended at the RP island. The second half took i
approximately 12 minutes to perform. The inspector concluded
<
that the guard adequately performed the duties of the roving fire watch.
l i
,_.
.
.
i
-
M (8)
Plant Chemistry - Chemical analysis results were reviewed for l
conformance with Technical Specifications and administrative control procedures.
.
(9)
Security
. Activities observed for conformance with regulatory requirements, implementation of the site security plan, and administrative procedures included vehicle and personnel access and protected and vital area integrity.
(10) Plant Housekeepina - Plant conditions and material / equipment storage were observed to determine the general state of
..
cleanliness and housekeeping..
(11) Radiation Protection Controls - Areas observed included control point operation, records.of licensee's surveys within the radiological controlled areas, posting of radiation and high
,
radiation areas, compliance with radiation exposure permits, personnel monitoring devices being properly worn, and-personnel frisking practices.
(12) Shift Turnover - Shift turnovers and special evolution
,'
briefings were observed for effectiveness and thoroughness.
One cited violation and one non-cited violation of NRC requirements were I
identi fied.
3.
Surveillance Testina - Units 2 and 3 (61726 and 71707)
'
Selected surveillance tests required to be performed by the Technical
.
Specifications were reviewed on a sampling basis to verify that:
1) the r
surveillance tests were correctly included on the facility schedule; 2) a
{
technically adequate procedure existed for performance.of the
_
,
surveillance tests; 3) the surveillance tests had been performed at the-
frequency specified in the Technical Specifications;.and 4) test results
,
satisfied acceptance criteria or were properly dispositioned.
Specifically, portions of the following surveillances were' observed by
the inspector during this inspection period:
l Unit 2
[
i Procedure Description
'
42ST-2DG01 Diesel Generator "A" Test
77ST-2SB07 CPC Channel "A" Functional Test
Emeraency Diesel Generator (EDG) loadina
On December 6,1993, the inspector observed portions of surveillance test-
,
42ST-2DG01, " Diesel Generator A Test 4.8.1.1.2.a."
During the performance of the test the inspector noted that the local kilowatt. (KW)L meter was reading 5600 KW and determined by reviewing the auxiliary i
,
.L
?
.
-
.
_,
_
.at
__
_
_
_ _
_
,
.
-
.
,
operator (AO) logs that the EDG had been operated at that loading for l
almost four hours.
Based on local indication this exceeded the 2-hour EDG load limit of 5500 KW. The inspector noted that the A0 logs were confusing in that they indicated a " maximum" load allowed of 5500 KW and -
a limit of 6050 KW. As a result of this confusion, the A0 did not inform the shift supervisor of the problem.
The licensee determined that the EDG was not overloaded based on control
room indication (5400 KW) and the total output from the breaker (21 MW, or approximately 5370 KW/ hour - the average EDG output over the time.that the EDG was loaded).
The control room indication, which is used for satisfying the surveillance test, was accurate to 121.KW, compared to the
- 21.0 KW accuracy of the local meter. Also, the licensee noted that:the EDG capability curve is drawn based on a load of 5500 KW and a power factor of 0.8, and that the EDG was run at a near-unity power factor,-
which allows a higher continuous load. The-licensee submitted a procedure change to ensure that out-of-specification readings are reported to shift management when they occur. The inspector concluded the operability determination of the EDG was correct and that the operations and engineering resolution of the problem was thorough.
Unit 3 Procedure Description 36ST-3SE01 Excore Safety Channel Log Calibration On November 15, 1993, the inspector observed Instrumentation and Control (I&C) technicians perform portions of Surveillance Test 36ST-3SE01,
,
"Excore Safety Channel Log Calibration." The inspector compared test equipment against the requirements in the procedure, checked the equipment calibration for currency, and inspected the general working conditions. These were found satisfactory.
During the inspector's observation of Section 8.2.3, " Test Circuit Card TCl-1 Calibration," of the procedure, the inspector questioned the method i
that the technicians used to perform Steps 8.2.3.5 through 8.2.3.7.
These steps measured and adjusted the rate of-TP-3 voltage increase to an optimum value of 7.000 volts in one minute. The inspector questioned the ability of the technicians to accurately measure the rate of voltage
increase on a rapidly changing digital volt meter (to three decimal i
places) using a hand-held stop watch. - Additionally, the inspector
,
questioned whether the voltage prior to or after the switch movement was the appropriate initial value because measured voltage dropped several percent after switch movement. The technicians appropriately stopped the test to resolve the questions raised by the-inspector. The inspector-will review the licensee's resolution in a future inspection (Followup Item 50-530/93-48-03).
No violations of NRC requirements or deviations were identified.
l
.
!
-
,
,
-
i 4.
Plant Maintenance - Units 1. 2. and 3 (62703)
During' the inspection period, the inspector observed and reviewed I
selected documentation associated with maintenance and problem i
investigation activities listed below to verify compliance with regulatory requirements, compliance with administrative and maintenance procedures, required quality assurance / quality control department i
involvement, proper use of safety tags, proper equipment alignment and use of jumpers, personnel qualifications, and proper retesting.
The inspector verified that reportability for these activities was correct.
Specifically, the inspector witnessed portions of the following maintenance activities:
Unit 1
'
Troubleshoot SIAS relay K408
Set CCI drag valve on auxiliary feedwater pump "A" miniflow line
Unit 2 Emergency diesel generator "B" generator brush inspection
Replace emergency mode fuel control valve on emergency diesel
generator "B" Sample oil on charging pump "B"
Receipt of diesel fuel oil
Replace pressure regulator on atmospheric dump valve SG-178
On November 23, 1993, the inspector observed the replacement of the Emergency Mode Fuel Control Valve, DGB-UV-012, on the "B" EDG. The Quality Control (QC) inspector present questioned the type of electrical splice the technicians intended to use and independently measured the wire gauge. The QC inspector determined that a different type of electrical splice should be used and a different size heat shrink would
' '
be needed. The inspector concluded the QC inspector demonstrated strong i
involvement in this maintenance evolution.
,
i Unit 3 Replace oil seal on charging pump "B"
,
,
No violations of NRC requirements or deviations were identified.
5.
Liftina of Main Steam Safety Valve (MSSV) - Unit 1 (37700 and'71707)
On November 20,1993, while performing atmospheric dump valve testing in
Mode 3, one Unit 1 MSSV lifted at an indicated steam generator pressure r
of approximately 1210 psia. 'The setpoint of the valve is required by
,
Technical Specifications to be 1250 psig 11 percent. The valve reseated
'
at approximately 1160 psig after being open approximately 1 minute and 50 seconds. The licensee subsequently declared the MSSV. inoperable. There
,
are 20 MSSVs installed in each unit (i.e.,10 per steam generator) and
'
,
-
w n
a w
e
,
m
-
.
.--
-
-
..
.
.
.
I
-
,
,
.
power operation is permitted with a maximum of four MSSVs per steam e
generator inoperable. The inspector concluded that operator response to manually reduce steam generator pressure was appropriate.
~
All of the MSSVs had been refurbished and tested using live steam at the Westinghouse facility prior to installation during refueling outage IR4 which had just been completed. The MSSV that lifted (MSSV 552) had been J
stored at the Palo Verde warehouse for 19 months prior to installation.
Plant engineering personnel reviewed the test data for MSSV 552 and determined that the valve had a wide variation in setpoints during
testing and had an unexpected change in the. direction of the setpoint during one adjustment. As a result, the licensee reviewed the test data for all 20 MSSVs installed in Unit 1 and determined that two other valves had similar anomalies during testing as MSSV 552.
- The licensee performed in-place setpoint testing of the other two MESVs
'
using the Furmanite Trevitest method to validate their expected
.
'
,
setpoints. The licensee applied an offset to the Westinghouse setpoint to account for differences between the Trevitest method and the Westinghouse method. NRC Inspection Report 50-528,529,530/93-40 describes the apparent offset between the two test methods. The Trevitest showed that'one of the MSSVs had an expected setpoint; however, the other MSSV had an average value that was outside the expected setpoint range. The licensee determined that this test was unsatisfactory and declared the valve inoperable. This resulted in two-inoperable MSSVs, one per steam generator. The licensee gagged both these valves and intends to leave them out of service during the current operating cycle. Technical Specifications 3.7.11 allows operation at
,
98.2 percent power with two MSSVs inoperable.
The inspector _ concluded.that plant engineering personnel conducted a
,
thorough review of the MSSV setpoint test data. Additionally, the decision to test the questionable valves and the operability recommendations were prudent. However, the inspector noted that the root cause of the valve lifting early had not been determined and that-the
,
latest test data showed that five of the MSSVs in Unit I had as-found
-
test results that were outside of the~i3 percent band requested by the licensee in a pending Technical Specification amendment. The inspector will review the Licensee Event Report -(LER) on this event and include this issue in the review of LER 50-528/91-05. Additionally, assistance in assessing the differences ~ between the Westinghouse and Trevitest methods will be requested from the Office of Nuclear Reactor Regulation (Followup Item 50-528/93-48-04).
No deviations or violations of NRC requirements were identified.
6.
Main Steam Isolation Valve (MSIV) Ooerability Determination - Unit 1 (71707)
..
While reviewing the unit logs for November 19, 1993, the inspector noted that during the nitrogen pre-charge check of one train of the hydraulic -
i system for MSIV 180, the pressure for both accumulators on MSIV 180 l
.
.
.
.
.
-
.
-
--
!
.
.
.
dropped below 3500 psi. This also occurred during the testing of MSIV 171.. While a drop in pressure was expected for the accumulator being tested, a drop-in pressure was not expected for the companion
,
accumulator.. There are two hydraulic trains for_each MSIV.
The i
pre-charge checks were being performed using procedure 410P-ISG01, "MSIV
'
, Operation," as a prerequisite for performing procedure 41ST-ISG01, " Main
'
Steam Line Isolation Valves Surveillance."
i As a result of the accumulators of MSIV 180 and 171 being less than 5000 psi, the licensee declared both MSIV 180 and MSIV 171 inoperable and then recharged the accumulators. Once the accumulators were recharged, the licensee declared the valves operable. The existing documentation
,
was not clear as to why the valves were declared operable; therefore, the
!
NRC inspector discussed with the shift supervisor the reason for declaring both trains of the hydraulic system operable when there could be a problem that affected both hydraulic trains. The shift supervisor
,
gave the following reasons for declaring the valves operable:
The accumulator pre-charge check had been satisfactorily performed
on each train (the pre-charge check determines if there is enough nitrogen pressure to fast close the MSIV)-
When the accumulators were recharged, they were able to perform
their safety function of fast closing the MSIVs; Since both accumulators passed the pre-charge check, he did not
,
'
think there was a problem common to the accumulators; and Based on the initial information available, the operators were
unable to determine which accumulator may have a problem.
Based on this information, the inspector concluded that the licensee had reasonable assurance that the valves would perform their safety function and that the licensee's initial operability determination was
.
appropriate. However, the inspector noted that the shift supervisor's
reasons for declaring the valves operable were not clearly documented in the unit logs.
The inspector discussed this with the operations supervisor who agreed with the inspector's comment and had already discussed the problem with the shift supervisor.
The inspector noted that the next operations ~ shift determined that
<
several air leaks caused one train of the hydraulic system to bleed down prior to testing the other train. The.affected train was subsequently'.
declared inoperable (the MSIV remained operable with one train of the hydraulic system operable) and the air leaks were repaired. The
.
inspector concluded that the troubleshooting efforts were thorough and -
that the discussion of the corrective actions and operability determination were clearly described in the unit logs.
j i
No violations of NRC requirements or deviations were identified.
.
-
.
,
.
.
_._
,_
._
_ _
_.
.
_ _
-
'
.
,
,
i
-
r
.
_
7.
Unmonitored Reactor Coolant System (RCS) Drainino - Unit 1 (92700 and (
93702)
.
Event Sum _m_any
!
'
At 1:20 a.m., on November 3,1993, with, Unit 1 in Mode 5 and RCS level at
!
about 114' (i.e., the vessel flange elevation), the primary reactor operator (P0) began a routine draining operation to reduce RCS level to
,
about 112'-6".
The evolution was performed by opening the recirculation
valve from the "B" low pressure safety injection (LPSI) pump to the
.
refueling water. tank (RWT). The primary operator became distracted with '
other plant activities and was not monitoring RCS level via the two channels of refueling water level indicating system (RWLIS).
Approximately eight minutes later, at about 1:28 a.m., the secondary operator noticed that the RCS level was indicated to be at 108'-5", about 5 feet over the top of the hot leg (103'-1"), but below the reduced
_
inventory threshold (111') defined in Procedure 410P-1ZZ16 "RCS Drain.
Operations." The operators immediately stopped the draining evolution.
RCS level was restored to 111' at 1:32 a.m. and subsequently to 112'-5".
The level was restored using a gravity feed path from the RWT through the non-operating shutdown cooling (SDC) suction header.
Detailed Description of Event
'
The inspector conducted a detailed review of the facts surrounding the event. This involved a review of plant records and procedures, and
,
intetviews with the control room operators and plant management.-'The fellowing questions were addressed during the review:
What precedure was the operator usinc tc perform the evolution?
The primary operator had the "RCS Drain Operations" procedure out by Panel B02 (this panel is at the front side of the control room near
,
the LPSI miniflow valve handswitch). He was using Section-5.3.7.5
'
of the procedure that provided instructions for draining RCS level l
within a pre-existing condition (either partial drain, reduced l
inventory,ormid-loop). The procedure stated that the LPSI mini-flow line could be used if the primary method (through the
purification system) or the alternate method [ throttling the SDC i
I heat exchanger (HX) 6" bypass valve] were unavailable.- Although other methods could be made available,.the operators preferred to e
use the LPSI miniflow line because it provided a higher flow rate a
than the purification system, but was more controllable than the SDC HX bypass line.
. Although the procedure allowed this interpretation, the inspector
concluded that the operators should have initiated a procedure a
'
change to indicate that there is not a single preferred method for conducting the draindown. The licensee agreed with the inspector's observation and issued a revision to the procedure clarifying the
preferred methods of draining on November 10,1993.
13-l
.
w-
-
e e
. - - - -
.-
=
.%-,
,-,.
,-,,w
<-a i
-
.
-
.
.-
-
-
.
What caused the primary operator to aet distracted durina the evolution?
The primary operator was involved with motor-oper.ated valve (MOV)
testing of circulation water (CW) valves prior to starting the
draining evolution. When he started the evolution he announced that he was lowering level. This general announcement was only acknowledged by the third reactor operator who was in the rear of the control room. The control room supervisor (CRS) and the secondary operator did not acknowledge the primary operator.
(Licensee management stated that they expected everyone in the control room to acknowledge general announcements.)
After the primary operator started the evolution, he went to the front of the control room, and monitored the level at the CRT on Panel B04. As soon as he got to the panel, he received a call from technicians concerning the MOV testing of the CW valves. -The primary operator went to radio the auxiliary operator involved and direct him to go assist with the M0V testing. After this, the primary operator went back to the front of the control room to monitor level, but was distracted when he began reviewing an RWT boron concentration calculation worksheet which had been handed to him earlier in the shift by the third reactor operator. This distracted the primary operator for about four minutes until the secondary operator noticed that level was low.
Step 5.3.7.5(3) of Procedure 410P-1ZZ16 requires.that operators monitor the reactor coolant system while draining. The inspector concluded the primary operator failed to properly monitor RCS level during the evolution as required procedure 410P-1ZZ16.
Technical i
Specification 6.8.1 requires that procedures be implemented.
The failure to follow Procedure 410P-1ZZ16 is a violation of TS 6.8.1 (Violation 50-528/93-48-05),
i
Why were ceriodic RCS level reductions reauired?
The operators were maintaining RCS level in a partially drained condition (i.e., below 15 percent pressurizer level. and. greater that 111 feet) and controlling the reactor water level between 112'-6" and 114'.
This_ band was selected because it optimized the amount of water in the reactor vessel without creating too high of a L
differential pressure across the steam generator nozzle dams.
- Periodic draindown had become necessary due to leakage past'the RWT isolation valves -to the LPSI pump suction line.
Although these
,
valves had been previously disassembled and checked for-seat ~
leakage, no problems were noted and the valves were satisfactorily l
retested. Due to the in-leakage, approximately 200 gallons of water had to be drained from the RCS about four-times a shift (normally a twominuteevolution).
The primary operator involved in the event had performed this evolution about nine times. After the event, based on information
,
.
- -
-
.-
.
-
._
-
_
-
.
.
-
,
.
.
,
,
from an operator in Unit 2, the isolation valves were manually torqued shut and an additional valve in the flow path was shut to provide two valve protection. This 'significantly reduced the number of draining evolutions that were required. The inspector concluded
'
that operators did not thoroughly evaluate other methods which had been used in other Palo Verde units to reduce the in-leakage-from
these valves. The licensee committed to evaluate. increasing the thrust settings of the motor operators for the valves to. improve valve seating.
Safety Assessment The inspector reviewed the licensee's evaluation of the safety
..
significance of the event. This evaluation was focused on determining.
o how long it would have taken to lose shutdown 1 cooling if operators
' !
continued to drain the reactor coolant system-(RCS) and how long it would
.
'
have taken to uncover the core once shutdown cooling was: lost.
The licensee determined that the actual volume of water that was transferred from the RCS to the RWT was 1030 gallons. This volume was determined from the computer point for RWT level. - Based on this
.
,
information, the lowest RCS level actually reached was 111'
10" and the drain rate was about 130 gpm. This is' an estimate and closely
~
approximates the setpoint of the LPSI mini-flow at about 110 gpm.
The
,
difference between actual and indicated levels.was due to pressure differences between the vessel head and the pressurizer.
The pressure
differences are a result of a slower venting rate of the reactor vessel
'
head compared to the pressurizer safety valves.
The inspector determined that this difference is always conservative.
(i.e., the actual level is higher while lowering-level and lower when
'
raising level) and that the magnitude of the difference depends on the drain rate and the volume of water that is transferred.
The inspector also determined that the draining procedure adequately addressed this y
phenomenon.
Based on a 130 gpm flow rate, the licensee determined it would have taken about 138 minutes to lose shutdown cooling. This is-j based on-a level change from'111'-10" to.the level where SDC is expected to be lost, 101'-4".
The shutdown risk analysis indicated that with a..
i loss of shutdown cooling 60 days after shutdown, the time to boil would'
conservatively be about 30 minutes, and the time _to uncover the core would be about.four hours. This assumes.that there is not any: new fuel
,
in the core and that there is a break in the cold leg. The inspector q
reviewed this calculation and determined that there was not an imminent i
safety concern associated with the event, but it demonstrated poor operator attention to a significant plant evolution.
Corrective Actions j
The licensee initiated a Category 2 incident investigation of the event'
and issued Incident Investigation Report 1-3-0636 on November 28, 1993.
The most significant initial corrective actions taken were:
.
'
.
5-y-
-
- s-'A tu w
.-
..
.
.
.
-
.
All control room staffs were briefed on the event and the need for
control of safety sensitive activities; Specific interim directions were issued to ensure positive control
'
of RCS inventory; The refueling water level indicators were set up to read out on a
trend recorder that was closer to the valve that is being used to
'
lower RCS level; and The crew involved was removed from shift to participate in the
!
investigation and to participate in high intensity team training
"
aimed at improvino teamwork and communications.
On November 15, 1993, licensee management also briefed Region V management on the results of their investigation and initial corrective actions (see NRC Meeting Report 50-528,529,530/93-50).
Conclusion
,
,
The inspector concluded that, although the plant was not close to losing shutdown cooling during the event, several significant concerns were evident.
First, the primary operator responsible for monitoring the evolution lost sensitivity to the safety significance of the evolution and allowed himself to become distracted. Second, the primary operator
,
did not have anyone to back him up because he made a general announcement r
and did not receive the proper acknowledgement from the rest of the control room staff. And third, operations supervision allowed evolutions to be conducted without maintaining expected communication standards.
l One violation of NRC requirements was identified.
'
8.
Missed Surveillance - Unit 2 (71707)
On~ November 23, 1993, the licensee failed to perform a required Technical Specification (TS) Surveillance Requirement following removal of the Unit 2 "B" EDG from service for maintenance at 4:43 a.m.
Technical Specifications Limiting Conditions for Operation (LCO) Action Statement
,
3.8.1.1.b requires that with one EDG inoperable, the operability of the
'
alternating current offsite sources be demonstrated by-performing Surveillance Requirement 4.8.1.1.1.a within one hour. The licensee identified the missed surveillance, due at 5:43 a.m., and subsequently completed it at 7:00 a.m.
The licensee evaluated the failure to perform the surveillance test in the required time in Condition Report /Dispost-
.
tion Request 2-3-0614 and concluded the root cause of the event was due to inattention-to-detail by the crew and took corrective action.
The'-
inspector concluded that the licensee's corrective actions were
appropriate. The failure to perform the TS surveillance requirement
within the allowed surveillance interval is a violation of TS 3.8.1.1.b.
I This violation is not being cited because the criteria specified in Section VII.B of the Enforcement Policy were satisfied (NCV i
50-529/93-48-06).
.
-
.
-.
.
.
.
.
-
.
.
l
'
-
.
One violation of NRC requirements was identified.
9.
Mid-loop Operations - Unit 3 (71707)
On December 3,1993, Unit 3 red ced reactor coolant system (RCS) ~1evel to l
the 101'-7", near the-center of the hot leg, to facilitate steam.
generator nozzle dam installation. The reactor had been shut down on
November 29, 1993, at 2:25 a.m., approximately 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> before. reduced inventory operations commenced.
The inspectors reviewed preparations for mid-loop operations, including inspecting the physical connections of the refueling water level indicating system (RWLIS) in containment, and found them to be
satisfactory. The inspectors also monitored control room activities and
operator command, control,. and communications during the drain down
.;
evolution.
Licensee management and oversight (Quality Monitoring
,
organization) involvement during the drain down were evident. The j
. inspectors also noted that the Independent Safety Engineering Group
!
'
(ISEG) had made recent recommendations for improvements to procedures and activities related to mid-loop operations.
,
Prior to the drain-down evolution, a miscommunication.and procedural-i weakness resulted in mechanics leaving RWLIS connections unconnected.
.
!
The. Instrumentation and Control (I&C) technicians discovered two of the four incomplete connections during performance of an I&C activity to fill
and calibrate the system. An auxiliary operator discovered the remaining
two incomplete connections during routine rounds. The licensee initiated
an investigation into the incident, completed the connection of the'.
!
system, and verified system integrity prior to commencing the drain down.
_
During the drain down, the operators developed a' correlation to facilitate utilization of digital level indications, and also ~ utilized level instrument uncertainty calculations to. determine the optimum
operating level. The inspector noted that the-licensee intended to
,
incorporate these enhancements into operating procedures prior to the upcoming Unit 2 mid-cycle outage.
.
r The inspector reviewed safety system availability and mid-loop related
maintenance activities, and observed that they appeared to be well-
controlled. The inspector also noted that unrelated activities were
.i minimized while the unit was at mid-loop. The inspector observed that
.
the pre-evolution briefing adequately addressed the safety-significance i
of mid-loop operations with hot fuel.
,
!
Coordination of activities between units was also observed.. Unit 1 delayed a planned down-power transient and Unit 2 delayed a scheduled I
essential cooling water system outage until the mid-loop operations were
completed. Also, during the mid-loop operations, a deficiency was l
Identified on switchyard breaker PL-928, which. feeds start-up transformer
,
NAN-X01 (and Unit 3 busses NAN-S06, NAN-S04, and PBB-SO4).
The licensee
intervened in Salt River Project's plans to immediately repair the
'
breaker in order to coordinate the repairs with Units 1 and 3.
The i
e
.
!
.
.
..
._
,
-
,
.
!
.
maintenance was successfully coordinated and performed on December 5,
'
1993, during the mid-loop operations. The inspector concluded that these activities were appropriately and adequately coordinated.
,
Operators increased RCS level immediately following completion of mid-loop maintenance activities.
Reduced inventory operations were concluded-on December 6, 1993.
Ths inspector concluded that operations activities were well-controlled and appropriate for the conditions and.that operators and maintenance personnel were sensitive to the safety significance of the evolution.
,
No violations of NRC requirements or deviations were identified.
10.
Control Element Assembiv (CEA) Subaroup Slio and Reactor Trio - Unit 3 (40500 and 93702)
i On November 3,1993, the inspector was observing Unit 3. operators reduce the main turbine generator output in preparation to repair a steam leak on a 1-inch drain line on a main steam lead downstream of turbine control Valve 2.
At approximately 4:16 p.m., the reactor was at about 24 percent
power when control element assembly (CEA) subgroup 5 of regulating
group 4 slipped into the core.
Control room operators quickly assessed
,
the situation and determined that subgroup 5 deviated from subgroup 22 by more than 9.9 inches. The shift supervisor ordered a manual reactor trip
'
to shut down the reactor. The plant responded normally and the operators
,
stabilized the unit in Mode 3.
.
Subgroup 5 consists of four CEAs and is one of two subgroups that make up_
regulating group 4.
Five regulating groups are used to control the neutron flux level in the reactor core during power operations.
Prior to
,
the event, regulating group 4 was used to control the axial shape. index t
and was being inserted at the time of the slip.
During the event, the inspector observed good command, control and communications and through plant stabilization. Operators displayed good
,
awareness of plant interactions by informing each other when their
,
actions on one panel. would produce an alarm on another panel.
l The licensee conducted troubleshooting, but did not determine the cause
of the event. The plant review board determined that' troubleshooting and
!
testing was thorough and_ recommended that the two components most likely to have caused the event be replaced. The licensee replaced the subgroup.
,
5 supply. breaker and the phase synchronizing cards and then proceeded-
- .
with reactor startup. The inspector concluded that the licensee had
'
performed an adequate root cause investigation and that the recovery i
plans were appropriate.
!
No violations of NRC requirements or deviations were identified.
j i
,
y a
,
e>
--
,
-
-
.
.
s
.
_
11. Followup on Previous 1v Identified items - Units 1. 2. and 3 (92701 and 92702)
a.
(Closed) Violation 50-528/93-12-04. Failure of Snubber Testing
_
Prooram to meet Technical Soecification Reouirements - Units 1. 2.
and 3 (92702)
This' item involved the failure of the licensee's snubber testing program to meet the requirements of the. surveillance requirements of Technical Specification 4.7.9.
The licensee modified Administrative Controls Procedure 73AC-9ZZ01, " Testing and Control of PVNGS Snubbers." The new procedure required that sample plan one be used for the functional test and also required that the snubber engineer randomly select the snubbers to be tested. The inspector concluded-that these actions should prevent a future recurrence of the violation. This item is closed.
b.
(Closed) Unresolved Item 50-528/93-40-02. Unlatched Core Element Assembly (CEA) - Unit 1 (92701)
This unresolved item involved the _ failure to properly latch CEA 34 during the removal of the upper guide structure'(UGS) on September 15, 1993. The inspector reviewed the licensee's evaluation of the event in Condition Report / Disposition Request (CRDR) 1-3-0479.
The inspector determined that the evaluation was thorough and identified several weaknesses that contributed to the event. The most significant of these was that the workers involved in latching
.
the CEAs had not been formally trained on how to perform the task.
!
One of the workers had never seen a CEA and the only training he received was during the pre-job briefing. During the briefing, the supervisor read the. applicable portions of procedure 31MT-9RC33,
" Reactor Vessel Upper Guide Structure Removal and Installation,"
which described the expected sequence of events. Step 4.5.4.3.1~of i
the procedure re mechanism (SLM) quired that the flange of the self-latching be verified in the down position, flush with the upper UGS support plate. The workers did not remember this requirement and since they did not have the procedure with them, j
they failed to notice that the flange for CEA 34 was above the UGS i
support plate.
The evaluation also included a calculation that determined that the l
CEA was about 21 inches above the fuel assembly when the UGS cleared the guide rods and was free to move horizontally.
Based on this.
determination, the inspector concluded that there was' minimal safety significance in lifting the UGS without having all the CEAs latched.__
As noted in NRC Inspection Report 50-528,529,530/93-40, the licensee's immediate actions to correct the situation were good and
prevented any serious safety-problems. The licensee's corrective-actions to improve personnel training include using a CEA and SLM mock-up. The licensee-identified violation is not being cited because the criteria specified in Section VII.B of the Enforcement i
h
-
_
.
.
.
_.
_ __.
.-
.
.
Policy were satisfied (NCV 50-528/93-48-07). This unresolved item is closed.
.
c.
(Closed) Followup Item 50-529/93-40-01. Core Operatina Limit
~
Supervisory System (COLSS) Database Errors - Unit 2 (92701)
This item was opened to review the licensee's eval'uation and
'
'
corrective actions related to the discovery of wrong fuel. cycle data in three of six COLSS databases, which resulted in a mismatch between COLSS and CECOR relative power calculations during power ascension -testing at 70 percent power.
i The inspector reviewed Condition Report / Disposition Request (CRDR)
2-3-0531, which documented the licensee's investigation.
In general, the licensee determined that cycle 5 data was properly loaded and tested in May 1993, but that later several errors resulted in Cycle 4 data being loaded, overwriting the Cycle 5 data.
The errors generally consisted of not producing enough backups of.
the large core storage (LCS), not recognizing that some of the r
backup tapes and disks still had Cycle'4 data, and using backup
tapes with Cycle 4 data to reboot the systems.
,
On August 20, 1993, following rebooting of both computers on which COLSS operates, errors were found in many of the COLSS addressable
>
constants, and incore detector strings needed to be deleted. These
,
discrepancies were corrected, but the COLSS databases were not
verified.
'
Differences between COLSS and CECOR calculated results'were first
.
detected during low power physics tests at 20% power on about September 1, 1993. One error was found in the exponent for a burnup constant, which reactor engineering personnel believed was the cause of the difference. The power ascension was resumed a few hours before the burnup constant was corrected. However,. correction of
the constant did not appreciably reduce the error. The licensee did
not thoroughly investigate the cause of the difference and verify
~
database information until after the 70 percent plateau was reached.
The licensee expected the differences to diminish almost completely.
as power increased, but the differences remained.
'
The licensee identified several corrective actions related to this
'
event. These include training personnel on the. loading of COLSS.
- database constants into core memory, logging media numbers, tracking the status of verification of COLSS database constants i
documentation; reviewing procedural requirements for COLSS database constants revisions; requesting that daily backups be performed during outages; revising the test procedure model to more.
j specifically identify the operational disk packs and tapes to which
LCS will be saved; and updating the model work order for installing-t and testing new COLSS constants to identify the process for loading.
the COLSS database constants into memory.
,
1
,
,.
.. ~ -
.
.-
.
.,
&
,
,
-
-
)
.
'
In October 1993, the inspector verified selected COLSS databases
'
installed in Unit 2, and found no discrepancies. This inspection j
was documented in NRC Inspection Report 50-528,529,530/93-43,
- >
Paragraph 18.d.
_
]
The significance of this event is limited primarily by.the close
,
attention given to power distribution and other parameters by-reactor engineering personnel and operators during startup from refueling. However, poor control of backup information could result
in incorrect data being loaded during system rebooting at any time,
'
when less attention is being given to' these parameters. The _
licensee's COLSS verification procedure provides some assurance that COLSS will not be declared operable with significant departures from
- '
expected values being calculated. Additionally, the licensee had established a process software configuration control task force in i
May 1993 to evaluate additional. configuration control methods. This task force issued.a summary report on November 19, 1993, - identi fyi ng significant recommendations for future implementation.
Tne inspector concluded that the licensee's CRDR evaluation was thorough and its corrective actions appropriate.
However, the licensee missed at least two opportunities to identify the actual
.
scope of the error, as licensee personnel failed to thoroughly
,
evaluate discrepancies in databases noted following the August 20,
!
1993, reboot of the computer systems, and during low power physics
,
testing at 20 percent power.
'
The licensee disagreed with the inspector, in that the licensee determined that the personnel involved pursued the differences as far as reasonable given the information available and their level of expertise. The inspector noted the licensee's comments.
Based on the actions completed by the licensee, this item is closed.
-
d.
(Closed) Followup Item 50-529/93-40-05. Steam Generator Tube Rupture
'
Emergency Procedure Modifications - Units 1. 2. and 3 (92701)
This item documented six commitments the licensee made to modify the steam generator tube rupture (SGTR) procedure and the excess-steam
-
demand (ESD) procedure based on lessons learned from review of the
Unit 2 steam generator tube rupture on March 1993. The inspector
-
reviewed the procedures for SGTR and ESD, 4XEP-XR003 and 4XEP-XR004 where 'X'
refers to the unit, for each unit. The inspector verified
.
that the commitments had been implemented but with one variation.
In. one commitment, the licensee had committed to modify the-safety r
function status' check section of the ESD procedure to guide operators to exit the functional recovery procedure based on indications of a main steam line break /SGTR event. The licensee
'
determined that_ a more effective location for this requirement was
!
in the event control section of the procedure. The event control section occurs prior to the safety function status check section.
,
,
-
w
--n y
~
.
-
-
-
-
---
r.
.
-:
-
.
'
,
The inspector determined that this variation met the intent of the original commitment. This item is closed, e.
(Closed) Followup Item 50-530/93-12-09. Emeroency Operatina Procedures - Units 1. 2. and 3 (92701)
This item involved three observations reported in a human performance study, conducted by the NRC's Office of Analysis and-Evaluations of Operational Data (AE0D) dated April 19, 1993, on the Unit 3 reactor trip of February 4, 1993. The first observation was
'
.
that the control room CRT trend displays -lagged actual plant
-
conditions by a considerable margin. The licensee responded that'-
the plant monitoring system was not designed to perform the trend display function during a fast transient. The computer system was designed to update functions on a priority basis. - Because the-function was not a critical function as defined in the Standard Specification for P1 ant Computer, N001-13.08-1, the trend display was assigned a priority of 60 and lags behind during a fast l
The second observation was that the auxiliary feedwater (AFW) and
)
safety injection (SI) systems do not have recorders to readily i
identify the amount of water injected for use during post event analysis. The licensee reviewed regulatory. commitments, licensing documents, safety reports, and regulatory guides and concluded.that they were not committed to record AFW or SI flows. However, the new emergency response facility data acquisition and display system (ERFDADS), currently installed in Units 1 and 2, does record system flows and can be used in a post-event analysis to integrate the amount of water injected during an event.
The third observation was that the emergency diesel generators (EDGs) were run unloaded for about four and one-half hours prior to
'
being shutdown. The EDG operating procedures and the Final Safety Analysis Report (FSAR) require that the EDGs be operated above 75 percent for 15-30 minutes after every six hours of no-load operation. The EDG manufacturer did not stipulate this requirement in the technical manual, but when contacted concerning this issue stated that there was no requirement for a loaded run after six-hours of no-load operation, but doing so would be beneficial to the engine.
The licensee did not intend to make any modifications based on these observations.- The inspector reviewed the licensee's response to the three observations and concluded-that the licensee met the regulatory and licensing requirements and that further review was not required. This item is closed.
One non-cited violation of NRC requirements was identified.
.
.
-
-
._
__
_
._
_
..
.
__
,
,
,
,..
_
l
-
t
.
12.
Review of Licensee Event Reoorts (LER)L - Unit ~ 3 (90712)
The following LERs were closed based on in-office review.
,
Unit 1 93-009 Revision 0 MSSV and PSV Setpoints Out of Tolerance 93-010-Revision 0 Inoperable Motor-operated Valves in Multiple Systems due to Various Deficiencies.Found During
Generic Letter 89-10 Testing Unit 3
'93-002 Revision 0 Spurious Opening of Steam Bypass Control Valves Caused' Reactor Power Excursion-93-004 Revision 0 Manual Reactor Trip Following Control Rod Misalignment No violations of NRC requirements or deviations were identified.
i
13. Exit Meetina (71707)
,
An exit meeting was held on December 9,1993, with licens'ee management.
.
and resident inspectors during which the observations and conclusions in
!
this report were discussed. The licensee had no additional comments to
!
the inspectors' findings. The licensee did not identify as proprietary t
any materials provided to or reviewed by the inspectors during the.
inspection.
t
5
.
'
i b
)
>l
,
e
i
'
a
.
.
p