IR 05000312/1989008
| ML20247J084 | |
| Person / Time | |
|---|---|
| Site: | Rancho Seco |
| Issue date: | 07/11/1989 |
| From: | Richards S NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| Shared Package | |
| ML20247J062 | List: |
| References | |
| 50-312-89-08, 50-312-89-8, NUDOCS 8907310232 | |
| Download: ML20247J084 (17) | |
Text
my-..,..,
-
,
<
-
[
', & j; I
'
m;
e ll * f.lg
'
'
+
-
-
.
.
.
b.
.f5
,
'Y.-
W ( #~ '
i j '..
,'
}y.
- q, H
,
q p."
U.;S. NUCLEAR REGULATORY COMMISSION
'
n
.,-
,
.
,
,
s s :-
is,
'
.,
.,
.
>1
}
s
/<J 4.
Y p v.l. ~,
REGION V
,
,
,
,
'
^
- .,
H ?!
I
, _.. -,
r_(-
w
,,..
' k.-
- l'
h.
[
c
9
-
i "--
': Report'N :
50-312/89-08
y
- n,
,
,
T.p
,
D'ocket No.
50-312.-
a
,
,
l
-
o..
,
"
, License No. DPR-54'
-
.
.
<
g Licensee:
Rancho.Seco Nuclear Generating Station
'+
. Sacramento Municipal Utility District 14440 Twin Cities Road Herald, California 95638-9799 Facility Name:. Rancho Seco Unit 1 Inspection.at:
Herald, California (Rancho Seco Site)
Inspection conducted:
April 22, 1989 through June 2, 1989.
Inspectors:
A. J. D'Angelo, Senior Resident Inspector C. J. Myers,. Resident Inspector P. M. Qualls, Resident l Inspector W..Ang, Regional Inspector F. Gee, Regional Ins tor Approved By:
,l 7-ll-fff s
S. Richards, Chief Date Signed Reactor ProjectsSection II Summary:
Inspection between April 22 and June 2, 1989 (Report 50-312/89-08)
Areas Inspected: This routine inspection by the Resident Inspectors, and in part by Regional Inspectcrs, involved the areas of operational safety verification, health physics and security observations, engineered safety features system walkdown, maintenance, surveillance and testing, quality assurance and followup items.
During this inspection, Inspection Procedures 71707, 71710, 61726, 62703, 93702, 92701, 92702 and 30703 were used.
i
f ;.
i 3907310232 890712,,
'
'
PDR ADOCK0500g.g O
.
K__
_ - _. _.
-.-
.-
c
- _ -_ _ __ _ ]
__- __ _
-
-
,
.
i j
'
7-
...
,
.
.
.
.
.
\\
.
a.
-2-
,.. ' '
'
.-?,._, 5,,
,' )
,
s
,
r"
-
,
,..
.
,
Qf ?
Results:
? - /)-
,_
>
- 9\\
, General Conclusions:
,-
i,:
- y
,
,i During.the inspection period the licensee operated the unit safely in
',
accordance with'their Technical Specifications.
The unit continues to
'
experience difficulty with the main feedwater pump controllers and has planned
-
. a modific' tion to address this issue.
,1 a
Significant Safety Matters:
None Summary of Violations or Deviations:
Two non-cited violations are discussed
,
,
in paragraph 9.b.
Open Items Summary:
Three open items were closed during the inspection.
-
<
_ ___
..
.
.
.
.
.
.
'
3;.
.
.
'
l DETAILS-
"
.
I l.
1.
Persons Contacted
'J. Fi Firlit', Chief Executive Officer, Nuclear
- D. Keuter, Assistant General Manager (AGM), Nuclear Plant Manager J. Shetler, AGM, Special Projects
- B. Croley, AGM, Technical Services
- J. Vinquist, AGM, Nuclear Quality and Industrial Safety P. Caudill, Acting AGM, Nuclear Support Services
- L. Fossum, Assistant Nuclear Plant Manager P. Lydon, Manager, Nuclear Operations D. Brock, Manager, Nuclear Maintenance
.
- S. Crunk, Manager, Nuclear Licensing R. Baim, Manager, Nuclear Cost Control and Plant Services M. Bua, Manager, Nuclear Radiation Protection J. Clark, Acting Manager, Nuclear Chemistry
- P. Bender, Manager, Material Control B. Kemper, Manager, Nuclear Scheduling and Outage Management
- W. Peabody, Manager, Nuclear Engineering-L. Houghtby, Manager, Nuclear Security J. Delezenski, Supervisor, Regulatory Coordination, Licensing.
Q. Coleman, Quality Engineering Supervisor J. Robertson, Licensing Engineer S. Carmichael, Maintenance Engineer B. Wilson, ICS System Engineer o
Other licensee employees contacted included technicians, operators, mechanics,. security, and office personnel.
- Attended the Exit Meeting on June 2, 1989.
2.
Operational Status of Rancho Seco During this inspection period, the plant operated continuously at various power levels between 92% and 60% power.
On April 22, 1989, power was i
reduced to 75% when the plant experienced Main Feedwater (MFW)
oscillations induced by a temporary' instrumentation recorder monitoring the "A" Main Feedwater Pump (MFP) controller (Lovejoy).
Following removal of the recorder, power was again escalated to 92% until May 13, when a failed module in the Lovejoy controller for the "A" MFP required a power reduction to 62%.
The plant remained at approximately 65% power for the remainder of the period for troubleshooting and modifications of the "A" MFP controller.
On June 7, 1989, after the end of this inspection period, the plant initiated a controlled shutdown and reached cold shutdown on June 9, 1989.
The shutdown was directed by the licensee as a result of the defeat of Measure K in a local voter referendum on June 6, 1989.
)
- _ _ _ _ _
- _ _ _ _ _
. _ _ _ -.
-___-_
_ - _ _ _ _ _ _ _. _ _.
l
[,
.
j
-
.
.
3.
Operational Safety Verification (71707)
The inspectors reviewed control room operations which included access
'
control, staffing, observation of system alignments, procedural
,
adherence, and log keeping.
Discussions with the shift supervisors and
'
s
- operators indicated an understanding by these personnel of the reasons for annunciator indications, abnormal plant conditions and maintenance 7 -
-work in progress.
The inspectors also verified, by observation of valve
-
'
and switch position indications, that emergency systems were properly j
aligned as required by technical specifications for the plant condition.
k
-
Tours'of,'the ' auxiliary, reactor,.and turbine buildings, including exterior ' areas, were made to assess equipment conditions and plant
,
.
'
conditions.' Also, the tours were made to assess the effectiveness of
" ' '
radiological controls and adherence to regulatory requirements.
The
.
inspectors also observed plant housekeeping and cleanliness, looked for
,
,,
potential fire and safety hazai 4, and observed security and safeguards
,
practic'es.
-
,
,
i During this inspection period, several isolated housekeeping
'
discrepancies were identified by the inspector which were promptly
,
resolved by the licensee.
.
During work activities, it appeared that the health physics managers were conducting plant tours and monitoring work in progress. They appeared aware of significant work which occurred during this period.
-
The inspector's Radiation Work Permit (RWP) review revealed that the RWP did include:
job description, radiation levels, contamination, airborne radioactivity (if expected), respiratory equipment, protective clothing, dosimetry, special equipment, RWP expiration, health physics (HP)
coverage, and signatures.
The RWP radiation and contamination surveys were kept current.
Employees understood the RWP requirements.
The inspectors observed that personnel in the controlled areas were wearing the proper dosimetry and personnel exiting the controlled areas were using the monitors properly.
Labeling of containers appeared appropriate.
The inspectors walked down portions of the protected and vital area boundaries to ensure that they were intact and that security personnel were properly posted where known deficiencies existed, The inspectors also observed protected area access control, personnel screening, badge issuing and maintenance on access control equipment.
Access control was observed.
Personnel entering with packages were properly searched and access control was in accordance with licensee procedures.
The inspectors observed no obstructions in the isolation zone which could conceal a person or interfere with the detection / assessment system.
Protected area illumination appeared adequate.
No violations or deviations were identified.
l
_
-_--_.--________-um___.-m_.._-__m___
. _ _ - _ - - - - _ _ _ _ - _ _ _ _ - _ _ _
i
.
'h
.
-
'
3,
-y, J
,, ;
.
,
'
!
O y.,
'4.
ESF System Walkdown (71710)
"
During the inspection period the Inspectors walked down the-Decay Heat j
E
. Removal.(DHR) and Contain.ent Spray (CSS). Systems for correct valve alignment and electrical switch position.
'
,Theeinspectors concluded that:
-
All observed hangers and supports'were properly made up ahd aligned.
l
Housekeeping was' adequate.
.
l
No excessive packing leakage was observed on valves.
'
.
J'
Major system components were properly labeled, lubricated and cooled.
No excessive leakage-was apparent.
i
..
,
,
'.
,
.
. Instrumentation appeared to be properly installed.
a
-
F
-
No out of calibration gauges were identified.
'
.l
- Flow path components appeared to be in the correct position.
,
!
.
- -
Required support systems were available.
,
Proper breaker and switch positions were verified.
.
No violations or deviations were identified.
..
5.
Monthly Surveillance Observation (61726)
i
.
,
.,
Technical Specification (TS) required surveillance tests were observed and reviewed to ascertain that they were conducted in accordance with Technical Specification requirements.
.
The following surveillance activities were observed:
SP.021 Monthly Motor Driven Auxiliary Feedwater Pump P-318 -
Inservice Test
SP.029A Monthly Decay Heat Removal Pump "A" Venting Surveillance
SP.56A Variable Diesel Generator (G-886A) Synchronization Surveillance Test.
The following items were considered during this review:
testing was in accordance with adequate procedures; test. instrumentation was calibrated; limiting conditions for operation were met; removal and restoration of the affected components were accomplished; test results conformed with TS and procedure requirements and were reviewed by personnel other than the individual directing the test; the reactor operator, technician or engineer performing the test recorded the data and the data was in agreement with observations made by the inspector, and that any
-- _ _ _ _ _ -
, _ - - -
__
-.- --
y a-e g
f -
'
,
I
- Y
.
}
s
. G, k ',, ; ' E
,
,
(
,.c
,
J'
'
i y
,
deficiencies _ identified during the testing were properly reviewed and resolve ^d by. appropriate management personnel.
No' violations or deviations were identified.
6.
Monthly Maintenance Observation /(62703)
.The Maintenance activities for the systems and components listed below were observed and reviewed to ascertain that they were conducted in
' accordance with. approved procedures, regulatory guides, industry codes or standards, and the Technical Specifications.
- Main feedwater pump maintenance on the pump's Lovejoy controllers.
- Routine maintenance on the Diesel Generator.
Routine maintenance on the Decay Heat Removal Pump.
The following items were considered during this review: The limiting conditions for operation were met while components or systems were removed from service; approvals were obtained prior to initiating the work; activities were accomplished using approved procedures and were inspected as applicable; functional-testing or calibration was performed prior to returnine components or systems to service; activities were accomplished by. m11fied personnel; radiological controls were implemented; and tire prevention controls were implemented.
No violations or deviations were identified.
7.
Cable Raceway Tracking System (CRTS)
NUREG 1286 Supplement 1 - Safety Evaluation Report related to the restart of Rancho Seco following the event of December 26, 1985, Section 4.8, documented the NRC staff acceptance of-the Rancho Seco CRTS problems and corrective actions.
As part of this acceptance, NUREG 1286 Supplement 1, Section 4.8.2.4 acknowledged licensee actions to correct cable discrepancies.
It further stated that the licensee had developed new procedures and controls based on recommendations resulting from the licensee's root cause evaluations and in turn submitted these corrective actions to the NRC via its December 4, 1987 Wire and Cable Prcblem submittal On January 6,.1989 and March 27, 1989, the licensee submitted to the NRC updates for the Wire and Cable Program.
The submittals were reviewed and discussed with licensee Nuclear Engineering Department
'
engineers.
An inspection of procedures and the CRTS process was
.
performed.
The above noted discussions and inspection of the licensee's revised CRTS process resulted in the following observations.
g a)
The licensee submitted Updated Safety Analysis Report Amendment 6 to the NRC on January 20, 1989.
This amendment included clarification of cable routing intermixing and tray weight licensee commitment ambiguities.
.
- _ _ _ _ _ - _
g
~.
,
tj['
%-
-
.
,
'%,;*,
4
,
G;p u
>
,
,
'
]
.
y
i b); -The. licensee purchased the' existing-(pre-1987) CRTS software and-
~
. data base from Contro1~ Data Corporation'(CDC).
A. software QA'
'
. program wasfunderway to' identify and correct any remaining software
!
-discrepancies.
gy
.
,
<
' c)
The licensee ve'rified, prior.to the March 1988 restart, the adequacy)
,
-of all' Class'1 cable database information for intermixing of power, control and? instrumentation cables.
For cable installations subsequent;to restart, the licensee had-added-a field in the
-
database to identify the-cable as power, control, or
'
instrumentation'.
This information, however, was not backfitted for previously. installed cable.
This information was not being used by-
.the CRTS software'or computer for detecting intermixing. The licensee's root'cause investigation ~87-09 for' unacceptable intermixing of' power / control and instrumentation cable recommended-that CRTS software be provided for verification that. intermixing, conditions do not exist.
The licensee's December 4, 1987 Wire and t
',.'
CabletProgram submittal to the NRC committed in Attachment 4,
'
'*
Item 29, Section 3.1.4, states that "CRTS software will be enhanced p
to screen raceways for service level mixing [P and/or C with 1] and block' proposed changes which will violate design criteria." In their January 6,~ 1989 Wire and Cable Program update, Attachment IV, page 2, the' licensee changed this commitment by stating that no CRTS
'
-
f( -software modification is necessary'due to the current process of
-
,
m; i ~
,
raceway designers performing manual checks to screen and prevent
+
- '
+
int'ermixing.
,
g
,
-[
L l)
'The' licensee verified the adequacy of all instrumentation cable d
'
strays and"all' Class IE power and control cable trays prior to the H, ~
M March'1988 restart.
At that time. ampacity checks also verified the
$
_,
,
q
',
adequacy of all Class 1E power and control trays. The CRTS software
,
'and computer determines percent fill for all trays.
It does not i.
" determine weight per foot of any given section of tray.
The
'
'
-
licensee initiated a process whereby the CRTS cable tray information
",
. is downloaded into a desktop Dbase III computer program. This in
>
turn calculates the weight per foot of any given section and is used
to generate a list of cable tray sections that weigh (a) 40-45 pounds, (b) 45-50 pounds and (c) greater than 50 pounds.
This in
.
turn is used to generate a list of " forbidden to use" trays, which is used by both the designers and CRTS personnel to verify the tray weight adequacy of new cable routin95.
The CRTS software and computer does not automatically determine tray weight nor does it screen for cable trays that could exceed specified weight limits.
The licensee's root cause investigation 87-05 recommended that adding a subroutine to the CRTS to calculate cable tray weights j
should be evaluated.
The licensee's Wire and Cable Program,
)
submitted to the NRC on December 4,1989, Attachment 4, committed that " enhancements planned for the CRTS software will, in the future, automatically block cable additions which exceed fill and weight limitations" and "CRTS software will be enhanced to
automatically block cable additions which exceed design limits."
l
<
The licensee's January 6,1989 Wire and Cable Program Update changed these commitments and stated that, due to the above described
!
_-- _ -
_ _ _ _ _ _ _ -
__
__ _
_
_
_ _ _ _ _ _ _ _ _. _ _ _ _ _ -
F;.
'.
VCc
..
.
process, no CRTS software modification for blocking cable additions that exceed fill and weight limitations was considered necessary.
L>
The above noted inspector observations and the licensee's January 6, 1989 Wire and Cable' Program Update will be forwarded to the NRC Office of Nuclear Reactor Regulation for-determination of its effect on the NUREG 1286 Supplement 1 SER and the acceptability of the licensee's corrective actions.
.The inspector discussed QA involvement with the-CRTS process with Nuclear Engineering and QA engineers / auditors.
The QA department >:ad performed QA surveillance in May (88-S-334), August (88-S-486) and October (88-S-569) 1988 on configuration control of CRTS software changes,
"
processing of CRTS cards, and CRTS software changts respectively.
The surveillance encompassed several segments of the total CRTS process.
The surveillance demonstrated a proactive mode for the Quality Department.
During further discussions with the AGM for Quality, the licensee agreed to consider including the total CRTS process, from designer to computer to field installation tu QC to computer and to designer, in a future QA audit.
The proactive surveillance and audit by the Quality Department appeared to be a strength.
No violations or deviations were identified.
8.
Main Feedwater System Review g
A.
Troubleshooting of Main Feedwater Transient
,On May 13, 1989, the plant experienced a main feedwater transient
'due to oscillations in the speed of the B main feedwater pump (MFP).
The inspector observed the conduct of the licensee's investigation and troubleshooting activities.
,.
Aided by diagnostic monitoring of the MFP controller, the licensee
.was able to determine that a function generator in the Integrated Control System (ICS) was inadequately calibrated for the range of input signals experienced during the transient.
The function generator module controls the output signal of the ICS to the MFP controller (Lovejoy).
The function generator normally receives an input signal of +/-10VDC and produces an output signal of +/-10 VDC which approximates a desired function curve by means of four straight line segments, each beginning with a breakpoint.
Calibrati.1 of the function generator per the vendor technical manual involved adjusting the breakpoint and slope of each of the four line segment channels.
The inspector observed the calibration of a replacement function generator in the I&C shop.
The procedure specified for the calibration was the vendor technical manual.
The inspector observed that the technicians deviated from the technical manual procedure by calibrating only two of'the four line segments channels for the expected range of input signals.
In an attempt to disable the two uncalibrated channels, the technicians adjusted the breakpoint of l
-
_ - - _ _ _ _._
_
. _ _ -.
__
-.
__
'
j
.
,
~
- r
'
.' T,
-
"
,
N.,'
'
f
T.-
'
'
g an
.. ;
.
Og those channels'to be generally above the maximum expected input
,
signal;'although not.specifically determined.
The inspector found.
~
h
- "
-
i'
-
'.
~ that(the vendor technical manual.did not direct a partial
i calibration'using only two of the four channels, Furthermore, the c-
'
Ldeviation was not documented as an exemption to the procedure in the
'
r
,
work request for the calibration of this non-safety related
'
')
equipment.
,
,
-
r
~
,
'
The licensee determined that signals as high as 12 VDC were input to
.
I
'
"
the function-generator during the transient.
The input signals in excess of the expected range during calibration were determined to
-
be proper signal levels resulting from the upstream signal
.
processing.
The excessive input signals caused the uncalibrated channels to affect the module output in an unpredictable fashion.
Recalibratior.of the function generator for the B MFP with input signals up to 15 VDC identified a decrease in the module output due to the uncalibrated channels.
After calibrating all channels of the function generator for the 15 VDC input range, the module output was observed to function as expected.
The licensee acknowledged the inspector's concern for the undocumented and unauthorized deviation from the calibration procedure specified in the vender technical manual.
The licensee reviewed the calibration controls for all other applications of function generators in the ICS and NNI systems and found no other cases of uncalibrated channels.
The inspector found the licensee's corrective actions to be adequate.
B.
Maintenance and Modification of Main Feedwater Pump Controller The inspector observed the modification and replacement of control i
circuit modules in the Lovejoy controller for the B main feedwater pump (MFP).
The work resulted from the licensee's troubleshooting of the May 13, 1989 feedwater transient. The scope of work included modifying the control circuit cabinet to install a dry air. purge and replacing several control modules and their bases.
The inspector
'
found that the work was performed successfully in a cautious and generally well controlled manner.
However the inspector noted several deficiencies in the work instructions.
1)
Although the work request referenced a procedure for installing a sealing compound as a moisture barrier in the conduits penetrating the cabinet, the procedure was not included in the work package. When questioned by the inspector, the maintenance personnel were found to be familiar and experienced
!
with the procedure for installing the sealing compound as a l
fire stop.
However since the installation was not for a fire
stop, the maintenance personnel deviated from the curing time requirements of the procedure under the vert al direction of the
,
supervising engineer.
The inspector was concerned about the j
adequacy of the precautions to prevent the compound from dripping into the cabinet.
,q
.
Q
\\
.
_ _
_
_
,,
_
,
,
e y(
v,
<
,;
.
..
,)
2)
The work instructions did not include specific precautions for cleanliness control for a field run length of air tubing being installed to supply a dry air purge to the cabinet.
The inspector did observe good workmanship and care in the preparation of the tubing for installation but noted no specific. directions to blow out the tubing prior to putting it in service.
The inspector was concerned that metal filings in the tubing could be blown'into the control cabinet when the air supply was initiated.
The. inspector identified his concerns to the Maintenance
-
Manager who acknowledged the deficiencies in the work controls and revised the work instructions in preparation for similar work on the A MFP.
The inspector found the licensee's actions to be adequate.
C.
Root Cause Determination Due to the continuing high failure rate of control circuit modules in the Lovejoy controller for the main feedwater pumps, the licensee pursued their investigation of the root cause of the problem by having the failed components examined by a consultant.
As a result of this examination, the licensee determined that due to the humidity and vibration to which the components were subjected a potential for erratic performance could be identified.
Soldering flux residue from the manufacture of the modules had not been completely removed from the circuit boards and was found to have migrated extensively within the module and along the contact pins of the module.
In a humid environment, the flux residue forms a acid
solution which allows an intermittent conductive leakage path along the circuit board and non-conductive corrosion products on the contact pins.
As a result of these findings, the licensee implemented modifications of the Lovejoy controller cabinets to supply a dry air
,
.
-
purge to the interior of the cabinet to minimize the humid j
?
environment.
Additionally, procurement of higher quality modules i
. specially ~ coated for humid conditions was initiated.
j The inspector reviewed the licensee's investigations and found them to be thorough.
The inspector noted that the identified problem was
similar to that found previously during the licensee's investigation
'
of. intermittent high contact resistance in Blue Ribbon connectors in i
'other control systems requiring careful cleaning techniques to t
resolve.
l D.
Long Term Modifications c
The Lovejoy controllers for the "A" and "B" feedwater pumps are to
!'
be replaced in an outage planned in late 1989.
The replacement is the Bailey Network 90, a programmable controller based system with an independent hydraulic fluid system.
One Bailey system is for each feed pump.
- _ - _ _ _ _ _ - - _ _ _ _ _ _ _ _ - _ - _ _ _
y y = y=
-
--
.
..
-
-
I.
.
t l"
The reasons-for the replacement of the Lovejoy controllers are as
]
follows:
1)
The Bailey design has an independent 1800 psi hydraulic fluid
~
system, which replaces the 180 psi lube oil system, for the
.
'
operation of the high and low pressure governor valves of the feed pump.
After the replacement, the hydraulic control fluid
.
of the feed pump governor actuators is not shared by the
'
actuators.of the stop valves and the turbine bearing
lubrication.
Possible contaminations of the hydraulic control fluid from water and bearing wear products are avoided.
This appears to have been a contributor to the current pump stability problems.
2)
The Lovejoy controller controls the high and low pressure
'
governors of the feed pump on a single control loop with a delay for the high pressure governor.
The Bailey system controls the high and low pressure governors independently.
3)
The Lovejoy electrical modules are mounted horizontally and are susceptible to vibration and the pulling force of the weight of the modules which could. generate intermittent signals.
The purpose of the NRC inspection of the system was to ensure that the operation of the feed pump controls does not induce unwarranted transients on the primary system due to the sensitive heat transfer property'of the Babcock & Wilcox steam generators.
The feed pump control system is a system closely related to other safety-related systems and has an indirect impact on the safety of the plant.
-The.. remote control panel located in the control room provides controls for the turbine speed, speed and demand indications, and alarms. A light on the panel indicates whether the control is at the remote panel in the contre 1 room or'at the local panel.
The local control panel houses the two redundant Motorola 68000, 12 MHz, microprocessors and the two redundant DC power supplies, each
- energized from a separate 120 VAC UPS power source. The operating
"j voltage is auctioneered from the two DC power supplied.
In addition to the indications on the remote panel, the local panel provides indications for the position of the icw and high pressure governor valves, alarms for.the failures of the power supplies, the speed sensors and the hydraulic unit, an overspeed test switch, and a control transfer; switch.
The two existing turbine speed sensors are used in the Bailey
' system.
The existing 0-10 volt input from the integrated control system is converted into a 4-20 ma DC signal for better electrical
~
noise immunity before it is sent to the Bailey system.
The existing common alann point for the Lovejoy system in the plant computer is used for the Bailey system.
I
-. - - - _ _ _ _
__ _ _ _.
, _ _ _ _
-..
_
-
__
__
_ _.
.
- 10~
.
.
.
k h
il The' Bailey. Network 90 is programmed to allnw fully automatic or
- manual startup and to provide independent control of the low and
'highlpressuregovernorvalves.
The system control logics are programmed into the microprocessors.
Each microprocessor has its own memory.
Data on the data bus is available to both microprocessors.
If the monitoring microprocessor senses that the controlling microprocessor is not functioning properly, the monitoring microprocessor will take over control
'immediately.
If both microprocessors fail, the control of the
_
. turbine will be manual.
u IThe l'ow pressure governor valve is approximately 100% open before
<
the high pressure governor valve begins'to open.
With the low
"
J
~
pressure governor fully open, the turbine speed is controlled by the
,
,
high pressure governor valve.
A single feed pump can possibly I
handle approximately 80% of reactor power.
The low and high pressure governor valves fail as is on the loss of both microprocessors.
The system alerts the operator and transfers to manual' control.
When the fault is cleared, the system will not automatically ~ transfer the control to "AUT0".
The system must be placed in "AUT0" by the operator.
On a loss of auxiliary and reheat
. steam, the sources of steam for the low pressure turbine, the low pressure governor valve goes 100% open, and the high pressure governor valve controls the turbine.
If a hydraulic servo slave module fails, the associated governor valve fails as is.
The feed pump turbine is tripped on the loss of hydraulic fluid pressure.
A solenoid trip valve in the hydraulic skid will trip the lube oil system on low hydraulic fluid pressure. Additionally, a trip.of the turbine lube oil system will trip the hydraulic skid and close the governor valves.
Each hydraulic fluid skid is equipped with an accumulator which provides hydraulic power for up to one minute of operation if both hydraulic pumps fail.
The failure of one pump will automatically start the other.
A pump alarm alerts the operator.
Three low-low level switches are provided to trip the operating hydraulic pump and to prevent the standby pump from starting on a two out of three logic.
The overspeed test switch on the local panel prevents the Bailey Network 90 overspeed trip from actuating and tripping the turbine.
The mechanical overspeed trip of the turbine is independent of the i
Bailey control system and will trip the turbine if an overspeed condition occurs due to a controller failure.
The Bailey Network 90 overspeed trip is set at a lower speed than the mechanical trip setting.
Two ramp rates are available, one for cold startup and one for hot startup.
A third ramp rate which is not select able is programmed into the control software for rapid acceleration through the
,
_ - - _ - - - -
- - - - - _ -
3g3
-
-
-
--
(p'
'S
.g a
_
j,y<F
- ta 4 $i
'
gp ' g,, 6 -
-
'
s,
,
ys c;
g;'
'
'
,
e r
cri tical. speed. zones.
The critical speed zones are those speeds
,
/
i
[:4 i
i,
't that excessive vibration of the turbine are observed.
The critical
,
.. speed zones are unique to each turbine.
,
In the~" AUTO START" mode and at the selected points, the turbine
/g speed is. held constant to allow for heat soaking to equalize turbine s
t internal temperatures. The " HEAT SOAK IN PROG" light indicates to e.
'
the operator such a process is in progress.
The integrated control system dictates the demand signal from
!
approximately 2800 RPM in the "Feedwater Auto" mode of operation.
The response time of the speed of the feed pumps to a step change of the demand signal is approximately 15 to 20 seconds for the Lovejoy-system.
The Bailey system is required in the procurement technical specification to perform 100% governor actuator travel in respond to a step change in demand in less than or equal to 3 seconds.
Conclusion At the present, there is no summary document to account for the usage of the hardware and software being purchased.
There is no account of the number and type of spare input / output points and the amount of memory available for startup or onsite modification.
The l
inspector was told that the input / output list can be drawn from the j
existing drawings and the initial purchase of memory size is for
'
2000 functional blocks. There is no account of how many functional blocks are not used.
A summary document should be readily available to account for the total number and type of input and output points
.and the usage of memory.
The input and output list (I/O list)
should account for point assignment, description, drawing references, the number of spare points of each type, and other pertinent information.
One of the inspector's concerns is possible electronic spikes during the control transfer and the time lag between the controlling and monitoring microprocessor.
The inspector was told that no observable effect resulted on the control output when the vendor pulled out one of the microprocessor cards to simulate a loss of a microprocessor in a demonstration.
The data on the data bus is available to both microprocessors, and each microprocessor is constantly updated. The transfer of control to the Integrated
'
Control System and the switching-in of the backup microprocessor are points to be observed in the startup testing.
The remote / local control transfer switch in the Bailey system is an input to the microprocessor and not mechanically independent of the
,
microprocessor.
The contro1 transfer is a sof tware programmed
'
function.
In the event of failure of both microprocessors, there is
,:
,
no guarantee where the control is at.
Furthermore, the demand I
sigr.sl from the integrated control system and the turbine speed signal are inputs to the microprocessor.
These signals are v
,
,
.t
_ _ _ _. _ _ _. _. _ _ _ _ _ _ _. _ _ _ _ - - - - - -
h + Q *< A ; 7:
.g
'
X
'
~
+
J s
,
NhR
- M, }Y_,1, '
. e
- ,
.
[') il
.
S
'
'
y- {
~
'
'
p. m r r3 3,,
/
> ' 12 '
.,
l m
y
.
,
1 s-W d)q ;[
- e/' '
-
-
f}g-
'
'J
'
<
Na_"@
_
^
y W t\\): p y
,
_
.
.
.
Jg'
a, necessary;for' manuali control of: the turbine and are probably 'notu
'*
~
."'available<after the=1oss of both microprocessors.
Engineering
..
?
'
changes might befrequired:to the exist.ing drawings to accomplish the
.
W f, f
,? manual"contro11of"the. turbine ~after the loss of both
.i
'
" microprocessors Similarly, the engineering staff was asked by the_
q
,
t'
. inspector how.the etaergency manual control of the turbine: speed,
,e
.
'
.
e
'
,after the" loss'of both microprocessors, is. accomplished f' -
electrically.' The manual raise and lowe'r pushbuttons are inputs to f*
the' microprocessor The inspector was told'that the vendor had sent-
.
< '
.in a modification to put in two additional pushbuttons, " EMERGENCY ~
LOWER" and " EMERGENCY, RAISE",-on the local and remote. control N
,
~
panels..The emergency. lower land raise pushbuttons:are enabled after.
?-
- the-loss of'both microprocessors.
The-licensee could not' explain how the indications'on the' turbine' speed and the demand signal were-
'
to be maintained after the loss of_both microprocessors. Thec s.
licensee agreedato.think through the process of how manual control
'
.is to be accomplished:and what indications and controls independent 2
,
of'the' microprocessors are needed on the control panels to
_
' accomplish the task.
Simulation of the emergency manual control
. " _
should be a major. item in startup testing.
'In genera'1, the Bailey replacement for the Lovejoy feedwater controller appears to be an adequate system.
Selection of. equipment and the-engineering'of the system appear to havelbeen' adequately performed.
Four items are of. concern to the inspector.
Two are_ technical, and the other two are administrative.
1)
A possibility exists that a failure of a microprocessor can be:
masked by the alarm on the local control panel.
An. alarm called the system alarm is located on both the local and remote control panels. The system alarm is a common alarm; The alarm is energized if'any of the alarms on the local panel or any of the microprocessor alarms are energized.
If the system alarm is energized-and there is no other alarm on the local panel. it is clear that the alarm is coming from the microprocessor related failures. The inspector presented a scenario ~in which a failure of one microprocessor may be ignored until both microprocessors. fail.
The scenario goes as follows.
The system alarm is energized, and one of the alarms on the local
panel is also energized. The roving plant operator goes to the local panel to investigate the alarm.
If there is no microprocessor alarm at the time but a second failure were to occur later in the microprocessor, this failure would be masked
,
by the current alarm scheme.
The conclusion is to bring out
'the microprocessor related alarms on an independent window-on the front of the panel and not to risk it being masked by the other alarm.
t
.c
.
Another concern of the inspector a % t the Bailey system is the 2)
,
s xf reliability of the air conditioning unit attached to the local
panel.
The air conditioning unit has to endure prolonged i
,
~
' continuous operation at the control enclosure.
Should the air m
y
,
V
' '
-
_
_
... _ _ _. _ _ _ _.... _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _
__M
y:
t
-
'.
-
-
h conditioning unit fail, the temperature in the local panel could rise to a point that a microprocessor can fail.
The second microprocessor would'probably fail at about the same time.
T.he, reliability of the associated HVAC system did not appear to have-been evaluated by the licensee.
A failure of s
the common HVAC system could potentia,1y lead to a failure of both feedwater pump controllers.
3)-
The informality of controlling dedign basis parameters is a-l concern for the inspector.
In the procurement technical specification for the.. Bailey system, there are desf n basis
parameters such as:
100% governor actuator travel in response-to a step change in demand in less than or equal to 3 seconds; maintaining the feed pump speed within + or - 15 RPM of the demand signal; and speed sensor with a resolution of + or - 1/2 FPM at the rated speed.
The inspector was told that all these parameters were the result s of a meeting with engineering and plant performance group.
These parameters were the best estimates for design t'
related information.
Even thoug5 the numbers were estimates, design basis parameters should be documented with the reasons I
described and with the signatures of the representatives of the concurring groups.
,
,
4)
The informality of controlling procurement design changes is
~
also a concern for the inspector.
A phone call was made to the vendor to change the source of power to the local panel.
The vendor submitted a drawing with the change.
The procurement technical specification was not immediately changed.
No document showed the authorization of the change.
No procureme.it document was issued to the vendor on this change.
Licensee senior management agreed with the inspector concerns and stated that a review and potential corrective actions would be established to correct the stated deficiency and review for potential generic concerns.
No violations or deviations were identified.
9.
Followup Items (92701)
a.
Enforcement Items (Closed) Violation 88-32-01 - Failure to Perform 10 CFR 50.59 Evaluation For Gagging Relief Valves PSV-26101 and PSV26110 Violation 88-32-01 identified that Decay Heat Removal system thermal relief valves PSV-26101 and PSV-26110 had been gagged and made inoperable without determining if the condition involved an unreviewed safety question and without an adequate 10 CFR 50.59 evaluation.
The licensee acknowledged the violation on
_ _ _ _ _ _ --- _ - - - _
- - - - - - - -
-. - -
-
-
--
-
__ _
~
.
.,
.
/
,
L i
.
November 23, 1989, and attributed the. condition to personnel error.
r" The' licensee response reported that a 10 CFR 50.59 evaluation had since been performed and the condition determined to not constitute
>
an unreviewed safety question.
The licensee response committed to train.10 CFR 50,59 reviewers to preclude recurrence of the
<
>
violation. 'NRC Region V acknowledged the licensee response by n
letter dated December 14, 1988. ;An inspection of the licensee's
~
'
F'
corrective action was performed to confirm the adequacy of the
,
,
. licensee response.
The licensee had re performed a 10 CFR 50.59 i
'
'
evaluation for the noted condition and had initiated training for
-
10 CFR 50.59 reviewers. The violation is closed.
e
,
,
.
b.
Licensee Event Reports
,-
(Ciosed)'LER89-05,"UnqualifiedAFWFlowTransmitters"
,
On May 18, 1989, the licensee issued LER 89-05 concerning flow
,
transmitters FT20501, 20502, 20503, 20504, 31802 and 31803, which
>
are'6 of the 8 transmitters in the Emergency Feedwater Initiation
.
and Control (EFIC) system.
The LER identified that these
!
transmitters did not meet the equipment qualification (EQ)
i requirements of 10 CFR 50.49 in that a plastic shipping plug was installed in the transmitters, rather than a metal plug, as specified in vendor. drawings.
The loss of FT 20502, 20503, 20504 and 20501 could have resulted in the loss of Auxiliary Feedwater (AFW). flow indication at flows less that 250 gpm and the loss of FT 31802 and 31803 could have resulted in a loss of flow indication in excess of 250 gpm for one of the redundant AFW lines.
This indication is used in the Emergency Operating Procedure E.05,
" Excessive Heat Transfer," for restoring a steam generator (OTSG) to
!
service after it.has boiled dry.
Licensee engineering had identified that the automatic function to start AFW was retained, however, followup action by the control room operators may have been impaired due to potential erroneous indications.
The indication may become impaired if a postulated high energv line break were to occur in the tank farm, which could produce a harsh environment for the transmitters.
These deficiencies were identified while the plant was in cold shutdown on February 18, 1989 and were corrected prior to plant restart on March 9, 1989. The plant operated in this condition at varying power levels from March 30, 1988 until February 18, 1989.
The vendor drawing which required the metal plugs were in the possession of the licensee, but were not incorporated inta the installation or maintenance process.
The licensee identified the problem and took prompt corrective action, as discussed in the LER.
The violation is not being cited because the criteria specified in Section V.G of the Enforcement Policy were satisf h 1 (non-cited violation 50-312/89-08-01).
10 CFR 50.73 requires that licensees report, via a Licensee Event Report (LER), within 30 days of discovery, any condition which was outside of the design basis of the plant.
The above condition was discovered on February 18, 1989.
Due to an evaluation error made by a Shift Technical Advisor, and later identified and corrected by the engineering organization, the condition was not reported until May
.--_
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
._
_ _ - _ - _ - _ - - -
--
.
.-
18,J 1989..This violation is also not being cited because the-
,
criteria specified in'.Section V.G of the Enforcement Policy were satisfied (non-cited violation 50-312/89-08-02).
>
~c.
-UnresolvediItems
,
>j TClosed) Unresolved Item 88-23-02 - Rockbestos Firewall SR and
.,
'
Firewall III-Cable
.
'
>1a-j
.
j M
Unresolved Item 88-23-02 identified a need for the licensee to
'
W
'
perform a thorough review and evaluation of test anomalies in
Rockbestos test) reports that were part of the licensee's
'
'
'i J
"En0ironmental, Qualification records for Firewall SR and Firewall III cable.
Subseciuent to the inspection, the licensee evaluated the
, test anomalies and documented this evaluation in Engineering Report M
ERPT-E0341.
The report concluded that despite the test anomalies,
, y
'
Rancho Seco's Rockbestos Firewall SR and Firewall III cable met the
.,
requirements'of 10.CFR 50.49.
The report, including supporting calculations, was" reviewed and discussed with the licensee.
The
-
report appeared to show that the tests, despite the anomalies, demonstrated the required performance characteristics and enveloped
~the applications and environment for the installed cables.
Based on the licensee's evaluations and engineering report, the Unresolved Item was closed.
The licensee was informed, however, that the Rockbestos test reports were still to be reviewed by the NRC Vendor Inspection Branch and that any applicable action that may be required as a result of that review would have to be later addressed.
Two non-cited violations were noted.
10.
Exit Meeting (30703)
!
The inspector met with licensee representatives (noted in paragraph 1) at
.i various times during the report period and formally on June 2, 1989.
The scope and findings of the inspection activities described in this report were summarized at the meeting.
Licensee representatives acknowledged the inspector's findings at that time.
l I
i i
i
!
I p
i
'
l
,
_
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ - - - - - - - -
. _ _
_
__.
_
-- J