IR 05000275/1990015
| ML17083C295 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon  |
| Issue date: | 08/10/1990 |
| From: | Morrill P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| Shared Package | |
| ML17083C294 | List: |
| References | |
| 50-275-90-15, 50-323-90-15, NUDOCS 9008290240 | |
| Download: ML17083C295 (32) | |
Text
U.
S.
NUCLEAR REGULATORY COMMISSION
REGION V
Report Nos:'0-275/90-15 and 50-323/90-15 Docket Nos:
50-275 and 50-323 License Nos:
Pacific Gas and Electric Company 77 Beale Street, Room 1451 San Francisco, California 94106 Facility Name:
Diablo Canyon Units 1 and 2 Inspection at:
Diablo Canyon Site, San Luis Obispo County, California Inspection Conducted:.
June 10 through July 21, 1990 Inspectors:
P.
P. Narbut, Senior Resident Inspector K.
E. Johnston, Resident Inspector Approved by:
orrs
,
s
,
eac or roJec s
ec ion a
e sgne Summary:
Ins ection from June 10 throu h Jul
1990 Re ort Nos.
50-275/90-15 and Areas Ins ected:
The inspection included routine inspections of plant opera )ons, maintenance and surveillance activities, follow-up of onsite events, open items, and licensee event reports (LERs),
as well as selected independent inspection activities.
Inspection Procedures 30703, 35502, 35702, 37700, 37701, 37702, 37828, 40500, 42700, 62702, 62703, 71707, 71710, 90712, 90713, 92700, 92701, 92702, 92720, and 93702 were.used as guidance during this inspection.
Safet Issues Mana ement S stem SIMS Items:
None 9003290240 90031$
PDR ADOCK 05000275
~Q PDC
Results:
General Conclusions on Stren th and Weaknesses:
Areas of Weaknesses:
o 'ngineering Instructions Design and plant engineering each issued poorly prepared engineering instructions for craft and operator implementation.
The first situation was due to a design change for diesel generator controls which contained major errors.
It appears that desiqn engineering ignored a conflict between vendor and licensee drawings, did not resolve the conflict prior to the design change issue to the field, and did not alert the field to the conflict (see Section 7).
The other situation involves the testing of the discharge check valve for feedwater pump 1-2 (see Section 4b).
In this case plant engineering issued a temporary test procedure with ambiguous
'cceptance criteria and instructions.
Areas of Strength:
Electrical maintenance technicians displayed. skill and caution during the implementation of the droop relay design change (see Section 7).'espite significant problems with the design change package, the craft personnel installed the droop relays correctly.
Si nificant Safet Matters:
None.
Summar of Violations and Deviations:
None 0 en Items Summar
,One unresolved item and one follow-up item were opene DETAILS Persons Contacted
- J.
D. Townsend, Vice President, Diablo Canyon Operations
& Plant Manager
- D. B.
Miklush Assistant Plant Manager, Operations Services
"M. J.
Angus, hssistant Plant Manager, Technical Services
- B.
W. Giffin, Assistant Plant Manager, Maintenance Services
"W.
G. Crockett; Assistant Plant Manager, Support Services W.
D.
Bar khuff, Actinq guality Control Manager
"T. A. Bennett, Mechanical Maintenance Manager D.
A. Taggert, Director guality Support T.
L. Grebel, Regulatory Compliance Supervisor
"H. J. Phillips, Electrical Maintenarice Mariager J.
S. Bard, Work Planning Manager
~R.
C. Washington, Instrumentatson and Controls Manager J.
A. Shoulders, Onsite Project Engineering Group Manager M.
G. Burgess, System Engineering Manager
'.
R. Fridley, Operations Manager R.
Gray, Radiation Protection Manager E.
C. Connell, Assistant Project Engineer
"J.
E. Tompkins, Director Nuclear Regulatory Affairs The inspectors interviewed several other licensee employees including shift foremen (SFM), reactor and auxiliary operators, maintenance personnel, plant technicians and engineers, quality assurance personnel and general construction/startup personnel.
"Denotes those attending the exit interview on. July 20, 1990.
0 erational Status of Diablo Can on Units 1 and
At the beginning of the inspection period, both units were at lOOX power.
On June 14, 1990, Unit 1 experienced a reactor trip on positive reactor flux rate following a loss of generator load (see Section 4a.).
On June 17, Unit 1 achieved criticality but experienced excessive feedwater pump vibration.
At the same time licensee personnel believed that the second feedwater pump had a stuck open check valve (see Section 4b).
Unit 1 operated, at approximately 50K power from June 20 to June 27 while repairs were performed on the feedwater pump which had experienced excessive vibration and bearing damage.
On June 27, two Unit 2 control rod drive mechanism cooling fan motors were replaced in containment.
The two motors, which had been placed in service only a month earlier, had apparently failed due to lack of lower bearing lubrication (see Section 4c).
On July 12, Unit 2 reduced power to 28K and on July 13 increased to 50'o facilitate repairs to non-safety systems (see Section 4e).
Unit 2 returned to full power on July 1.
0 erational Safet Verification (71707)
a.
b.
Genera1 During the inspection period, the inspectors observed and examined activities to verify the operational safety of the licensee's facility.
The observations and examinations of those activities were conducted on a daily, weekly or monthly basis.
On a daily basis, the inspectors observed control room activities to verify compliance with selected Limiting Conditions for Operations (LCOs) as prescribed in the facility Technical Specifications (TS).
Logs, instrumentation, recorder traces, and other operational records were examined to obtain information on plant conditions and to evaluate trends.
This operational information was then evaluated to determine if regulatory requirements were satisfied.
Shift turnovers were observed on a sample'asis to verify that all pertinent information on plant status was relayed to the oncoming crew.
During each week, the inspectors toured the accessible areas of the facil~ty to observe the following:
(a)
General plant and equipment conditions.
(b)
Fire hazards and fire fighting equipment.
(c)
Conduct of selected activities for'compliance with the licensee's administrative controls and approved procedures.
(d)
Interiors of electrical and control panels.
(e)
Plant. housekeeping and cleanliness.
(f)
Engineered safety feature equipment alignment and conditions.
(g)
Storage of pressurized gas bottles.
The inspectors talked with operators in the control room, and other plant personnel.
The discussions centered on pertinent topics of general plant conditions, procedures, security, training, and other aspects of the work activities.
Radiolo ical Protection The inspectors periodically observed radiological protection practices to determine whether the licensee's program was being implemented in conformance with facility policies and procedures and in compliance with regulatory requirements.
The inspectors verified
" 'hat health physics supervisors and professionals conducted frequent plant tours to observe activities in progress and were aware of significant plant activities, particularly those related to radiological conditions and/or challenges.
ALARA considerations were. found to be an integral part of each RWP (Radiation Work Permit).
C.
Ph sical Securit (71707)
Security activities were observed for conformance with regulatory requirements, implementation of the site security plan, and administrative procedures including vehicle and personnel access screening, personnel badging, site security force manning, compensatory measures, and protected and vital area integrity.
Exterior lighting was checked during backshift inspections.
No violations or deviations were identified.
4.
Onsite Event Follow-u 93702)
a ~
Unit 1 Reactor Tri Followin Load Re ection On June 14, 1990, at 3:55 p.m. Unit 1 experienced a high positive flux rate trip following a load rejection while operating at 100K power.
The load rejection occurred when the second of two output breakers (PBC 632) opened as, a result of an offsite 500 KV grid disturbance caused by a fire.
The first output breaker (PCB 532)
had been opened. earlier for preventive maintenance work.
In accordance with design, the main generator continued, to supply plant loads through the auxiliary transformers.
This design is to facilitate recovery from a load rejection without a reactor trip.
The loss of generator load caused the speed of the main turbine to increase before the turbine governor valves could compensate.
This increased generator frequency, which in turn caused the reactor coolant pumps (RCPs) to speed up.
The increased reactor coolant flow caused cold leg water to penetrate further into the core, thereby inserting positive reactivity.
This positive reactivity then caused reactor power to increase at a rate sufficient to provide a positive flux rate reactor trip.
The power increase appeared to be more pronounced than experienced during startup physics testing due to a more negative moderator temperature coefficient..
Subsequent to the reactor trip and expected turbine trip, the RCPs tripped on underfrequency.
This was due to the slowing of the main generator after it tripped.
RCPs and plant loads did not switch to the secondary offsite (startup)
power immediately which is in accordance with electrical design.
As a result, the operators initiated shutdown on natural csrculation.
Operators manually initiated a transfer to startup power.
However, the circulating water pump designated to automatically transfer did not transfer due to the differences between the frequency of the grid and the slowing main generator.
This resulted in the loss of the main condenser for steam dump operation.
Due to leakage from steam dumps, feedwater heater and moisture separator reheater drain tank level control valves, as well as other steam sources, the
condenser pressurized and one low pressure turbine rupture disc blew out.
The main steam isolation valves (MSIVs) were subsequently closed by the operators without further incident.
As a result of the electrical bus transfer to startup power, the momentary undervoltage condition caused diesel generator 1-1 to automatically start.
Auto start of diesel generator 1-1 on transfer to startup power has occurred in the past and was previously evaluated as an acceptable event.
Operators manually shut down the unloaded diesel generator when the plant conditions stabilized.
Operators subsequently restarted the four RCPs and stabilized the Unit in Mode 3.
Auxiliary power was returned at 6:00 p.m..
As follow-up to the event, the licensee initiated NCR DCl-90-OP-N045 and issued LER 50-275/90-08.
The licensee took the following actions to address some of the consequences of the event:
o The event was reviewed with all operations personnel in the form of an Operations Incident Summary.
A simulator training session was planned to recreate the'equence of events for all licensed operators.
Revisions to emergency operating procedures E-0 and E-O.l were initiated to include steps to isolate the main steam lines and break condenser vacuum if both circulating water pumps are lost following a reactor trip.
o A design change review was initiated to provide a Unit trip (which opens output breakers and transfers loads to startup)
fo116wing a turbine trip initiated by a load rejection.
The high.positive flux rate trip had not been anticipated by the
'icensee.
However, during follow-up discussions between the NRC, the licensee, and the nuclear steam supply system vendor (Westinghouse),
the vendor indicated that this was not an unexpected response to a load rejection and its consequences were bounded in the accident analysis.
Unit 1-Main Feedwater S stem Problems On June 17 and 18, 1990, while at low power during startup of Unit 1 (following the June 14 trip, see Section 4a); the licensee noted excessive vibration of main feedwater pump (FW pp) l-i, and observed parameters that suggested that the discharge check valve (FW-1-506)
for FW pp 1-2 had stuck open.
The vibration of FW pp 1-1 was caused by severe bearing damage and turbine rubbing caused by water in the turbine.
The indication of a stuck check valve proved to be false and based on an apparent lack of understanding of system operatio Check Valve FW-1-506 Unit 2 was critical and ready to increase power.
Operators had been feeding the steam generators with auxiliary feedwater.
During the initial startup of the main feed water pumps operators noted what they considered to be abnormally high recirculation flow on FW pp 1-2 and considered this to be an indication of a stuck or sticking discharge check valve on FW pp 1-. 2.
Operators tried several operations to seat the check valve, including tripping feed water pump 1-2.
This evolution was stopped when feedwater pump 1-1 started to have severe vibrations (which are explained later in this section).
Subsequently, operators used main feed water pump 1-2 while the damaged pump 1-1 was repaired.
At that time actions were taken to verify the check valve operability.
These actions are discussed below.
To investigate the apparent stuck check valve, the licensee
. performed a boroscope inspection, attempted to recreate the sequence of events on the simulator; and performed a test to verify the check valve was properly seating.
Borosco ic ins ection:
The inspection found that the hinge pin,
'e asnsng psns, an isc were all in piece and seated as expected; Simulator test:
The simulator test confirmed that a change in,
~pp.
td h
g i
th p iti ith pp it pumps recirculation valve.
Check valve test:
The, license issued a temporary procedure (PEP o
es
-
-1-506 to ensure it was operating correctly.
In essence, the procedure had operators take data with FW pp 1-2 running and FW pp 1-1 in a standby mode.
Operators were then instructed to reduce, FW pp 1-2 load and allow FW pp 1-1 acquire the load in automatic.
If there were indications that FW-1-506 was leaking, the procedure required the FW pp 1-2 discharge isolation valve be closed, and. if a FW pp l-l flow decrease was then observed this would confirm that FW-1-506 was indeed leaking.
The inspector reviewed PEP 03-01 after it had been approved by the Plant Safety Review Committee (PSRC).
The procedure contained a
number of steps which were ambiguous and had acceptance criteria which were not specific.
The following are examples:
o Step 12.2 required operators to take initial instrumentation readings with FW pp 1-1 "idling."
The inspector discussed with the system engineer and shift foreman their interpretation of the definition of "idling" and observed that it could mean anything from the turbine spinning on condensate flow to running at just less than the speed required for it to assume some of the loa I
Recorded suction flow could vary depending on the extent of leakage of the governor valve and the interpretation of. idling.
o Step 12.6 required that FW pp 1-2 be brought to 3000 RPM with FW pp 1-. 1 maintaining 50K load.
It noted that "if the value of flow on FI-56 (FM pp 1-1 suction flow) is increasing to 0.5 MLBm/hr greater than the value in step 12.5 and/or 12.2..."
then check valve FW-1-506 may be stuck open.
The information in steps 12.5. and 12.2 contained different values from different measuring instruments and different operating conditions.
It was not at all clear what the procedure's author had intended.
o In discussions with the system engineer, he stated that the plan included provisions for stationing a number of people in various places, including a person stationed at the check valve with a stethoscope to listen for the. check valve disc to seat.
These instructions were not provided in the procedure.
The inspector discussed the procedure weaknesses with the Assistant Plant Manager (APH) for. Technical Services.
The APH felt that while there were some ambiguities to the test procedure, the presence of.:
the system engineer and the general plan of action described by the test procedure, would provide confirmation that the check valve was either operating as expected or was sticking open.
The inspector expressed concern that it was the general lack of data and ambiguity of results on June 17 that had provided some question as to the operation of FW-1-506.
To perform a test with a similar lack of precision might provide additional confusion rather than confirmation.
The APH revised the procedure prior to the performance of the test.
PEP 03-01 was performed on June 28 and the licensee determined that FM-1-506 was operating as designed..
Feedwater Pum l-l Excessive Vibration On June 18, 1990, at. 12:37 a.m.,
FM pp 1-1 was-tripped due to excessive vibration.
Upon disassembly it was observed that bearings had been destroyed as well as the seals between each of the stages.
The cause. of the damage was determined to be a bowed turbine rotor, coming in contact with seal surfaces.
The licensee and the pump vendor (Westinghouse)
determined that the bow had resulted when a
condensate drain line had backed up and cooled the pump casing.
The casing contracted, causing the rotor to bow.
The licensee and vendor determined that the following had contributed to the backed up condensate drain.
o An orifice in the drain line appeared to have been partially clogged with rust and scale, preventinq it from fully draining to the condenser.
The orifice and drain line had never been cleaned based on the licensee's record o The turbine had been at low load service, then it was tripped, and then was placed into low load service again.
The licensee determined that this type of service would contribute td the build up of condensation in the drain line.
o At low load, the first stage pressure of the turbine, which supplies the motive force to drive the condensate to the condenser, is at its lowest.
The licensee cleaned the drain line and the orifice for FW pp 1-1.
Cleaning of the other FW pp drain lines is reportedly scheduled for the next outage of sufficient duration.
This was based on the licensee's conclusion that while the drains of the other FW pps may be in similar conditions, at higher steady state power there is adequate motive force to drive condensate to the condenser, condensate generated would be less, and to accomplish the task at power could increase the dissolved oxygen levels in the condensate.
Conclusions The licensee concluded that the indications of a stuck open check valve may have been due to a misunderstanding of main feed pump interactions at low speed.
At the conclusion of the inspection the licensee was reviewing systems test results to determine if the system would respond as operators stated it did.
The licensee-intends to enhance feedwater training if such is indicated when they conclude these studies.
The licensee concluded that a number of corrective actions were necessary to prevent recurrence of vibration damage to the main feedwater pump turbines.
These actions included prohibitions of operation't critical vibration speed and establishment of preventative maintenance for the turbine casing drains.
Unit 2 CRDM Fans Installed Without Bearin Grease On June 8, 1990, Unit 2 control rod drive mechanism (CRDM) cooling fan 2 E-14 tripped due to thermal overload.
A subsequent restart attempt was unsuccessful.
On June 23, 1990, CRDM fan 2-E 11 also tripped on thermal overload and attempts to restart it were likewise unsuccessful.
All eight motors for the Unit 1 and Unit 2 CRDN fans (there are four fans per unit) had been replaced during the cycle 3 refueling outages.
The previous motors had exhibited a high failure rate and in the case of Unit 1, were replaced while the Unit was at 100K power.
The fans and their motors are inside the reactor containment.
The fans do not serve any safety related functions and are not assumed to perform in accident analyses.
However, the fans are needed to prevent high CROM temperatures which would damage the rod control system.
It was determined that at steady state conditions, two fans could sufficiently cool the CRDM d.
e.
During an inspection of two warehouse spare CRDM fan motors, the lower bearings were found to not have any grease.
It was suspected and later confirmed, that the installed CRDM fan motors had been installed without grease in the lower bearings.
The Plant Safety Review Committee approved a temporary procedure (TP TD 9010)
and a safety evaluation to allow the replacement of CROM fans 2 E-11 and 2 E-14 at power.
The replacement was completed June 27, 1990.
The licensee concluded that the motors had been supplied without grease, contrary to the. licensee's specifications to the vendor.
Although the vendor normally supplies its motors without bearing lubrication, the design and lubrication of the lower bearing had been specifically addressed during the purchase of the motors.
Diesel Generator 1-2 S eed Sensin Rela Failure On July 12, 1990, during testing of Diesel Generator 1-2 following the completion of a des>go change (see Section 7), operators received control room indication that the diesel fuel oil (DFO)
priming tank level was falling.
The small priming tank is designed to supply DFO when the D/G is first started and to be isolated when the D/G reaches 100.rpm.
At 100 rpm a shaft driven booster pump is able to supply fuel from the larger DFO day tank.
However, on this start, the priming tank did not ssolateb As a result, the D/G received fuel from both the priming tank and the day tank.
Operators discovered that the solenoid operated priming tank isolation valve (SV-714) had not closed.
The D/G was subsequently shut down and declared inoperable.
Electrica'1 Maintenance determined by that a "Tach Pak" speed sensing relay had failed to provide a close signal to SV-714.
During the trouble shooting of the Tach Pak it was determined that a capacitor in the Tach Pak logic had failed.
The licensee procured a
replacement part of similar design, dedicated it for equality Class I use, installed and tested it prior to the expiration of the Diesel Generator 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> Technical Specification Action Statement.
The licensee initiated non-conformance report (NCR) DC1-90-EM-N049 to address the cause of the failure, potential generic concerns, and corrective action to prevent recurrence.
The inspector will review the NCR during subsequent routine inspection.
Unit 2 Power Reduction For Balance of Plant Corrective Maintenance On July 12 1990, Unit 2 reduced power to 2BX to perform balance of plant (BOP corrective maintenance on the following items:
o Feedwater pump (FW pp) 2-1 had developed a pinhole steam leak directly below the pump casing on a pipe weld for a drain lin o FW pp 2-2 had developed a steam leak on the casing equalizing line.
Both the FW pp leaks were discovered July 10, 1990 by an auxiliary operator performing rounds.
o The No.
2 Heater Drain Tank pump had developed excessive motor vibration.
o The feedwater regulating valve FCV-520 was oscillating which resulted in approximately a 4 Megawatt power swing.
Investigation of this problem required that FCV-520 be closed and its bypass valve, FCV-1520, be used to control flow.
Work was completed on July 16, 1990 and the unit returned to full power.
V No problems were identified with the control of FCV-520.
Following return to full power, -FCV-520 was again observed to be oscillating.
At the end of the inspection period, the licensee still pursuing the cause which was believed to be in the digital feedwater control system logic.
Repairs of the two leaks on the feedwater pumps were determined to=
be rework, since both pipes had recently been repaired.
The inspector had previously asked the licensee if it would not be prudent to perform a review of past history of leaking pipes and welds to determine i'f there was a commop cause..
The licensee concurred and committed to have a review complete by August 31, 1990.
(Follow-up Item 50-275/90-15-02).
Unit j. Control Rod Drive Mechanism Coolin Fan Motor Failure On July 7', 1990, a Unit 1 CRDM fan motor failed, apparently due to the failure of its lower bearing..
The Unit 1 CROM fan motors were of the new design and had been operating for approximately 6 months.
The bearings had been lubricated about a month prior to this failure, unlike the unlubricated bearings discussed in paragraph 4d of this report.
On July 19, 1990, the licensee replaced the motor with a motor of the old design since the two spares of the new design had already been used on Unit 2.'t the end of the report period, the licensee had not established the root cause of the bearing failure of the Unit 1 CRDM fan motor.
The. inspector will followup the licensee's review of the CROM fan motor problems during routine inspection activities.
No violations or deviations were identified.
5.
Maintenance (62703 The inspectors observed portions of, and reviewed-records on, selected maintenance a'ctivities to assure compliance with approved procedures, technical specifications, and appropriate industry codes and standard Furthermore, the inspectors verified maintenance activities were
~
~
~
~
~
erformed by qualified personnel, in accordance with fire protection and ousekeeping controls, and replacement parts were appropriately certified.
The inspector reviewed maintenance activities related to the repairs of FM pp l-l which was damaged due to excessive vibration (see Section 4b),
the investigation of the FM pp 1-2 discharge check valve which was thought to be stuck open (see Section 4b), the CRDM fan motor replacement (see Section 4c), the'/G 1-2 speed sensing relay replacement (see Section 4d),
and the replacement of the D/G 1-2 droop relay (see Section 7).
No violations or deviations were identified.
Sur veil 1 ance 61726 By direct observation and record review of selected surveillance testing, the-inspectors assured compliance with TS requirements and plant procedures.
The inspectors verified that test equipment was calibrated, and acceptance criteria were met or test results appropriately dispositioned.
The inspector reviewed portions of temporary and post maintenance test procedures related to the F'H pp'-2 discharge check valve testing (see Section 4b), the CROM fan motor replacement (see Section 4c),
and the replacement of the D/G droop relay (see Section 7).
No violations or'eviations were identified.,
Diesel Generat r 1-2 Drop Rela Re lacement 37700 The inspector examined Design Change Notice (DCN) DC1-EE-43712, which.,
involves Emergency Diesel Generator droop relay replacement.
The inspectors observed portions of this DCN's implementation for D/G 1-2.
Based on this review, the inspector developed concerns regarding the adequacy of the design review associated with the droop relay replacement, the adequacy of the design change instructions, and the adequacy of a work package which placed responsibility on the skill of the craft for successful completion.
~History On May 11, 1989, D/G 1-2 was declared inoperable following excessive load swings that were observed during a test where the D/G was paralleled to its 4KV bus.
The D/G droop relay, which functions to take the D/G from isochronous mode to droop mode when it is to be placed in parallel with offsite power during testing, was found to not have completely actuated all of its eight contacts.
NCR DC1-89-EM-N053 was initiated to review the root cause of the failure and to initiate corrective action.
Subsequent to the failure of the D/G 1-2 droop relay, other D/G droop relays also failed to completely actuate.
Some contacts were found to be dirty which created increased resistance.
The root cause was determined
to be a misapplication of the relay.
The licensee found that (1) a slight misalignment of mechanical components internal to the relay could prevent all the contacts from actuating and (2)
some contacts did not see high currents and therefore actuation did not
'wash" the contacts.
Normal oxidation and dust buildup were not cleaned from the relay contacts which caused increasing electrical resistance over a period of time.
One corrective action included in the NCR was to initiate a design change to change the existing relays with a model more suited for the application.
A DCN was issued at the end of 1989 replacing one relay with two relays of a different model.
Two relays were used because the relays of the desired model had seven contacts versus eight required in the plant design.
The new model provided sealed contacts so that dust and oxidation should not present a problem.
On March 16, 1990, the first DCN was implemented on D/G 2-1.
Durinq the implementation, the field electrician discovered that the DCN contained error s requiring a field change (FC) to the DCN.
On July -ll, 1990, the inspector observed the replacement of the final droop relay on 0/G 1-2.
No Vendor Su lied of As-Built Dia ram of connections The licensee did not have a diagram of connections for the 0/G exciter cabinets where the droop relays were located.
Additionally, no
"as-built" wiring diagrams were obtained from the plant prior to writing the DCN.
The DCN was written based on the schematic drawings supplied by the vendor and subsequent PG8E drafts of the same drawings.
Included in the DCN package was a sketch, which had been prepared by the PG8E design engineers, of 'a diagram of connections for the replacement relays.
However, the sketch was labeled "info only" and included a note to the craft to refer to the schematic drawings for specifics.
The "info only" sketch included in the DCN did not identify "pass through" wiring which had been terminated on the droop relay.
Pass through wires are wires of the same potential as the terminal they are landed on, but not related to the function of the specific relay (as an example, a ground connection on relay B may land as a "pass through" wire on the ground terminal for relay A which is in turn landed on the cabinet ground terminal).
The inspector noted three pass through wires which had not been identified in the DCN but were later identified in the field.
The licensee did not subsequently assure that the functions provided by the pass through wiring had been tested.
The licensee committed to review the pass through leads and assure that either the routine D/G test or the post modification test (PMT) addressed the operability of these functions.
The inspector discussed the design change with design management.
The assistant Project Engineer stated that while diagrams of connections are available for most equipment cabinets, there were some cabinets supplied by vendors for which only schematic diagrams were available.
Additionally, he stated that an adequate design change can be performed
based on schematic drawings.
The licensee recently performed an inspection of the 0/G control cabinets to obtain as-built wiring diagrams.
The inspections were done following the identification of discrepancies between the control wiring diagram of connections and installed wiring configurations.
The D/G exciter cabinets were not included in this inspection.
The basis for not inspecting the exciter cabinets was that the exciter cabinets were constructed by a different vendor, did not include a diagram of connections, and no discrepancies
'ad been previously noted.
Ade uac of the Desi n Review As stated above, the designers relied on two sets of schematic drawings, the vendor drawings and the licensee drawings, to develop the DCN.
The licensee's drawings, which were based on the vendor drawings, contained an error with respect to one droop relay contact.
Whereas the vendor drawing described the contact as being normally closed, the licensee drawing described it as being normally open.
The drawing discrepancy was discovered during the design process and was not resolved by the.
designers.
The designers based the DCN on the PG&E drawing apparently assuming it to be correct.
Had the DCN been implemented as written, the droop relay would not have>
allowed the D/G to transfer from isochronous to droop mode.
The inspector questioned why the error was not discovered as a logic error during the design phase.
The licensee responded that a check of the logic was not deemed to be necessary since on a contact level, the replacement was essentially a one for one replacement.
No mention of the discrepancy between the vendor and the PG8E drawings was included in the DCN or work order.
This error was discovered by the implementing electrician.
It also appears that the discrepancy would have been identified during the post modification test.
The inspector questioned the designer's judgement in light of the discrepancies between the two drawings.
One drawing was assumed to be correct when a study of the system logic would have identified the error.
Additionally, a preimplementation walkdown could have identified the error.
Worst of all, the design change was issued without informing the field that a.discrepancy could exist.
As a result, the design organization needlessly challenged the skill of the electrician and the adequacy of the post modification test.
Ade uac of Instructions Su lied to the Field Step 9 of the work order (WO) stated
"TERMINATE NEW DROOP RELAYS, DR-A-12 AND DR-8-12 PER DCl-EE-43712. AND FC E-14059.
REF.
MP E-57.2B" MP E-57.2B describes standard practices for wiring terminations and splices.
The DCN, excluding attachments, included 35 pages.
The FC included
pages.
The inspector observed that to accomplish WO step 9 the electrician needed to integrate the information on three drawings in the FC and three in the DC ~,
Presumably to make this task simpler, the DCN included an "info only" sketch of a diagram of connections for the replacement relays.
This
"info only" sketch was revised by the FC.'owever, wiring could 'not be accomplished in accordance with the FC "info only" diagram.
It included two instances where (contrary to standard wiring practices)
more than three leads were landed on one terminal.
As a result, the electrician performing the work on D/G 2-1 had to land these leads in "functionally equivalent" locations and produce an "as-left" drawing of his work. It was this as-left drawing that was used as a blueprint for the work on the other D/Gs.
The electrician performing the work on D/G 1-2 had copies of the as-left drawings for both D/G 1-. 1 and D/G 1-3.
At this point, however, the sketch was a markup of a markup of a poorly drafted original
"info only" sketch included in the DCN and was barely legible.
The whole episode appeared to be complicated by design engineering issuing the diagram of connections as an "info only" drawing instead of the usual action of making the diaqram of connections a offscial design drawing.
It appears that the drawing was issued as "info only" because the designer was reasonably sure that the diagram. would need changes.
By making the diagram of connections an "info only" drawing, the drawing was not controlling and the field was free to deviate from this drawing.
Conclusion In summary, the original DCN was inadequate in that it contained a
fundamental error which would have prevented, the droop relays from performing their design function.
Further, it did not consider the as-built configuration of the cabinet and the effects of pass through wiring.
As a result, the, adequacy of the post modification testing to verify all functions affected by wiring which was disturbed is in question.
Finally, the work package which was sent to the field lacked specificity and required craft skill for its successful accomplishment.
This item is unresolved pending a review of the function of the pass through wiring and a review of the licensee's actions regarding the adequacy of design change instructions (Unresolved Item 50-275/90-15-01).
No violations or deviations were identified.
0 en Item Follow-u 92703 92702 a.
Fuel Handlin Buildin Brid e Crane Hoist Problems Unresolved Item Inspection report 50-275/90-08 discussed problems experienced during the Unit 2 refueling outage with the fuel handling building bridge
- crane hoist.
To address these, questions, the licensee initiated an NCR (DC2-90-TN-N046).
At the close of this inspection period, the Technical Review Group (TRG) had not completed its review of the issues related to this item.
The inspector will followup this item at the conclusion of the TRG revie.
Unresolved Items Unresolved items are matters about which more information is required in order to ascertain whether they are acceptable items, violations, or deviations.
An unresolved item disclosed during this inspection is discussed in Section 7 of this report.
10.
Exit (30703)
On July 20, 1990, an exit meeting was conducted with the licensee's representatives identified in paragraph l.
The inspectors summarized the scope and findings of the inspection as described in this report.