IR 05000528/1991032
| ML17305B723 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 08/20/1991 |
| From: | Wong H NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| Shared Package | |
| ML17305B722 | List: |
| References | |
| 50-528-91-32-MM, 50-529-91-32, 50-530-91-32, NUDOCS 9109090003 | |
| Download: ML17305B723 (70) | |
Text
U.S.
NUCLEAR REGULATORY COMMISSION REGION V
Report Nos.
50-528/91-32, 50-529/91-32, 50-530/91-32 Docket Nos.
50-528, 50-529, 50-530 License Nos.
NPF-41, NPF-51, NPF-74 Licensee:
Arizona Public Service Company P.O.
Box 53999, Station 9012 Phoenix, Arizona 85072-3999 Facility Name:
Palo Verde Nuclear Generating Station Units 1, 2, and
Meeting Location:
Arizona Public Service Company Corporate Office Phoenix, Arizona Meeting Conducted:
August 2, 1991 Prepared by:
J.
A. Sloan, Resident Inspector H. J.
Wong, Chief, Reactor ProjectsSection II Approved by:
~Summar
.
Wong, C i Reactor ProjectsSection II
~
2-o g/
Date Signed A management meeting was held on August 2, 1991, at the APS Corporate Office to discuss plant status and current issues, on-line maintenance practices, engineering issues, the APS PVNGS Business Plan and NRC Diagnostic Evaluation Team followup, and Unit 2 outage preparations.
O~OVOOOS 9 WS~S 1O82i PDR ADOCK 0 PDR G
DETAILS 1.
-Meetin Attendees Arizona Public Service Com an APS M. DeMichele, President/Chief Executive Officer J. Levine, Vice President, Nuclear Production E. Simpson, Vice President, Nuclear Engineering and Construction J.
N. Bailey, Vice President, Nuclear Safety and Licensing B. Ballard, Director, guality Assurance R. Adney, Plant Manager, Unit 3 R. Flood, Plant Manager, Unit 2 W. Ide, Plant Manager, Unit 1 D. Mauldin, Director, Site Maintenance and Modifications G. Overbeck, Director, Site Technical Services P. Caudill, Director, Site Services J. A. Bailey, Director, Nuclear Engineering E. Dotson, Director, Site Nuclear Engineering S. Guthrie, Deputy Director, Nuclear Safety J. Scott, General Manager, Site Chemistry E. Firth, General Manager, Nuclear Training T. Foster, General Manager, Outage Planning and Management P. Hughes, General Manger, Site Radiation Protection J. LoCicero, Manager, Independent Safety Engineering M. Powell, Manager, Nuclear Licensing D. Marks, Manager, Nuclear Safety M. Reeve, Manager, Outage Planning and Management R. Page, Manager, Management Services A. Johnson, Supervisor, Compliance M. Fallon, Senior Media Representative C. Emmett, Senior Coordinator, Owner Services L. Bullington, Acting Supervisor, Nuclear Engineering Assurance Department Y. Lukic, Principle Consultant, Nuclear Engineering Assurance Department Nuclear Re ulator Commission J. Martin, Regional Administrator R. Zimmerman, Director, Division of Reactor Safety and Projects M. Virgilio, Assistant Director for Regions IV and V, NRR D. Kirsch, Chief, Reactor Safety Branch G. Yuhas, Chief, Reactor Radiological Protection Branch H. Mong, Chief, Reactor ProjectsSection II D. Coe, Senior Resident Inspector J. Sloan, Resident Inspector S. Friedenthal, Engineering Intern G. Cook, Senior Public Affairs Officer Others I.
M.
J.
Troncoso, Senior Vice President, El Paso Electric Company Benac, Manager, El Paso Electric Company Draper, Site Representative (SCE/LADWP/PNM)
R. Henry, Site Representative, Salt River Project S. Olea, Chief Engine'er, Arizona Corporation Commission J.
Brown, Engineer, Arizona Corporation Commission 2.
Plant Status and Current Issues Mr. Martin opened the meeting by stating that the NRC and APS periodically meet to discuss items of mutual interest and characterized the format of the meeting as basically an APS presentation.
Mr. Levine summarized the current plant status by stating that all three units were operating at 100 percent power and that they were all running well with no significant problems.
A.
Unit 3 Inadvertent Containment Spray Event Mr. Adney presented a summary of the inadvertent containment spray event of June 19, 1991, including the decision to trip the reactor and stop the reactor coolant pumps.
Mr. Zimmerman asked if the Notification of Unusual Event (NUE) declaration was timely and if the assessment criteria were followed.
Mr. Adney stated that the initial assessment was timely and that management later conservatively decided to declare an NUE based on the circumstances of being in natural circulation after a containment spray event.
Mr. Levine stated that based on further review, current procedures were determined not to require the NUE declaration.
Mr. Adney described the basis for the decision to restart the unit.
He stated that the assessment of conditions in containment following the event found nothing detrimental to plant operations, and that no abnormal indications had been observed since the restart.
He also discussed the cause of the event and the actions planned to prevent recurrence.
Mr. Martin commented that he was unsure of the decision to trip the reactor and stop the reactor coolant pumps solely on the basis of the inadvertent containment spray actuation.
It appeared that prompt followup to determine the appropriate course of action for containmnet spray events was warranted.
APS personnel noted that there is not an industry consensus regarding the response to this situation, but this matter was being evaluated by APS.
B.
Personnel Performance Issues Mr. Ide discussed APS efforts to improve personnel performance.
He stated the desirability of correcting the cause of non-consequential events in order to prevent the occurrence of more significant events.
He presented trend data of reported events which showed that about 50 events were reported per month under the pre-June 1990 system of Problem Resolution Sheets and guality Deficiency Reports, but that 119 events were reported under the new Condition Report/Disposition Request (CRDR) system.
The increase in number of events was indicated to be due to a substantially lower reporting
threshold, which is consistent with APS efforts to correct non-consequential events.
Mr. Ide added that inadequate or improper work practices were major contributors to events.
Nr. Ide also described APS's program of identifying and reviewing recurring events.
He identified events which had recently been reviewed by the Plant Review Board.
He discussed other performance improvement efforts including training impact reviews, work practice training, and the Positive Discipline Program which included coaching employees in the desired performance.
A brief discussion followed of an event in which workers began work on a filter on the wrong train of the reactor coolant pump seal injection system.
This was used as an example in recent maintenance briefings in which workers were admonished to stop, look, touch, verify, anticipate, manipulate, and observe as they do work.
3.
On-line Maintenance Practice Mr. Mauldin described APS efforts to integrate the Individual Plant Evaluation ( IPE)/Probabilistic Risk Assessment (PRA) techniques and results into the maintenance and on-line scheduling program.
He stated that APS currently limits on-line maintenance of safety systems to preventive maintenance (PM), Corrective Maintenance (CM) and Surveillance Tests (ST)
necessary to maintain system operability and reliability.
Work control guidelines are to schedule no more than 50$ of the allowed outage interval.
Elective maintenance is scheduled coincident with other maintenance, but is not allowed to extend the scheduled system unavailability.
He also discussed the planning and review provided by the 12-week Schedule program.
Nr. Martin commented that reduction of the hours of unavailability is not the goal, but that good, reasonable management is the intent.
This should not get out of balance such that maintenance is not performed when needed.
Mr. Mauldin described the status of PM task force reviews for PRA dominant scenarios, which are resulting in PM program changes.
He stated that APS intends to complete the reviews'nd implement appropriate changes by mid-1992.
Mr. Levine commented that PM program changes and better scheduling and planning have resulted in substantial reductions in unavailability of the systems tracked for the Institute of Nuclear Power Operations ( INPO)
Safety System Performance Indicators.
Nr. Mauldin presented Business Plan goals for future performance in this area, including the intention to limit unavailability for other safety significant systems.
APS intends to clearly distinguish between system availability and Technical Specification operability, and to use PRA results in the 12-week scheduling process.
Mr. Martin asked what the most risk-significant systems were, based on the PRA results to date.
Mr. Bailey replied that High Pressure Safety
Injection was very low risk, and that Auxiliary Feedwater, Emergency Diesel Generators, Low Pressure Safety Injection, and Essential Spray Ponds were the leading contributors to core melt risk.
Mr. Martin commented that as the analysis continues, APS should examine the results to determine whether the APS focus is on the appropriate safety significant systems.
He also noted that certain combinations of unavailable systems may also be found to be exceptionally high contributors to risk.
4.
En ineerin Issues Mr. Overbeck presented the status of APS efforts with respect to current engineering issues.
He described the evaluation, corrective actions, and unresolved aspects of the Control Element Drive Mechanism (CEDM) cooling fan, fan bolting, and exhaust stack bolting failures, as well as the status of the root cause of failure evaluation, testing, and corrective actions resulting from the failure of Westinghouse ARD relays.
Progress toward implementation of plant modifications affecting DC equipment related to the PRA findings were also presented.
Mr. Overbeck briefly discussed the status and benefits of the Maintenance Engineering program.
In response to questions from Mr. Wong, he described the controls in place to ensure adequate review of engineering decisions made in the field by these engineers.
Mr. Overbeck admitted that no administrative controls or second checks exist, but that the maintenance engineers have substantial experience at Palo Verde and that they have been given guidance as to what they can and cannot do.
Mr.
Wong expressed caution on the loss of ability to review the engineering decisions.
Mr. Overbeck described the erosion/corrosion evaluation and control program, and stated that while some deficiencies have been identified, APS does not have any known significant problems in this area.
5.
Business Plan/Dia nostic Evaluation Team Issues Mr. Bailey presented the status of Business Plan items related to implementation of the NRC's Diagnostic Evaluation Team corrective actions.
In response to questions from Mr. Coe, Mr. Simpson reaffirmed APS's commitment to timely completion of the design basis reconstitution effort, currently targeted for completion in 1994.
6.
Unit 2 Outa e Pre arations Mr. Flood and Mr. Reeve presented the approach being taken in preparation for the Unit 2 refueling outage scheduled to begin in October 1991.
Outage considerations included:
PRA techniques have been applied in developing the schedule, fully operational train swap-overs are planned, outage management is planned via a central outage control office, and 100 percent constructibility walkdowns for modifications are being required.
There are no planned mid-loop operations with fuel in the reactor vesse Mr. Martin noted that Unit 3 appeared to have a very large number of field changes for plant modifications during its recent outage.
Nr.
Simpson stated that APS has reviewed each one to determine the cause in an effort to prevent recurrence.
7.
Meetin Conclusion Nr. Martin commented that APS appeared to be on an improving trend and that he is cautiously optimistic.
He said he is impressed with indications that Engineering is increasing its effectiveness.
Nr. Martin noted that there have been some recent discouraging events.
He stated that further review of the reactor coolant pump seal injection filter issue may reveal more graphic lessons to be learned, that the recent missing of a chemistry sample shows that people are not sensitive to the, issue, and that the inadvertent containment spray event is rare in the industry.
He stated that each of these is inconsistent with how a plant should be run, and that they are troubling indicators.
He warned that people should not be allowed to relax their vigilance.
Nr. Nartin observed that several things looked positive.
The CRDR program and the evidence of APS management's rising expectations were highlighted as good examples.
Mr. Martin stated that the challenge is how to keep people interested in improving their performance.
Nr. Martin noted that while industry-wide, performance seems to be improving, he gave examples of recent serious problems from each plant in Region V and stated that APS should be looking for other safety significant situations.
He stated the importance of maintaining a
questioning attitude and a broad perspective on issue APS/NRC MANAGEMENTMEETING AUGUST 2, 1991
APS/NRC MANAGEMENTMEETING AUGUST 2, 1991 AGENDA I.
PLANT STATUS AND CURRENT ISSUES UNIT 3 CONTAINMENTSPRAY PERSONNEL PERFORMANCE II.
ON-LINE MAINTENANCEPRACTICES J. M. LEVINE R. J. ADNEY W. E. IDE C. D. MAULDIN IPE/P RA CONSIDERATIONS TECHNICALSPECIFICATION CONSIDERATIONS III.
ENGINEERING CEDM COOLING FANS ARD RELAYS STATUS OF DC MODIFICATION MAINTENANCEENGINEERING EROSION/CORROSION G. R. OVERBECK IV.
BUSINESS PLAN/DIAGNOSTIC ISSUES STATUS J. N. BAILEY.
V.
UNIT 2 OUTAGE PREPARATIONS R. K. FLOOD/W. J. REEVE SHUTDOWN OPERATION CONSIDERATIONS LESSONS LEARNED FROM UNIT 3
UNIT 3 CONTAINIVIENTSPRAY INITIALACTIONS 06/19/91 MODE 1, 100% POWER PLANT PROTECTION SYSTEM ST IN PROGRESS CTMT SPRAY ACTUATIONSIGNAL RECEIVED CS PUMPS START, CS DISCHARGE VALVES OPEN CTMT ISOLATIONVALVES CLOSE VERIFY NON-VALIDCONDITION - NO HIGH CTMT PRESSURE AND TERMINATEDCTMT SPRAY FLOW STOPPED CS PUMPS CLOSED CS HEADER ISOLATION VALVES MANUALLYTRIPPED REACTOR STOPPED 4 RCPS VERIFIED NATURALCIRCULATION DECLARED NUE 1002 1003 1004 1007 1022
'119 CTMT INSPECTION/RCPS AND CTMT FANS MEGGARED RCP RESTARTED, NUE TERMINATED PLANT STABILIZEDIN MODE 3 1339
UNIT 3 CONTAINMENTSPRAY INITIALACTIONS RESTART ACTIVITIES EVALUATIONOF CAUSE LONG TERM ACTIVITIES CONCLUSION
RESTART ACTIVITIES INITIATEDINVESTIGATION (CAT 2)
ASSESSMENT OF PLANT CONDITIONS AND ACTION PLAN TO EVALUATECONSEQUENCES OF EVENT ON SYSTEMS AND COMPONENTS IN CTMT REVIEWED PREVIOUS CSAS INDUSTRY EVENTS ESTIMATED 5000 GAL DISCHARGED INTO CS HEADER AND PIPING PERFORMED CTMT WALKDOWNAND INSPECTION (JUNCTION BOXES, MOVS, SNUBBERS)
EVALUATEDPOTENTIAL CORROSION EFFECTS ALERTED TO GROUNDING PROBLEMS WITH EQUIPMENT OPERATION AND CONTROL ROOM INDICATION
EVALUATIONOF CAUSE ELIMINATEDPOSSIBILITY OF PERSONNEL OR PROCEDURAL ERROR INITIATEDCOOLDOWN TO MODE 5 FOR TROUBLESHOOTING ROOT CAUSE OF FAILURE PERFORMED TEST PUSH BUNION FOUND TO BE INCORRECTLY ASSEMBLED PUSH BUNION REPLACED PUSH BUNION IN ALL3 UNITS INSPECTED 06/22/91, MODE 2, RESTARTED UNIT
LONG TERM ACTIVITIES HEIGHTENED AWARENESS TO POTENTIAL ADVERSE CONDiTIONS THAT MAYDEVELOP AS A RESULT OF THE EVENT, IN PARTICULAR, INDICATIONOF GROUNDING INSPECT REACTOR HEAD FOR BORIC ACID RESIDUE DURING NEXT REFUELING OUTAGE BORIC ACID WALKDOWNAND CLEANUP OF POLAR CRANE SNUBBERS FOUND SUSCEPTIBLE TO INTERNAL CONDENSATION WILLBE TESTED AND ASSESSED EVALUATERESPONSE TO BE TAKEN FOR
!NADVERTENTCSAS (REACTOR AND RCP TRIP}
CONCLUSION NO ADVERSE EFFECTS OF CS ON SAFETY-RELATED SYSTEMS AND COMPONENTS NO EMERGENT WORK AS A RESULT OF THE EVENT ALLEQUIPMENT OPERATED AS DESIGNED EXCELLENT PERSONNEL PERFORMANCE AND TEAMWORK
PERSONNEL PERFORMANCE AN INDUSTRY AND PALO VERDE ISSUE INDUSTRY COMPARISONS CONDITION REPORTING PROGRAM RESULTS EFFORTS TO DETECT TRENDS AND IMPROVE PERFORMANCE
PERSONNEL PERFORMANCE INDUSTRY COMPARISONS PRIMARY CAUSAL FACTORS WORK PRACTICES WRITTEN COMMUNICATIONS VERBALCOMMUNICATIONS INPO HAS FOUND A CORRELATION BETWEEN THE CAUSE OF CONSEQUENTIAL AND NON-CONSEQUENTIAL EVENTS CORRECTING CAUSES OF NEAR MISSES AND NON-CONSEQUENTIAL EVENTS WILLREDUCE THE LIKELIHOODOF CONSEQUENTIAL EVENTS ONE OF THE MAJOR PURPOSES OF THE CONDITION REPORT DISPOSITION REQUEST PROGRAM WAS TO REDUCE THE THRESHOLD OF REPORTING
PROBLEMS REPORTED 140 i20 100
40
FEB MAY TREND PBS/QDR DATA JAN-MAY: CBDR JUNE
CONDITION REPORT DISPOSITION RE UEST CRDR HAS REDUCED REPORTING THRESHOLD 193 CRDR'S ISSUED IN TWO MONTHS RECOGNIZES LINE ORGANIZATIONRESPONSIBILITY TO IDENTIFYAND CORRECT PROBLEMS DISCREPANCIES ARE TRENDED PREVIOUSLY IDENTIFIED RECURRING EVENTS MISSED FIREWATCHES (12/90)
LUBRICATIONISSUES (4/91)
INCORRECT COLSS CONSTANTS (4/91)
CLEARANCE/TAGGING ISSUES (4/91)
RECURRING EVENTS IDENTIFIED SECOND QUARTER LOSS OF INSTRUMENT AIR PROCEDURAL COMPLIANCE WITH FITNESS-FOR-DUTY COLSS FAILURES WORK PRACTICES ERRORS
EFFORTS TO DETECT TRENDS AND IMPROVE PERFORMANCE TRAINING IMPACT REVIEW WORK PRACTICES TRAINING POSITIVE DISCIPLINE PROGRAM BRIEFING TO MAINTENANCEPERSONNEL STAND-DOWN ON FRIDAY, JULY 26, 't991 MANAGER AND SUPERVISOR EMPHASIS TRAINING ON ORGANIZATIONALAND PROGRAMMATIC ROOT CAUSE
IPE PRA TS CONSIDERATIONS PRA INTEGRATION INTO MAINTENANCE PM EVALUATIONLOGIC AND EFFECT ON PRA SAFETY SYSTEM PERFORMANCE INDICATORS FOR 1ST AND 2ND QUARTER 1991 - AF, DG, HPSI BUSINESS PLAN GOALS
IPE/PRA/TS CONSIDERATIONS PRA INTEGRATION INTO MAINTENANCE ON-LINE MAINTENANCE ONLYTO PERFORM REQUIRED PMs, STs, AND CMs NECESSARY TO MAINTAINTHE SYSTEM IN AN OPERABLE CONDITION.
MINIMIZELCO ALLOWABLEDOWN TIME ELECTIVE MAINTENANCETHAT WOULD MAKETHE SYSTEM OR COMPONENT INOPERABLE TO BE PERFORMED COINCIDENT WITH ABOVE. ENSURE OPPOSITE TRAIN AVAILABILITY TWELVE-WEEKSCHEDULING PROCESS IS BEING USED TO GROUP PMs, STs, AND CMs TOGETHER TO MINIMIZESYSTEM 0 NAVAILABILITY
PM EVALUATIONLOGIC AND EFFECT ON PRA PM TASK FORCE REVIEWS COMPLETED AND REVISIONS TO PM DATA BASE INCORPORATED FOR AF, DG, AND CLASS AC AND DC DISTRIBUTION SYSTEMS FOR PRA IDENTIFIED CRITICAL COMPONENTS PM TASK FORCE REVIEWS OPTIMIZING PM DATA BASE THROUGH RESEARCH OF VENDOR DATA, NPRDS, CFAR, EQ FILES, PVNGS PERFORMANCE HISTORY, SOERs, SERs, OERs, ETC. - WITH THE NET RESULT OF REDUCING ON-LINE MAINTENANCE HOURS AND IMPROVING PRA ASSESSMENTS
PVNGS Safety System Performance IndIcator (SSPI)
3-Unit Composite Auxiliary Feedwater System 0.040 0.035 0.032 0.030 O.
(g 0.025 0.020 I
0'~
E 0.0<5 C7 0.021 0.010 0.005 Cumulative Average SSPI SSPI Goal 0.000
PVNGS Safety System Performance Indicator (SSPI)
3-Unit Composite High Pressure Safety Injection System 0.040 0.035 0.030 CL g)
0.025 0.020 I
~~
E 0.015 O
Cumulative Average SSPI SSPI Goal 0.010 0.010 0.005 0.000
'-'='
1990-1991:-
AUXILIARYFEEOMATER SYSTEM QUARTER MAINT HRS CUMUL CRIT MAINT HRS HRS CUMUL CUMUL CRIT HRS SSPI 90-1 90-2 90-3 90-4 91-1 91-2 243 167 846 346
36 243 410 1256 1602
127 3047 2199 5900 6596 5254 5008 3047 5246 11146 17742 5254 10262
.0266
.0260
.0376
.0301
.0058
.0041 QUARTER MAINT HRS HIGH PRESSURE SI SYSTEM CUMUL CRIT MAINT HRS HRS CUMUL CUMUL CRIT HRS SSPI 90-1 90-2 90-3 90-4 91-1 91-2
85 176
150
24 3047 109 2199 285 5900 377 6596 150
5254 150 5008 3047 5246 11146 17742 5254 10262
.0039
.0104
.0128
.0106
.0143
.0073 MAINT QUARTER HRS EMERGENCY AC SYSTEM CUMUL CRIT MAINT HRS HRS CUMUL CUMUL:
CRIT HRS SSPI 90-1 90-2 90-3 90-4 91-1 91-2 525
729 670 363 232 525 608 1337 2007 363 595 6480 6552 6624 6624 6552 5008 6480 1.3032 19656 26280 6552 11560
.0405
.0233
.0340
.0381
.0277
.0257
SYSTEM UNAVAILABILITY TOTAL FOR ALL THREE UNITS (HOURS)
YEAR PM CM FAILURE TOTAL 1990 (1)
810 1991 456 1992
.176 1993
1994
1995
456 324 216 216 216216'36 360 360 360 360 360 1602 1140 752 612 612 612 SSPI TOTAL FOR ALL THREE UNITS (PERCENT)
YEAR PM CM FAILURE TOTAL GOAL 1990 1991 1992 1993
.1994 1995 1;52io 0.
81'.28'.
06io 0. 06io 0. 06io 0. 86'o 0. 58'.
0. 35'o 0. 34'o 0. 34'o.
0. 33'o 0.63'.64:
0.58:
0.57:
0. 57'o 0. 56io 3. 01'o 2. 03~o 1.
21'.97:
0.96:
0. 95'o 2. 10'..
1 ~ 27~o 0. 98'o 0. 98'.
0. 98'.
BUSINESS PLAN GOALS 1991 OBJECTIVE:
DO NOT EXCEED THE FOLLOWING SAFETY SYSTEM UNAVAILABILITY:
HPSI AF EMERG. AC POWER 0.0 t 0 0.032 (0.021 REVISED}
0.025 1992 OBJECTIVE'O NOT EXCEED THE FOLLOWING SAFETY SYSTEM UNAVAILABILITY:
HPSI AF EMERG. AC POWER 0.0100 0.0'j 27 0.0200 5 YEAR GOALS:
DO NOT EXCEED THE FOLLOWING SAFETY SYSTEM UNAVAILABILITY:
HPSI AF EMERG. AC POWER 0.0 t 00 0.0098 0.0150
OBJECTIVE 1.D.3 ADDED FOR 1992:
IMPLEMENTA PROGRAM TO LIMIT UNAVAILABILITYOF SYSTEMS THAT HAVE A SIGNIFICANT IMPACT ON NUCLEAR SAFETY WORK SCHEDULING PROCEDURE WILLBE REVISED TO IDENTIFYSYSTEMS IMPORTANTTO PRA SAFETY, AND INCORPORATE SOUND OPERATING JUDGEMENT AND PRA INSIGHTS TO DETERMINE SYSTEM OUTAGE FREQUENCY
CEDM COOLING FANS RECENT EVENTS DESIGN FAILURE HISTORY PREVIOUS CORRECTIVE ACTIONS POTENTIAL CAUSES UNDER INVESTIGATION IMMEDIATEACTION INTERIM CORRECTIVE ACTIONS NEXT OUTAGE CORRECTIVE ACTION TO PREVENT RECURRENCE
CEDM COOLING FANS RECENT EVENTS UNIT 1 FAN TRIPPED ON GROUND FAULT INSPECTION REVEALED THATTHE FAN SEPARATED FROM ITS MOUNT UNIT 2 FAN TRIPPED ON GROUND FAULT INSPECTION REVEALED 15 OF 16 FAN 8 AND 10 OF 16 FAN D EXHAUST STACK MOUNTING BOLTS NfERE BROKEN OR MISSING
DESIGN TWO {2) 100'/o CAPACITYAIR COOLING UNITS (ACU} EACH WITH COOLING COILS AND TWO
{2} FANS FOUR (4) VERTICALLYMOUNTED FANS WITH FOUR (4} EXHAUST STACKS ONE (I) FAN REQUIRED TO COOL CEDM NORMALLYRUN TWO (2) FANS FANS EXHAUST INTO EXHAUST STACKS EXHAUST STACK MOUNTED TO FLANGE ON ACU HOUSING FAN MOUNTED TO FLANGE IN ACU HOUSING FAILURE HISTORY FAN BEARING FAILURES FAN MOTOR GROUNDINGS FAN MOTOR MOUNTING BOLT FAILURES FAN EXHAUST STACK MOUNTING BOLT FAILURES
PREVIOUS CORRECTIVE ACTIONS ADDITIONOF BACKDRAFTDAMPERS MODIFICATIONOF THRUST BEARING GREASE PORT INSTALLATIONOF VIBRATIONMONITORING EQUIPMENT ON UNIT 1 FAN B AND D FOR PERIODIC EVALUATION POTENTIAL CAUSES UNDER INVESTIGATION RESONANT VIBRATIONFREQUENCY OF STACK ASSEMBLY MATCHES FAN OPERATING VIBRATIONFREQUENCY CEDM FAN MOTOR BEARING FAILURE 8EARING ORIENTATION MOUNTINGALIGNMENT
IMMEDIATEACTION UNIT 2 EXHAUST STACKS RESTRAINED UNIT 2 OPERABLE FANS WERE LUBRICATED UNITS 1, 2, AND 3 EXHAUST STACK AND FAN BOLTS WERE CHECKED EXCEPT FOR OPERATING UNIT 3 FANS UNIT 2 EXHAUST STACK BOLTS REPLACED WIRE ROPE SLINGS INSTALLED INTERIM CORRECTIVE ACTIONS NEXT OUTAGE REPLACE BEARINGS ON ALLCEDM FANS REPLACE FAN AND EXHAUST STACK MOUNTING BOLTS INCREASE THE GRIP LENGTH OF BOLTS USED AND PRELOAD APPLY LOCTITE TO MOUNTING BOLT NUTS iNSTALLSHEAR/ALIGNMENTPINS ON FAN MOUNTING FLANGE RETAIN WIRE ROPE SLINGS ON EXHAUST STACKS
CORRECTIVE ACTION TO PREVENT RECURRENCE COMPLETE ROOT CAUSE OF FAILURE INVESTIGATION DEVELOP ACTION PLAN BASED ON ROOT CAUSE OF FAILURE INVESTIGATION EVALUATEINSTALLATIONOF VIBRATION MONITORING EQUIPMENT AND ALARMS REVISE BEARING LUBRICATIONPM'S
ARD RELAYS PVNGS FAILURE EXPERIENCE APPARENT CAUSE OF FAILURE PVNGS APPLICATION TESTING RESULTS ROOT CAUSE CONCLUSIONS LONG TERM ACTIONS
ARD RELAYS PVNGS FAILURE EXPERIENCE STEAM GENERATOR BLOWDOWN ISOLATION VALVE SAFETY INJECTION VALVE APPARENT CAUSE OF FAILURE PLUNGER AND COIL BONDED TOGETHER WESTINGHOUSE CONTACTED PROBABLE CAUSE DUE TO EPOXY SEEPAGE WESTINGHOUSE 10CFR21 DATED 6/24/91 PVNGS APPLICATION RELAYS INSTALLED IN AUXILIARYRELAY CABINETS RELAYS PERFORM CONTROL AND ALARM FUNCTIONS
TESTING RESULTS THIRTY-FOUR SPARE RELAYS TESTED IN ACCORDANCE WITH WESTINGHOUSE 10CFR21 FORTY-SEVEN IN-SERVICE RELAYS CYCLED AND RESPONSE TIME TESTED ROOT CAUSE CONCLUSIONS EPOXY SOFTENING DUE TO IMPROPER MIX RATIO DURING MANUFACTURE UPON ENERGIZING EPOXY SOFTENS AND BEGINS TO RUN INTO ARMATURE FOREIGN MATERIALIN ARMATURE/PLUNGERAREA, ENOUGH TO CAUSE RELAYTO FAIL A
TIME RESPONSE TESTING APPROPRIATE TESTING METHOD LONG TERM ACTIONS IN - HOUSE DEDICATION IDENTIFYPOSSIBLE REPLACEMENT
STATUS OF DC MODIFICATION DC EQUIPMENT ROOM HIGH TEMPERATURE ALARMSTATUS TRAIN A DOWNCOMER ISOLATION VALVES FAIL OPEN MAINSTEAM ISOLATION VALVES AND FEEDWATER ISOLATION VALVES FAIL CLOSED NON-ESSENTIAL AUXILIARYFEED PUMP CONTROL POWER
STATUS OF DC MODIFICATION DC E UIPMENT ROOM HIGH TEMPERATURE ALARM STATUS DESIGN CHANGE FORECASTED 08/19/91 SCHEDULE FOR INSTALLATION UNIT 1 UNIT 2 UNIT 3 12/31/91 3RD REFUELING 12/31/91 TRAIN A DOWNCOMER ISOLATION VALVES FAIL OPEN DESIGN CHANGE FORECASTED 09/01/91 SCHEDULE FOR INSTALLATION UNIT 1 UNIT 2 UNIT 3 4TH REFUELING 3RD REFUELING 3RD REFUELING
MAINSTEAM ISOLATION VALVES AND FEEDWATER ISOLATION VALVES FAILCLOSED DESIGN CHANGE FORECASTED 09/03/91 SCHEDULE FOR INSTALLATION UNIT 1 UNIT 2 UNIT 3 4TH REFUELING 3RD REFUELING 4TH REFUELING NON-ESSENTIAL AUXILIARYFEED PUMP CONTROL POWER DESIGN CHANGE FORECASTED 12/02/91 SCHEDULE FOR INSTALLATION UNIT 1 UNIT 2 UNIT 3 3RD REFUELING 4TH REFUELING 3RD REFUELING
MAINTENANCEENGINEERING PURPOSE STATUS BENEFITS
MAINTENANCEENGINEERING PURPOSE DIRECT ENGINEERING SUPPORT TO MAINTENANCEAND WORK CONTROL EARLY ENGINEERING INVOLVEMENTIN PROBLEM IDENTIFICATIONAND INVESTIGATION TIMELYRESOLUTION ENGINEERING PRESENCE IN SHOPS STATUS THREE (3)'AINTENANCEENGINEERS ASSIGNED MARCH 1991 FOUR (4) ADDITIONALMAINTENANCE ENGINEERS ASSIGNED AUGUST 1991 THREE (3) ADDITIONALMAINTENANCE ENGINEERS ASSIGNED DECEMBER 199 BENEFITS IMPROVED COMMUNICATIONSAND VISIBILITY IMPROVED TEAMWORK FIELD RECOGNITION OF WHEN ENGINEERING INVOLVMENTIS NEEDED TROUBLE SHOOTING ASSISTANCE IDENTIFICATIONOF MATERIALDEFICIENCIES COLLECTION OF MAINTENANCERELATED DATA AND MATERIALFOR ROOT CAUSE OF FAILURE MINIMIZEDUPLICATE EERS FEEDBACK ON USABILITYOF MNCR AND EER DISPOSITIONS
EROSION CORROSION PROGRAM GOALS FEATURES RESULTS
EROSION/CORROSION PROGRAM GOALS MAINTAINPERSONNEL AND PLANT SAFETY PREDICT AND LOCATE EROSION/CORROSION PROVIDE PREDICTIVE TOOL FOR ENGINEERING TO DISPOSITION TEST RESULTS INCORPORATE INDUSTRY EXPERIENCE FEATURES CHEC/CHECMATE PROGRAM MODEL - SINGLE-PHASE AND TWO-PHASE EROSION/CORROSION PLANNING INDUSTRY INFORMATION-CHEC/CHECMATE USERS GROUP {CHUG)
NON-MODELED DATA BASE - SMALLBORE PIPING AND VALVEWALKDOWNS PVNGS EXPERIENCE AND INSPECTION RESULTS
RESULTS MAJOR AREAS OF EROSION/CORROSION
't2" FEED WATER EXTRACTION LINES OFF OF HP TURBINE MINOR AREAS OF EROSION/CORROSION 18" MAINSTEAM LINES TO HEATERS
't2" CONDENSATE LINES
BUSINESS PLAN DIAGNOSTIC ISSUES STATUS
BUSINESS PLAN/DIAGNOSTIC ISSUES STATUS INTEGRA'TION OP PROGRAMS/INITIATIVESINTO PVNGS 1991 BUSINESS PLAN OCTOBER 1989 IN PREPARATION FOR THE DIAGNOSTIC EVALUATIONIN NOVEMBER t989, PVNGS DEVELOPED LISTING OF PROGRAMS/INITIATIVESUNDERWAYAT PALO VERDE FEBRUARY - MARCH 1990 PROJECT CONDUCTED TO REVIEW AND PROVIDE SUMMARIES OF THE VARIOUS PROGRAMS/INITIATIVES.
APRIL 1990 PROGRAM/INITIATIVESMANUALWHICH INCLUDEDTHE ABOVE INFORMATIONWAS DISTRIBUTED TO SENIOR MANAGEMENT
COMMITMENTS
~~~~~~~%@(PROGRAM/INITIATIVE;TITL'E"".gP<..,',-":.'5g'im.'-'.
integrated Scheduling/12 Week Schedule MOV Program Procedures Parallel Maintenance Activities PM Task Force Site Engineering Excellence Simulator Upgrade Training-Action Plan Upgrade Maintenance Policies, Procedures Work Control Process Improvement Unit/EED interface Meetings Engineering Excellence Program Site Facilities Plan Commercial Grade Dedication Vendor Technical Manuals Design Basis Reconstitution Oversight Group Consolidation (Self-Assessment)
Emergency Operating Procedure Upgrade Material and Equipment Configuration Program Commitment Action Tracking System (CATS) (01GB-OCQ01)
Incident investigation Program (79PR-OIP01)
Plant Modification Committee (05GB-OZZ'1 0)
.k..8!j '.~'BUSINESS" PLAN
.'~+<=,g~; ~.:.<."-".OBJECTIVES <&'.
'.G.3; 3.G.4 1.C.1 Major Task 2 3.C.'1; 3.C.2; 3.C.3 3.C.2 3.D.1; 3.D.5; 3.D.6 3.H.5 2.A.2; 2.A.3 3.C.2; 3.C.3; 3.G.2 3.C.7; 3.G.4 2.C.1; 2.C.2; 2.C.3 3.D.1; 3.D.5; 3.D.6 Included as part of capital budget (4.A.4)
3.A,2 3.H.1 Major Task 2 3.H.1; 3.H.2 1.C.3; 3.8.1 2.B.5 Major Task 2 3.H.1 1.C.'1; 1.C.2 3.B.2 3. INITIATIVES
'jp~'qadi~:"')"";g~pjjgPROGRAM/INITIATIVE'TITL'E'."'w",:""
"'"..:.',6-'.
Central Maintenance Support Group SIMS Improvements & Task Force Central Depot Electronic Article Surveillance (EAS)
Material Control Laser Engraver Off-Line Development 8 Testing Computer Systems Team Building (Units)
Operations QA Plan Strategic Info Plan (IXS Study)
- BUSINESS'PLAN:-'-'.
"-'-w,'.."..:
..',',
.'."OBJECTIVES'<'-"--".
3.G.1 through 3.G.4 3.C.7 3.C.5 3.C.6 3.C.5 To be included in 1992 Business Plan Objective 3.D.6 2.C.1; 2.C.2 QA plan submitted 3. ~
~
~ e ~
e
~
e e ~
e
~
~ -
-
~ ~
-
~
~ ~
~
I e ~
II
~
~ --
~
-
~ ~
I
~ I ~ ~
~ ~ ~
~ ~ ~ ~ ~
~
~
~
~
-
~ ~
~ ~
-
~ I '
~ ~ ~
~
~ ~
~
~ ~
-
~
~
~
~ I '
~ ~ ~ ~
~ ~
~
~ I '
~ ~ ~
~ -
-
- ~ ~
~ ~
~ '
OOI
~
~
~ I ~
~ OUI
~ ~
~
~
~
~ ~
~ ~ ~
~ ~ ~ ~
~ ~
~ ~
~
~ ~
e
~
I
~ ~
~
~
~
~
~
~
~
~ ~
~
~
~ '
~
~
~ I ~
~ -I ~
0'
~
~
~
~
~
~ ~
~ ~
~
~
~ ~
~
~ ~.
I
~
I
~
~ ~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
'I
~
~ ~
~
~
~
~
MAY1990 COMMITMENTMANAGEMENTTASK FORCE (CMTF) REVIEWED PROGRAMS/INITIATIVES AND MADE RECOMMENDATIONS MAY-JUNE 1990 DEVELOPMENT OF 1991 BUSINESS PLAN STRATEGIES AND OBJECTIVES WAS INITIATED 1991 BUSINESS PLAN OBJECTIVE 3.D.6:
DEVELOP A COMMON ENGINEERING WORK MANAGEMENT SYSTEM BY 1992 OBJECTIVE ORIGINALLYDEVELOPED AS A RESULT OF ACTIVITIESINCLUDED WITHIN THE ENGINEERING EXCELLENCE PROGRAM AND SITE ENGINEERING EXCELLENCE PROGRAM OBJECTIVE WAS MODIFIED IN JUNE 1990, TO ESTABLISH A COMMON ENGINEERING WORK MANAGEMENTSYSTEM A USERS GROUP WAS ESTABLISHED EARLY IN 199'f WITH REPRESENTATIVES FROM VARIOUS ENGINEERING AND COMPUTER SERVICE ORGANIZATIONS
AT THE END OF FIRST QUARTER 1991, THE ORIGINALOBJECTIVE WAS ACCOMPLISHED DURING SECOND QUARTER 199 t, COMPLETION OF THE USER REQUIREMENTS, FUNCTIONAL SPECIFICATIONS AND IMPLEMENTATION PLAN WAS ACCOMPLISHED. THE OBJECTIVE HAS BEEN REVISED/EXPANDED TO ENCOMPASS ALLAPPLICABLE PVNGS ORGANIZATIONS
Goal 3:
PRODUCTION - Maximize electric generation and improve organizational effectiveness.
991 Objective 3.D.6:
Identify and evaluate the various work management systems in use within the engineering organization.
Establish the successful attributes of an engineering work management system and develop an action plan to achieve implementation by 1992.
Outlook for Objective Task Plan Performance First Quarter 1991 Planned YTD % Complete Status
% Complete a
Actual Stan
~taaaaS 'rTD Catap4ttoa
10 20 30 40 60 60
80 90 100 Original objective for 1991 has been accomplished.
Based on 3.B.9, the scope is being increased to all PVNGS units.
Support for these actions should be obtainable, as the 1991 objective has been achieved.
A S 0 N
D J
1990 FMAMJJASOND 1991 Orlglnal P an
PALO VERDE NUCLEAR GENERATING STATION Unit 2 Third Refueling Outage Preparations SAFETY QUALITY S CHEBULE
UNIT 2 OUTAGE PREPARATIONS OUTAGE OBJECTIYE OUTAGE PRIORITIES LESSONS LEARNED PROGRAM SUMMARY
UNIT 2 OUTAGE PREPARATIONS OUTAGE OBJ ECTIVE PERFORM A ROUTINE REFUELING SEQUENCE AS WELLAS ALLNECESSARY PREVENTIVE, CORRECTIVE, MODIFICATION,AND SURVEILLANCE TASKS TO SUPPORT CONTINUED SAFE AND RELIABLE POWER OPERATIONS
OUTAGE PRIORITIES PRIORITY 'f
'EACTOR,PLANT, AND PERSONNEL SAFETY RISK REDUCTION EFFORTS OPERATIONS INVOLVEMENTIN SCHEDULE DEVELOPMENT PRA TECHNIQUES APPLIED TO SCHEDULE CONTINGENCY PLANS IDENTIFIED BASED UPON RISK NO LOADED CORE, REDUCED INVENTORY OPERATIONS PLANNED RISK CHANGE ASSESSMENTS MADE DURING OUTAGE INDUSTRY EVENTS AT SHUTDOWN; PUBLICATIONAND MANDATORYREVIEW FULLYOPERATIONALTRAIN SWAP-OVERS MANAGEMENT OPERATIONS MANAGEMENTIN OUTAGE CONTROL MANDATORYMANAGERTOURS DAILYPRODUCTION MEETING, TOPIC 4 1 IS SAFETY SHIFTLYTURNOVER MEETINGS, "TIME OUT FOR SAFETY" PERSONNEL SAFETY APPROACH CHANGES
PRIORITY 2:
QUALITYOF WORKMANSHIPAND EXECUTION PRE-OUTAGE P REPARATIONS DETAILED PROCEDURE REVIEWS FIELD WALKDOWNSOF ALLACCESSIBLE WORK ORDERS IDENTIFICATIONAND SCHEDULING OF ALL SUPPORT ACTIVITIES TEAMS ASSIGNED FOR HIGH RISK WORK FIELD WALKDOWNSFOR EACH DESIGN CHANGE VERIFICATIONOF PARTS PRE-STAGING OPERATIONS DEPARTMENT INVOLVEMENT WORK WINDOW CONCEPT WORK STRICTLY SCHEDULED BY WORK WINDOW EXPANDED EARLYTEST SCHEME RETURN TO SERVICE OF SYSTEM AT END OF WINDOW OPERATIONS INVOLVEMENTIN CONTENT AND SCHEDULE
MANAGEMENTINVOLVEMENT PROVlDE "DO IT RIGHT THE FIRST TlME" ATMOSPHERE DAILYPRODUCTION MEETING, TOPIC 4 2 IS QUALITY SHIFTLYTURNOVER MEETING "DAILY QUALITYTOPICS" MANDATORYMANAGEMENTTOURS TAILBOARDSESSIONS {PRE-JOB, POST-JOB)
FACE-TO-FACE TURNOVERS 24 HOUR PER DAY MANAGEMENT PRESENCE IN THE FIELD PROV)DE A REASONABLE SCHEDULE WITH INPUT FROM FIELD PRIORITY 3 PERFORMANCE OF TASKS AS SCHEDULED NO SURPRISES, EXECUTE THE PRE-PLAN MANAGE FROM THE FIELD, FOR THE FIELD 24-HOUR MANAGEMENT,GUIDANCE, ENGINEERING 100/o VERIFICATIONTASK IS READY TO WORK PROACTIVE COORDINATION TO PREVENT
"ROAD BLOCKS" MANDATORYMANAGEMENTTOURS FOR QUALITYAND SAFETY DO IT RIGHT THE FIRST TIME
LESSONS LEARNED PROGRAM INPUT TO SYSTEM CRITICALFEEDBACK ENCOURAGED ON ALL WORK LESSONS LEARNED FORMS AND BOXES IN PLANT UNIT 2 PERSONNEL PARTICIPATED IN UNIT 3 OUTAGE POST-OUTAGE FORMAL CRITIQUES WITH WORK GROUPS VALIDATION MANAGEMENTREVIEW OF EACH COMMENT OUTAGE MANAGERVALIDATESAND COLLATES OP&M DISTRIBUTES REPORT DISPOSITION EACH LESSON ASSIGNED TO SINGLE POINT WITH A DUE DATE EACH LESSON TRACKED BY COMPUTER AIDS
REVIEW OUTAGE MANAGER REVIEWS/APPROVES RESPONSES NEXT OUTAGE MANAGERTRACKS COMPLETION ASSIGNMENT OF COORDINATORS TO FOLLOW EXECUTION EXAMPLES MANAGEMENT IMPROVE TURNOVER EFFECTIVENESS IMPROVE SHOP INPUT TO SCHEDULE REDUCE RATE OF MINOR INJURIES FORMALIZEAPPROACH TO SHUTDOWN RISKS TECHNICAL ICI CUTUP MACHINE DESIGN PROBLEM 0 DEGREE F HEATUP/COOLDOWN RATE LIMITBELOW SG DEGREE F FOREIGN MATERIALEXCLUSION IMPROVEMENTS ELECTRICAL BUS OUTAGE EFFECTS ON OTHER WORK
SUMMARY AGGRESSIVE 70 DAYSCHEDULE ACHIEVABLE THROUGH LESSONS LEARNED AND OUTAGE STANDARDIZATION RISK REDUCTIONS ACHIEVABLETHROUGH ALTERNATIVESCHEDULING AND RISK RECOGNITION SHUTDOWN RISKS ASSESSED, DOCUMENTED, AND TRAINING PROVIDED TO INCLUDE CONTINGENCY PLANS lh CONTINUE MANAGEMENTINVOLVEMENTIN DETAILS, DECISIONS, AND RISK CHANGE PRIORITIES: SAFETY - QUALITY-SCHEDULE