IR 05000206/1992017
| ML13329A170 | |
| Person / Time | |
|---|---|
| Site: | San Onofre  |
| Issue date: | 05/19/1992 |
| From: | Wong H NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| Shared Package | |
| ML13329A169 | List: |
| References | |
| 50-206-92-17-MM, 50-361-92-17, 50-362-92-17, NUDOCS 9206160087 | |
| Download: ML13329A170 (42) | |
Text
U. S. NUCLEAR REGULATORY COMMISSION
REGION V
Report No /92-17, 50-361/92-17, and 50-362/92-17 Docket No, 50-361, 50-362 License No DPR-13, NPF-10, NPF-15 Licensee:
Southern California Edison Company Irvine Operations Center 23 Parker Street Irvine, California 92718 Facility Name:
San Onofre Units 1, 2, and 3 Meeting Region V Office, Walnut Creek, California Report Prepared By:
B. J. Olson, Pro ct Inspector Approved By:
f42J LA4 0-.Wong, Chi5 f
0a-e 5i0 Reactor Projects Section 2 Summary A management meeting was held on April 30, 1992, to discuss the status of the licensee's program for engineering improvement and to discuss recent plant issues. These discussions included the overspeed of a Unit 2 auxiliary feedwater pump, leakage identified in the Unit I refueling water storage tank and two emerging issues dealing with check valve testing and the recent repairs to a reactor coolant pump in Unit 3. The licensee also provided information pertaining to the "Partners for Success" program, steam generator eddy current testing, and the operating status of Unit 1. A copy of the slides used during the licensee's presentation is enclose PDR ADOCK 05000206 G
DETAILS Meeting Attendees Southern California Edison Company H. Ray, Senior Vice President, Nuclear B. Katz, Manager, Nuclear Oversight J. Reilly, Manager, Nuclear Engineering and Construction M. Short, Manager, Station Technical R. Ornelas, Manager, Licensing L. Cash, Manager, Maintenance D. Brevig, Supervisor, Onsite Nuclear Licensing J. Mundis, Site Support Services San Diego Gas & Electric B. Lacy Nuclear Regulatory Commission J. Martin, Regional Administrator K. Perkins, '
puty Director, Division of Reactor Safety and Projects S. Richards, Chief, Reactor Projects Branch L. Miller, Chief, Reactor Safety Branch G. Yuhas, Chief, Emergency Preparedness and Radiological Protection Branch H. Wong, Chief, Reactor Projects Section 2 A. Gody Jr., Acting Chief, Engineering Section C. Caldwell, Senior Resident Inspector D. Corporandy, Reactor Inspector B. Olson, Project Inspector Details Engineering Improvement Program Mr. Ray initiated Southern California Edison's (SCE) presentation of the engineering improvement program by indicating that the program has been one of their larger areas of work. He turned the presentation over to Mr. Reill Mr. Reilly provided an overview of the engineering improvement program, established in 1988. In addition to improving the quality of design work, one of the program goals was to reduce the reliance on architect/engineering firms. As such, the licensee's nuclear engineering staff grew from 203 in 1988 to 466 in 1992. The majority of the growth occurred in the nuclear engineering design organization, however; the licensee still relied on contract personnel to fill approximately fifty percent of the positions in the design organizatio Mr. Reilly indicated that the experience level of the design engineering organization had increased through the addition of experienced personnel
and through training. Mr. Reilly explained that a job task analysis for design engineers had been performed, and a three phase training program had been developed. The training program provided entry level training, system design engineer training, and supervisory training. The training program will be completed in 1992, one year ahead of schedul Independent audits of design packages had shown that the design organization correctly applied codes and standards, complied with procedures, and achieved the design objectives. Mr. Reilly indicated the audits found that improvement could be made in the quality of written documentation and that calculations could be more rigorously performed and documented. Mr. Reilly also provided audit information to show that errors in design packages had decreased. In response to a question by Mr. Martin, Mr. Reilly indicated that the improvements in the design packages were attributed to increased effort in the formulation of the package. Mr. Short added that plant operators now participate more in the development of design change Mr. Reilly described the licensee's root cause analysis of calculation problems associated with NRC Generic Letter (GL) 89-10, "Safety Related Motor Operated Valve Testing and Surveillance." The licensee's review was a result of an NRC inspection which found inconsistencies ard errors in GL 89-10 design calculations. Mr. Reilly acknowledged that the errors existed and stateu that followup calculations showed there had been sufficient margins in design assumptions to accommodate the calculation errors. The cause of the errors was attributed to deficiencies in supervisory methods and management processes. After the licensee's corrective actions were discussed, Mr. Richards asked if the oversight organization had personnel who could review special calculations, such as those required for GL 89-10. Mr. Katz replied that the oversight organization did have personnel who could review the design calculation After Mr. Reilly completed his presentation, Mr. Martin indicated that he had observed a number of changes in the way that SCE performs technical work. He made specific reference to the past practice of using outside engineering firms to develop designs and the new practice of developing the designs in-house. In response to a question by M Martin regarding the affects of the changes in engineering processes and organization, Mr. Ray indicated that SCE has seen that design changes are better integrated than before and that by performing the work in house, SCE personnel have a better understanding of the plant and plant issue Turbine Driven Auxiliary Feedwater Pump Overspeed Trip Mr. Short provided information pertaining to an overspeed trip of the Unit 2 turbine driven auxiliary feedwater pump (TDAFP), which occurred on February 26, 1992. The overspeed trip was induced by water that had collected in the steam line leading to the turbine. The licensee concluded that the water collected in the line due to a steam trap that had become clogged and failed to operate properl Mr. Short described the system for providing steam to the turbine and stated that two steam traps are installed to remove water that forms in the piping. In 1990, several pump overspeed trips occurred because one of the steam traps was inadvertently left isolated following an outag The 1990 trips were the subject of an NRC escalated enforcement actio SCE implemented changes as a result of the 1990 overspeed trips, including checking the steam traps to determine whether they were ho Mr. Short explained that these checks were made to ensure that the traps were in-service and operatin After the recent overspeed trip, the licensee determined that although the steam trap was not isolated, an internal strainer was found to be about 75 percent obstructed due to the buildup of scale. Mr. Short indicated that condensate would not drain through the trap as a result of the obstruction. The other steam trap in Unit 2 and the two traps in Unit 3 were inspected and found to be free of obstructio Mr. Short explained that the traps were not included in the reliability centered maintenance (RCM) program. Had the traps been in the RCM program, an evaluation of potential failure mechanisms would have been performed, and the potertial for strainer clogging might have been discovered. Mr. Shc.,t indicated thdt the traps are now bypassed, and SCE plans to re-design the steam supply to the TDAFP and eliminate the steam trap Mr. Ray expressed his disappointment that this event occurred. He indicated that the event should not have happened and that he would not have expected it to happen. Mr. Richards stated that if personnel had been a little more intrusive regarding steam trap operation, then the event might have been prevented. Mr. Ray agreed with Mr. Richard Refueling Water Storage Tank Leakage Mr. Reilly discussed the licensee's on-going investigation into leakage observed from the Unit 1 refueling water storage tank (RWST). Evidence of leakage was discovered at nine locations near the tank base. SCE performed an ultrasonic examination of the leak areas, assumed a worst case defect, and performed a stress analysis of the tank. SCE concluded that the RWST was operable. Mr. Reilly stated that the defects are believed to be a result of external corrosion due to the tank being exposed to salt in the air. A remote inspection was performed of the tank's interior, and the inspection indicated that internal defects were not causing the leakag Mr. Reilly indicated that SCE is continuing to characterize the size of the defects, although this is difficult because of the limited access to the affected area at the base of the tank. As new information becomes available, the stress analysis of the tank is re-performed. In addition, SCE has submitted a relief request to the NRC for not performing tank repairs as specified in Section XI of the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Cod The licensee plans to meet with NRC Headquarters personnel at the SONGS site in early May 1992 to discuss this issu Emerging Issues Mr. Short briefly discussed two emerging issues. The first issue pertained to testing normally closed check valves located in the auxiliary feedwater (AFW) system. SCE had performed tests, as required by ASME codes, to demonstrate that normally closed AFW check valves would open and pass sufficient flow. Tests had not been performed to demonstrate that the AFW check valves would close against reverse flow because SCE considered that normal operation of the plant showed that the valves would close successfully. When SCE discovered that tests should be performed to demonstrate adequate valve closure, the tests were performed. The test results were satisfactory. SCE is investigating why the closure tests were never performed for the AFW valve The second issue pertained to the recent failure of six bolts for a reactor coolant pump baffle plate in Unit 3. In April 1991, SCE discovered that bolts for a reactor coolant nuimp baffle plate in Unit 2 had lost their preloa SCE ccrected the Un't 2 problem by using a special technique to assure adequate preload which involved torquing then loosening the bolt. During the last Unit 3 refueling outage, work had been performed on two reactor coolant pumps. When the baffle plates were reinstalled in the pumps, the bolt installation technique developed from Unit 2 was not used. Mr. Short indicated that the bolt installation technique had been thought not to be necessary because a longer bolt had been used in the Unit 3 pumps. Subsequently, six bolts in one of Unit 3's pumps failed, and the bolts in the other pump were found to have lost their preload. The licensee indicated that these bolts would be reworked prior to unit startu Partners for Success Mr. Cash provided a description of this program, which had been implemented for Unit 1. The program teams work planners with maintenance foreman and craft personnel to follow a maintenance activity from planning through completion. Mr. Cash indicated that the program has resulted in improved schedule performance and in a reduction of human errors. SCE plans to implement the program for Units 2 and 3 in 199 Steam Generator Eddy Current Inspection Program Mr. Mundis described the eddy current inspection program. At least 20 percent of the steam generator tubes are inspected during each refueling outage. SCE has not observed problems with stress corrosion cracking that have been observed at other plants, notably Trojan. To date, only about 3 percent of the steam generator tubes have been plugged out of a limit of about 11 percen Unit 1 Status Mr. Ray stated that a proposed settlement for Unit 1 is before the California Public Utility Commission. He indicated that if the settlement is approved then Unit I would not operate beyond October or November 1992. If the settlement is not approved, SCE would seek NRC approval to operate Unit 1 using re-shuffled fuel which is in the spent fuel poo Closing Remarks Mr. Martin thanked Mr. Ray and members of his staff for the opportunity to meet and discuss issues of mutual interest. Mr. Ray stated that he also appreciated being able to discuss current issue NRC MANAGEMENT MEETING APRIL 30, 1992 SOUTHERN CALIFORNIA EDISON
Status Report Nuclear Engineering Organization. Goals Established 1988 - Five Year improvement Program
Improve the Quality of Design Ongoing
Make One Organization Accountable for the Design Complete
Reduce Reliance on Full Service Contractors (AEs)
Complete
Questioning Attitude and Attention to Detail Ongoing
Provide Experienced, Trained Resources 90% Complete
Develop a Comprehensive Design Change Process Complete
TOTAL ENGINEERING STAFF (SCE + CONTRACTORS)
600 500-466 400
- ENGRS 344 300 203 200 120 120 100
STEC NEDO TOTAL STEC NEDO TOTAL 1988 1992
SCE ENGINEERING STAFF EXPERIENCE 100 -....................................
88 80 -.........
..................
- ENGRS
63 60 -...
4 0 -- - - - -3 7...
..
3 6, - - - -
- -
40....3 3............
~6
~
.
0 E-3 E-2 E-1 SR ASS'T E-3 E-2 E-1 SR 1988 1992
TRAINING PROGRAM DEVELOPMENT For NEDO Engineers PERCENT 100 TASKS COMPLETED EDELIVER PHASE Il/III
DELIVER PHASE I & DE E HIRE STAFF & PLAN DE O VALID. JA & START TRNG 60 0I SSFI & JOB ANALYSIS 4 0.............- - -
.....
..
..
88
90
92 YEAR April 1992
PHASE COMPLETION AS OF APRIL 1992 ATTENDEES 250-244.....
....
....
NEDO TRAINING PHASES OIRE/RE TRAINING SCH OIRE/RE TRAINING ATT EISDE/IPRE TRAINING SCH 200 ISCEIPRE TRAINING ATT U SUPERVISOR TRAINING SH W SUPERVISOR TRAINING AT 150 120 100
50
22
PHASE I PHASE 11 PHASE III (61 hr.)
(177hr.)
(35hr.)
ENGINEERING ACCOUNTABILITY
Increase Supervisor Accountability by Requiring Their Participation in the Design Change and NCR Processes
Develop the System Design Engineer Concept
Increase Design Engineering Involvement in Operational Problems
ENGINEER INVOLVEMENT IN OPERATIONS AND MAINTENANCE APPROXIMATE RATE
NCR 2000/YR 25% NEDO/STEC 75% STEC
SPR 500/YR 50% NEDO 50% STEC
FCN 1000/YR 75% NEDO 25% STEC 2 YRS AGO 25% NEDO 75% STEC
NUCLEAR OVERSIGHT ASSESSMENT OF SAFETY EVALUATIONS - 10CFR50.59
- Evaluations performed to standards of NSAC1 25
- In-line review and approval of safety evaluations by Safety Engineering
- Non-Conformance Reports ISEG/Root Cause Group
- Design Changes - Nuclear Safety Group
- Assessment
- Meets requirements of NSAC125
- Performed to high quality standards
- Improvements under evaluations:
- Perform in-line review of safety evaluations for Temporary Field Modifications (TFMs)
INDEPENDENT SAFETY EVALUATION GROUPS ASSESSMENT OF NCRs UNIT 2 CYCLE 6
Reviewed Approximately 300 NCRs
Accepted Approximately 285 NCRs on First Submittal
Found Few if Any Significant Safety Concerns
Significant Increase in Quality Over Past Few Years MGMTPRES.MB1
ANALYSIS UPGRADE
- Mechanical/Civil - 11 Computer Codes Added to Li Thermal Analysis 1 & 2 Phase Flow Analysis ASME Class 1, 2 & 3 Stress Analysis Finite Element Analysis Fracture Mechanics Analysis
- Electrical - 4 Computer Codes.Added to Li Protectioi Coordination of Power Dist Transient Analysis of 3 Phase Power Dist Computer Aided Electrical Dejign
- Nuclear Safety Analysis - 6 Computer Codes Added to Li Containment/Enclosure P/T Analysis Dose Analysis
THIRD PARTY AUDITS OF DCP'S BY CYGNA, UNITED ENGINEERS & UNITED ENERGY SERVICES Summary of Findings
13 audits have been performed to date
Good procedural compliance
Codes and standards are being correctly applied
Major improvement in the 50.59 evaluations since 1989
Designs achieved objectives and would perform intended function
Areas noted for further improvement:
Quality of written documentation Calculations need to be more rigorously performed and documented
NUMBER OF FIELD CHANGES PER REFUEL OUTAGE 2500 2014 2000
- IFCNS 1500
. 1300 1000 970 1000
...
..................................
531 500 -....
430....
U1C1o U2C5 U3C5 UlCil U2C6 1988 1989 1990 1990 1991
REVIEW OF DCP CHANGE CAUSES UNIT 1 E-Error Editorial E-Error Scope 0.04 C-Error Improv Editorial Improv C-Ease 0.48 C-Error 0.11 Scope U1C10 - 1988 UlC11 - 1990
REVIEW OF DCP CHANGE CAUSES UNIT 2 C-ErrorC-Ease.5 Editorial Scope 0.46 0.03 E-Error 0.85 Scope E-Error improv.63 C-Ease 0. 5 8C-Error Editorial 0.04 U2C5 - 1989 U2C6 - 1991
ENGINEERING DESIGN STANDARDS SCOPE The completed set will contain 104 Standards, consisting of 95 Component Standards and 9 Topical Standard Scheduled completion is the end of 199 * Total Standards 104
- Prepared
- Issued
- In Progress
Encompass all engineering disciplines:
- Electrical Engineering
- Mechanical Engineering
- Civil/Structural/Architectural Engineering
- Controls and Instrumentation Engineering
- Nuclear Engineering
ENGINEERING DESIGN STANDARDS OBJECTIVE Provide an organized series of manuals which provide practical guidance to an engineer or designer on issues that must be considered in the design proces * Improve technical performance of the individual and the organizatio * Expand the knowledge base of the organizatio * Improve the efficiency and the productivity of the individual and the organizatio BASIS The content of the standards is based on the following:
- Current industry practic * Technically proven methodologies and practice * Standarization where appropriat * Focused on the unique requirements of a nuclear plant and specifically SONG * Focused on retrofit as compared to new constructio ENGINEERING DESIGN STANDARDS CONTENT In addition to general topics, each standard considers the following:
- Operation
- Maintenance
- Surveillance
- Testing
- Economic
- Construction
- Analytical Methodologies
- Calculational Examples
- Interface
- Procurement
USE OF SFA STANDARD In July 1991, Edison prepared acid issued a Topical Design Standard on Single Failure Analysi RESULTS
- Potential safety enhancements for the containment isolation function of SIS and Recir System identifie * Topical Design Basis Document on Single Failure identified other similar opportunities for safety enhancements of plant system * Updating the UFSAR Failure Moles and Effects Tables will result in improved documentation of plant compliance with NRC safety criteri GENERIC LETTER 89-10 OPPORTUNITIES FOR iMPROVEMENTS
- Initial problem identification
- Effect of results on engineering product
- 6% non-conservative
- Sufficient margin to accommodate non-conservatisms
- Root Cause Categories
-
Supervisory Methods Breakdown
- Assessment of nature of work
- Allocation of experienced engineers
- Inappropriate work tools
- Review process ineffective
-
Management Processes Breakdown
- Identification of scope and [is complexity
- Timely resolution or organizational issues
CORRECTIVE ACTION
- Reassignment of superviso * Detailed review of additional calculations (8).
- Reviewing management process that initiates/controls work for complex tasks that are not project controlle * Enhancement of supervisory training/selectio * Direction on use of engineering tool * Engineering guidelines for 89.10 wor * Calculations will be redon * Reemphasis to supervisors/engineers on management's expectation * Better in process monitorin SAN ONOFRE UNITS 2 & 3 TURBINE DRIVEN AUXILIARY FEEDWATER PUMP WATER INDUCED OVERSPEED TRIP February 26, 1992
RECENT HISTORY
FALL 1990 - OVERSPEED TRIPS DUE TO WATER INTRUSION
TRAP S02-F207 ISOLATED
ASSOCIATED STEAM SUPPLY 2HV-8201 NORMALLY CLOSED
CONDENSATION BUILDUP BEHIND 2HV-8201
PROCEDURES DID NOT CONTROL PRE-TEST POSITION OF 2HV-8201
FALL 1991 - CHECK VALVE DAMAGE OBSERVED
TRACED TO STEAM LINE WARMUP PROCEDURE (9/90)--2MU1258 OPENED PRIOR TO STEAM LINE BLOWDOWN
CHANGED POSITIONS OF 8201 & 8200 FOR S02--MINIMIZE DELTA P THAT ACCELERATES WATER SLUG
OVERSPEED EVENT 2/26/92 FINDINGS
REALIGNMENT OF 8200/8201
IST 4 DAYS LATER
ROUTINE IST ON 2/26
TRAP OPERATION CHECKED BY OPERATOR
PROCEDURE MODIFICATIONS MADE FOLLOWING '90 EVENT TEST SYSTEM IN AS-FOUND CONFIGURATION (No steam line blowdown)
OVERSPEED TRIP FROM OBVIOUS WATER INTRUSION
TRAP F-209
MARGINALLY LOW INLET TEMPERATURE
STEAM SYSTEM TRAPS NOT YET REVIEWED BY RELIABILITY CENTERED MAINTENANCE (RCM) PROGRAM
NO DAMAGE TO STEAM LINE COMPONENTS AND TURBINE
SCRATCH PLATES SHOWED NO SIGNIFICANT MOTION
DISASSEMBLY AND INSPECTION OF CHECK VALVES
OVERSPEED EVENT 2/26/92 CONCLUSIONS
TRAP F209 DEGRADED OPERATION
TRAP INSPECTION - STRAINER CLOGGED
QUALITATIVE OPERATOR MONITORING NOT CAPABLE OF DETECTING DEGRADED PERFORMANCE
RCM PROGRAM WOULD HAVE IDENTIFIED TRAP PERFORMANCE REQUIREMENTS
SCHEMIATIC OF AUXILIARY FEEDWATER PUMIP STEAMI SUPPLY PIP11UG 211VBZOD 211VO201 EL. 43'-8*
EL. 43 -6" 503-26*
36.3-34*
STEAN S.TE Alti HEADER II 1ADE1R E tO" V LINlE Ll:NE r
s-1 111003 6-LINE20 FORu93 4 L UT RACV PURPOSE ONLYV-71
OVERSPEED EVENT 2/26/92 CORRECTIVE ACTIONS
UNIT-2/3 TRAPS F207 and F209 TRAPS OPERATING IN BYPASS UNTIL PLANT MODIFICATIONS IMPLEMENTED INSPECTION OF S03 STEAM SUPPLY TRAPS
RESTORED ALIGNMENT OF 8200/8201
QUANTITATIVE ACCEPTANCE CRITERIA FOR TRAP PERFORMANCE
RE-DESIGN OF STEAM SUPPLY AND OVERSPEED TRIP SETTING PLANNED FOR NEXT REFUELING OUTAG SIG PON-To Main E-089 Steam System To SIG M0*
P4E-089 MUMU-293 FM206Fro EEK0P1
- - -- N--
Fro-onen Condensere MU-co_
To Main E-088SteamSyste U,
K-007P-5
MU-ts M 2<*Condnser Storae Tank S/G0 To Main E-088 StamESyste MUI
UNIT 1 RWST INVESTIGATION
INITIAL RWST INVESTIGATION
Visual Inspection External by STEC ana NEDO Engineers
External UT of Excessible Area Around Defect
Developed a Non-Code Repair to Eliminate Tank Weapage
Order of Magnitude Stress Analysis Done on Tank Defect Location
Review Corrosion Mechanism With Onsite Metallurgist
- Concluded That Corrosion From Outside
Completed Initial Operability Assessment
Initiated Discussion with NRC
CONFIRMATION RWST NVESTIGATION
Best Effort Defect Characterization
Obtained Independent Corrosion Expertise and TK Vendor Expertise
Performed Detail Stress Analysis of Tank
Internal Video Inspection
Based Upon Internal Inspection Performed Detailed External UT of Tank
Historical Records Search
Multiple Discussions with NRC Staff Conclusions to Date:
Initial Evaluation Remains Valid
-
Tank Structurally Operable
-
Submitted Relief Request to NRR for Weapage Condition
STEAM GENERATOR EDDY CURRENTTESTING PROGRAM Refueling Outage Interval Inspections o
20% of tubes (plus "batwing" region and previous
>20% through-wall indications) inspected with bobbin coil probe o
100 tubes/steam generator inspected with profilometry o
All indications evaluated regardless of amplitude o
"3rd Party" review of all data (experienced analysts)
o Eddy current signals are "clean" (i.e., very little interference with interpretation)
Experienced Steam Generator Engineers o
Supervisor -
15 years of steam generator experience o
Senior Engineer -
Level III in eddy current testing
-
10 years of steam generator experience o
Well developed network of peer/industry contacts o Keep abreast of experience at other plants
PARTNERS FOR SUCCESS
GOAL: Create a Spirit of Teamwork and Accomplishment through Commitment, Ownership, Empowerment, and Accountability
Key Elements of the Program
- Planner and Foreman Partnership
- Craftsmen a Part of the Team
- Managing Human Effectiveness Training
- Frequent Meetings with Management
HISTORY OF THE PROGRAM
Why was it needed?
- Personnel Errors
- High Turnover Rate
- Personnel Contaminations
- Poor Schedule Performance
- Grievance Activity
Partners for Success Implemented Unit 1 July 1991
RESULTS ACHIEVED
Improved Communication between Management and the Work Force
Better Quality Work Plans
Reduced Human Errors
Improved Schedule Performance
Reduced Maintenance Order Backlog
FUTURE OF THE PARTNERS PROGRAM
Implement the Program in SONGS Unit 2/3 Maintenance
- Reorganization and Relocation of Planners
- Train Unit-2/3 Maintenan.'e Manager and Maintenance Superintendents
- Form Teams and Train Supervisors and Planners
- Train Craftsmen and Include in Teams
- Include Other Organizations