IR 05000272/1994002
| ML18100A974 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 03/25/1994 |
| From: | Ruland W, Scholl L NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML18100A973 | List: |
| References | |
| 50-272-94-02, 50-272-94-2, 50-311-94-02, 50-311-94-2, NUDOCS 9404040020 | |
| Download: ML18100A974 (9) | |
Text
U.S. NUCLEAR REGULATORY COMMISSION
REGION I
DOCKET/REPORT NO /94-02 50-311/94-02 LICENSE NO LICENSEE:
FACILITIES:
INSPECTION DATES:
INSPECTOR:
APPROVED BY:
DPR-70 DPR-75 Public Service Electric and Gas Company P.O. Box 236 Hancocks Bridge, New Jersey 08038 Salem Nuclear Generating Station -
January 24 - 28, 1994 February 7 - 10, 1994 Larry L. Scholl, Reactor Engineer Electrical Section, EB, DRS William H. Ruland, Chief Electrical Section, EB, DRS
~3R4040020 940328 G
ADOCK 05000272 PDR zfr!t.
ate
Areas Inspected: The performance of the Unit 1 and 2 rod control systems, following the augmented inspection team (AIT) in June 1993 was reviewed. Modifications to the rod control systems, including those already implemented and the modifications planned for future outages were reviewed. The implementation of plant program improvements that were made as a result of the NRC AIT and the PSE&G Significant Event Response Team (SERT)
findings were also reviewe Results: The Unit 1 and 2 rod control systems have operated reliably and required little correctiv~ maintenance since the AIT in June 1993. PSE&G has implemented modifications to Unit 1, and plans similar modifications for Unit 2, to replace the system firing cards and group step counters with more reliable components. Several program changes have been implemented in areas such as the control of troubleshooting, system testing prior to plant startups and the determination of system operability following repeat component failures. -
The development of a comprehensive program for root cause determinations is not scheduled to be fully in place until mid-199 **
DETAILS BACKGROUND During the last Unit 2 plant refueling outage, PSE&G contracted Westinghouse to perform preventive maintenance and testing on the rod control system to improve system reliabilit Following the maintenance, system testing identified several failed components on rod control system printed circuit board_s. During subsequent reactor startups, additional circuit board failures occurred. When the system failures continued to occur over several days, PSE&G established a Significant Event Response-Team (SERT) to determine the root causes of ihe failures and to assess the appropriateness of the decisions to proceed with the reactor startups when a root cause of the system failures had not been establishe On June 5, 1993, the NRC established an Augmented Inspection Team to determine the cause of the rod control system failures, to identify generic concerns and to evaluate PSE&G's performance in addressing these failures. The team found the predominant cause of the rod control system failures was that the solid state components were subjected to high -
- voltage spikes produced by the group step counters, coincident with the loss of a surge suppression circuit. Additional component failures were due to poor work practices during troubleshooting and testing of the system. The team identified weaknesses associated with station policy and procedures relative to the determination of root cause of component failures. Another observation of the team was that the initial troubleshooting efforts lacked *
- clear leadership and delegati<m of responsibilities. The AIT also deterniined that a single rod control system component failure could result in unplanned control rod withdrawal, a condition that was outside of the design basis of the plan The SERT review identified similar weaknesses as those identified by the AIT, and provided numerous additional corrective action recommendations. Actions taken as a result of the AIT and SERT reviews were reviewed during this inspectio.0 ROD CONTROL SYSTEM OPERATION AND RELIABILITY The inspector reviewed the corrective maintenance history for the rod control systeins on both Salem units to assess the system reliability since the AIT inspection in June 1993. The rod control systems on both units have operated reliably with only two circuit card failures, one firing card on each unit, during this time period. The Unit 2 failure was an existing card while the Unit 1 failure occurred shortly after installation of enhanced design model firing cards. Both failures were identified during current trace testing prior to plant startu The pre-startup current trace testing was initiated by PSE&G after the Unit 2 rod control system problems that were the subject of the AIT inspection. The Unit 1 card was returned to Westinghouse (the supplier) to determine the root cause of the failure. The card failure was intermittent, until a transformer on the card short circuited, causing the card to remain in a failed condition. The transformer was replaced and the circuit card then functioned properl During the Unit 1 refueling outage, the electro-mechanical group step counters were replaced with digital counters that utilize liquid crystal displays (LCDs). The new counters were installed to increase reliability and to enhance the availability of replacement part However, several counter failures were encountered during testing performed following the replacement. On three different occasions, a counter locked up (stopped counting), ramped up in counts without an up signal, or lost track of the existing count. PSE&G promptly involved the vendor to assist in the identification of the cause of the failures. The root cause for these failures was determined to be a high resistance electrical contact between the LCD module and the printed circuit card to which it mounts. The connections between the contacts on the LCD and the circuit board contacts are made using elastomeric strips. When the LCD module is mounted onto the circuit board the elastomeric strips are compressed to form a seal around the electrical contacts to reduce the potential for oxidation of the contact The high resistance connection was determined to be due to inadequate cleaning of the contact surfaces prior to assembly and the failure of map.ufacturing personnel to seat the LCD display properly on the circuit board during assembly. When the failed counters were properly cleaned and assembled they functioned as designed. The remainfog counters were functionally tested using self-contained ramp up and count down switches. No additional failures were identified and all of the counters operated properly during subsequent control rod movement The inspector concluded that the rod control systems were operating reliably. The addition of suppression diodes at the time of the AIT and other actions appear to have been successful in preventing more damage to rod control system components. The initiative to perform current trace testing before reactor startup identified two system component failures. When failures of newly installed components occurred, PSE&G promptly involved the vendors to ensure the root cause of the failures was identified and corrected before proceeding with a plant startu.0 SYSTEM MODIFICATIONS To improve the reliability of the rod controls systems, PSE&G implemented two modifications on the Unit 1 rod control system. The changes were the replacement of the firing cards and the group step counters. Similar modifications are planned for Unit 2 during the next refueling outag Firing Card Replacement An enhanced design rod control system firing card was developed by Westinghouse and is a one-for-one replacement of the existing firing cards. The upgraded design provides the following improvements:
additional test points are available on the front edge of the cards to allow access to more circuit points during troubleshooting,
the physical layout and shapes of components permit increased air flow around the card, resulting in lower operating temperatures,
the component layout scheme minimizes heat transfer to the heat sensitive components,
the use of state-of-the-art components provide improved circuit performance,
- including more stable operation of the oscillator circuits over the range of expected
. operating temperatures, and
the use of redundant components in critical circuits was incorporated to improve reliability. For example, additional oscillators permits t):le loss of one oscillator circuit without dropping control rods, as was the case with the original card Group Step Counters As discussed in Section 2.0 of this report, the electro-mechanical group step counters on Unit 1 have been replaced with electronic counters using LCDs. The new counters reduce the number of electrical coils that can produce counter electromotive forces (EMFs) such as those that damaged solid-state components in Unit 2. The digital counters do not use coils in the counting mechanism; however, the counters have a "clicker" to provide the operators an audible indication of control rod motion. The clicker is a solenoid-operated metal arm that taps the counter case whenever the control roos step in or out. The design of the clicker coil circuit has included surge suppressing components to prevent counter EMFs from damaging the solid-state components of the rod control syste In addition to these two modifications, PSE&G also is working with the Westinghouse Owners Group (WOG) to develop a hardware modification that will eliminate the design deficiency that can permit rod withdrawal due to a single component failur The inspector concluded that PSE&G is taking appropriate measures to implement modifications to improve the reliability of the rod control system.0 CORRECTIVE ACTIONS/PROGRAM IMPROVEMENTS RESULTING FROM THE AIT/SERT REVIEWS In addition to performing the system hardware modifications, PSE&G has taken actions to address the findings of the NRC AIT inspection and the PSE&G SERT review. These actions include:
Current trace testing is performed prior to each reactor startup. This testing provides a check of the integrated rod control system performance and has successfully identified several problems, including two defective firing card *
Training for instrumentation and control (I&C) systems technicians and operators relative to the rod control systems has been upgraded. The criticality of the circuit card edge connector condition is now stressed to the technicians and their supervisor Spread pins on the edge connectors caused the loss of the surge suppression* circuit on Unit 2 in 199.
Poor electrical jumpering techniques that were the probable cause of at least two of the rod control system failures in 1993 were eliminated._ Difficult jumpering tasks were simplified by replacing the use of a small, hard-to-control wire jumper with a dummy circuit board that can be easily inserted into the card slot to accomplish the necessary circuit jumperin *
NUREG/CR-5555, "Aging of Westinghouse Pressurized Water Reactor Rod Control Systems," was evaluated and as a result preventive maintenance tasks are being added for the rod control system These include the performance of additional rod control system visual inspections, periodic replacement of the lift coil fuses and thermographic inspections of electrical components to identify potential problems in the early stages of developmen *
Additional guidance was provided to plant operators, via the standing night orders, relative to the assessment of system operability with a particular emphasis on systems operability following numerous component failure *
Procedure SC.IC-GP.ZZ-0006(Q), "Controls Equipment - Troubleshooting," was revised to improve the control of troubleshooting activities. Improvements include the following:
Risk levels must be assigned for each troubleshooting effort and additional supervisory oversight is required as the risk level increases. Direct supervision of activities is specified for risk levels above "Medium." These.
would be activities that could result in a possible load reduction, plant transient, reportable event or plant tri Directions have been added to instruct personnel to evaluate the results of each troubleshooting step to determine if it is correct to p~ocee Test personnel are directed to hold failed components for inspection by the system enginee.0 ROOT CAUSE ANALYSIS (RCA) PROGRAM/PROCEDURES One of the primary findings of the AIT was that there were no well defined station policies or procedures to provide direction for performing root cause analysis of component failure A PSE&G Comprehensive Performance Assessment Team evaluation, performed after the AIT inspection, reached similar conclusions regarding root cause determination policy and procedure PSE&G has formed a team to improve the root cause determination process and procedure T~e team's responsibilities include the development of a process and procedures for identifying and documenting root causes for all failures resulting in corrective maintenance work. The target date for completing this activity is June 1994~ Technicians and supervisors would then be trained and the system would be implemented on activities related to systems that are subject to the NRC Maintenanee Rule by December 1994. Implementation for the remaining systems would be accomplished by May 1995. The team has also been tasked with developing generic root cause analysis procedures, by December 1994, that would provide guidance for performing RCA on a variety of types of problems of varying significance and co~plexitie During this inspection, -the quality of_ root cause analyses reviewed by the inspector were varied. The efforts to identify the root cause of newly installed rod control system components, the Unit 1 firing card and digital counters, were good. Other root cause evaluations, discussed below, did not take advantage of root cause evaluation techniques alieady available and, in some cases, were not effective in identifying the cause of event During a review of RCA methods currently available, the inspector found that Procedure NC.NA-AP.ZZ-0006(Q), "Incident Report/Reportable Event Program and Quality/Safety Concerns Reporting System," was revised, in June 1993, to add root cause evaluation forms for performing Change Analysis, Barrier Analysis and Causal Factor Analysis. Step 5.2.2 of-the procedure directs the department manager/engineer or his designee to perform detailed investigation of the event and to identify the root cause(s) by completing, as a minimum, one of the :root cause technique form The inspector reviewed several incident reports to assess the effectiveness of the root cause analysis fonns in assisting the personnel in detennining the root cause of events. The inspector found, by this review of completed incident reports and through discussions with station personnel, that in many cases the forms were not used. Also, completed forms indicated that the personnel using the forms may not fully understand the root cause analysis process. For example, one event was analyzed using causal factor analysis techniques. One of the questions evaluated during this process is whether there was a task interruption~ The response was yes, the job was stopped after the event occurred, rather than assessing whether a task interruption was a cause of the event. The inspector also noted that for two significant events, involving the loss of breathing air to a worker, and the loss of a vital bus, the investigations were not able to clearly establish a root caus During the inspection, the inspector was informed that future incident reports will be returned to the investigating departments if the required root cause analysis documentation has not been provide Since there is no specific regulation that requires licensees to implement a root cause analysis program, no violation will be cited for the failure of PSE&G personnel to utilize the root cause analysis techniques specified in Procedure NC.NA-AP.22-0006(Q).
The inspector concluded that the root cause analyses are of varied quality and acknowledged that PSE&G is taking measures to develop and implement a more rigorous RCA progra.0 CONCLUSIONS The overall conclusions of this inspection were that the rod control systems were operating reliably with a minimum number of component failures. PSE&G is implementing plant modifications to enhance the systems reliability and is working with the WOG to resolve the single-failure issue. Root cause analysis policies and procedures have been slow to develop and existing programs are not being properly implemented, indicating the need for additional management attention in this are.0 EXIT MEETING At the conclusion of the inspection on February 17, 1994, the inspector met with PSE&G representatives denoted on Attachment 1. The inspector summarized the scope of the inspection findings at that time. The facility representatives acknowledged the NRC inspector finding *
Public Service Electric and Gas D. Best D. Budzik
- L. Catalfomo
- R. Griffith R. Heaton R. Heller S. Mannon L. Mayer
- M. Morroni K. Myers J. Vender
- E. Villar ATTACHMENT 1 Persons Contacted U. S. Nuclear Regulator_y Commission
- S. Barr
- W. Ruland
- J. Wiggins
- Denotes those present at exit meeting.