ML18087A716
| ML18087A716 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 03/09/1983 |
| From: | Office of Nuclear Reactor Regulation |
| To: | |
| Shared Package | |
| ML18087A715 | List: |
| References | |
| NUDOCS 8303160068 | |
| Download: ML18087A716 (23) | |
Text
I I' Abstract I.
Purpose and Scope II.
Background
III. Issues A.
Equipment Issues Contents *
- 1.
Safety Classification of Breakers
- 2.
Identification of Cause of Failure
- 3.
Verification Testing
- 4.
Maintenance and Surveillance Procedures B.
Management Issues
- o;RAFT MAR 9 1983
- 1.
Operating Procedures for Anticipated Transients Without Scram (ATWS) and Reactor Trips
- 2.
Operator Response
- 3.
Operator Training
- 4.
Post-Trip Review
- 5.
Master Equipment List
- 6.
Work-Order Procedures
- 7.
Timeliness of Event Notification
- 8.
Updating Vendor-Supplied Information
- 9.
Involvement of QA Personnel with Other Station Departments
- 10.
Post Maintenance Operability Testing IV.
Conclusions Appendix A Appendix B
~
8303160068 830309 PDR ADOCK 05000272 P
PDR Salem Restart Report 1
Abstract A report for assuring the readiness of Unit 1 of the Salem Nuclear Generat-ing Station* for restart is presented based on the NRC evaluation of the events of February 22 and 25, 1983, when there were failures of the automatic reactor trip system following receipt of valid signals from the Reactor Protection System.
The manual trip system was used to shut down the reactor.
It was determined that the failures to automatically trip were caused by malfunction of the undervoltage trip attachments in both reactor-trip circuit breakers.
A number of issues have been identified as having contributed to causing the events.
Short-term actions have been identified to resolve them prior to resumption of operation, as well as long-term actions needed following restart.
The issues are categorized as 11equipment issues 11 and 11management issues.
11 The equipment issues are (1) the safety classification of the circuit breakers, (2) verification testing of operability, and (3) maintenance and surveillance procedures.
The management issues are (1) operating procedures, training, and response; (2) post-trip review of failure of the automatic system; (3) the quality assurance and work order procedures; (4) timeliness of reporting; (5) updating vendor-supplied documentation; and (6) post-maintenance equipment operability testing.
The staff has reviewed the proposed corrective actions and has determined that they are appropriate and acceptable.
Salem Restart Report 2
Salem Restart Report I.
Purpose and Scope This report briefly describes the,NRC actions to address and resolye equipment and management issues identified by the NRC evaluation of the two events at Unit 1 of the Salem Nuclear Generating Station that resulted in failure of the reactor to trip automatically upon a valid signal.
The second event occurred on February 25, 1983 and led to the realization that a similar event had occur-red on February 22, 1983.
Based upon NRC evaluation, the cause of the failure has been identified and is attributable to the lack of proper attention given to a device in the breaker assembly.
Replacement of such devices with new ones and conducting proper tests and surveillances, in conjunction with resolution of related issues, provides reasonable assurance that Salem Unit 1 can be restarted.
An NRC task force has beer established to conduct a separate longer range study of the broader implications of the events.
NRC long-term actions noted herein are those related only to Salem.
The NRC task force will determine generic actions needed for other facilities.
For the Salem facility, longer term actions developed by this task force may supersede or complement some of the long-term actions identified herein.
I I. Background On February 25, 1983 an event occurred at Unit 1 of the Salem Nuclear Generat-ing Station when the reactor-trip circuit breakers failed to automatically open following receipt of a valid trip signal from the Reactor Protection System (RPS).
The manual trip system was used to shut down the reactor.
Subsequently, it was concluded by the licensee that the failure to trip was caused by a malfunction of the undervoltage cuy) trip attachments in both reactor-trip circuit breakers.
These UV trip attachments translate the electrical signal from the RPS to a mechanical action that opens the circuit breaker.
On February 26, 1983, an NRC team was onsite to conduct initial followup and to collect preliminary information.
As a result of NRC. inquiries,.the licensee determined that both reactor-trip circuit breakers had similarly failed to open upon receipt of a valid trip signal on February 22, 1983.
The failure to auto-matically trip on February 22 was not recognized by the licensee until the com-puter printout of the sequence of events was reexamined in more detail on February 26.
Further evaluation of these events and the circumstances leading up to them revealed a number of issues that require resolution by the licensee and/or the NRC.
This report identifies those issues and the short-term actions proposed to resolve them prior to resumption of operation at Salem Unit 1* and the long-term actions that are needed on a defined schedule following restart.
The. short-term actions required for Unit 1 will also be implemented on Unit 2 prior to restart of Unit 2.
- Salem Unit 2 is presently shut down for refueling and is not presently scheduled to resume operation before Unit 1.
Salem Restart Report 3
The licensee met with NRC staff on February 28 and March 5, 1983 to present the results of initial evaluations related to the events.
Based on licensee sub-mittals of March 1 and March 8, 1983 and on the findings of the NRC evaluation of the Salem events, the issues are categorized as equipment issues and manage-ment issues.
They are discussed in detail in Section III of this report.
III. Issues A.
Equipment Issues Three of the issues relate to the affected equipment, that is, the reactor-trip circuit breakers (Westinghouse DB-50 circuit breakers).
These issues are 1) safety classification of the circuit breakers, 2) 'identification of the cause of the failure, and 3) verification testing of the circuit breakers.
- 1.
Safety Classification of Breakers During the initial NRC evaluation of the February 25 event, it was determined that maintenance was conducted on the Salem Unit 1 reactor-trip circuit breakers in January 1983, following a failure of one reactor-trip circuit breaker to trip upon receipt of an RPS signal at Salem Unit 2 on January 6,*1983.
The work orders authorizing the.. January 1983 maintenance identified the maintenance as not safety related and not requiring quality assurance review.
The reactor-trip circuit breakers contain both a UV trip attachment and a shunt trip attach-ment, but only the UV trip attachment is operated by an RPS trip signal.
As a result, it was not clear on February 26, 1983 what portion, if any, of the reactor-trip circuit breakers was considered safety related by the licensee.
Action/Evaluation This issue has been resolved.
Section 7.2.1.1 of the Salem Updated Final Safety Analysis Report (UFSAR), Revision 0, indicates that the Reactor Trip System includes the reactor-trip circuit breakers and the UV trip attachment.
The Westinghouse Solid State Logic Protection System Description (WCAP-7488L) also defines the scope of the system as including the reactor-trip circuit breakers and the UV trip attachments.
The UV trip attachment and the reactor-trip circuit breaker are safety-related equipment in that they are essential features of the Reactor Trip System, which is necessary to prevent or mitigate the con-sequences of a design-basis event that could result in exceeding the offsite exposure guidelines set forth in 10 CFR Part 100.
The shunt trip attachment of the reactor-trip circuit breakers is not required by present NRC regulations and, although it is provided to perform the manual trip function, no credit is taken for this design feature in the safety analysis (a manual reactor trip also actuates the UV trip attachment).
The licensee in a March 1, 1983 letter to NRC concurred in this understanding.
Hence, the specific issue with regard to the safety classification of the reactor-trip circuit breakers is considered resolved.
Other issues concerning the manner in which the reactor-trip circuit breakers were treated from a procurement and maintenance standpoint at Salem.
are addressed under Management issues (Section III B).
The licensee has made a commitment to install new UV trip attachments on all four Unit 1 circuit breakers prior to restart and to verify that the new circuit breakers have been properly serviced and tested.
Salem Restart Report 4
- 2.
Identification of Cause of Failure The licensee 1s initial determination of the cause of the failure of the reactor-trip circuit breakers (as documented in a March 1, 1983 letter) was that there was binding and excessive friction of the vertical latch lever of the UV trip attachment due to a lack of proper lubrication.
This conclusion was concurred in by Westinghouse representatives and was based on visual inspection of the UV trip attachment, in-place testing performed after the failures, and previous Westinghouse experience.
Because of the importance of the reactor-trip circuit breakers and UV trip attachments, however, the NRC staff has prepared a more structured approach to resolving this item.
The NRC has conducted an initial determination of the cause of the failure based on inspection of the failed trip attachments and interviews with cognizant maintenance personnel on how the devices were maintained.
The inspection indicates that there were* possibly multiple contributing causes of failure.
Possible contributors are (1) dust and dirt; (2) lack of lubrication; (3) wear; (4) more frequent operation than intended by design; (5) corrosion from improper lubrication in January 1983; and (6) nicking of latch surfaces caused by vibration from repeated operation of the breaker.
The contributors appear to be cumulative, with no one main cause.
The initial investigation indicates that the failure was age related and that a new device would perform properly.
Many surfaces of the latch mechanism are worn and the additional friction tended to prevent proper operation.
Proper l~brication throughout the life of the device might have prevented the wear that can be seen on the sample.
These initial findings confirm that the UV trip ~ttachment failed from binding and excessive friction.
A laboratory testing and examination program will attempt to determine the precise cause of failure, if possible.
Appendix A describes the NRC inspection effort and extent of additional examination and testing to be done by NRC.
NRC Action - Short Term NRC conducted an initial investigation of the cause of the UV trip attachment failures by visual examination of the devices by qualified personnel and de-termined how the devices were maintained (See Appendix A).
NRC Action - Long Term NRC will conduct laboratory testing and examination of the failed attachments to determine the precise cause of failure, if possible.
Test and examination results will be used as a basis for future maintenance surveillance and/or requirements for the UV trip attachments.
- 3.
- Verification Testing On August 20, 1982, one reactor-trip circuit breaker on Unit 2 failed to operate during surveillance testing.
A UV trip attachment was replaced on this circuit breaker, the circuit breaker was reinstalled, and subsequent post maintenance testing established operability.
Similarly, on January 6, 1983, a reactor trip occurred at Salem Unit 2 due to a low-low steam generator level, but one Salem Restart Report 5
reactor-trip circuit breaker failed to open.
The licensee concluded that the circuit breaker failure was due to binding from dirt and corrosion in the UV trip attachment.
The UV trip attachment on the Unit 2 circuit breaker, as well as the UV trip attachment on all Unit 1 reactor-trip circuit breakers, was cleaned, lubricated and readjusted under supervision of a Westinghouse representative.
Since the circuit breakers again failed on February 22 and 25, adequacy of the verification testing to ensure circuit breaker operability is an issue. Verification testing following reactor-trip circuit breaker maint-enance or initial installation should be sufficiently comprehensive to provide reasonable assurance that the circuit breaker will function as needed.
Licensee Action - Short Term The licensee has proposed a program to verify proper operation of the reactor-
~**
trip circuit breakers prior to returning them to service.
The program will involve preinstallation testing of UV trip attachments 25 times by the vendor.
After installation on the trip breakers, the UV trip attachment and trip breaker will be tested ten more times.
Following this testing, a time response test of the breaker actuated through the RPS will be performed.
This issue is sufficiently resolved to*permit restart of the plant pending a commitment to develop and implement a program comparable to that described under Long Term.
Licensee Action - Long Term Although the licensee has not yet proposed a long-term.program, the NRC staff proposes an extensive bench test of a reactor-trip circuit b.reaker and UV and shunt trip attachments as an integrated unit.
The test is to involve cycling (a total of 2000 cycles) under simulated environmental service conditions to determine if a properly maintained circuit breaker and its attachments can operate for an extended number of cycles.
The testing should be performed by the licensee or appropriate industry owners group or vendor.
NRC Action - Short Term NRC will verify satisfactory completion of the licensee's short-term preopera-tional testing program.
NRC Action - Long Term NRC will review the licensee's long-term operational verification program for the reactor-trip circuit breakers to assure that the following points are included:
- 1.
a sufficient number of cycles is included to provide statistically meaning-ful results.
- 2.
the test exercises both UV and shunt trip attachments (not simultaneously),
as well as the circuit breakers.
- 3.
the test is conducted under environmental conditions similar to those seen by the circuit breakers.
Salem Restart Report 6
- 4.
sufficient delay time is included between cycles to allow return to steady-state conditions.
- 4.
Maintenance and Surveillance Procedures During the investigation, it was determined that no specific maintenance pro-cedure existed to conduct preventive or corrective maintenance on the reactor-trip circuit breakers.
The maintenance conducted in January 1983 was not per-formed in accordance with the latest Westinghouse recommendations, which were contained in Westinghouse Technical Bulletin NSD-74-1, as amended by technical data letter NSD-74-2.
Additionally, no program of preventive maintenance had been conducted on these circuit breakers since original installation.
With respect to surveillance testing, the licensee conducted a functional test of one of the two reactor-trip circuit breakers every month, so each circuit breaker was tested once every two months.
The surveillance tests, which involved tripping a circuit breaker by use of the UV trip attachment, met the technical specification requirements.
The licensee also operated the circuit breakers weekly by exercising the shunt trip attachment.
In view of the number of reactor-trip circuit breaker failures at Salem, it appears that the periodic surveillance testing was ineffective in assuring reactor-trip circuit breaker operability.
The licensee has now developed a maintenance procedure verification program.
The NRC staff initial review of gram identified certain deficiencies (see Appendix B).
pending further review.
Licensee Action - Short Term and preoperational the procedures and pro-Thi s issue is unresolved The licensee has now developed a specific preventive maintenance procedure for use on the reactor-trip circuit breakers (including the UV trip attachment),
wRich is based on all applicable vendor maintenance recommendations, appropriate quality assurance (QA) requirements, and post maintenance testing.
The licensee has proposed monthly testing of the main reactor-trip circuit breakers by use of the UV trip attachment and weekly testing of the reactor-trip circuit breakers by use of the shunt trip attachment.
Licensee Action - Long Term The NRC intends to require that the licensee incorporate results of a long-term verification testing of the reactor-trip circuit breaker into maintenance and
.surveillance programs.
In July 1982, the licensee had embarked on a managed maintenance program to thoroughly review and update the preventive maintenance program for certain systems and components.
The licensee should continue this program and complete it in a timely manner.
NRC Action - Short Term The NRC staff has completed an initial review of the surveillance and mainte-nance program and its procedures.
Certain deficiencies have been identified Salem Restart Report 7
(see Appendix B)~ The licensee will be required to complete action necessary to resolve the identified deficiencies prior to restart.
With regard to the licensee 1s managed maintenance program described above, the NRC staff will verify prior to restart. that the licensee 1s program also includes the reactor trip system, emergency core cooling systems (including activation systems) actuation systems, the auxiliary feedwater system, and containment isolation systems.
The licensee 1s proposed surveillance test requirements on the circuit breakers will also be reviewed.
NRC Action - Long Term NRC will evaluate the licensee 1s proposed lubrication requirements for the UV trip attachments (i.e., type of lubricant, frequency of lubrication, points of application, etc.).
NRC will also assure that results of long-term verifica-tion testing of the reactor-trip circuit breakers are adequately incorporated into maintenance and surveillance programs to determine testing frequency, inspection requirements, and lifetimes.
The evaluations will be conducted with the assistance of the Franklin Research Center (FRC) and the Brookhaven National Laboratory (BNL).
B.
Management Issues Based on examination of the circumstances associated with the events involving reactor-trip circuit breakers, certain issues have been identified relative to procedures, training, etc. that are not solely related to the reactor-trip cir-cuit breakers.
The extent to which such issues impact other systems, components or operations at the Salem facility needs to be examined.
These are categorized as management issues.
They are
- 1.
Operating procedures for ATWS and reactor trips
- 2.
Operator response
- 3.
Operator training effectiveness relative to the RPS and associated indicators
- 4.
Post-trip review
- 5.
Issues r~lated to the Master Equipment List associated with the licensee 1s QA program
- 6.
Work order procedures
- 7.
Timeliness of 10 CFR 50.72 reporting
- 8.
Updating vendor-supplied documentation
- 9.
Involvement of QA with other station departments
- 10.
Post maintenance equipment operability testing These issues are discussed in the sections below.
Salem Restart Report 8
- 1.
Operating Procedure for Anticipated Transients Without Scram (ATWS) and Reactor Trips Interviews with control room operators were conducted by NRC staff, and a review of the operating procedure for ATWS and reactor trip (EI-4.3) have revealed that a) the operators do not take immediate action to initiate a manual trip based on reactor-trip 11 first-out 11 annunciators, b) they were not directed to do so by the procedure; however, the procedure did require a manual trip if an automatic reactor trip did not occur.
The procedure required only evaluation of reactor power level remaining high and/or multiple control rods failing to insert, c) at least one operator questioned the appropriateness of the ATWS procedure's step to tr.ip the turbine, without first verifying that the reactor had tripped, since that results in a loss of heat sink, and d) the revised procedure dated March 4, 1983, would not have substantially changed the operators* response due to a perceived need to evaluate plant status from control room indications.
Licensee Action - *short Term
- 1.
The licensee shall identify the indications in the control room that provide positive indication, without operator analysis or verification, that an ~utomatic reactor trip demand is present.
- 2.
The licensee must revise procedures to direct the operators to insert a manual trip whenever positive indication of an automatic trip demand is present without delaying to evaluate the overall plant status.
- 3.
The licensee must review the basis for the ATWS procedure steps and order of priority in light of the operators* concern, revise the procedure as necessary, and brief the operators on the basis for the procedural steps and importance of procedural compliance.
- 4.
All operators must be trained on the revised procedures prior to restart of Unit 1.
Licensee Action - Long Term Incorporate any procedural changes for Unit 1 into Unit 2 procedures and retrain Unit 2 operators on revised procedures prior to Unit 2 ~estart.
NRC Action - Short Term NRC will review the licensee's revised procedures and basis for the procedural steps and order of priority.
NRC Action - Long Term NRC will incorporate review of ATWS basis into the review of the Westinghouse Owners Group, Emergency Operating Procedure Guidelines review.
Salem Restart Report 9
.AFT
- 2.
Operator Response Interviews with operators on shift for the February 22 and 25, 1983 events and with I&C and maintenance personnel disclosed the following:
- a.
In both events, the operators too_k 20 to 30 seconds to determine the overall plant status and initiate a manual reactor trip.
For the first event, this evaluation time was necessary because of the large number of alarms and equipment and indicators lost with the electrical bus transfer failure and was nearly identical to the time it took for the plant condi-tions to degrad causirig the RPS to respond.
For the second event, the evaluation of the plant status began when the reactor trip annunciator actuated and the evaluation determined that a reactor trip was in fact necessary based on plant parameters and RPS indicators.
This time could have been shortened had the operators recognized that a valid trip was ca 11 ed for by the* RPS,.
- b.
Information provided in the control room (i.e., first out panel alarms, illuminated RPS displays, and safety grade instruments) is adequate to immediately identify an ATWS event.
I&E and maintenance personnal indicated that the first out panel and the RPS logic are highly reliable.
- c.
During the first event, after an operator was directed to manually trip (scram) the reactor, the switch handle was not operated correctly.
When the SRO called for a manual trip, the control inadvertently was pulled off the board and had to be reinserted to perform the manual trip.
Because of the near coincident automatic trip signal, this may have contributed to the operator's failure to reoognize that the automatic trip system had
.called for a trip and had failed to trip the reactor prior to the manual
- trip,
- d.
In spite of the positive indication of the reactor protection system failure during the second event, the operators did not understand or trust the indications.
Because of this the operators unnecessarily reevaluated plant status.
The operators manually tripped the reactor in response to their evaluation of the plant status and RPS indicators and not due to recognition of the failure of the reactor protection system.
The NRC staff concluded that, given the operators' understanding of the reactor protection system, their lack of confidence in the annunciators, their perceived need to determine the overall plant status, and their use of procedures, the response of the operators was prompt and adequate to protect the reactor for both events.
Licensee Action - Short Term
- 1.
Operators must be cautioned on the use of the manual trip 11J 11 handle contra 1.
Licensee Action - Long Term
- 1.
The licensee should evaluate alternative means to permanently secure the 11J 11 handle.
Salem Restart Report 10
- 3.
Operator Training Interviews conducted by NRC with the licensed operators who were onshift during the two events indicate a lack of familiarity with the functions of the annuni-cators and indicators associated with RPS.
The interviews also revealed that the operators who were onshift during the February 25 event did not recognize that a malfunction of the RPS had occurred until approximately 30 minutes after the event.
Specifically, the operators interviewed were not able to describe whether the reactor-trip-indicator* light (red) on the RPS mimic status panel indicated a demand or confirmation of a breaker trip.
Interviews also indicated that at least some operators questioned the validity of _annunciators until they could be confirmed by independent indication.
Based on a review of calibration testing incidents in 1982, where the reactor trip annunicator was actuated by a signal but no reactor trip occurred, there may be instances that operators need to verify reactor trip annunciators. This need to verify caused the operators not to take immediate action to trip the reactor based on annunicator indication alone on February 25, 1983 as discussed in management issues 1 and 2.
Testing conducted by the licensee in response to NRC questioning confirmed that short-duration signals (less than 10 milliseconds) could produce a reactor trip annunication without tripping the reactor.
Initial followup of review of this testing indicates that the system is functioni"ng as designed, requiring trip signals of more than 10 to 12 milliseconds to actuate the reactor-trip circuit breakers.
In any event, it is apparent that training in the areas of the RPS and associ-ated indications and alarms is warranted.
This issue is unresolved pending further review.
Licensee Action - Short Term The licensee will conduct additional training on the RPS and associated indica-tions and alarms (specifically whether these are demand or confirmatory and the use of this information), and to review the February 22 and 25 events with all operators.
Licensee Action - Long Term The licensee will assure that RPS training and associated subjects in the operator qualification and requalification program address the areas of (1) logical function of the RPS and (2) operation of the RPS and associated indications.
NRC Action - Short Term NRC will evaluate the adequacy and completion of remedial training prior to Unit 1 and Unit 2 restart.
NRC Action-Long Term NRC staff will audit the licensee 1 s requalification program.
Salem Restart Report 11
- 4.
Post-Trip Review*
The licensee did not determine that there had been a failure to trip on February 22 until the computer printout of the sequence of events was reevalu-ated on February 26, as a result of NRC inquiries.
Although the licensee con-ducted a review of each trip, there was no formal procedure for conducting a systemat'ic review.
By letter dated March 1, 1983, the licensee made a commit-ment to develop a post trip and safety injection review procedure.
The proce-dure wi 11 specify the review and documentation necessary to determine the cause of the event and whether equipment functioned as designed.
Other key elements of a post-trip review procedure are 1) necessary management authorization for restart, 2) debriefing of affected operators, 3) verification that reporting requirements were completed, and 4) followup review by safety committees.
Licensee Action - Short Term The licensee will develop and issue post-trip and post-safety-injection review procedures and train all Operations Department personnel on the requirements prior to Unit 1 restart.
Licensee Action - Long Term The licensee will evaluate the effectiveness of the above procedure.
~RC Action - Short Term NRC will review the licensee 1s post-trip and safety injection procedures.
- 5.
Master Equipment List The licensee maintains a Q list that identifies activities, structures, com-ponents, and systems to which the Operational Quality Assurance (QA) Program applies.
A Master Equipment List (MEL) is used by the _licensee as the source document for determining the safety classification of individual equipment:
The MEL is intended to be a comprehensive list of all station equipment and identifies each item as nonsafety related or safety related.
When preparing maintenance work orders, the MEL is consulted to determine if QA coverage of the work is necessary.
Licensee and NRC review identified three problems associated. with the MEL.
These problems are, 1) the accuracy and completeness of the document, 2) issuance as a noncontrolled document, and 3) lack of understanding of its proper use.
The MEL was deri~ed from a construction program document called Project Directive 7 (PD-7) and was provided to station personnel by the Engineering Department as a reference document in July 1981.
Prior to issuance of the MEL, the PD-7 was used as the reference document.
The MEL, however, was not issued as a controlled document, therefore verification of its accuracy and complete-ness on issuance was not assured, and it was not updated in the plant as neces-sary.
The reactor-trip circuit breakers and the RPS were not included in the MEL.
In addition, some personnel were not familiar with how to use the MEL for determining the classification of a particular piece of equipment.
Maintenance personnel acknowledged that r~ference was made to PD-7 on occasion during the January - February 1983 period.
Salem Restart Report 12
Licensee Action - Short Term
- 1.
Verify the MEL is complete and accurate with respect to emergency core cooling (ECCS), including actuation systems, RPS, auxiliary feedwater, and containment isolation systems.
- 2.
Indoctrinate appropriate personnel in the purpose and use of the MEL.
Licensee Action - Long Term The licensee will verify the completeness and accuracy of the MEL and reissue it as a controlled document.
NRC Action - Short Term NRC will perform sampling review of the MEL on the above systems.
NRC Action - Long Term NRC will confirm completion of the licensee 1s long~term action.
- 6.
Work Order Procedures The review identified that the personnel preparing maintenance work orders were not complying with instructions contained in the station administrative procedures.
Specifically, for the work performed on the reactor-trip circuit breaker in January 1983, the engineering department was not consulted to verify safety classification, and an erroneous nonsafety determination was made.
Such consultation is required if equipment is not listed in the MEL.
There was, therefore, no independent review within the maintenance organization, and the Quality Assurance Department was not involved in the work.
Historically, there was no requirement for QA personnel to be involved in the review of work orders as they were processed to assure that appropriate steps were taken to assign classification.
Licensee Action - Short Term The licensee has made a commitment to have the QA Department review all non-
_ safety related work requirements prior to starting work, and to implement a program and training to ensure that work orders are properly classified.
Licensee Action - Long Term The licensee will review work orders written since issuance of the MEL for proper classification and will evaluate safety consequences of those found improperly classified.
NRC Action - Short Term NRC will review licensee 1s work order classification program.
Salem Restart Report 13
DR An
- 7.
Timeliness of Event Notification On three occasions between January 30 and February 25, 1983, the licensee notified NRC of significant events belatedly.
In each case, the notification was approximately 30 minutes late.
Two of these reports were for the February 22 and 25 events.
Furthermore, in the February 22 event, the first.notifi-cation did not contain known significant information regarding actuation of engineered safety features and opening of the power operated relief valves.
This additional information was provided approximately 40 minutes later.
The notification procedure used by the licensee warrants further evaluation as to the priority assigned for NRC notification.
Licensee Action - Short Term The licensee will reemphasize reporting requirements with all shift and on-call manqgement personnel and will reevaluate notification priorities.
NRC Action NRC will confirm that licensee 1s short-term action is completed.
- 8.
Updating Vendor Supplied Information As a result of the February 25, 1983 event and NRC IE.Bulletin 83-01, the licensee indicated not being aware of the existence of two Westinghouse technical service bulletins that provided preventive maintenance recommenda-tions for the reactor-trip circuit breakers.
The two documents in question were published by Westinghouse in 1974.
The licensee has requested documenta-tion for all Westinghouse equipment and will incorporate this information into station documents.
An NRC staff concern is whether a similar situation exists with respect to documentation for other vendor-supplied safety-related equip-ment and how the licensee will maintain vendor-supplied information current in the future.
Licensee Action - Short Term The licensee has made a commitment to a program to update existing documentation on safety equipment and to ensure that vendor documentation is under a con-trolled system.
Licensee Action - Long Term The licensee will complete the above program in a timely manner.
NRC Action - Long Term NRC will perform inspections to verify the implementation of licensee 1 s program.
- 9.
Involvement of QA Personnel With Other Station Departments The Quality Assurance Department did not review maintenance work orders asso-ciated with repair of the reactor-trip circuit breakers in January 1983 because Salem Restart Report 14
DRAfl the work was not designated safety related.
Further examination determined
.that the QA Department does not review for proper determination of classifica-tion the work orders designated nonsafety related by other departments.
Dis-cussions with the licensee indicate that the QA Department has been somewhat isolated from the activities of other departments.
As a result of prior decisions, the licensee had initiated steps in January 1983 to relocate the QA Department from the corporate offices in Newark, N.J.
to the site and is, taking steps to increase QA Department involvement in other station activiti~s.
Licensee Action - Short Term The licensee has made a commitment to institute a program to more fully integrate QA activities into the overall activities.
Licensee Action - Long Term I'),
The licensee will complete the above QA integration program.
NRC Action - Long Term Monitor licensee 1 s implementation of the above QA integration program.
- 10.
Post-Maintenance Operability Testing Past practice at Salem for post maintenance operability testing has varied.
Such testing may be *Specified by the preparer of the maintenance work order or left to the discretion of maintenance personnel.
For safety-related equipment, generally, post-maintenance surveillance testing is done before returning the equipment to service.
Additional functional post-maintenance and repair testing of equipment, such as surveillance testing; may need to be performed to demon-strate operability as an integral part of the larger component or system in which it must function.
Licensee Action - Long Term The licensee will review and revise procedures and practices as necessary to ensure that functional testing of the overall components or system is performed to demonstrate operability prior to returning the equipment to service follow-ing maintenance and repair.
Measures will be revised, as necessary, to assure that operations department personnel review the testing prior to returning such equipment to service.
NRC Action - Long Term NRC will review licensee 1s revised procedures and their implementation to assure that appropriate postmaintenance operability testing is being accomplished before equipment is returned to service.
IV.
Canel us ions The issues discussed in this report were developed from examination of the information revealed during numerous interviews, document reviews, and meetings Salem Restart Report 15
conducted by NRC staff and contractor personnel with licensee representatives.
Based upon the staff 1 s conclusion that the cause of the failures is attributed to the failure of the UV trip attachment to automatically trip the reactor and deficiencies in its maintenance and care, restart of Unit 1 should be permitted subject to the following:
- 1.
Replacement and operational verification of the UV trip attachments;
- 2.
Modification and implementation of procedures associated with operator response to RPS trip signals; and,
- 3.
Satisfactory resolution of those other issues identified as 11 short-term 11 in this report.
11 Long-term 11 issues involve more comp re hens i ve action that do not* have an immediate safety implication; furthermore, these long-term issues will be reconsidered in light of the results of the generic evaluations being conducted by an NRC Task Force.
In the interim the staff intends to establish commitments for corrective actions and imple-mentation schedule, and intends to assume timely implementation of these at the Salem facility.
The staff also believes that the long-term corrective actions related to the management issues at the Salem facility can be further evaluated as to com-pleteness a.nd applicability by an independent and more in-depth assessment.
Accordingly, the Office of Inspection and Enforcement is planning to perform a Performance Appraisal Team (PAT) inspection of the licensee within the next few months.
Salem Restart Report 16
Appendix A RESULTS OF NRC STAFF INVESTIGATION OF EVENTS AT SALEM NUCLEAR GENERATING STATION I.
IDENTIFICATION OF CAUSE OF FAILURE Summary and Initial Findings Initial inspection of the UV trip attachment indicates a possiblity of multiple contributing causes of failure.
Possible contributors are (1) dust and dirt; (2) lack of lubrication; (3) wear; (4) more frequent operation than intended by design; (5) corrosion from improper lubrication in January 1983; and (6) nick-ing of latch surfaces caused by vibration from repeated operation of the breaker.
The contributors appear to be cumulative, with no one main cause.
The initial investigation indicates that the failure was age related and that a new device would perform properly.
Many surfaces of the latch mechanism are worn and the additional friction tended to prevent proper operation.
Proper 1 ubri cation througho.ut the 1 ife of the device might have prevented the wear that can be seen on the sample.
The tests and examinations proposed by the staff and its contractor will
- attempt to determine the cause of failure and if possible reproduce it.
The following summarizes the initial findings and lists the proposed tests.
Discussion and Circumstances A site visit was conducted on March 3, 1983 by NRC and Franklin Research Center personnel to inspect the type DB-50 circuit breaker undervoltage trip attache-ment in an effort to determine the most probable cause of failure.
The reactor trip circuit breaker rooms for Units 1 and 2, each of which contain four DB-50 circuit breakers, were visually inspected and the following observations were made:
- 1.
All four DB-50 Unit 1 circuit breakers and UV trip attachments were removed from the circuit breaker cabinets.
The enclosures were generally clean and free of dust.
The ambient temperature was between 85 and 95°F, with warm exhaust air from inverter cabinets being directed at the DB-50 circuit breaker cabinets.
The spacing between cabinets is approximately 3 feet.
- 2.
All four DB-50 Unit 2 circuit breakers were also inspected.
The UV trip attachments were removed, however.
The circuit breaker cabinets contained a layer of loose dust approximately 1/16 inch thick.
The ambient temper-ature was in the 70°F range.
UV trip attachments are mounted on the top of the circuit-breaker platform, to the right of the shunt trip attachment, which is several inches from the bottom of the circuit breaker cabinet.
Interviews were conducted with an electrical maintenance supervisor who discussed the circumstances of the removal of the circuit breakers that were involved with the incident on Unit 1, and an electrical supervisor who had also worked on the circuit breakers in question in August 1982.
The information received Salem Restart Report 17
was that the circuit breakers and their UV trip attachments had been operated frequently and had operated during surveillance testing within a few days prior to the incident.
A request was made to Salem management to provide one of the UV trip attach-ments and a shunt trip attachment for testing at Franklin Research Center (FRC).
This request was complied with, and an investigation of these devices is now under way at FRC.
Results of Initial Examination Initial investigations indicated roughness in the operation of the trip latch.
There is some dragging of the mechanism, and portions of the latch mechanism have obvious signs of wear.
Possible contributing factors to the failure to operate are a lack of lubrication, wear, jarring of the UV attachment as a result of circuit breaker operation and more frequent operation of the UV trip attachment than was intended during design.
It is postulated that under most industrial applications, the UV attachment would be used very infrequently and probably would be operated only during test sequences at perhaps yearly or longer intervals.
Therefore, in industrial applications, it would operate only a few times, perhaps 20 or 30 cycles during its lifetime, and would not be a normal tripping mechanism for the breaker.
However, in its use at Salem and other nuclear power plants, it is the prime tripping device for the circuit breaker, and is therefore called upon to operate on the order of 50 times per year.
This would mean that at its current age, in 1983, there would have been possibly 400 to 500 trip operations of this device.
During the initial investigation, it was noted that the shunt trip attachment has been operated once every seven days since August 1982, rather than at longer intervals.
This means that the circuit breaker is tripped and closed every seven days.
This causes jarring of the entire mechanism of the circuit breaker and its attached relays and coils due to the normal operation of the breaker.
This may or may not be significant, but it should be noted that the UV attachment stayed energized duri*ng these trips, and its 1 atch mechanism was jarred somewhat by operation of the circuit breaker.
This possibly added to the friction built up in the latch mechanism from normal operation by causing the latch mechanism to just slightly nick the surface that it rides on and thereby tend to prevent operation.
Further investigation will try to determine whether this is \\indeed a problem.
It appears from initial inspection of the device that wear and roughness of mating surfaces in the trip latch are present.
Proper lubrication might have prevented the current situation or could have reduced the roughness to the point where proper operation could occur.
Further investigation will attempt to determine whether the CRC-2-26 lubricat-ing and, cleaning spray added to the operating problem by either causing corrosion or removing all residual lubrication from initial construction*and possible caking of dust and dirt. It appears that from the time of initial construction of the UV trip attachments up until January of 1983, no lubrfca-tion procedures had been performed, and then, in January of 1983, lubrication procedures were undertaken by the maintenance personnel and a Westinghouse technician.
At this time, the CRC-2-26 lubricant cleaner was sprayed on all Salem Restart Report 18
four UV trip attachments associated with the Unit 1 circuit breaker.
lubricant is being procured by FRC for testing purposes.
List of Investigations To.Be Performed by NRC Contractor (FRC)
This
- 1.
The first test will be to perform various deenergizations and energiza-tions of the UV trip attachment and monitor the device under various conditions.
- 2.
The second test will be to disassemble the latch mechanism to observe the surfaces of*the variou~ parts of the latch and to photograph these sur-faces through a microscope to determine the various levels of wear on these surfaces.
- 3.
The third test is to determine the effects of CRC-2-26 spray on the various types of metals used in this devices.
An attempt will be made to use metals other than those in the actual attachment.
If possible, the chemical consistency of this spray will be determined from the manufacturer.
To prove that the sample UV trip attachment is identical to all such Salem devices, a visual inspection of all existing Salem Unit 1 and 2 UV trip attach-ments will be performed.
This can take place at Salem, with no disassembly needed.
The inspection can be made with the devices mounted on the circuit breakers or loose.
These inspections should be done as soon as possible, and Tuesday, March 8, 1983 is recommended.
If further tests are required they will be based on the results of these initial tests.
All tests will be nondestructive such that the device can be used for further testing and returned to the utility.
Additional Test To Be Conducted by the Licensee, as Revised by NRC Staff*
This test will require the use of a spare circuit breaker.
The UV trip and shunt trip attachments will be mounted on the breaker, and the breaker will be operated repeatedly to determine the effect on the shunt and UV trip attach-ments.
It is surmised that while the attachments are energized and the breaker trips and closes a number of times, additional friction of the trip latch may occur from the vibration.
This test is described in detail in the following section.
II.
REVISED SURVEILLANCE OF REACTOR-TRIP CIRCUIT BREAKER OPERATION AND VERIFICATION TESTING The licensee proposed the following increased surveillance of reactor-trip circuit breaker operation:
- 1.
Main and bypass breakers will be shunt-tripped weekly.
- 2.
Main breakers will be UV-tripped monthly.
The acceptability of this revised surveillance of reactor-trip circuit breaker operation has been evaluated by NRC staff.
Based on an analysis conducted by NRC staff, which considered reactor-trip system unavailability, reactor-trip Salem Restart Report 19
circuit breaker failure rates, and test intervals, the following conclusions were drawn.
First, the proposed test of each reactor-trip circuit breaker UV trip attachment once every 30 days is acceptable.
Second, the proposed test of the shunt trip attachment once every seven days is considered to be excessive and may impact on the reliability of the reactor trip system by increasing the potential for a single failure.
During testing, a single failure in the logic portion of the reactor trip system could prevent an automatic SCRAM.
Thus, it is recommended that the shunt trip attachment be tested on the same schedule as the UV trip attachment; that is, once every 30 days.
It is also recommended that the UV trip of the bypass breakers be tested prior to restart and every refueling thereafter.
Discussion The acceptability of the proposed test intervals for the reactor-trip circuit breakers was based on NRC staff review of reactor-trip circuit breaker *failure rate data obtained from Licensee Event Reports (LERs).
The generic RPS unavailability of 3 x 10-5 (used in both NUREG-0460, 11Anticipated Transients Without Scram for Light Water Reactors, 11 and by the ATWS Task Force and Steering Group in the development of the proposed ATWS Rule) was used in evaluating the licensee's proposed test intervals.' In addition~ the following considerations were incorporated into the NRC staff recommendation:
_l.
The shunt tr1p attachment provides a diverse means of tripping the reactor-trip circuit breaker, which is electrically independent of the UV trip attachment.
The UV trip attachment is supplied by a 48-V de source and is deenergized to,trip.
The shunt trip attachment is supplied by a 135-V de source and is energ~zed to trip.
- 2.
The shunt trip attachment is an energize-to-actuate device and is not 11 fail safe 11 in that a loss of power will not cause a trip.
However, the shunt trip is powered from a reliable Class lE battery-backed source.
- 3.
Sihce the shunt trip attachment is an energize-to-actuate device, it is not subject to the constant heating effects that the continuously*
energized UV trip attachment experiences.
The heating effects may contribute to the higher failure rate of the UV trip attachment.
- 4.
The mechanical construction of the shunt trip attachment is less complex than that of the UV trip attachment.
The shunt*trip attachment does not rely on the successful operation of the complex latching mechanism that has been determined to be the source of the majority of the failures of the UV trip attachment.
- 5.
The majority of the electrical circuit breakers used in the high-voltage electrical distribution system have de-powered energize-to-actuate shunt trip attachments.
These circuit breakers are used for manual, as well as automatic, trip functions for l.oad shedding and power switching.
Relia-bility of energize-to-actuate shunt trips in similar applications through-out the nuclear power industry has been shown to be significantly higher than for devices that are constantly energized.
Salem Restart Report 20
- 6.
Over 70% of the known reactor-trip circuit breaker failures were caused by UV trip attachment failures.
- 7.
Most of the concerns relating to the events at Salem on February 22 and 25, 1983 are related to the operation of the UV trip attachment.
During the events at Salem, the shunt trip attachment functioned properly.
- 8.
The bypass breakers are required to trip in response to a UV trip demand signal should this occur when the main breakers are being tested.
Since the test frequency of the main breakers has been increased, the bypass breakers should be tested to verify the capability to perform their backup safety function.
Verification Testing It is recommended that a bench test be performed on one DB-50 reactor-trip circuit breaker.
The purpose of the test will be to cycle the DB-50 with the UV trip and shunt trip attachments in place for a total of 2000 cycles to determine if any adverse effects can be identified and, if there are no adverse effects, show that a properly maintained breaker and its subcomponents can operate for an extended number of cycles.
The breaker will be tripped, with each cycle being alternated with the UV and shunt trips.
The ambient temperature should be 100°F to simulate the expected service environment, and the circuit breaker should be cycled no more often than once every 30 minutes to allow for return to steady-state conditions.
The results of each circuit breaker operation will be documented and a visual check made.
Additional details for this type of test will be provided at a later time.
Salem Restart Report 21
Appendix B INITIAL NRC STAFF REVIEW OF LICENSEE 1S MAINTENANCE PROCEDURE
' AND PREOPERATIONAL VERIFICATION PROGRAM NRC staff reviewed the licensee 1s maintenance procedure, Salem Generating Station Maintenance Department Manual Maintenance Procedure M3Q-2, Revision 1.
This document includes a procedure for verifying proper operation of the UV trip attachment and t~sting of the UV trip attachment coil following replacement.
NRC staff also reviewed the licensee 1s proposed reactor-trip circuit breaker operational verification program, which references Procedure M3Q-2.
The following comments and recommendations were made concerning these documents:
- 1.
- The maintenance procedure does not specify whether the maintenance and testing described are applicable to both the main and bypass breakers.
It _
should specify that it does.
- 2.
The maintenance procedure should specify required actions to be taken in the event any acceptable tolerances, as identified in Enclosure 7 of,
M3Q~2. are not met.
- 3.
The frequency of all maintenance ~nd testing specified in the procedure, with the exception of the verification testing identified following UV trip attachment replacement, should be specified.
- 4.
The procedure should be modified to require cleaning of the entire circuit breaker room, the removal of all four circuit break~rs and cleaning of the cabinets by vacuuming, and cleaning of the breakers during every refueling outage.
- 5.
Section 9.7.2.1 of Procedure M3Q-2 specifies that the UV trip attachment is to be cleaned with a standard solvent.
The procedure should specify the exact solvent to be used.
NRC will request FRC and aNL to determine the adequacy of the proposed solvent and any potential adverse effects from its use.
(This evaluation need not be completed prior to plant startup).
- 6.
Section 9.7.2.2 specifies the composition of the lubricant to be applied to specific points of the UV trip attachment.
This specification should state whether the mechanism is to be lubricated each time maintenance is performed.
NRC will request FRC and BNL to determine the adequacy of the lubricant and the points of application specified, as well as the fretjuency of lubrication.
- 7.
Any UV trip attachment that does not successfully complete the 25 consecutive cycles of testing to be performed by W~stinghouse should not be accepted or installed by the licensee.
- 8.
Section 9.7.4.15 specifies the testing to be performed on the UV trip attachment coil following its replacement.
The maintenance procedure Salem Restart Report 22
should be revised to require that all replacement UV trip attachments successfully complete 25 consecutive cycles of testing prior to installation in the plant and start of the ten test cycle specified in the maintenance procedure.
The time between each of the ten tests should be specified.
NRC recommends 30 minutes for the reasons specified in Appendix A.
NRC staff believe the increase in test cycles, and the acceptance criteria specified if any failures occur during this testing, are reasonable and should be incorporated into maintenance procedure M3Q-2.
- 9.
Technical Department Procedures Nos. IIC-18.1.011 and IIC-18.1.010, referenced by the licensee, should be reviewed and their acceptability determined by NRC staff.
Following revision of the maintenance procedure and the associated proposed reactor-trip circuit breaker operational verification program to incorporate the above comments and recommendations.
NRC staff will reevaluate the documents and provide another report that will include the results of the NRC contractor's evaluations and ~ill document the final NRC evaluation and conclusions concerning the adequacy of the maintenance procedure and preoperational verification program.
Salem Restart Report 23