ML18087A721
| ML18087A721 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 03/14/1983 |
| From: | Public Service Enterprise Group |
| To: | |
| Shared Package | |
| ML18087A720 | List:
|
| References | |
| PROC-830314, NUDOCS 8303160463 | |
| Download: ML18087A721 (62) | |
Text
PUBLIC SERVICE ELECTRIC AND GAS COMPANY CORRECTIVE ACTION PROGRAM RELATED TO
- l 8303160463 830314 PDR ADOCK 05000272 S
PDR REACTOR TRIP BREAKER FAILURES FEBRUARY 22 AND 25, 1983 UNIT NO. 1 SALEM GENERATING STATION DOCKET NO. 50-272 March 14, 1983 i
TABLE OF CONTENTS
1.0 INTRODUCTION
AND CONCLUSION 2.0 EQUIPMENT ISSUES 2.1 Safety Classification of Breakers 2.2 Identification of Cause of Failure 2.3 Verification Testing 2.4 Maintenance and Surveillance Procedures 3.0 MANAGEMENT ISSUES 3.1 Overall Management Performance 3.2 Operating Procedures 3.3 Operator Response 3.4 Operator Training 3.5 Post-Trip Review 3.6 Master Equipment List 3.7 Work Order Procedures 3.8 Timeliness of Event Notification 3.9 Updating Vendor -
Supplied Information 3.10 Involvement of QA Personnel With Other Departments 3.11 Post Maintenance Operability Testing 3.12 Safety Review Group Performance 3.13 Preventive Maintenance 3.14 Procurement ii 1-1 2-1 2-2 2-4 2-5 3-1 3-4 3-7 3-11 3-18 3-22 3-25 3-26 3-28 3-29 3-33 3-35 3-37 3-40
(.
APPENDICES ATWS Training Program Maintenance Department Procedures Technical Department Procedures Emergency Instructions Integrated Operating Procedures Administrative Directives Operations Directives Field Directive Administrative Procedures Quality Assurance Instructions Appendix A
B c
D E
F G
H I
J Quality Assurance Nuclear Operations Policy L
Quality Assurance Training Program M
iii
1.0 INTRODUCTION
AND CONCLUSIONS On February 22 and 25, 1983, the ~eactor trip breakers on Unit 1 of Salem Generating Station failed to open automatically following receipt of signals from the Reactor Protection System.
The manµal trip system was used to shut down the reactor.
It was determined that the failures to automatically trip were caused by malfunctioning of the undervoltage trip attachments in the reactor trip breakers.
Both the Company and the NRC began investigations of the incidents, their cause and ramifications.
PSE&G met with the NRC staff on February 28 and March 5, 1983 to present the results of initial evaluations related to the events.
Each meeting was followed by submittals, on March 1 and March 8, 1983, evaluating the malfunction, proposing corrective actions, discussing future actions and responding to equipment and management issues defined at the meetings.
On March 2, 1983, the-NRC staff briefed the NRC Commissioners on the Salem events and status of their evaluations.
1-1
This report which supplements the March 1 and 8, 1983 letters, addresses the issues related to the incidents and discusses those actions which have been or will be taken as a result of the PSE&G investigation and evaluation.
There has been sufficient investigation and evaluation to determine the cause of the incidents and to ascertain the necessary corrective actions to permit the restart of Salem Unit 1.
In particular, PSE&G has concluded that the cause of the reactor trip breaker failures is attributable to the lack of proper lubrication on latch mechanism of the undervoltage trip attachments.
It is further concluded that the corrective actions described in this report provide assurance that Unrt 1 can, with the maintenance and surveillance programs described, be operated safely.
Moreover, programs are either underway or have been the subject of specific cornrni t~ents *to assure that the follow-up actions are undertaken on a timely schedule
- l with adequate attention to* detail.
1-2 I
2.0 EQUIPMENT ISSUES 2.1 Safety Classification of Breakers The reactor trip breakers are part of the Reactor Trip System which is a safety-related system.
In the design and construction of Salem Generat1ng Station, PSE&G considered as safety-related those structures, systems and components that prevent or mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of the public.
Salem UFSAR, Section 7.1.1.l states that the Reactor Trip System consists of equipment which initiates reactor trip or activates engineered safety features.
Included is equipment ranging from sensors to actuating devices.
The reactor trip breakers and the undervoltage attachment are safety-related.
The shunt-trip attachment is not a functional part of the automatic reactor trip system.
This matter is considered closed.
2-1
2.2 Identification Of Cause Of Failure Immediately following the February 25, 1983 event, tests were conducted which identified that the problem was not in the Reactor Protection System Logic, but isolated it to the undervoltage trip attachment on the breakers.
Based on additional field investigation, combined with the input of Westinghouse, the manufacturer, PSE&G has determined that the failure of the UV trip attachment resulted from lack of proper lubrication on the latch of the undervoltage trip attachment.
PSE&G is utilizing its Research Laboratory to continue the investigation in order to confirm the failure mechanism to assure that there were no other contributing elements.
To date, parts have been examined to determine conditions of wear and the unit was tested for operability over several hundred cycles.
Westinghouse, in its letter of March 1, 1983 to the Director of Nuclear Reactor Regulation, stated that based on its inspectiony "the lack of proper lubrication either caused or greatly contributed* to the failure of the device to function."
It also committed to conduct a thorough evaluation and testing program of the undervoltage coil trip attachment
- 2-2
PSE&G is continuing its program, and will utilize either the manufacturer's resources or an independent testing laboratory, in an effort to confirm the failure mechanism.
A long term operational verification program for the reactor trip breakers will be submitted for NRC review by May 1983.
2-3
2.3 Verification Testing The following preoperational verification program has been satisfactorily completed:
- 1. The manufacturer has satisfactorily electrically tested each undervoltage trip attachment on a test circuit breaker twenty-five consecutive times without experiencing failure.
- 2. After installation of the undervoltage trip attachment, the reactor trip breakers were successfully tested ten times in accordance with the criteria of Maintenance Procedure M3Q-2 (see Appendix B.
A 30-minute time interval was utilized between each test.
- 3. After installation into the appropriate breaker compartment, a response time test of the breaker, actuated through the Solid State Protection System (SSPS) was performed in accordance with the Technical Department test procedure lPD-18.4.002 or lPD-18.4.005 (see Appendix Co 2-4
2.4 Maintenance And Surveillance Procedures As a result of the evaluation of the incidents, maintenance, surveillance and operability testing of the reactor trip and bypass breakers has been augmented and performed as described below.
These actions are in addition to the completed verification testing discussed in Section 2.3.
Semi-annual Inspection and Testing of Reactor Trip and Bypass Breakers The following semi-annual maintenance is to be performed in accordance with Maintenance Procedure M3Q-2, "Reactor Trip and Bypass ACB Semi-Annual Inspection and Testing".
A. Insure mechanical integrity of the breaker, control relays, shunt trip and undervoltage trip attachment by providing for:
- 1. Inspection for cleanliness and specific cleaners to be utilized.
- 2. Lubrication requirements and specific lubricants where needed.
- 3. Mechanical inspection for stress, arcing, loose bolts and nuts, pitting, wear and cracks.
- 4. Adjustments if required, including necessary clearances.
2-5
- 5. Part replacement.
B. Insure electrical integrity of the breaker, control relays, sh~nt trip and undervoltage *trip attachment by providing for:
- 1. Coil voltage tests on undervoltage trip attachment.
- 2. Breaker timing, both closing and opening.
- 3. Megger readings for phase to phase insulation and integrity.
- 4. Coil resistance measurements.
Testing of Breakers Following Completion of Maintenance The following testing is to be performed in accordance with Maintenance Procedure M3Q-2, "Reactor Trip and Bypass ACB Semi-Annual Inspection and Testing," on all Reactor Trip and Bypass Breakers following completion of maintenance.
The following is a general outline of the procedure.
A. The A and B bypass breakers are removed and maintained in accordance with Maintenance Procedure M3Q-2.
B. The A bypass breaker is returned to its cubicle, racked into position and closed.
2-6
C. The A trip breaker is opened, racked out and.
removed in preparation of performance of Maintenance Procedure M3Q-2.
D. The B bypass breaker is placed into the A trip breaker cubicle and UV and shunt trip checked in accordance with Technical Department Procedures 1IC18.l.010 and 1PD18.l.004 respectively, then left closed.
E. The A bypass breaker is opened and racked out.
F. At the completion of maintenance, the A trip breaker is returned to its respective cubicle and UV and shunt trip checked in accordance with Technical Department Procedures 1IC18.1~010, and 1PD18.l.004 respectiveiy.
G. The B bypass breaker is returned to its respective cubicle, racked in and closed.
H. The B trip breaker is opened, racked out and removed in preparation of performance of M~intenance Procedure M3Q-2.
I. The A bypass breaker is placed into the B trip breaker cubicle and UV and.shunt trip checked in accordance with Technical Department procedures 1IC18.l.Oll and 1PD18.l.005 respectively and left closed.
J.
The B bypass breaker is opened and racked out.
2-7
K.
Upon completion of maintenance on the B trip breaker the breaker is returned to its cubicle and tested in the same manner as the A trip breaker as described above.
The A bypass breaker is now returned to its designated cubicle.
Routine Surveillance Testing Of Breakers -
To Be Performed Once Each Month.
A. In accordance with the Technical Specifications, one reactor trip breaker will be UV trip checked by performance of the applicable SSPS train functional procedure, lPD-18.1.008 (lPD-18.1.009), "SSPS Function Test Train A(B)..... The remaining.trip breaker will be UV trip checked by performance of Technical Department Procedure lIC-18.1.010 or lIC-18.1. 011; whichever is applicable.-
B. Each reactor trip breaker will be shunt trip checked by performance of Technical Department Procedure 1PD18.l.004 and 1PD18.l.005, wTrain A(B)
Reactor Trip Breaker and P-4 Test."
The design does not permit in-place UV trip testing of the bypass breakers while.in service.
A-UV trip signal generated while a bypass breaker is in the closed racked 2-8
in position would also trip open the opposite trip breaker.
This would result in a reactor trip.
- However, the bypass breakers do not need to be UV trip check tested monthly due to the following reasons:
Bypass breakers are utilized only to allow testing of the main trip breakers.
Technical Specifications for Salem Unit l allow a time period of one hour to accomplish the testing.
Salem Generating Station -
Updated Final Safety Analysis Report Section 7.2.1.7 allows the following exception to the single failure criterion, "One-out-of-two systems are permitted to violate the single failure criterion during channel bypass provided that acceptable reliability of operation can be otherwise demonstrated and bypass time interval is short.
11 The proposed maintenance schedule and breaker testing scheme should demonstrate acceptable reliability,* and the bypass interval is no longer than allowed by Technical Specifications.
Routine Surveilljnce Testing -
To Be Performed Every 18 Months.
A. The reactor trip and bypass breakers will be time response tested and UV trip checked by performance of Technical Department Procedures 1PD18.4.002, and 1PD18.4.005, "Response Time Test of SSPS Logic -
Reactor Trip Train A(B).
11 2-9
Implementation of Verification Testing Results.
In addition to the changes in testing already implemented, the station will review all recommendations made by the Nuclear Engineering Department at the completion of the long term verification testing of a sample reactor trip breaker.
The accepted recommendations will then be incorporated as changes to either Maintenance Procedure M3Q-2 or the interval of surveillance testing of the breakers, whichever is applicable.
2-10
3.0 MANAGEMENT ISSUES 3.1 Overall Management Performance PSE&G continually reviews its policies, programs and procedures associated with the operation and support of Salem Generating Station in order to improve effectiveness and responsiveness.
A prime example of this effort is the recent reorganization and formation of the Nuclear Department in October 1981.
This represents a major commitment by PSE&G to consolidate its resources in a program to ensure a dedicated and consistent support exists for our nuclear operations.
The establishment of the Nuclear Department at the Salem site substantially improves the Company's ability to respond to the needs of our nuclear operations.
Many of the personnel that staff the Nuclear Department were drawn from the company organizations previoµsly responsible for those functions.
The result has been a dedicated team of management, professional, and technical personnel with considerable experience in plant operations, engineering, nuclear saf~ty, and quality assurance.
The effectiveness of the reorganization has been demonstrated by the enhanced ability of the Company to respond to the present situation.
3-1
Among the inputs utilized by PSE&G in evaluating management effectiven~ss are those prepared by INPO and by the NRC in its SALP program.
Since September 1979, there have been three full Systematic.Assessment of Licensee Performance (SALP)
Reports and one abbreviated SALP Report assdciated with Salem Generating Station.
The periods covered by the full reports are as follows:
September 1979 through August 1980, July 1980 through June 1981, and September 1981 through August 1982.
A mini-SALP Report was performed for the peiiod of September 1980 through February 1981, as a means of monitoring our corrective action efforts for those areas identified as needing improvement in the initial report.
An assessment of the these evaluations ~ith respect to the specific functional areas indicates a continued rate of improvement.
To date the Institute of Nuclear Power Operations (INPO) has conducted two evaluations of Salem Generating Station, one in 1981 and one in 1982.
A review of these evaluations indicates improvement.
Management has.taken action on INPO's recommendations to further improve the, effectiveness of its nuclear operations
- 3-2
As part of the Company's evaluation of the circumstances associated with the recent incidents involving the reactor trip breakers, the Company has examined the potential for broader implications.
We have concluded that the fundamental management programs, and procedures are adequate.
We have identified some areas where personnel required additional training to ensure adherance to procedures and modifications to some procedures have been made to facilitate their implementation.
. t t I 3-3
3.2 Operating Procedures The Emergency Instruction in use at the time of the events of February 22 and 25 was Revision 6 to EI-I-4.3, Reactor Trip.
This Emergency Instruction, approved and issued in September 1980, contained steps to address ATWS in accordance with the Westinghouse Emergency Operating Instruction Guidelines which were in existence at the time.
Subsequent to the two events, EI-I-4.3 was revised to better delineate for the operator the steps required to initiate a reactor trip and achieve control rod insertion.
The means available to the operator in the control room to achieve control rod insertion include: 1) initiating a manual trip with either of the manual trip levers, 2) opening the reactor trip breakers utilizing the "OPEN" pushbuttons on the console, 3) manually initiating a turbine trip to provide an additional input to the SSPS, and 4) opening the 4KV feeder breakers which supply power to the 460V busses that power the Control Rod Drive MG Sets.
If all of these fail to achieve control rod insertion, an operator can be dispatched to the el~ctrical equipment room to manually open the reactor trip breakers and to de-energize the Control Rod Drive MG Sets.
3-4
The "Subsequent Actions" section of EI-I-4.3 was revised to include a requirement to perform a post-trip review prior to startup of the unit.
The requirements for the post-trip review and the authorization for startup are defined in Administrative Directive AD-16, Post Reactor Trip/Safety Injection Review.
The steps specifying emergency notifications have also been revised to better delineate the required notifications.
Other procedures which have been reviewed and revised to reflect changes similar to those described for EI-I-4.3 are EI-I-4.0, Safety Injection Initiation, EI-I-4.2, Recovery From Safety Injection, IOP-2, Cold Shutdown to Hot Standby and IOP-3, Hot Standby to Minimum Load.
The remaining Emergency Instructions, Integrated Operating Procedures, and Alarm Response Procedures have also been reviewed to determine if it was necessary to make any changes to assure consistency with AD-16.
It was determined that additional changes were not required because the procedures either adequately addressed the requirements directly or directed the operator to a procedure where the requirements are addressed.
All procedure reviews and revisions were conducted for both Unit 1 and Unit 2 concurrently to assure consistency_
between the procedures.
In addition, a two-year review 3-5
of all Operating Procedures and Alarm Response Procedures is being conducted at this time.
In support of the procedure revision effort, the control room instrumentation, indications and alarms have been reviewed and evaluated and it has been determined that there is sufficient information available to immediately identify and terminate an ATWS event.
3-6
3.3 Operator Response As stated in Section 3.1, it has been determined that there is sufficient information available to the operators to immediately identify an ATWS event.
Licensed operating personnel have been informed of the details and evaluation of the February 22nd and 25th events and the procedural and policy changes that have been developed as corrective actions via Information Directives.
Procedure Reviews Administrative Directive AD-13, Independent Review of Operations Department Documents, prepared and issued in November 1982, describes the procedures and other documents which are to be classified as safety-related.
This directive lists the reference documents utilized to make the determination as to which procedures and documents were to be classified as safety-related.
A review of System Descriptions was conducted to ensure that all systems have appropriate operating instructions.
Systems which do not have specific operating instructions will be evaluated as to whether a detailed instruction is required.
These systems will be evaluated as part of the ongoing procedure review to determine whether specific procedures should be developed.
3-7
In January 1983, an intensive effort to rewrite the
(
existing Emergency Instructions to incorporate the latest information available from the Westinghouse Owners Group (WOG) was begun.
The WOG Emergency Response Guidelines are being used as the base document.
This effort is scheduled to be completed by June 1984.
This effort includes validation of the procedures on the Salem simulator and training of operators before the procedures are implemented.
The existing Emergency Instructions will be replaced by new procedures.
The new procedures will. include a set of Emergency 6perating Procedures (EOP's), which will deal with the accidents procedures and a set of abnormal Operating Procedures which will address transients.
The Emergency and Abnormal Procedures will be issued at the same time, after operator training is completed.
There is also an ongoing bi-annual review of departmental procedures.
Procedures are being _revised to conform to criteria developed for the EOP effort.
These include use of a standard author's guide. and abbreviations list.
These guides were developed usjng NUREG-0899 and the documents developed by the EOPIA group and INPO
- 3-8
Overhead Annunciator Overhead alarms have been reviewed to confirm which alarms on the First Out Panel are demand indications for a reactor trip and which are confirmatory.
This matter was reviewed with all licensed shift operators.
Various means to distinguish between demand and confirmatory
~ndications will be investigated.
Recommendations will be reviewed as part of the control room design review, currently being performed as directed by NUREG-0700, prior to implementation.
Directives Operations Directive OD-10, Removal and Return of Safety Related Equipment to an Operable Status, establishes formal departmental policy on the requirements for the removal and return of safety related equipment to service.
Additional guidance is also provided on responsibilities, considerations, and documentation of testing.
Operations Directive OD-15, Use of Operations Department Procedures, has been evaluated and found to be satisfactory in providing guidance to the operators in the proper use of procedures.
The content of OD-15 has been reviewed with all licensed operators during the training conducted following the events of February 22nd and 25th.
3-9
During the February 22nd event when the SRO called for a manual trip, the control handle which is removable, by design was inadvertently removed and was reinserted to perform the manual trip.
Operating personnel have since been cautioned on the use of the manual trip "J" handle.
Additional methods of precluding this from recurring are being investigated.
' fr 3-10
3.4 Operator Training While we believe that our operators had an adequate understanding of the Solid State Protection System (SSPS), additional training will be conducted prior to startup to reemphasize and strengthen their understanding of the SSPS and the significance of associated alarms and indicators.
The Nuclear Training Department, in conjunction with Station personnel, developed and conducted a refresher training program for licensed operators covering the following areas:
background information on ATWS, Westinghouse Owner's Group/Emergency Response Group procedure and basis for ATWS, the Salem events of February 22nd and 25th, and the associated sequence of events printout, DB-50 Circuit Breaker and operation of associated-shunt and undervoltage trip devices, revisions to EI-I-4.3, Reactor Trip Emergency Instruction, Post Reactor Trip/Safety Injection and Unit Startup Approval Requirement and a review of the SSPS emphasizing the difference between confirmatory and demand signals.
Licensed operator understanding of the material was verified by written examinations administered by training personnel at the completion of the individual training 3-11
sessions.
Training has been completed for licensed operators except those unavailable.
Makeup sessions will be conducted as necessary.
The retraining program as presented is provided in Appendix A.
Additional information concerning the Solid State Protection System and the differences between demand and confirmatory signals will be incorporated into the licensed operator training programs for initial license candidates and for requalif ication.
Specific objectives will be added that require licensed operators to identify the alarms associated with various systems and to indicate whether a particular alarm represents a* demand for a function, such as reactor trip, or a confirmation that the action has occurred *. A detailed review of signal source, indications associated with the Reactor Protection System and Emergency Safeguards Actuation System will_ be included.
Training materials to supp~rt these programs will be revised beginning with the next requalification class (March 28, 1983) and the next initial licensed operator training class (June 1983).
Additionally, independent of recent events, on February 16, 1983, the Nuclear Training Department initiated a study group. to review the licensed operator 3-12
Requalif ication Program and to recommend any revisions to the program based upon an independent needs analysis and the INPO Job/Task Analysis.
Results.of the study_ are expected by May 27, 1983.
In addition, PSE&G's Nuclear operator training courses have been reviewed by the New York State Board of Regent's external degree program and recommendations have been made by that body for college level credits to be awarded to individuals who successfully complete those programs.
All operator training courses will be reviewed by the In~titute of Nuclear Power Operations for their accreditation during the week March 21, 1983.
Simulator Training Simulator training has been conducted on simulators not identical to the Salem control room but which represented similar power systems (i.e., Westinghouse 4 loop plants).
This has been an accepted procedure in the industry to increase the training effectiveness over "walk and talk through" training in an operating power.
plant.
When training is conducted on non-specific simulators, the principal objective is *not to become familiar with control configuration, but to train the 3-13
student on principals of operation of a power plant and reaction to unusual or emergency situations, training which cannot be effectively done in an operating plant.
Salem specific training is then conducted at Salem Generating Station.
When non-specific simulators are used, however, every effort is made to eliminate or clearly identify dissimilarities by:
- a. Use of Salem procedures,
- b. Furnishing critical reference documents to the simulator staff,
- c. Visit to Salem control room by simulator staff,
- d. Providing difference training to the students in the classroom before and after simulator practice, and
- e. Clearly identifying significant differences at the control board during training.
The new Salem specific simulator, which duplicates the Salem controls, will be in use at the PSE&G Nuclear Training Center to further enhance operator training by September 1983.
This will eliminate any concerns regarding training on a simulator that is different from the actual unit
- 3-14
Training Effectiveness Effectiveness of training is evaluated by review of instructor capabilities and course content and effectiveness.
l*. Instructor Certification Each instructor must qualify in accordance with th~
PSE&G Nuclear Training Center Instructor Development Plan which requires a review of both instructional ability and technical knowledge.
Once an instructor has documented and demonstrated his or her abilities and technical cbmpetence, an instructor certification is issued.
Areas of weakness identified by this procedure are documented and self improvement goals are set.
Review of each instructor's progress and development is accomplished annually in conjunction with the Corporate performance appraisal system.
Contract instructor qualifications are reviewed in accordance with the criteria in the Instructor Development Plan before instructional support contracts are let
- 3-15
- 2. Course Evaluation Upon completion, all training programs are evaluated by the students.
The evaluation forms are submitted to the instructor's supervisor for review and analysis with the instructor.
Any improvements that are needed are documented and the recommendations forwarded to the department head for necessary action.
A follow-~p evaluation is made approximately six months after course completion.
The trainee and his or her supervisor are asked for their responses.
The completed forms are used by the Nuclear Training Department to determine if any revisions or modifications to the course are required to meet trainee and station needs.
Periodic training meetings are conductedibetween the Nuclear Traininq Department and Station per:sonnel to develop and evaluate all training programs.
- f The licensee requalification program, in/addition to the normal evaluation process, is evaluat~d on a continuing basis to determine the effectiveness of the program in meeting designated requirernen\\ts and to 3-16
)
1
identify the job performance results attributable to training.
Changes to the requalification program as a result of these evaluations are formalized and implemented.
Evaluations/Mechanisms are:
- a. Inspection, audit and evaluation reports of requalif ication training by outside organizations and by facility personnel,
- b. Review of oral and written examination results,
- c. Review of regulations and standards affecting licensed operator retraining, and
- d. Solicitation of input from plant management.
Licensee requalification program curriculum deficiencies and licensed operator retraining needs determined by the review are identified and recommended corrective actions are forwarded to the department head for Operations Training.
An annual written evaluation is forwarded to the Manager -
Nuclear Training on all present license requalif ication programs.
3-17
3.5 Post-Trip Review At the time of the February 22 and 25, 1983 incidents, informal post-trips were conducted.
Since that time,
- Administrative Directive AD-16, Post Reactor Trip/Safety Injection Review and Startup Approval Requirements, was issued.
The directive establishes a formal post trip review for all Reactor Trip and Safety Injection events.
The directive also establishes requirements and criteria that must be met prior to startup of either unit following a trip event or after scheduled outages.
The directive provides for the following:
- 1. Personnel and management responsibilities with regard to. the conduct, evaluation, and review of the sequence of events report prior to startup, along with document retention requirements.
- 2. Specific items to be reviewed during the investigation:
. -~*
- a. Condition of the unit prior to the event;
- b. Personnel assignments at the time of event;
- c. Evolutions in progress prior to the event that could have contributed to itr 3-18
- d. All equipment, control and protection systems that were out of service or inoperable at the onset of the event;
- e. Mode of event initiation;
- f. Computer sequence of events;
- g. Control room recorder charts;
- h. Alarms received which were out of the 0rdinary for the event and alarms which should have been received and we~e not;
- i. Specific check off sheets for reviewing the computer sequence of events; and
- j. Required corrective actions to be completed prior to startup.
3 ** Approval requirements f.or reactor startup:
- a. Startup after controlled shutdowns; the Operations Manager may permit startup; b~ Startup after trip events where the cause of ~he event is. clearly identified and it is known that all equipment and systems functioned as designed.
The Post Trip Review Report is to be reviewed with the Operations Manager*.
When he is satisfied that all equipment and systems functioned as designed, and the cause of the event is correctly identified, he may grant permission for the unit to be started up; and 3-19
- c. Startup after trip events where the cause of the event is unclear or it is not known why all equipment and systems did not function as designed.
In this event, the Station Operations Review Committee (SORC) will review the event and recommend further followup actions.
When SORC is satisfied with the additional followup actions, a recommendation is made to the General Manager -
Salem Operations concerning restart of the unit.
- 4. Verification that all reporting requirements were met.
- 5. Subsequent reviews by the SORC and the Nuclear Engineering Department.
- 6. Specific involvement in the review process by the individual on shift who is Shift Technical Advisor qualified.
The following operating procedures for both Units 1 and 2 were revised to incorporate the r~quirements of AD-16 prior to reactor startup:
- 1. EI-I-4.3, Reactor Trip, Revision 9.
- 2. EI-I-4.2, Recovery from Safety Injection, Revision 11.
- 3. IOP-2, Cold Shutdown to Hot Standby, Revision 1.
- 4. IOP-3, Hot Standby to Minimum Load, Revision 1.
3-20
Licensed shift personnel have been trained in the requirements and content of AD-16 and the procedures to assure proper post-trip review prior to any unit startup.
This information was disseminated through the issuance of Information Directive ID83-17.
Additionally, the content and requirements of AD-16 were addressed during the training sessions conducted to brief personnel on the events of February 22 and 25 and corrective actions.
Senior Reactor Operator licensed personnel will be given specific in-depth training for interpretation of the sequence of events printout.
This training will be completed by June 1983.
The adequacy of the directive will be evaluated periodically to determine if improvements are required to increase its effectiveness.
If additional modifications are found necessary, the directive will be revised
- 3-21
3.6 Master Equipment List Those activities, services, structures, components and systems to which the Operational Quality Assurance Program applies are identified in the Q-List.
Part of the Q-List is a document called the Master Equipment List (MEL).
The MEL lists specific plant components which are part of the Q-Listed systems.
The MEL was prepared as a reference document to be utilized in determining safety classification for station work orders and on station procurement documents.
The MEL was developed as a successor to a construction program document called Project Directive No. 7 (PD-7).
The Salem Unit 1 MEL was issued on 'July 15, 1981 and the Salem Unit 2 MEL was first issued on November 3, 1980 with a revision issued on January 27, 1981.
The orginal purpose of the MEL was to provide a convenient listing of safety related components in one document.
The MEL data was taken from various source documents such as the FSAR, system descriptions, equipment specifications and engineering drawings.
3-22
Prior to restarting Salem Unit 1 the following actions will be completed:
- a.
To facilitate utilization of the MEL for work order classification, a list of systems abstracted from the Q-List will be incorporated into the MEL.
- b.
The completeness of the sets of the MEL currently issued to the Salem Station will be confirmed *
- c.
A Field Directive providing both background and instructions on the use of the MEL will be issued and incorporated into the copies of the MEL in use by the Salem Station and confirmed in b. above.
(See Appendix H)
- d.
Training has been provided to appropriate personnel in Nuclear Engineering, Quality Assurance, Nuclear Construction Support, and Station who utilize the MEL.
Further detail of this training c6nducted in conjunction with the Quality Assurance Department is discussed in Section 3.9.
3-23
In addition, a program has been undertaken to update, review and reissue the MEL as a controlled document.
This progra~
will include a detailed review of the MEL data to determine completeness and to validate the classifications of data contained therein.
A procedure will be developed to provide instructions on updating the MEL and the frequency of the updating.
The completeness and accuracy of the following systems will first be determined:
ECCS (including actuation systems),
Reactor Protection System, Auxiliary Feedwater System and Containment Isolation System.
Upon completion of these systems, a partial reissue of the MEL will be made on a controlled distribution basis.
This will be completed by March 17, 1983.
The completeness and accuracy of the remainder of the Q-List systems will then be determined.
Following completion of that work, a complete reissue of the MEL will be made on a controlled distribution basis.
This will be completed by May 1983.
3-24
3.7 Work Order Procedures Administrative Procedure AP-9, "Control of Station Maintenance" has been revised to ensure that work orders are properly classified.
Personnel designated to review work orders prior to issue are now required to contact the sponsor engineer when equipment or systems do not appear on the Master Equipment List (MEL), or clarification as to the classification is required.
An independent review of all non-safety related work orders will be performed by Quality Assurance prior to implementation to ensure proper classification.
A Quality Control Inspection stamp will be affixed to the non-safety related work orders prior to issue to ensure QA concurrence with classification.
Indoctrination of appropriate personnel on the above changes has been completed.
Administrative Procedure AP-9 and Quality Assurance Instruction QAI 10-6 which i
provides guidance for the QA review of Station work I
orders are provid~d in Appendix J.
, I Detailed training in the initiation, processing and
\\
closeout of the work orders will be conducted to reemphasize Quality Assurance requirements, test/retest requirements and interdepartmental coordination.
This training will be completed by September 1, 1983.
3-25
3.8 Timeliness of Event Notification The importance of adhering to the reporting requirements of 10CFR 50.72 has been emphasized to all operating personnel.
Emergency Plan Procedures have been reviewed and revised to rearrange the priority of notifications to assure that all initial contact calls are made within the required time periods.
The Emergency Plan Procedures require the designated personnel to immediately start making the required notifications and reading the initial contact messages upon the classification of the event.
Training on the modifications will be conducted for personnel* involved in the implementation of the Emergency Plan prior to startup.
In addition to the above, the emergency communication systems are being evaluated and recommendations have been made to consolidate communication devices in the control room areas, Technical Support Center and Emergency Operation Facility.
The upgrade of the communication systems will be completed by December 1983.
3-26
Upon completion of the installation of the upgraded communication system, the Emergency Plan will be reviewed and revised to incorporate the modifications.
Training will be conducted at that time for personnel involved in the Emergency Plan relative to system modifications and procedure changes.
3-27
3.9 Updating Vendor-Supplied Information Copies of all previously issued Westinghouse NSID (NSD)
Technical Bulletins and NSID (NSD) Data Letters have been obtained on a controlled copy basis from Westinghouse.
A review will be made to ensure that applicable documents are incorporated into Station procedures, where appropriate, by July 1, 1983.
Future issues of these documents will be reviewed by Engineering and formally issued to the Station.
A review is in progress to ensure that all vendor manuals in use are incorporated under a controlled system.
Completion of the review is scheduled for April 1, 1983.
Incorporation of any uncontrolled manual under a control system is scheduled for completion by December 1983.
A review will be made to determine that PSE&G has controlled vendor manuals for all major safety equipment.
Vendor manuals will be requested from vendors where necessary.
This review is scheduled for completion by December 1983.
3-28
3.10 Involvement of Quality Assurance Personnel With Other Departments Through a recent reorganization, all personnel in the Operational Quality Assurance Organization are in the process of being relocated to the site.
The purpose of this change is to promote increased involvement by Quality Assurance personnel in the functions of the Nuclear Department as evidenced by the QA interfaces shown in the individual responses to the other corrective action items in this report.
In addition, the newly issued QA organization policy (see Appendix L) places greater emphasis on verification of Quality Assurance Program implementation through increased observation and monitoring.
The current comprehensive review of QA Program implementing procedures and any necessary changes identified is expected to be completed by August, 1983.
Additional improvements identified through subsequent evaluations will be taken into consideration and implemented in a timely manner.
In order to emphasize existing QA Program requirements as well as newly instituted procedural changes, an indoctrination/training program was conducted for all appropriate personnel regarding Classification, Work 3-29
Orders, and Procurement.
The following specific items were identified in the training program (see Appendix M):
Use of the MEL List of Systems for classification of all station work orders.
Requirement for all non-safety related work orders to be reviewed by QA prior to implementation.
Use of the MEL List of Systems and Component List for classification of items for procurement.
Review of the requirements and philosophy of Commercial Catalog_ Items (CCI), stressing the fact that CCI classification pertains only to procurement.
Re-emphasis of the basic QA Program objectives and reponsibilities.
Short Term Training To ensure proper classification of work orders and procurement documents, indoctrination was provided to appropriate personnel with emphasis on the specific criteria to be used in the determination of safeiy classifications.
Guidelines were provided and discussed during the various indoctrination sessions to ensure that appropriate personnel understand and effectively apply 3~30
the djfferent guidelines presented for classifying work orders and procurements.
Work Orders For the determination of classification on work orders appropriate personnel were directed to consult the recently established List of Systems section of the MEL, which identifies those specific systems which are governed by the PSE&G QA Program.
Any ~ork activity performed on these systems or identified portions thereof is to be classified safety-related and conducted in accordance with the provisions of the QA Program.
Procurement For the determination of procurement document classification, appropriate personnel were instructed to consult both the MEL List of Systems and component listing sections and to follow the prescribed criteria on the use of the listings.
Any item classified as safety-related or CCI from the MEL listings will be processed and controlled in accordance with the provisions of the QA Program.
Appropriate personnel were further instructed to consult Nuclear Engineering in the event of any question in the application of the MEL
- With respect to CCI's, personnel were indoctrinated as to the criteria identifed by QAI 4-3, which must be 3-31
satisfied to support the classification of CCI.
Specific examples of CCI's and non-CCI's were provided and discussed.
Personnel were advised that the QA Program controls are applied to these items throughout the life cycle including procurement, receipt inspections, storage, installation, maintenance and test.
It was emphasized that the CCI classification is utilized only as a method of procurement and has no bearing on work order classification.
Further, it was emphasized that the QA Program controls applied to CCI are an acceptable method of procurement since they represent a replacement "in kind" and provide the necessary assurance that the item (CCI) may be installed in a safety-related application.
Following completion of the training program, selected attendees were interviewed to ascertain the degree of comprehension and understanding of certain key elements of the presented material.
The interviews demonstrated that the key program objectives were achieved.
Long Term Training Proficiency in determination of classification of work orders and procurement documents will be maintained by on-going periodic training and indoctrination sessions
- 3-32
3.11 Post Maintenance Operability Testing The following three procedures have been revised to ensure adequate emphasis on quality assurance, test/retest and interdepartmental communications requirements:
Operations Directive OD-10,.Removal and Return of Safety Related Equipment to an Operable Status, has been revised to ensure that testing in accordance with the Technical Specifications is completed prior to declaring equipment operable.
As a minimum, an equipment functional test will be performed and appropriate parameters verified.
Technical Department -
Instrumentation and Controls Procedure 14.1.001, General Procedure for Troubleshooting, Rework and Repair of Installed Equipment, has been revised to ensure completion and documentation of all test and retest requirements and to ensure proper interdepartmental coordination on all retests.
Maintenance Department Procedure M3Z, Electrical Equipment Troubleshooting and Repair, has been revised to include Quality Assurance/Quality Control inspection hold points, test/retest requirements, and interdepartmental notification of retest requirements.
3-33
In addition, Maintenance Department Test Manual A-21 has been developed to ensure that prior to issuance of a work order, retest requirements are determined.
In addition, retest requirements will be identified on the work order.
Appropriate maintenance personnel will be trained on the revised procedure prior to startup.
A review of vendor and engineering recommendations and current testing procedures will be made.
Based on this review, changes will be incorporated into departmental documents by January 1, 1984.
Test/retest requirements will include adequate Quality Assurance/Quality Control involvement.
3-34
3.12 Safety Review Group Performance The Safety Review Group (SRG) is composed of five dedicated full-time engineers and functions to examine plant operating characteristics, NRC issuances, industry advisories, LER's and other sources of information which may indicate areas for improvement of plant safety.
The SRG is responsible for maintaining surveillance of selected plant activities to provide independent verification that these activities are performed correctly.
Recommendations for improving plant safety which result from such reviews, are reported to management.
These formal SRG recommendations are tracked as open items until they are resolved to the satisfaction of the SRG.
Mechanisms exist for escalating SRG open items through higher management if necessary.
In addition, the SRG has free access to all levels of management within the Nuclear Department to discuss issues requiring immediate attention and to discuss any areas having a potential impact on safety.
These activities provide independent feedback on the overall safety of plant operations
- 3-35
The SRG has actively followed resolution of the issues associated with the reactor trip breaker events through participation in technical and management meetings, discussions with key individuals and review of documentation.
The Safety Review Group has been effective at Salem.
SRG recommendations have resulted in action being taken to improve plant safety and management has been kept informed with regard to safety of plant operations.
A formal report evaluating the effectiveness of the SRG, with suggestions for any needed improvements, will be submitted by July 1, 1983
- 3-36
3.13 Preventive Maintenance The Station preventive maintenance program is described in Administrative Procedure AP-10 "Inspection Order System."
The Inspection Order System is a computerized system designed to provide timely notification of required tasks of a recurring nature.
These tasks include items such as Surveillance Activities, Inspection Activities, and Lubrication Activities.
The inspection order issued to the responsible department consists of the following:
Unique Identification Number Responsible Department Type of Order (Surveillance, Inspection, Lubrication)
Issue Date 3-37
Due Date Applicable System Code (System of which component is a part of)
Applicable Procedure required to complete task Description of required task Signoff s for both performer and reviewer of task In addition to scheduling tasks, the Inspection Order System produces documentation which allows management to control the timely completion of each task and schedule the task for future maintenance.
The inspection order request form is the means by which a department can specify a specific task to be included in the system.
. t!
The inspection order syste'm has proven to be a valuable
- l tool in the scheduling and control of both required (as specified by the Technical Specifications) and preventive
.i maintenance tasks.
The e!fectiveness of the system has
/
been evaluated by the Ins~itute of Nuclear Power Operations and classified as a Good Practice.
3-38
In addition to the above, a managed maintenance program is being developed with the support of Westinghouse.
The program includes inspections, maintenance and spare parts.
All systems will be reviewed and priorities rearranged based on recent events.
Specifically the program includes the reactor trip system, emergency core cooling systems, auxiliary feedwater system and containment isolation systems.
Target date for completion of this program is January 1, 1984.
3-39
3.14 Procurement Program The procurement system utilized at the Salem Generating Station has been reviewed to assess the effectiveness and adequacy of procedural implementation of the controlling system instructions.
This review has identified areas requiring additional attention to verification of classification and for improved adherence to existing procedures.
The current system requires the initiation of a Material Order Item Classification form (MOIC) to identify the required item, quantity and description.
In addition, the MOIC contains provisions for the identi~ication of detailed item classification parameters i.e., seismic class, Nuclear Class, lE, 10CFR Part 21 applicability to procurement and QA provisions.
In addition to providing ordering information, the MOIC was developed to provide a convenient tool for MEL update.
It is important to note that the MOIC form is an internal document and not the contract document utilized for procurement.
Pertinent information is extracted from the MOIC (safety related, non-safety related, CCI) and transcribed onto a purchase requisition and finally the purchase order which is the procurement contract document.
As emphasized in Section 3.9, the classification guidelines for procurement are for procurement item classification only and are not 3-40
- I I
utilized during the work order classification process.
For example, an item procured as CCI would be installed in a safety related application utilizing a safety related work order and respective QA Program controls.
In emergency or confirming order situations, the current procurement system allows for items which meet the CCI requirements to be procured in parallel with engineering review of the MOIC.
Short Term PSE&G's review of procurements for all Westinghouse DB-50 circuit breakers and parts purchased since 1980.
revealed inconsistencies in processing material order forms (MOIC's) and inconsistent application of the CCI classification requirements.
The inconsistent application of the CCI classification criteria involved the fact that in some cases replacement parts of the subject breakers were not listed in the supplier's catalog (Westinghouse Spare Parts Report).
- However, the top assembly, DB-50 Circuit Breaker, was listed and identified as a commercial item, and accordingly, it was inferred that all components thereof could be procured to the same classification level.
Classification and procurement of the UV Coil (P0885297 dated 3-1-83) as a CCI was not consistent with the prescribed criteria for CCI classifications due to the fact that the item was not listed in the supplier's 3-41
catalog; however, it has been determined that the item is classified by Westinghouse as a commercial item and should have been so listed.
In the future, Westinghouse will be notified of items of omission of this nature for correction.
With respect to CCI classification, personnel were indoctrinated as described in Section 3.9, Involvement of QA Personnel With Other Departments.
To provide additional assurances in the short term that procurement classifications were correct and consistently applied, procurement records associated with replacement of non-safety related and CCI NSSS equipment supplied by Westinghouse, were examined for compliance with the prescribed criteria for item classification.
This effort involved the examination of those procurement documents processed since issuance of the MEL associated with items purchased as either non-safety related or CCI.
The results of this examination substantiates our confidence that procurement classifications were appropriate and consistent with the prescribed criteria
- 3-42
.....,J
"'..-.;. *"' ~-
In addition, the procurement classification guidelines presented in Section 3.9 for utilization of the MEL will be implemented as stated, and monitored by QA Nuclear Operations.
Long Term PSE&G is in the process of evaluating the existing procurement system in an effort to establish flow sequence changes and controls that will provide additional assurances that procurement classifications are appropriate for the identified application, retrievable and consistently applied for subsequent procurements.
This will be accomplished by the establishment of new or revised procedures identifying the methods for specifying and controlling all procurement.
This system will provide for Nuclear Engineering establishment of item classification through a controlled and verified MEL or determination of those item classifications not on the MEL.
In addition, proficiency in the application of item classification techniques will be maintained through continual periodic training and indoctrination sessions.
The above actions will be accomplished by July 1983.
3-43
~UMMARY OF CORRECTIVE ACTION COMMITMENTS 2.0 EQUIPMENT ISSUES 2.1 Determine Safety Classification of Breakers (complete) 2.2 Identify Cause of Failure (complete)
- a.
Short Term Testing (complete)
- b.
Long Term Testing (ongoing) May 83 -
Long Term Operational Verification Program Report -
Submit by May 83 2.3 Preoperational Verification Program (complete)
- 1.
Manufacturer will electrically test U/V trip attachment on Test CB 25 times
- 2.
After installation, U/V trip attachment tested 10 times
- 3.
After installation in appropriate breaker compartment, Response Time Test 2.4 Maintenance/Surveillance Procedure
- 1.
Semi Annual Inspection and Test Reactor Trip/Bypass Breakers (M3Q-2)
(complete)
- 2.
Post Maintenance Testing (M3Q-2)
(complete)
- 3.
Monthly Surveillance Test on Breakers -
U/V Trip Check -
a Technical Specification Requirement (complete)
- 4.
18-month Surveillance Test -
Time Response and U/V Trip Check (complete)
S.
Implement Verification Testing results -
any changes identified will be incorporated in M3Q-2
SUMMARY
OF CORRECTIVE ACTION COMMITMENTS - Cont'd 3.0 MANAGEMENT ISSUES 3.1 N/A 3.2 Operating Procedures
- a.
Modify procedure EI-I-4.3 to clarify the steps required to initiate a reactor trip and achieve control rod insertion (complete)
- b.
Add requirement to "Subsequent Actions" section of EI-I-4.3 to perform post-trip review prior to each restart of the unit (complete) 3.3 Operator Response
- a.
Review and evaluate control room indication, instrumentation and alarms (complete)
- b.
Incorporate latest.WOG information in EI 1 s, validate procedures on Salem simulator, train operators in new/revised procedures (June 1984)
- c.
- 1. Overhead First Out Alarms -
review for determination of which are demand indications and which *are confirmatory of a reactor trip (complete)
- 2. Investigate means of positive/permanent identification of which alarms are demand and which are confirmatory of reactor trip -
review with c_ontrol room design review
- d.
- 1. Instruct operating personnel in use of Westinghouse "J" handle mechanism on manual reactor trip (complete)
- 2. Investigate methods for preventing recurrence of removing handle from reactor trip switch 3.4 Operator Training
- a.
Additional training on SSPS and associated alarms, ATWS, DB-50 circuit breaker design, and procedural changes since February 22 and 25 incidents.
(complete, except for those on vacation/sickness
- b.
Modify licen~ed operator requalification andinitial training to include SSPS, alarm significances.
(Requalification March 28, 1983) (Initial licensed operator June 1983)
SUMMARY
OF CORRECTIVE ACTION COMMITMENTS -
Cont'd 3.5 Issue formal Post-Trip Review Procedures (complete) 3.6
- a.
Incorporate in Master Equipment List (MEL) systems extracted from Q-List for Work Order classification (complete) 3.7
- b.
- c.
- d.
- e.
- f.
- a.
Confirm that sets of MEL at Salem Station are complete.
(complete)
Issue Field Directive on use of MEL (complete.)
Provide training on purpose and uses of MEL (complete)
Verify MEL data for certain safety systems by 3/17/83.
Update and reissue MEL as controlled document (May 1983)
Revise Administrative Procedure AP-9 and QA instruction QA! 10-6 to ensure proper classification of non-safety related work orders (complete)
- b.
Conduct training in the initiation, processing and closeout of work orders emphasizing QA requirements, test/retest requirements, and :
interdepartmental coordination (September 1~83) 3.8
- a.
Review Emergency Plan Procedures and rearrange priority of notifications to ensure required calls are made within required time periods (complete)
- b.
Investigate communications system (December 1983) i
- c.
Review/revise Emergency Plan to incorporate*f upgraded communication system, train personn..e*l in system/procedure changes following completiqn of upgrading the system.
t l 3.9
- a.
Review Station Procedures for incorporation of applicable vendor-supplied information (July 1983)
I
- b.
Review all vendor manuals that are under the controlled document system (April 1983)
- c.
Incorporate any uncontrolled vendor manuals into controlled system (December 1983)
- d.
Review to determine that all major safety equipment have controlled vendor manuals and obtain copies from vendors where necessary (December 1983)
SUMMARY
OF CORRECTIVE ACTION COMMITMENTS - Cont'd 3.10 N/A 3.11 Review vendor and engineering recommendations on Post-Maintenance Testing and current testing procedure and incorporate change in departmental documents -
include QA/QC involvement in test/retest requirements (January 1984) 3.12 Submit formal report on Safety Review Group effectiveness (July 1983) 3.13 Complete development of managed maintenance program (January 1984) 3.14 Establish new/revised procedure identifying methods for specifying and controlling procurement. (July 1983)