ML18101B166

Responds to NRC 951016 Ltr Re Violations Noted in Insp Repts 50-272/95-02,50-272/95-07,50-272/95-10 & 50-311/95-02, 50-311/95-07 & 50-311/95-10.Corrective Actions:Cap Revised as Described in Cover Ltr to Attachment

ML18101B166
Person / Time
Site:	Salem
Issue date:	11/15/1995
From:	Eliason L Public Service Enterprise Group
To:	Lieberman J NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I)
Shared Package
ML18101B165	List: ML18101B166
References
NUDOCS 9601160030
Download: ML18101B166 (50)

Text

Public Service Electric and Gas Company Leon R. Eliason Public Service Electric and Gas Company P.O. Box 236, Hancocks Bridge, NJ 08038 609-339-1100 Chief Nuclear Officer & President Nuclear Business Unit NOV 1 5 1995 *

• LR-N95196 United States Nuclear Regulatory Commission Document Control Desk Washington, DC 20555 Attn: Mr. James Lieberman Director -Off ice of Enforcement Gentlemen:

RESPONSE TO NRC NOTICE OF VIOLATION INSPECTION REPORT NOS. 50-272/311/94-32, 50-272/311/95-02, 50-272/311/95-07 AND 50-272/311/95-10 SALEM GENERATING STATION UNIT NOS. 1 AND 2 DOCKET NOS. 50-272 AND 50-311 On October 16, 1995, the Nuclear Regulatory Commission (NRC) issued a Notice of Violation (NOV) and proposed a $600,000civil penalty for violations identified by the NRC during four inspections that occurred between December 5, 1994 and June 23, 1995. The NRC issued to Public Service Electric & Gas (PSE&G) reports for these inspections on March 30, April 7, May 24, and July 14, 1995. A predecisional enforcement conference was held on July 28, 1995. PSE&G does not dispute the violations cited in the October 16, 1995 NOV. Therefore, pursuant to 10CFR2.201, PSE&G submits its reply to the October 16, 1995 NOV. An electronic transfer of funds payable to the Treasurer of the United States in the amount of the proposed civil penalty will be made on November 15, 1995. As the NRC is aware, PSE&G management realized that significant steps were necessary to reverse the P,erformance decline at Salem. Therefore, on June 7, 1995, a decision was made to maintain Salem Unit Nos. 1 and 2 shutdown -until performance improves to acceptable levels. The self-imposed shut down sent a significant message to PSE&G employees.

PSE&G management is 9601160030 951227 PDR ADOCK 05000272 G PDR I . ** *

• Document Control Desk LR-N95196 i\!OV-1 5 1995 serious about the changes necessary for plant safety, personnel performance, and process improvement.

PSE&G evaluated the apparent violations and broader concerns identified in the four inspection reports. Based on this evaluation, our July enforcement conference presentation focused on three critical broad areas that had to be improved before acceptable and long-lasting changes at Salem could occur. These areas are: (1) establishment of a culture that will facilitate improvement, (2) improvement of self-assessment capabilities, and (3) ensuring timely and thorough problem assessment and resolution.

These focus areas and their underlying problems are a subset of concerns being addressed in the Salem Restart Plan. The details of the Restart Plan will be formally submitted on the and discussed with you during the public meeting presently scheduled for December 1995. In addition to our response contained in Attachments 1 through 5, we provide below a discussion of our progress in addressing the three focus areas . Culture Change Improved personnel and organizational performance is currently and *will continue to be a focal point for the new management team and is considered essential in establishing the proper safety culture within the Nuclear Business Unit (NBU) . To aggressively change the culture of the NBU, most of its top management has been replaced.

This change signals the most important factor that distinguishes*present activities from those of the past. One of the key characteristics of the new managers is the ability to lead by example. Personnel selected for this team have demonstrated the necessary leadership capabilities as well as the high standards necessary to develop a quality organization.

Most of the individuals come from nuclear units which have had successful performance turn-arounds and operate at an excellent level. NBU management has placed an emphasis on the development and communication of roles and responsibilities to the organization, as well as establishing expectations for individual performance.

The following are examples of initiatives which have been established to drive the process of change. 95-4933

• Document Control Desk LR-N95196 NOV 15 1995 First, several of the action plans developed to support restart recognize the need for improved definitions of organizational and individual roles and responsibilities.

For example, a Conduct of Operations document is now being finalized which communicates management expectations and, as importantly, establishes the ethic of the Operations organization.

Roles and responsibilities for system engineers have already been defined and communicated to support the System Readiness Review Process and system engineer improvement initiatives.

The goal of these communications is to establish the necessary standards against which personnel and organizational performance can be measured and held accountable.

Secondly, a Performance Ranking process has been instituted to assess individual performance in the following behavioral areas; and Leadership, Initiative and Results Achievement, Job Knowledge, Communication, and Adaptability and Flexibility.

Individuals will develop improvement plans appropriate to their overall standing.

This process is designed to identify and confront substandard performance that has gone undetected or unchallenged to date. In addition, personnel who fail to make prescribed improvements will be held accountable, up to and including discharge.

This ranking process represents the first of four performance review efforts to be conducted within the NBU over the next 18 months. This focus on performance is intended to re-emphasize the responsibility of managers and supervisors to set and enforce proper s.tandards and revise substantially the quality and productivity of the workforce.

Finally, managers and supervisors are being provided training to assist them in identifying, confronting and correcting performance issues. The process being utilized has been implemented successfully at other nuclear plants, as well as non-nuclear companies.

NBU management has established the expectation that line managers and supervisors attend this training and utilize this process. Two protocol groups have been established to ensure the process is being ir.,plemented uniformly and consistently.

The Managers protocol group has recently developed the course content and identified significant issues to be addressed.

The Executive protocol group has evaluated the course content and training to ensure that expectations for this process have been satisfied.

In the longer term, the Managers protocol group will evaluate .

• implementation of the process to promote consistency and make r I . 95.4933

• • *

• Document Control Desk LR-N95196 NOV 1 5 1995 appropriate recommendations on policy issues to the Executive protocol group. These actions, effectively implemented, are expected to improve individual and organizational performance and will provide the infrastructure for the proper safety culture within the NBU. As the impacts of these actions are measured, appropriate changes in approach and method will be made to achieve the lasting and profound changes being targeted.

Self-Assessment Improvement The long-term objective for this focus area is to develop an organization which instinctively takes necessary steps to improve through effective self-assessment and timely corrective action. A program defining expectations for assessment during routine operations has been developed.

Each Salem department has identified specific representatives to support this program. These representatives have been trained on the program and its expectations.

To date, all but one Salem Station department has performed a self-assessment using this program. The remaining departmental self-assessment will be completed in the near future. Issues identified during these assessments will be reviewed and incorporated into the Salem Restart Plan, as appropriate.

A second program, which provides guidance on conducting self-assessments for readiness to return to operation following refueling outages, is being developed.

Salem personnel are demonstrating their willingness to identify deficiencies and to initiate actions necessary for correction.

Indications of this can be seen in the March 24, 1995, "Organizational Effectiveness Assessment Report for Salem Nuclear Generating Station," and our presentation during the July 28, 1995 enforcement conference.

This continues to be shown by system walkdown results, backlog review, and most notably, the number of condition reports being generated on a daily basis. NBU management has and will continue to monitor, and to the extent necessary intervene, when self-assessment related expectations are not met . 95.4933

• • *

• Document Control Desk LR-N95196 Timely/Appropriate Resolution NOV 1 5 1995 A consolidated Corrective Action Program (CAP) has been implemented to communicate NBU management expectations on timely problem identification and resolution and provides clear definition of roles and responsibilities.

The CAP was designed using input from other utilities which have effectively managed program consolidations as measured by improved program and station performance.

The consolidated program includes a low threshold for reporting problems, provides aggressive problem assessment/root cause determination expectations and places management in charge of root cause and corrective action completion times. Results to-date indicate that personnel are not hesitant to raise issues through the process. The Director -Quality Assurance/Nuclear Safety Review has oversight responsibility for the CAP. He has dedicated resources, *_under the Manager -Corrective Action and Quality Services, to fulfill that responsibility.

Measures have been established to monitor the performance of the corrective action process. Recent data indicate overall improvement in evaluation completion times and a reduction in overdue corrective actions. Station management receives daily reports on overdue evaluations

-most of which have resulted from the volume of issues generated by system walkdowns.

Accountability for CAP implementation rests with station line management.

As such, station managers review root cause evaluations for completeness and adequacy.

A Corrective Action Review Board (CARB) has been established at Salem and the General Manager -Salem Operations is its chairman.

Completed root cause assessments for significant issues are presented to the CARB where the adequacy of the cause determination and selected corrective actions are evaluated:

A performance measure has been established which tracks the acceptance/rejection rate for CARB presentations.

This indicator is included in the monthly report to senior management . 95-4933

• • *

• Document Control Desk LR-N95196 NOV 15 1995 A new element, being incorporated under the CAP improvement area, is the Operational Experience Feedback (OEF) Program. This program is under review to identify needed improvements in the processing of internal and external OEF information.

This review includes a validation of actions taken in response to past OEF items. Improvements to the OEF process itself will include the establishment of well defined roles and responsibilities, and standards of performance for implementing organizations.

Performance measures will also be established to allow NBU management to monitor program effectiveness and assign accountability if performance standards are not satisfied.

These changes are being made in order to better integrate the OEF program into the operation of the stations.

NBU management recognizes that, in addition to the changes already described, culture improvements and self-assessment capability improvements are essential to anchoring the CAP as an integral part of sustained performance improvement.

We will establish and achieve appropriate performance standards for the CAP at Salem prior to restart . Summary We agree with the NRC. that performance within the NBU must improve. Our commitment to maintain the Salem Units shutdown until required performance improvements are demonstrated, changes to the NBU management team, and our aggressive actions to strengthen the safety culture within the NBU., illustrate the fundamental differences between our present actions and those of the past. We will not restart the Salem Units until the hardware, important processes and programs, and organizational and individual performance reach acceptable levels. Changes will continue, as needed, to ensure that expectations continue to be met after resumption of power operation . 95-4933 I . ** *

• Document Control Desk LR-N95196 NOV 1 5 1995 If you have any questions regarding this submittal, please do not hesitate to contact me. Sincerely, Attachments 95-4933

• Document Control Desk LR-N95196 C /Mr. T. T. Martin, Administrator

-Region I U. S. Nuclear Regulatory Commission 475 Allendale Road King of Prussia, PA 19406 NOV 15 1995 Mr. L. N. Olshan, Licensing Project Manager -Salem U. S. Nuclear Regulatory Commission One White Flint North 11555 Rockville Pike Mail Stop 14E21 Rockville, MD 20852 Mr. C. Marschall

-Salem (S09) USNRC Senior Resident Inspector Mr. K. Tosch, Manager, IV NJ Department of Environmental Protection Division of Environmental Quality Bureau of Nuclear Engineering CN 415 Trenton, NJ 08625 95-4933

• REF: LR-N95196 STATE OF NEW JERSEY SS. COUNTY OF SALEM L. Eliason, being duly sworn according to law deposes and says: I am Chief Nuclear Officer & President

-Nuclear Business Unit of Public Service Electric and Gas Company, and as such, I find the matters set forth in the above referenced letter, concerning the Salem Generating Station, Unit Nos. 1 and 2, are true to the best of my knowledge, information and belief .

and before me this /5-t.h day of /puvm/JJ.A_ . , 1995

/biGW£. Notary Jersey My Commission expires on KIMBERLY JO BROWN NOTf\RY PUBLIC OF NEW JERSEY My Commission Expires April* 21, 1998 I I " ** *

• Document Control Desk LR-N95196 ATTACHMENT 1 VIOLATION I. 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, requires, in part, that conditions adverse to quality are promptly identified and corrected; .and in the case of significant conditions adverse to quality, the cause of the condition shall be documented, appropriately reported to levels of management, and corrective action taken to preclude repetition.

• A. Contrary to the above, a significant condition adverse to quality existed at the Salem Unit 2 facility from January 26, 1995, until June 7, 1995, in that the Licensee was aware that the No. 22 Residual Heat Removal (RHR) pump minimum recirculation flow valve would not open on low RHR flow as required to prevent pump failure. Similarly, the Licensee was aware that the same significant condition adverse to.quality existed at the facility from February 9, 1995, until June 7, 1995, for the No. 21 RHR pump minimum recirculation flow valve. However, prior to June 7, 1995, the Licensee failed to determine the cause of the valve failures or initiate corrective measures.

(01013) This is a Severity Level III Violation (Supplement

1) Civil

$100,000 RESPONSE -DESCRIPTION OF CIRCUMSTANCES

• psE&G does not dispute the violation.

On January 26, 1995, and 9, 1995, different operating crews identified failure of the automatic open feature for the Residual Heat Removal (RHR) pumps minimum flow recirculation valves 21RH29 and 22RH29. Both failures occurred as Salem Unit 2 was nearing completion of its eighth refueling outage (2R8) . Each failure was observed while the console *operator (a licensed Reactor Operator (RO)) was reducing RHR flow in preparation to align RHR as an Emergency Core Cooling System (ECCS) flowpath.

In both cases, the operating crew initiated an Action Request (AR) . Troubleshooting for these valves was subsequently scheduled for August 2, 1995 and June 27, 1995, respectively.

Although Operations personnel recognized that valve operability was Mode-dependent, they did not establish mode change constraints when the failure of the automatic open feature was recognized .

• • *

• Document Control Desk LR-N95196 Attachment 1 (cont'd) In June, 1995, following Operations department identification of 54 open work orders with potential operability concerns, these valves were targeted for immediate operability assessment.

Once valve operability was questioned, the RHR system was operated to test and evaluate valve response.

Valve 21RH29 failed to operate and was declared inoperable.

When tested, valve 22RH29 opened on pump start. The Engineering Analysis Group (EAG) was tasked with performing a follow-up operability assessment.

The results of follow-up engineering evaluations did not provide sufficient basis to confirm 22 RHR loop operability.

As a result, with both RHR loops inoperable, at 18:27 hours on June 7, 1995, the operating crew entered Technical Specification 3.0.3 and commenced shutdown of Salem Unit 2. At the time of the initial valve misoperation events, an Operations Standing Order and Operability Determination (OD) Flowchart were in place to guide Operations personnel in making Operability Determinations.

Licensed operators had received training on the use of the OD flowchart during the 1994 fall training segment. Although the Standing Order and OD Flowchart were available on January 26, 1995, and February 9, 1995, the operating.crews did not perform an Operability Determination when the operation of the RH29 valves came into question . ROOT CAUSE ASSESSMENT The RH29 valve control relays were tested and the most probable cause for misoperation was attributed to failure of the Struthers-Dunn low flow interlock relay. PSE&G has determined that the root cause of the failure to *identify and correct this condition adverse to quality was inadequate management commitment to the Operability Determination process. This was demonstrated by the following:

1. The implementation of NRC Generic Letter (GL) 91-18 operating philosophy was not timely and effective in improving Operability Determinations.

2. The implementation of Operations Department procedures (Operability Flowchart and Operations Department Directive SC. OP-DD. ZZ-OD02 (Q) (OD-2) , 11 Operability Determinations")

to improve Operability Determinations was ineffective.

3. Less-than-adequate safety culture within the Operations, Technical Engineering, and Station Planning organizations, whi.ch was manifested by a tolerance for equipment problems and insufficient follow-through to correct these problems .

• • *

• Document Control Desk LR-N95196 CORRECTIVE STEPS THAT HAVE BEEN TAKEN Attachment 1 (cont'd) The Struthers-Dunn valve control relays for valve 21RH29 were replaced.

The 22RH29 valve control relays passed in situ functional testing and will be replaced prior to Unit restart. Salem Unit 2 was shutdown to comply with Technical Specification requirements.

To address the less-than-adequate safety culture issues, PSE&G management decided that Salem Units 1 and 2 will remain shutdown until performance improves.

The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

OD-2, "Operability Determinations" has been revised to provide better guidance and expectations for performance of Operability Determinations.

Operator awareness of NRC GL 91-18 is being reinforced during Salem licensed operator training.

These actions assure that management expectations regarding roles and responsibilities in the Operability Determination process are clearly understood and consistently applied. As an interim measure, the Operations department reviews Operability Determinations (OD's) periodically to ensure that actions and contingencies are progressing and/or completed.

The review process is directed by Operations procedure OD40 {Q), "Shift Routines." To assess the effectiveness of the OD process, the Safety Review Group (SRG) is, on an interim basis, independently evaluating the OD's and providing feedback to Operations management.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE The Operability Determination process, including the OD-2 procedure, is being further enhanced to: 1) improve the Engineering and Operations departmental interface;

2) ensure consistency between OD-2 and NC.NA-AP.ZZ-0006(Q) (NAP-6) "Corrective Action Program";

and 3) ensure tracking of Operability Determination status. These improvements to the process will be completed by March 1, 1996 . f

• • *

• Document Control Desk LR-N95196 DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Attachment 1 (cont 1 d) PSE&G has identified and corrected the cause of the valve failures.

PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

• • *

• Document Control Desk LR-N95196 ATTACHMENT 2 VIOLATION B. Contrary to the above, a significant condition adverse* to quality existed at the Salem Unit 1 facility from December 12, 1994, until May 16, 1995, in that the No. 12 safety related switchgear ventilation supply fan failed on December 12, 1994, and the Licensee did not initiate resolution of the condition or effect any corrective measures to resolve the condition promptly. ( 02013) This is a Severity Level III Violation (Supplement

1) . Civil Penalty -$100,000 RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

In Deceinber, 1994, the No. 12 Switchgear Penetration Area Ventilation System (SPAVS) supply fan tripped on overload protection.

Further investigation revealed that the fan motor bearings had failed. Repair of the fan motor bearings necessitated that the fan motor assembly be removed from the system. A Temporary Modification (T-Mod) was required to maintain system/plenum integrity with the fan motor assembly removed. As a result of poor planning and lack of communication, corrective actions had not been taken to repair the No. 12 SPAVS supply fan when the No. 13 SPAVS supply failed on May 12, 1995. At the time of these failures, no spare supply fan motors were available.

Troubleshooting reve.aled that the second fan motor had developed an internal short to ground. In accordance with the Salem Updated Final Safety Analysis Report (UFSAR) , normal system operation requires two of the three 50% capacity SPAVS supply fans to be in service, with the third fan available in a standby mode to accommodate failures.

With the failure of No. 12 SPAVS supply fan motor in December, 1994, .station personnel failed to recognize that SPAVS was operating outside the UFSAR assumptions.

On May 12, 1995, two of the three supply fans became and System Engineering personnel were unable to clearly establish the system's ability to fulfill its intended safety function.

A shutdown of Salem Unit 1 was initiated on May 16, 1995 .

• • *

• Document Control Desk LR-N95196 ROOT CAUSE ASSESSMENT Attachment 2 (cont'd) PSE&G has determined that the root cause of this event was ineffective corrective action. Involved personnel failed to recognize the significance of losing redundant, important to safety components.

Due to a less-than adequate safety culture, prompt corrective actions, consistent with the safety significance of the equipment, were not initiated as evidenced by: 1. Failure to repair the first failed SPAVS supply fan motor in a timely manner. 2. Lack of communication in the System Engineering organization.

3. Failure to complete the work planning for repair by the issuance of a T-Mod which was not accomplished prior to the second SPAVS supply fan motor failure. The Action Program (CAP), in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be identified and resolved in a timely manner. That same program did not provide clear guidance on the need to perform nor the required content of assessments to support continued assurance of equipment operability . The following contributing factors were also identified:

1. Adequate Preventative Maintenance (PM) program tasks were not established for these fan motors. Opportunities to establish appropriate PM's were missed due to lack of follow-through with regard to industry experience notifications and a previous SPAVS fan motor failure. 2. Lack of clear understanding by Operations and Engineering personnel of the SPAVS design basis. 3. Operations did not have a tracking system to assure that inoperable Technical Specification systems or support systems would be corrected in a timely manner. CORRECTIVE STEPS THAT HAVE BEEN TAKEN On May 16, 1995, Salem Unit 1 was shutdown to comply with Technical Specification requirements when reasonable assurance of system operability could not be established .

• • *

• Document Control Desk LR-N95196 Attachment 2 (cont'd) The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

Preventive Maintenance Change Requests (PMCR's) were generated to create new PM Recurring Tasks to replace the SPAVS fan motor bearings on a regular basis. All three Salem Unit 1 SPAVS supply fans were inspected and the fan motors replaced.

• operations Department procedure SC.OP-DD.ZZ-ODlO(Q} "Removal and Return of Nuclear Safety Equipment" has been issued. This procedure provides guidelines for removal and return to service of all Technical Specification related equipment.

OD-2, "Operability Determinations" has been revised to provide better guidance and expectations for performance of Operability Determinations.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE A_ll SPAVS supply fans on Salem Unit 2 will be inspected and the fan motor bearings will be replaced, on an as-needed basis, prior to unit restart . Process improvements for the Operating Experience Feedback Program (OEF) are presently under evaluation.

This activity is being managed under the Corrective Action Program element of the Salem Restart Plan. The Technical Specification Action Tracking procedure has been revised to require the NSS to verify and initial for completed Technical Specification actions and allow for tracking of potential Technical Specification entries. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The No. 12 and 13 SPAVS supply fans were repaired.

PSE&G have achieve.d compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

--

I . ** * ** Document Control Desk LR-N95196 ATTACHMENT 3 VIOLATION C. The Licensee was informed by Westinghouse on March 15, 1993, of a significant condition adverse to quality involving nonconservatisms in the setpoint methodology for the Pressurizer Overpressure Protection System (POPS) for low temperature overpressure*

transient conditions.

1. Contrary to Criterion XVI, the Licensee took nine months of analysis, from March 1993 to December 1993, to conclude that the corrected peak transient pressure would exceed pressure/temperature (P/T) limits as described in each unit's technical specifications limits. After completing the analysis, from December 30, 1993, and continuing for approximately one month, the Licensee dispositioned the matter of the nonconservatism in the setpoint methodology for the POPS by 1) administratively limiting RCS operation to two reactor coolant pumps when the RCS was less than 200° F and 2) increasing each unit's P/T limit by 10%; the latter corrective action was inadequate because it utilized as a basis an unauthorized ASME Code Case (N-514), which the Licensee was aware was not acceptable pursuant to 10 CFR 50. 55 (a) . ( 03 013) This is a Severity Level III Violation (Supplement

1) Civil Penalty -. $100,000 2. Contrary to Criterion XVI, in January 1994, following the Licensee recognizing the unacceptability of using unauthorized Code Case N-514 as a corrective action to disposition the POPS setpoint methodology, the Licensee elected to implement corrective action by taking credit for the relief capacity provided by RHR system suction relief valve RH3 to augment POPS relief capacity .

I . ** *

• Document Control Desk LR-N95196 Attachment 3 (cont'd) However, as the Salem FSAR (Section 7.6.3.2) describes the POPS system to include two Power Operated Relief Valves (PORVs) and does not describe Valve RH3, this corrective action was inadequate because an evaluation was not performed to determine the acceptability of the use of Valve RH3 as part of the POPS system. In addition, the Licensee failed to identify that on the receipt of a safety injection (SI) signal, a previously operating positive displacement charging pump's discharge, combined with the discharge from the high head safety injection pump that starts on receipt of the SI signal, could have injected water mass into the RCS at a rate that could have prevented POPS from performing its function.

(04013) This is a Severity Level III Violation (Supplement I) Civil Penalty -$100,000 RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

On March 15, 1993, Public Service Electric & Gas (PSE&G) was advised by Westinghouse of a generic issue involving a conservative setpoint calculation in the analysis of the Pressurizer Overpressure Protection System (POPS) . The Low Temperature Overpressure Protection System (LTOPS) protects the Reactor Pressure Vessel (RPV) against pressurized thermal shock events as required to comply with 10CFR50 Appendix G criteria ("Fracture Toughness Requirements").

PSE&G requested Westinghouse to perform a Salem plant-specific analysis for the cases of one, two or four reactor coolant pumps running. On September 29, 1993, PSE&G received the plant*-specif ic Westinghouse results and had evidence that a non-conservative setpoint (375 psig) could lead to violating the Technical Specifications and Appendix G pressure/temperature limits. Over a period of three months (September to December, 1993), Nuclear Engineering personnel performed calculations to address this concern .

I . ** Document Control Desk LR-N95196 Attachment 3 (cont'd) On December 30, 1993, the Nuclear Engineering department issued an evaluation (MEC-93-917) which restricted operations in Mode 5 to two reactor coolant pumps. The recommended restrictions were implemented via revisions to the plant's Integrated Operating Procedures (IOP's). Nuclear Engineering personnel improperly took credit for American Society of Mechanical Engineers (ASME) Code Case N-514 (which had not yet received NRC approval) as part of dispositioning this issue. Despite the involvement of multiple departments during this evaluation process, numerous opportunities to recognize the reportability requirements for this issue were missed and, as a consequence, the condition was not reported to the NRC. On May 26, 1994, another evaluation (MEC-94-630) was issued whi.ch further restricted the number of operating Reactor Coolant pumps from two to one pump in Mode S. The new calculated transient values showed that Salem Unit 2 pressure did not exceed specified limits. However, it was recognized that Salem Unit 1 could exceed its pressure limit during a mass addition transient below 2oo*degrees F. Involved personnel failed to evaluate the calculated deviation from the specified limit against reportability requirements.

Likewise, there was a failure to recognize the need to establish justification for continued operation while this condition existed and the need to report

• that justification to the NRC.

• On June 13, 1994, Nuclear Engineering issued calculation S-C-RC-MDC-1358.

This calculation inappropriately took credit for use of a relief valve (RH3) in the Residual ijeat Removal (RHR) system. On November 17, 1994, it was determined that Salem Unit 1 could operate outside of the design/licensing basis for the POPS analysis if the following conditions existed: 1) a Safety Injection (SI) signal was initiated;

2) Reactor Coolant System (RCS) temperature was below 200 degrees F; 3) a Reactor Coolant pump was in service; 4) a Positive Displacement Charging Pump was in service; and 5) power remained available to a maximum of one Centrifugal Charging Pump. This discovery resulted in the issuance of Licensee Event Report (LER) 272/94-017 .

I . ** *

• Document Control Desk LR-N95196 Attachment 3 (cont'd) On February 7, 1995, the NRC approved PSE&G's use of ASME Code Case N-514. At that time, appropriate 10CFR50.59 Safety Evaluations were performed for the resultant changes to the Updated Final Safety Analysis Report (UFSAR) . Implementation of the ASME Code Case provided additional margin (10%) and higher pressure/temperature limits for POPS during the LTOP conditions and re-established plant operation within its design and licensing bases. In April, 1995, PSE&G issued Incident Reports to identify and evaluate the organization's inappropriate actions and their causal factors. ROOT CAUSE ASSESSMENT PSE&G has determined that the root causes of this event were: 1. 2. 3. Lack of understanding of the regulatory significance and reportability implications of the Westinghouse analysis results. Specifically, the organization became too focused on the technical resolution aspects of the issue without adequate consideration of regulatory requirements.

Lack of supervisor/management sensitivity to the need to impJement existing procedures and processes which require timely entry of issues into the Corrective Action Program (CAP) . Monitoring of the Corrective Action process by was insufficient.

Inadequate training of engineering personnel on the use of ASME Code Cases, requirements of lOCFRS0.59 and requirements for regulatory reporting.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment .

• • *

• Document Control Desk LR-N95196 Attachment 3 (cont'd) NBU Management has re-emphasized the expectation that supervisory personnel must assess issues objectively.

Specifically, supervisory personnel must maintain their oversight role. The Manager -Nuclear Engineering Design (NED) has verbally reinforced this expectation to the engineering design organization.

The Nuclear Engineering Design organization was surveyed relative to any past reliance on unapproved ASME Code Cases. Based on this survey, no other instances of unapproved ASME Code. Case use were identified.

Personnel involved in this occurrence have received appropriate reinforcement on procedure compliance, their responsibility for compliance with regulatory requirements, and problem reporting.

Management has re-emphasized by internal memorandum and follow-up review with engineering personnel that the potential impact on the UFSAR must be considered whenever design basis calculations, evaluations or assumptions are revised. Departmental procedures provide clear guidance on these requirements.

The expectation

• for procedural adherence was also reinforced.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Engineering Design and Licensing

&_Regulation management will reinforce expectations for organizational interface to their personnel.

This will be completed by March 15, 1996. Lessons learned from this issue will be disseminated to Engineering Support personnel during 4th quarter Operating Experience Feedback (OEF) training.

This will be completed by January 15, 1996. Process improvements for the Operating Experience Feedback Program (OEF) are presently under evaluation.

• This activity is being managed under the Corrective Action Program element of the Salem Restart Plan. Specific training on the ASME Code and NRC restrictions on its use will be provided to appropriate engineering support personnel.

This will be completed by January 31, 1996 .

• • *

• Document Control Desk LR-N95196 Attachment 3 (cont'd) The Engineering Qualification training program is being revised to assure that job qualifications are consistent with job requirements and that Engineering personnel are trained consistently.

Required personnel training in Code Job Packages will be incorporated into the Engineering Qualification Guide. This* training will include ASME Code Cases, NC.NA-AP.ZZ-002B(Q) "Code Job Packages" procedure requirements and regulatory reporting requirements.

The revised Engineering Qualification Guides will be completed by January 31, 1996. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The request to use ASME Code Case N-514 at Salem station was approved by the NRC. PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

I Document Control Desk .* LR-N95196 *

• ATTACHMENT 4 -lST EXAMPLE VIOLATION D. Contrary to the above, on several occasions, conditions adverse to quality existed, but were not identified and promptly corrected, as evidenced by the following examples:

1. On June 7, 1994, the Licensee identified that material management documentation for limit switches related to the reactor head vent valves, improperly classified the components as non-safety related. A nuclear design discrepancy evaluation form (DEF) identified that a switch short circuit could render two head vent valves inoperable since the components were powered from the same common circuit. Notwithstanding, the DEF did not identify any concern relative to operability or safety. In February 1995, the Licensee determined that non-safety related limit switches were actually installed in reactor head vent valves 1RC41 and 1RC43 at Salem Unit 1. Subsequently, the Licensee failed to* perform and document an engineering evaluation to demonstrate the acceptability of continued Salem Unit 1 operation with non-safety-related parts installed in a safety-related application.

RESPONSE -DESCRIPTION.OF CIRCUMSTANCES PSE&G does not dispute the violation.

On June 7, 1994, a Discrepancy Evaluation (DEF) was written to resolve an apparent conflict in safety classification between the Reactor Head vent valves and their corresponding position indicating limit switches for Salem Units 1 and 2. On March 3, 1995, it was determined that related limit switches were installed in two Salem Unit 1 Reactor Head vent valves. Investigation into this occurrence indicates that, in April, 1992, an opportunity to resolve the noted discrepancy was missed when a different DEF on the same subject was dispositioned.

The identified corrective actions in that DEF were not carried through to completion .

• • *

• Document Control Desk LR-N95196 ROOT CAUSE ASSESSMENT Attachment 4 (cont'd) PSE&G has determined that the root cause of this occurrence was the erroneous classification of the Reactor Head vent valve limit switches as non-safety related. Due to personnel error, these switches were incorrectly assigned a non-safety related purchase class during a spare part Folio Classification initiative in 1986. This error in classification initiated a sequence of events which resulted in the installation of non-safety related limit switches in an application originally designed to use safety related components.

The root cause of the failure to resolve this condition adverse to quality in a timely manner is attributed to an inadequate Corrective Action Program (CAP). The CAP, in effect at that time, lacked sufficiently low thresholds to ensure that conditiqns adverse to quality would be identified and resolved in a timely manner. That same program lacked centraliz'ed oversight of the various mechanisms to identify and resolve discrepancies.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN In March, 1995, Nuclear Engineering Design issued an assessment to resolve the outstanding DEF. This assessment concluded that the non-qualified switches did not affect the operability of the Reactor Head vent valves.

• The following changes were made in the Nuclear Procurement and Material Management (NP&MM) system: The Purchase Class 4 (PC4) Limit Switch Folio parts were put 11 0n Hold" and were re-classified as 11 obsolete.11 A New Purchase Class 1 Limit Switch Folio was created. The Reactor Head vent valve limit switch component ID's were removed from the computerized Managed Maintenance Information System (MMIS). Separate component ID's were determined to be unnecessary as the Bill of Materials (BOM) for the valves contains the Folio information for the limit switches .

• • *

• Document Control Desk LR-N95196 Attachment 4 (cont'd) The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Non-safety related limit switches in the Reactor Head vent valves will be replaced prior to restart of Salem Unit 1. Outstanding DEF's are being reviewed for impact on plant systems, including operability issues. This will be completed prior to restart of Salem Units 1 and 2. PSE&G is currently conducting a review of the MMIS database to determine if there have been other occurrences of safety related components being purchased as non-safety related. The scope of _this review will include components acquired under purchase class "PC4" (non-safety related).

Any additional occurrence(s) of safety related parts in safety related applications, discovered during this review, will be dispositioned under the current CAP which include documentation of Operability Determination and evaluation for reportability, when appropriate . . This review will be completed prior to restart of either Salem Unit 1 or 2. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The Engineering department has dispositioned the outstanding DEF. PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

I . ** *

• Document Control Desk LR-N95196 VIOLATION ATTACHMENT 4 -2ND EXAMPLE 2. On February 24, 1995, Unit No. 1 operators placed control of a PORV in the manual mode, rendering it inoperable, and failed to adhere to the Technical Specification 3.4.3 action statement which required operators to close the block valve within one hour. A shift supervisor discovered that the PORV had been erroneously placed in the manual mode and corrected it on February 25, 1995, about 23 hours2.662037e-4 days <br />0.00639 hours <br />3.80291e-5 weeks <br />8.7515e-6 months <br /> later. RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

On February 24, 1995, Salem Unit 1 was in the process of raising Reactor.Coolant System (RCS) pressure using Integrated Operating Procedure 2 (IOP-2). To support a controller inspection, the Pressurizer pressure master controller was removed and pressure control was placed in manual. This action rendered Operated Relief Valve (PORV) 1PR2 inoperable and required closing of PORV block valve 1PR7. The operator did not close valve 1PR7 and the oversight went unnoticed for approximately 22 hours2.546296e-4 days <br />0.00611 hours <br />3.637566e-5 weeks <br />8.371e-6 months <br />. Although a pre-job brief was performed prior to this evolution, it did not cover all TS required actions. Specifically, the briefing did not discuss closing valve 1PR7. The Nuclear Control Operator (NCO) and the Nuclear Shift Supervisor (NSS) failed to conduct adequate self-checking.

The NSS failed to maintain the proper supervisory overview to insure that the Technical Specification action was completed.

ROOT CAUSE ASSESSMENT The root cause of this event has been attributed to personnel error on the part of the 'supervisor (NSS) and the control operator (NCO) . A contributing cause to this event was inadequate guidance in the Technical Specification Action Tracking Log. This log did not prompt operators to verify that TSAS are completed when the action statement is entered .

• • *

• Document Control Desk LR-N95196 CORRECTIVE STEPS THAT HAVE BEEN TAKEN Attachment 4-2ND example (cont'd) Appropriate disciplinary actions were taken for the individuals involved.

The NSS primary work location has been moved into the respective Control Room area as of March 3, 1995, to improve oversight and management of control room activities.

The Technical Specification Action Tracking procedure has been revised to require the NSS to verify and initial for completed Technical Specification actions and allow for tracking of potential Technical Specification entries. An Information Directive 95-017 and two separate shift briefings were completed for each of the Operations crews. The Operations Department has re-emphasized the use of checking techniques, peer verification and expecta.tions for NSS oversight.

Operations management re-emphasized the conditions under which the PORV's should be declared inoperable during Licensed Operator Requalification (LOR) training in segment 4, 1995. Understanding of Technical Specification actions by Operations personnel were verified through LOR examinations.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE The Operations Department is developing an Operations Standards document which will reference appropriate procedure guidance for conducting pre-job briefings.

The Operations Standards document will be implemented by November 21, 1995. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The PORV block valve was closed to comply with TS requirements.

PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

• • *

• Document Control Desk LR-N95196 Attachment 4-3RD example (cont'd) VIOLATION ATTACHMENT 4 -3RD EXAMPLE 3. On July 6, 1994, safety-related reactor head vent valve 2RC40 failed to operate (stroke open) during testing while Unit No. 2 was in cold shutdown.

Subsequently, the valve was returned to normal service on July 10, 1994, without any review or assessment in accordance with established procedures; that is, the Licensee failed to process this occurrence in accordance with the applicable "Work Control Process" procedure.

Consequently, this failure of a safety related component was never documented and formally assessed relative to preventive maintenance, operability, actions to prevent recurrence, or generic implications.

RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

On July 6, 1994, the 2RC40 valve failed its post-maintenance testing due to indications of reduced flow and dual position indication problems.

Subsequent investigation, including consultation with the vendor, indicated that the most probable cause of the valve failing to stroke open was due to boric acid solidification around the pilot plug. The boron solidification was suspected to be the result of valve seat leakage. The Maintenance Engineer recommended backflushing of the valve with demineralized water and increasing the Reactor Coolant System (RCS) temperature to 180 °F to dissolve the boron. This resulted in proper valve operation and supported the original root cause supposition.

Therefore, it was concluded that a temporary condition could develop at low RCS temperatures and pressures that could result in boric acid binding of the valve. On July 8, 1994, the valve was placed back into service with a recommendation to evaluate the need for additional valve preventive maintenance .

• • *

• Document Control Desk LR-N95196 Attachment 4-3RD example (cont'd) In December, 1994, the Salem Unit 2 head vent valves were replaced as a result of excessive seat leakage. Similar conditions had been previously observed on the Salem Unit 1 valves 1RC40 and 1RC42 and prompted their replacement in May, 1994. In May, 1995, the vendor disassembled and inspected valve 2RC40, which had been removed in December, 1994, to identify any material condition that could have caused the valve's failure to open. The test results on valve binding were inconclusive but indicated that the reported leaking of the valve could be attributed to steam cutting between the valve and the pilot valve disc due to normal wear. On April 5, 1995, an Incident Report was initiated and a root cause analysis undertaken which arrived at much the same conclusions as that of the vendor. The root cause of the failure of valve 2RC40 to stroke was indeterminate.

The type of degradation experienced by the head vent valve would not have alone prevented it from stroking.

Degradation of the valve internals, however, was identified as a causal factor in both the valve leakage and failure to stroke and was attributed to a lack of preventive maintenance.

PSE&G has determined that this failure mode is applicable only to the Reactor Head vent valves. In April, 1995, a re-analysis of the Preventive Maintenance*

requirements for these valve internals was completed and a 54-month inspection was recommended.

ROOT CAUSE ASSESSMENT The final root cause for the failure of valve 2RC40 to open was inconclusive.

Probable causal factors include: 1. Lack of preventive maintenance on valve internal components.

2. Accumulation of boric acid precipitate on valve pilot plug. The root causes for the failure to identify and correct this condition adverse to quality were: 1. An inadequate Corrective Action Program* (CAP). The CAP, in effect at that time, failed to establish sufficiently low reporting thresholds to ensure that conditions adverse to quality would be identified and resolved in a timely manner. 2. Management failure to establish and enforce high expectations for equipment and personnel performance .

• • *

• Document Control Desk LR-N95196 CORRECTIVE STEPS THAT HAVE BEEN TAKEN Attachment 4-3RD example (cont'd) The Reactor Head vent valves 1RC40/42 and 2RC40/41/42/43 have been replaced in May, 1994, and December, 1994, respectively.

Valves 1RC41 and 1RC43 are being replaced during the current outage. The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

Appropriate Operations Department procedures have been revised. These revisions include guidance to preclude boric acid accumulation in the valve body. An Action Request to identify any solenoid operated valves other than the reactor head vent valves that serve as a Reactor Coolant System (RCS) pressure boundary and could potentially be subject to the same or a similar failure mode, such as boric acid binding to*seat leakage, has been completed.

PSE&G has determined that this failure mode is applicable only to the Reactor Head vent valves. NC.NA-BP.ZZ-0002(Z), "Root Cause Analysis Guidelines," has been developed to provide additional information and guidance in the use of various root cause analysis techniques which have been proven effective in resolving both human and equipment performance problems.

Within the Salem Maintenance Department, PSE&G has established dedicated resources to conduct required root cause analyses, develop _and recommend appropriate corrective actions, and assure their proper implementation and overall effectiveness through followup assessments.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE New PM Recurring Tasks (RT's) have been initiated to implement a 54-month PM to open and inspect the Reactor.Head vent valve internals and to repair as needed. A new Maintenance Department procedure has been issued to provide guidance on the disassembly, inspection and refurbishment of the Reactor Head vent valves. These corrective actions will be completed prior to restart of the affected unit .

• • * ** Document Control Desk LR-N95196 Attachment 4-3RD example (cont'd) DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED This condition was documented and a root cause analysis was completed.

PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

• • *

• Document Control Desk LR-N95196 VIOLATION ATTACHMENT 4 -4TH & STH EXAMPLES 4. An oil sample laboratory report, dated August 4, 1994, recommended resampling and changing the oil on the No. 21 high-head safety injection pump based upon a ten-fold increase in wear particle concentration.

An oil analysis, dated November 28, 1994, identified high wear particle concentration in the No. 22 high-head safety injection pump speed increaser oil. In both these cases, the system engineer, though aware of the findings of the lab reports, did not initiate any follow-up evaluation or corrective measure, nor establish a bases for operability or reliability in view of the apparent degraded condition of the equipment.

The degraded nature of the equipment was not entered into the Equipment Malfunction Identification System (EMIS) until March 20, 1995. 5. A lab report, dated October 6, 1994, recommended resampling the No. 23 Auxiliary Feedwater (AFW) turbine lube oil due to a detectable amount of water contamination and an increase in wear particle concentration.

However, the degraded nature of the equipment was not entered into the EMIS until March 27, 1995, and the system engineer did not initiate review, and evaluation, or establish any basis for equipment operability or reliability.

RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

PSE&G acknowledges the issues identified in this violation were not addressed in a timely fashion. Documentation of equipment status was deficient and inadequately maintained .

• • *

• Document Control Desk LR-N95196 Attachment 4-4TH & STH (cont'd) ROOT CAUSE ASSESSMENT PSE&G attributes the root cause of these occurrences to: 1. Management's failure to enforce expectations regarding individual's responsibilities for the Performance Monitoring program. 2. The lengthy turnaround .time for laboratory analyses (including radioactive material handling) challenged the ability of the System Engineer to make timely decisions.

The root cause for the failure to identify and correct these conditions adverse to quality is: 1. An inadequate Corrective Action Program (CAP). The CAP, in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be resolved in a timely manner. That same program did not provide clear guidance on the need to perform nor the required content of assessments to support continued assurance of equipment operability.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN The 23 AFW Pump was declared inoperable.

This action was completed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of when PSE&G received notification from the laboratory-that the follow-up oil sample had been confirmed to be the wrong grade for the component.

The Corrective Program (CAP) has been revised as described in the cover letter to this Attachment.

Roles and responsibilit.ies within System Engineering have been defined and communicated as described in the cover letter to this Attachment.

Within System Engineering, a component reliability group was established to provide improved focus on equipment performance and reliability issues. The Manager -Component Reliability will define and communicate roles and responsibilities for tracking I . ** *

• Document Control Desk LR-N95196 Attachment 4-4TH & STH (cont'd) and trending of performance monitoring data. This will be completed by January 15, 1996. Lube oil abnormalities from this occurrence have been documented by an Abnormal Condition Report to the System Manager from the Lube Oil Analysis Program Manager. This process will remain in place to document future reports of abnormal indications.

PSE&G has contracted with a lube oil analysis laboratory capable of handling radioactively-contaminated lube oil samples. The laboratory's ability to handle contaminated material will reduce the time from sample collection to condition determination by reducing the required count time per sample. CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE The Lubricating Oil Program is being assessed to identify recommendations on a comprehensive lube oil program. The program recommendations are due by the end of the fourth quarter, 1995. These will be evaluated and an Implementation Plan for approved recommendations will be established by the end of first quarter, 1996. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The abnormal Lube oil conditions were documented, reviewed and evaluated for operability impact. PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related -processes have been effective at identifying and conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

• • *

• Document Control Desk LR-N95196 VIOLATION ATTACHMENT 4 -6TH EXAMPLE 6. LER 95-05 identified seven instances, between May 8, 1990 and January 14, 1995, of Pressurizer safety valves (PSVS) being beyond the 1% tolerance required by TS 4.0.5 for Unit 1. Four instances were identified between November 14, 1994, and January 14, 1995, which involved 2 of the 3 installed PSVS. In all instances, the vendor notified the appropriate system engineer by telephone and written follow-up reports. However, the responsible system engineer never initiated an Incident Report. Consequently, root cause, operability, and reportability actions were not accomplished.

-DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

Beginning in May 8, 1990, eight (8) occurrences (total for both Salem Units) of Pressurizer code safety valves (PSV's) exceeding the 2485 psig +/-1% lift set pressure were identified.

Seven of those instances were cited within the Notice of Violation.

An eighth occurrence was self-identified and reported to the NRC via LER Supplement 272/95-05-01, dated October 31, 1995. These occurrences were identified during testing required by Technical Specification (TS) 4.0.5. Failure to report these anomalies resulted from personnel error in that Incident Reports (IR's) were not written in accordance with NC.NA-AP.ZZ-0006(Q) procedure requirements.

ROOT CAUSE ASSESSMENT The causes of the lift setpoint variances are a combination of variability due to individual valve performance characteristics and random test variations which are common for these valves. The specific causes for Salem Station's variation are: 1. Minor test loop instrument error. 2. Valve design limitations.

3. Applied loads from the discharge piping .

' . ** Document Control Desk LR-N95196 Attachment 4 -6th example (cont'd) PSE&G has determined that the programmatic root cause of this violation was management's failure to clearly and adequately communicate expectations regarding when an IR was required.

Specifically, the System Engineers did not recognize the requirement to initiate IR's for these lift setpoint anomalies, in accordance with Nuclear Administrative Procedure NC.NA-AP.ZZ-0006 (Q), "Corrective Action Program" in effect at the time. They also failed to recognize the reportability implications for the out-of-tolerance valve performance data. Consequently, the testing anomalies were not reviewed against the lOCFRS0.73 "Licensee Event Report" (LER) reporting criteria and required LER reporting did not occur. CORRECTIVE STEPS THAT HAVE BEEN TAKEN PSE&G has performed engineering evaluations as part of the Fuel Upgrade Margin Recovery program. The thermal-hydraulic analysis indicates that PSV lift setpoint variances up to 3% are acceptaple.

The structural analysis is more limiting, indicating that variances of +2.2 to -3.0% are acceptable.

Analyses within the Fuel Upgrade Margin Recovery program are continuing.

The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

,_., Lessons learned from this violation were incorporated into the

• third quarter Operating Experience Feedback (OEF) training for Engineering Support personnel.

Appropriate discipline was taken with personnel involved in the failure to initiate IR's. Applied loading on the PSV's from the discharge piping has been* reduced. CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE A single point of contact within the PSE&G organization will be established to ensure coordination of activities associated with PSV testing. This will be completed by March 29, 1996 .

• • *

• Document Control Desk LR-N95196 Attachment 4 -6th example (cont'd) DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The lift setpoint variance conditions were documented, reviewed and assessed to demonstrate acceptability.

PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved . l I I . *

• Document Control Desk LR-N95196 -l -VIOLATION ATTACHMENT 4 -7TH EXAMPLE 7. On March 6, 1995, May 3, 1995, and May 8, 1995, the Salem Unit l staff failed to determine the cause, correct, or prevent recurrence of failure of the Containment 100 foot elevation personnel airlock to pass its local leak rate test. RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

On March 6, 1995, the Salem Unit l Containment personnel airlock on the 100 foot elevation failed its local leak rate test (LLRT) A work request was initiated for the Maintenance Department to investigate and correct the problem. Maintenance technicians inspected the door seals and identified no obvious seal damage but noted that dirt had accumulated on the seal surf ace near the bottom of the door. The door seal was wiped clean with a damp rag and the LLRT was successfully rerun. Following the incident, Maintenance and Operations agreed to have Operations personnel wipe down the door seal and retest the airlock in the event of another airlock failure prior to contacting the Maintenance Department.

This was noted as the corrective action in Incident Report (IR) #95-204. On May 3, 1995, the airlock again failed the local leak rate test. Operations personnel wiped down the door seal and satisfactorily retested the airlock. The Operations Department initiated IR #95-518 and Action Request (AR) #950503088 to evaluate and document the occurrence.

On May 5, 1995, an LLRT was satisfactorily conducted indicated an elevated leakrate.

On May 8, 1995, the airlock failed its LLRT for the third time. The door seal was wiped down and the LLRT was successfully rerun. Operations initiated IR #95-551. and AR #950508110 to troubleshoot and correct the recurring condition.

Subsequent investigation revealed a .significant buildup of dirt and hardened grease in the groove on the seal surface which was caused by the gasket set. Seal surface wipedown would not have been effective in removing

• this buildup .

• • *

• Document Control Desk LR-N95196 ROOT CAUSE ASSESSMENT Attachment 4 -7TH example (cont'd) The root cause of this event has been attributed to adequate management expectations of system performance as demonstrated by: 1. Inexperienced personnel were assigned to perform the initial inspection and corrective actions. 2. Inexperienced personnel were assigned to perform the initial root cause evaluation.

Deficiencies in both the preventive maintenance and surveillance test procedures also contributed to this event. CORRECTIVE STEPS THAT HAVE BEEN TAKEN The gaskets have been replaced on the Salem Unit 1 containment personnel airlock on the 100 foot elevation and the leakage test was performed.

The Salem Unit 1 containment personnel airlock (130 foot

• elevation) and equipment hatch gaskets will be replaced prior to restart of Salem Unit 1 . The Salem Unit 2 gaskets on the containment personnel airlocks and equipment hatch will be replaced prior to restart of Salem Unit 2. The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

Procedure NC.NA-BP.ZZ-0002(Z), "Root Cause Analysis Guidelines" has been developed to provide additional information and guidance in the use of various root cause analysis techniques proven effective in resolving both human and equipment performance problems.

Within the Salem Maintenance Department, PSE&G has dedicated resources to conduct required root cause analyses, develop and recommend appropriate corrective actions, and assure their proper implementation and overall effectiveness through followup assessments.

Appropriate Salem Maintenance procedures have been revised to include specific guidance on seal inspection, cleaning, and maintenance to assist in troubleshooting of leakage problems .

• • *

• Document Control Desk LR-N95196 Attachment 4 -7TH example (cont'd) Appropriate Maintenance procedures will be revised to change the airlock seal lubricant specification from Dow Corning 111 to Dow Corning 3451 in accordance with Nuclear Engineering recommendations.

The above procedure revisions will be completed prior to restart of either Salem Unit 1 or 2. CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE A Preventive Maintenance Change Request (PMCR) has been initiated to evaluate the need for additional Preventive Maintenance (PM) tasks for the containment airlock gaskets. The PMCR recommends PM's following the six-month Structural Integrity Test and gasket replacement at the end of each refueling cycle. Appropriate Operations Department procedures will be revised to provide guidance on maintaining seal surf ace cleanliness and for performing leak rate testing, including discrete leakage criteria for determining when additional corrective action is required.

These corrective actions will be completed prior to restart of the affected unit . DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The cause of the airlock seal f ailu*re occurrences was documented and evaluated, and the condition was corrected.

PSE&G will have achieved compliance with lOCFRSO Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved .

• • *

• Document Control Desk LR-N95196 VIOLATION ATTACHMENT 4 -BTH EXAMPLE 8. From February 29, 1992 until June 7, 1995, Salem Unit 1 staff failed to correctly determine the cause or take action to preclude recurrence of failures of instrument lines connected to the jacket water cooling system for the No. lB and No. lC emergency diesel generators.

RESPONSE -DESCRIPTION OF CIRCUMSTANCES On June 1, 1995, during a lB Emergency Diesel Generator (EDG) Surveillance Test, a jacket water leak was identified at the threaded connection of a 1/4" pipe nipple to an elbow upstream of instrument root valve 1DA46B. The failed component was subsequently replaced in kind. As part of the root cause analysis for the 1/4" nipple failure, natural frequency tests were performed on all EDG's (at both Salem Unit 1 and 2) at locations congruent to-this failure. The'test results showed that piping at specific locations on this and other EDG units could potentially experience damage or fail in response to induced vibrational stresses.

The testing indicated locations with natural vibration resonance frequencies very close to an integer multiple of the frequency which corresponds to the EDG shaft operating speed. The affected EDG's were declared inoperable pending further analysis.

A review of the past failure and maintenance history of the Salem Unit 1 and 2 EDG's was performed to identify occurrences of similar failures.

PSE&G's analysis indicates that there have been repeated failures due to yibration-induced fatigue and the recurrent nature of these failures was not recognized.

Failure to recognize this repetitive problem was due to inadequate root cause analyses and the fact that the failures were attributed to a wide variety of causes. Recommendations stemming from this analysis included design change activities t6 create more vibration-tolerant configurations and maintaining failed components for subsequent laboratory analysis.

Corrective actions taken in the past were ineffective at resolving the vibration-induced component failures, as evidenced by the recurring nature of these problems .

• • *

• Document Control Desk LR-N95196 ROOT CAUSE ASSESSMENT Attachment 4 -BTH example (cont'd) PSE&G attributes the root cause of the piping nipple failure to a design which did not adequately include tolerance for vibrational stresses.

A contributing cause was a lack of specifications for dimensions potentially critical to vibration tolerance in the manufacturer's documentation.

The root cause of the failure to identify and correct these component failures was an inadequate Corrective Action Program (CAP). The CAP, in effect at that time, had numerous program elements which lacked adequate capacity for integration and oversight.

The CAP did not facilitate detection of common failure elements nor did it ensure that conditions adverse to quality were assessed for impact on Operability in a timely manner. CORRECTIVE STEPS THAT HAVE BEEN TAKEN All affected EDG's were declared inoperable but available, pending resolution of the potential for vibration-induced failure. Interim contingency plan guidance was provided to the Operations*Department.

This guidance established requirements to maximize the availability of the demineralized water supply to fill the EDG jacket water system in the event of a postulated failure. A short-term adjustment to the cantilever length of the affected piping was made. This action reduced the potential for resonance between this piping and the engine/header.

A vibration tolerance design review of the EDG's and peripheral equipment was conducted.

This review resulted in recommendations for appropriate enhancement modifications to harden the diesel engines against vibration-related concerns.

The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

A "lessons-learned" memorandum relative to this issue was issued by the Manager -Nuclear Engineering Design (NED) to all appropriate NED personnel.

rolldowns to NED personnel have communicated the "lessons learned."

• • *

• Document Control Desk LR-N95196 Attachment 4 -BTH example (cont'd) CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Jacket Water Pressure transmitting tubing runs will be redesigned to eliminate the piping nipples and associated piping isolation valves. This work has been completed for lB EDG. Modification packages have been prepared for the remaining five EDG's. The Design Change Process (DCP) checklists will be revised to include specialty engineering review for vibration-induced failure issues. These changes will be incorporated at the next revision to the appropriate procedures.

Maintenance and Planning Department training programs will be revised to include specific information regarding the general nature of fatigue failure and system vibratory response.

These corrective actions will be completed prior to restart of either Salem Unit 1 or 2. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The cause of the instrument line failures was identified and actions were taken to reduce their susceptibility to induced failures . PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart Salem Units 1 and 2 until performance in this and other areas has improved.

i ! .

• • *

• Document Control Desk LR-N95196 VIOLATION ATTACHMENT 4 -9TH EXAMPLE 9. From July 11, 1992 until June 10, 1995, Salem staff failed to determine the cause, evaluate the potential safety consequences, and establish corrective action for an abnormal condition affecting the No. 21 Residual Heat Removal discharge manual isolation valve (21RH10) associated with impact noise from the interior of the valve. (05013) RESPONSE -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

On July 11, 1992, during mode 5 operation, unusual noise was identified coming from Residual Heat Removal (RHR) system valve 21RH10. A slightly lesser noise was heard from valve 22RH10 and from the Unit 1 operating RHR loop No. 21. A Salem Technical Department Memo (92-138) was issued to inform the Operations Department that this noise may be caused by flow-induced vibrations from existing play in*the male/female discs and/or disc arm. On April 21, 1993, a maintenance activity to open and inspect the valve was completed and wear marks were found on the downstream seat in two locations.

The cause of the wear marks was attributed to "wedge banging against seat ring." No internal pa.rts were found in need of replacement with the exception of packing and a gasket. On June 10, 1995, valve 21RH10 was again reported making metallic banging noise internally.

A maintenance supervisor did an in-field observation of the valve and concluded that the noise was abnormal.

An Action Request (AR) was written documenting the noise; however, a formal Operability Determination to assess the impact of the noise on system functional capability was not documented prior to June, 1995 .

• • * ** Document Control Desk LR-N95196 ROOT CAUSE ASSESSMENT Attachment 4 -9TH example (cont'd) PSE&G has determined that the root causes of this event were: 1. Inadequate performance by the System Engineer regarding record keeping and tracking/trending of equipment malfunctions. . 2. Inadequate Corrective Action Program (CAP) as indicated by: -Inadequate management and supervisory oversight of equipment failure follow-up.

-Lack of documented engineering analysis of the physical condition of the valve. The Corrective Action Program (CAP), in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be identified and resolved in a timely manner. That same program did not provide clear guidance on the need to perform nor the required content of assessments to support continued assurance of equipment operability.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN On June 15, 1995, Salem System Engineering completed a Follow-up Assessment of Operability and determined that the valve noise did not *adversely affect the functional capability of the RHR system. Work Orders have been issued to open and inspect valve 21RH10 to determine the reasons for the noise currently being experienced.

Valve 21RH10 is scheduled to be opened and inspected after Unit 2 core off-load.

For the purpose of trending, vibration data on valve 21RH10 is being taken periodically and reviewed by the System Manager. This will continue until the loop is taken out of service following core off-load.

The vendor for the valve was contacted to obtain recommendations on actions to be taken. The vendor stated because of the valve design and its location in a turbulent flow area, impact noises can be expected.

The vendor did not recommend any periodic preventive measures.

The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.

L

• • *

• Document Control Desk LR-N95196 Attachment 4 -9TH example (cont'd) System Engineering Department roles and responsibilities have been identified and clearly communicated to all System Engineering personnel.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Additional corrective actions, if any, will be identified after valve disassembly and inspection, as stated above. System readiness reviews are currently underway and include assessment of the readiness of plant systems to support unit restart. Self-assessments of the effectiveness of the system engineering organization to carry out its roles and responsibilities will be conducted.

These corrective steps will be completed prior to restart of either Salem Unit 1 or 2. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Salem System Engineering issued their Followup Assessment of Operability for this valve condition on June 15, 1995 . PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner. PSE&G will not restart either Salem Unit l*or 2 until performance in this and other areas has improved .

• • *

• Document Control Desk LR-N95196 ATTACHMENT 5 VIOLATION II. 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures, and Drawings", requires that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings of a type appropriate to the circumstances, and shall be accomplished in accordance with these instructions, procedures and drawings.

Instructions, procedures, or drawings shall include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily accomplished.

Contrary to the above, following a modification in May 1993, that installed a drain system for the Salem Unit 2 Pressurizer code safety loop seals, the Licensee did not ensure that an activity affecting quality was satisfactorily accomplished in that the procedure that directed the installation of the modification to the Pressurizer code safety loop seals drains did not adequately ensure that the drain valves were properly positioned prior to plant startup after the modification.

Specifically, valve 2PR66, a valve in a common drain line for the 2PR3, 2PR4, and 2PR5, Pressurizer safety valves, was left closed throughout the operating cycle between May 1993 and October 1994. (06013) This is a Severity Level III Violation.

I) Civil Penalty -$100,000 RESPONSE ** -DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.

During the 2R7 outage, a design change package (DCP) was implemented to *add drain lines and drain valve 2PR66 to the Pressurizer Overpressure Protection system. Valve 2PR66 was installed to drain the line downstream of the Pressurizer Safety Valve ioop seals in order to prevent potential water hammer .

I : ** *

• Document Control Desk LR-N95196 Attachment 5 (cont'd) Final testing of the newly installed drain lines was completed on April 27, 1993. On October 19, 1994, in preparation for the Salem Unit 2 eighth refueling outage (2R8), valve 2PR66 was discovered to be closed. Valve 2PR66 being left in the closed position prevented drainage of the Power-Operated Relief Valve (PORV) and Pressurizer Safety Valve loop seal lines and established the loop seals, thus defeating the purpose of the design change. After valve 2PR66 was discovered closed, the computer-based Tagging Request and Inquiry System (TRIS) was checked to confirm the expected valve position.

The normal position for this valve is "open" in accordance with TRIS. The exact time when valve 2PR66 was closed and why this occurred is indeterminate.

The most probable period when valve 2PR66 was manipulated and left closed was determined to be after flushing activities were performed as part of DCP testing. The DCP .included verification of the valve positions during and at the end of the testing portion of the modification.

Valve 2PR66 was documented to be open after the testing. Subsequent to the testing, there was a final acceptance walkdown of the system pr1or to turnover to Operations.

The DCP did not require a written component list which documented valve positions during the walkdown.

As a result, valve 2PR66 was not verified to be open after the DCP, when the system was turned over to Operat.ions.

The Operations DCP Coordinator understood that, in order to approve the design package turnover to Operations for TRIS revision, it was only necessary to verify that the component change had been made in the computer database.

The Operations DCP Coordinator signed off the "Change Package Turnover to Operations" checklist for DCP 2EC-3190, without ensuring that a temporary valve position lineup (referred to as an "auxiliary lineup") had been or* would be performed prior to returning the to power operation.

The Operations DCP Coordinator had not received any training related to expected roles and responsibilities for providing or receiving the final component configurations after modification .

• Document Control Desk LR-N95196 Attachment 5 (cont'd) At the time of this event, there existed an excessive TRIS backlog of 6000 changes waiting to be processed.

The Operations Staff supervision failed to take prompt action when the TRIS backlog became unmanageable.

TRIS database maintenance received an inappropriately low priority.

This was compounded by the fact that the TRIS Coordinator was assigned other collateral duties. The TRIS Coordinator did not create an auxiliary lineup in accordance with SC.OP-DD.ZZ-OD16, "TRIS Operations." Procedure SC.OP-DD.ZZ-OD16 does not specify a time limit for performing an auxiliary lineup. However, an auxiliary lineup was expected to have been performed prior to declaring TRIS database complete. "RC-MECH-001" is a standard valve lineup used to restore affected systems to a ready condition in preparation for plant startup. The revision of SC.OP-DD.ZZ-OD16, in effect* at the time valve 2PR66 was added to the database, specified that the auxiliary lineup pe completed and confirmed in TRIS before the component is added to its applicable standard lineups. The auxiliary lineup for valve 2PR66 was delayed and eventually never performed.

As a result, valve 2PR66 was not added to the RC-MECH-001 lineup in a timely manner . ROOT CAUSE ASSESSMENT PSE&G has determined that the root cause of this event was inadequate commitment to the DCP turnover process and TRIS maintenance program by Operations Management as demonstrated by the fallowing: .

• 1. Operations had less-than-adequate turnover acceptance of DCP's. Roles and responsibilities were not clearly defined. Supervision failed to communicate expectations effectively.

2. The Operations department allowed the TRIS database to become unmanageable.

The backlog was accepted.

The safety significance of the backlog on system design and operability was not adequately evaluated .

• Document Control Desk LR-N95196 CORRECTIVE STEPS THAT HAVE BEEN TAKEN Attachment 5 (cont'd) The Operations Department reviewed TRIS database change requests initiated from DCP 1 s completed during the time period from the beginning of 2R7 to the present. This review encompassed 485 DCP's which have gone to "Part A" closure since the beginning of 2R7. Part A closure signifies that the activity has been installed and the DCP has been turned over to Operations.

PSE&G has evaluated the elements of the DCP process which ensure that Operations procedures and the TRIS database are updated. This evaluation has determined that the process is adequate.

The TRIS backlog was reduced to zero in May of 1995. The backlog is being maintained at zero. Operations has assigned additional personnel as TRIS coordinators.

The Coordinators are responsible for all TRIS interfaces including procedure SC.OP-DD.ZZ-OD16.

Operations Senior Reactor Operators (SRO's) have been assigned ownership of plant systems. The SRO interacts with the project managers and System Managers associated with the DCP from conception.

The SRO accepts responsibility for system turnover *to Operations.

CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Operations procedure SC.OP-DD.ZZ-OD16 is being revised. The revision will emphasize Operations management's expectations and incorporate the auditing process for TRIS revision requests.

This will be completed by December 1, 1995. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Valve 2PR66 was correctly positioned for existing plant conditions.

PSE&G has evaluated the DCP process relative to accomplishing appropriate valve positioning after modification activities are complete.

This evaluation indicates that the process is adequate.

• l , l . [