IR 05000272/2003008
| ML040330007 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 01/30/2004 |
| From: | Lanning W Division of Reactor Safety I |
| To: | Richard Anderson Public Service Enterprise Group |
| Scholl L | |
| References | |
| IR-03-008 | |
| Download: ML040330007 (36) | |
Text
ary 30, 2004
SUBJECT:
SALEM GENERATING STATION - NRC SPECIAL INSPECTION REPORT 05000272/2003008, 05000311/2003008
Dear Mr. Anderson:
On November 7, 2003, the U.S. Nuclear Regulatory Commission (NRC) completed a special inspection at your Salem Generating Station. The enclosed report documents the inspection findings which were discussed on December 16, 2003, with you and other members of your staff.
This special team inspection was sent to review your actions in response to the July 29, 2003, event involving a Unit 1 reactor trip and partial loss of offsite power to both units. The event involved significant unexpected electrical system interactions.
This report documents three NRC identified findings of very low safety significance (Green).
These issues, each of which reveal weaknesses in design control activities constitute violations of NRC requirements. However, because of their very low safety significance and because they have been entered into your corrective action program, the NRC is treating these findings as non-cited violations (NCVs) in accordance with Section VI.A of the NRC Enforcement Policy.
If you deny any of these non-cited violations, you should provide a response with basis for your denial, within 30 days of the date of this inspection report, to the Nuclear Regulatory Commission, ATTN: Document Control Deck, Washington, DC 20555-0001; with copies to the Regional Administrator, Region I; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at the Salem Generating Station.
In addition, the team found examples of inconsistent application of your corrective action program that are similar to those identified in previous NRC inspections and discussed in both the annual and mid-cycle performance review letters dated March 3 and August 27, 2003, respectively. We expect to closely monitor your effort to address weaknesses in this area.
Mr. Roy In accordance with 10 CFR 2.790 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC website at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
If you have any questions, please contact John F. Rogge of my staff at 610-337-5146.
Sincerely,
/RA/
Wayne D. Lanning, Director Division of Reactor Safety Docket Nos.: 50-272; 50-311 License Nos.: DPR-70; DPR-75 Enclosure: Inspection Report 05000272/2003008 and 05000311/2003008 cc w/encl:
C. Bakken, Senior Vice President Site Operations J. T. Carlin, Vice President Nuclear Assurance D. F. Garchow, Vice President, Engineering and Technical Support W. F. Sperry, Director Business Support S. Mannon, Manager - Licensing C. J. Fricker, Salem Plant Manager R. Kankus, Joint Owner Affairs J. J. Keenan, Esquire Consumer Advocate, Office of Consumer Advocate F. Pompper, Chief of Police and Emergency Management Coordinator M. Wetterhahn, Esquire State of New Jersey State of Delaware N. Cohen, Coordinator - Unplug Salem Campaign W. Costanzo, Technical Advisor - Jersey Shore Nuclear Watch E. Zobian, Coordinator - Jersey Shore Anti Nuclear Alliance
Mr. Roy
SUMMARY OF FINDINGS
IR 05000272/2003-008, 05000311/2003-008; 08/18/2003 - 11/07/2003; Salem Generating
Station, Units 1 and 2; Special Inspection; Problem Identification and Resolution.
The NRC special inspection was conducted by a five person team comprised of regional specialist inspectors, a senior resident inspector and a senior reactor analyst. The team was accompanied by a representative of the State of New Jersey, Department of Environmental Protection. The inspection identified three Green issues which were determined to be non-cited violations (NCVs). The significance of most findings is indicated by the color (Green, White,
Yellow, Red) using Inspection Manual Chapter 0609, "Significance Determination Process" (SDP). Findings for which the SDP does not apply may be "Green" or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 3, dated July 2000.
The team found examples of inconsistent application of your corrective action program that are similar to those identified in previous NRC inspections as discussed in both the last annual and mid-cycle performance review letters dated March 3 and August 27, 2003, respectively.
NRC-Identified and Self-Revealing Findings
Cornerstone: Initiating Events
- Green.
The team identified a violation of 10 CFR 50, Appendix B, Criterion III,
Design Control, for design control inadequacies during plant modifications, setpoint changes and revisions of calculations associated with the 4160 volt electrical power system. These electrical system design deficiencies caused the two offsite power sources not to be independent of each other as required by 10 CFR 50, Appendix A, Criterion 17, Electric Power Systems.
The finding was more than minor because it affected the design control attribute of the Initiating Events Cornerstone objective and resulted in an increased likelihood of a loss of offsite power (LOOP) event. The finding was determined to be of very low safety significance (Green) based on a the results of a phase 3 SDP analysis which evaluated the increase in core damage frequency (CDF) due to the increased likelihood of a LOOP caused by the design deficiencies.
(Section B.1.b.1)
- Green.
The team identified a violation of 10 CFR 50, Appendix B, Criterion XVI,
Corrective Action, for the failure of the licensee to implement adequate corrective actions to address design issues identified following the July 29, 2003, loss of offsite power event. When performing an operability evaluation to support plant restart, the licensee failed to identify that the lower operating voltage limit for the 4.16 kV buses needed to be increased to prevent recurrence of a similar event.
The plant was restarted and operated from August 4 to August 22, 2003, until the issue was identified by the NRC and corrected by the licensee.
ii
The finding was more than minor because it affected the design control attribute of the Initiating Events Cornerstone objective and resulted in an increased likelihood of a loss of offsite power event (LOOP). The finding was determined to be of very low safety significance (Green) based on a the results of a phase 3 SDP analysis which evaluated the increase in core damage frequency (CDF) due to the increased likelihood of a LOOP caused by the failure to take appropriate corrective actions prior to plant restart. (Section 4OA2.b.1)
Cornerstone: Mitigating Systems
- Green.
The team identified a violation of 10 CFR 50, Appendix B, Criterion III,
Design Control, for failure of the licensee to translate design change information into plant procedures. Following the installation of a plant modification to provide a cross connect between the Unit 1 and 2 chemical and volume control systems (CVCS), instructions for utilizing the cross connect feature were not included at the appropriate steps in the associated procedures.
The finding was more than minor because it affected the design control attribute of the Mitigating Systems Cornerstone objective. The issue was not a design or qualification deficiency that the licensee had evaluated in accordance with GL 91-18, and was determined to be of very low safety significance (Green)because it did not result in an actual loss of safety function of a single train for internal or external event initiated core damage sequences. (Section C.b.2)
Licensee Identified Violations
None.
iii
SUMMARY OF UNIT 1
PLANT STATUS
On July 29, 2003, with the Salem Unit 1 reactor at 100% power, the reactor automatically tripped as a result of a turbine trip. The turbine trip was caused by the actuation of a circuit breaker ground overcurrent fault detection relay that isolated bus section 1 in the 500 kV switchyard. This also caused a loss of power to several station power transformers (SPTs),and, due to design deficiencies, resulted in the loss of all offsite power to the Unit 1 vital buses and resulted in the buses being supplied power from the emergency diesel generators (EDGs).
Following an event investigation and system repairs PSEG restored Unit 1 to 100% power on August 4, 2003.
A. CHRONOLOGY OF EVENT In 1992, PSEG initiated a major redesign of the Salem switchyard configuration and completed associated electrical system modifications in 1995. The intent of the redesign was to increase the reliability of the offsite power supply to the vital buses by providing separate SPTs for the vital, non-vital and circulating water (CW) pump buses. Prior to completion of the project, PSEG made a decision to place the CW and the vital buses on the same SPTs. Engineering revised the supporting design calculations and switchyard model to account for this change, but did not properly consider the impact of this change on 10 CFR 50, Appendix A, General Design Criteria (GDC) 17, Electric Power Systems, requirements.
In July 1993, during an electrical distribution system functional inspection (EDSFI), PSEG determined that the Unit 1 and 2, 4160 volt vital bus second level undervoltage protection system (SLUPS) relay setpoints were not conservative to ensure an adequate voltage to all components under degraded grid conditions. In December 1994, the NRC approved a Technical Specification (TS) amendment to raise the SLUPS setpoints for Units 1 and 2 to 94.6%.
In August 1993, PSEG completed installation of penetration seals in the conduits in the walls of the turbine building to eliminate ground water leakage into the building. Minimal guidance was provided to maintenance personnel regarding installation of these seals, particularly with respect to the removal of old damming material (silicone type caulk). It is likely that damage to cable insulation occurred during this process. The cables that pass through these penetrations are submerged because of a high water table and damage to the cable insulation eventually resulted in electrical short circuits and grounds.
On November 16, 2001, Unit 2 operators identified that they were unable to adjust 22 SPT voltage using the load tap changer (LTC) in manual. Troubleshooting determined that a shorted wire in the control power cable for the LTC had affected manual control. The repair for this cable required a splice and the use of a spare penetration because the cable could not be removed. Although the exact location of the short circuit could not be determined, electrical testing indicated that the failure occurred within 3 to 4 ft of the turbine wall penetration seal installation.
In the first quarter of 2003, a control power problem was discovered with the 23 SPT LTC. The licensees investigation determined that a cable to the LTC was damaged where the cable passed through the penetration seal in the turbine building wall. The licensee concluded that the damage most likely occurred during the installation of the penetration seal. The cable was repaired and returned to service.
On July 29, 2003, at 1329 hours0.0154 days <br />0.369 hours <br />0.0022 weeks <br />5.056845e-4 months <br /> the Salem Unit 1 reactor tripped as a result of a turbine trip.
The turbine trip initiated after the 500 kV 1-5 circuit breaker ground overcurrent fault detection relay actuated. This actuation denergized 500 kV bus section five by opening the generator output breakers 1-5 and 5-6, and 500 kV bus section one by opening 500 kV breakers 1-8, 1-9, and 13 kV breakers 3-4 and 4-5.
The loss of 500 kV bus section one deenergized the 500 kV to 13 kV station power transformers (SPTs) 2 and 4 that deenergized 13 kV north ring bus sections 3, 4, and 5 and 13 kV south ring bus sections C, D, and E.
The loss of the 13 kV north ring bus sections deenergized 12 SPT and 22 SPT that supply offsite power to the non-vital group buses, 1F, 2F, 1G and 2G. The group buses on both units are normally powered from the output of the main generators. Since Unit 2 generator did not trip the Unit 2 group buses remained energized. However, following the Unit 1 turbine trip and the loss of the 12 SPT, the 1F and 1G group buses were denergized.
The loss of the 13 kV south ring bus section C and E deenergized the 14 and 23 SPTs. In a normal lineup each of these transformers powers one of two circulating water (CW) pump buses and either one or two of three vital buses. On July 29 the 14 and 23 SPT each were suppling power to two vital buses (1B, 1C and 2B, 2C respectively), and one CW bus (14CW and 23CW respectively). When the 14 and 23 SPTs were lost, the associated vital and CW buses transferred to the 13 and 24 SPTs. This placed all three vital buses and both CW pump buses for each unit on one SPT. Following the transfer, the 24 SPT voltage recovered as expected and the Unit 2 vital buses remained on offsite power. However, voltage on the 13 SPT did not recover as expected, and as a result, the SLUPS relays actuated. This relay actuation caused the stripping and loading of all three Unit 1 vital buses onto the Unit 1 emergency diesel generators (EDGs).
Following the initial electrical transient all equipment responded as expected. Operators immediately entered the emergency operating procedures (EOPs) and completed the applicable portions of 1-EOP-TRIP-1 and 1-EOP-TRIP-2. The operations shift supervisor (OSS) declared an Unusual Event at 1401 due to a loss of both sources of offsite power to the vital buses for greater than 15 minutes.
Operators exited the EOPs at 1453, and began the process of recovering the electric plant. At 1515 operators manually adjusted the LTC to return 13 SPT secondary voltage back into its required operating range.
At 1600, after isolating the fault to the 500 kV 1-5 breaker, operators reenergized 500 kV bus section one by closing breakers 1-8 and 1-9. The deenergized sections of the 13 kV north and south ring buses were restored, and the 14 SPT was reenergized at 2201. At that time both sources of offsite power were available to all three Unit 1 vital buses and the OSS terminated the Unusual Event. At 2251 operators placed the last of the three Unit 1 vital buses on the 13 SPT.
The Transient Assessment Response Plan (TARP) teams initial engineering analysis determined that the separation of the Unit 1 vital buses from offsite power occurred due to the transfer of the non-vital 14 CW bus and the B and C vital buses to the 13 SPT. The transfer of this bulk load caused the SLUPS relays to operate and time out before the transformer secondary voltage could recover above the relay reset value. The TARP team recommended disabling the auto-transfer feature for the CW buses on both units. This reduced the amount of non-vital loads transferred to the remaining source of offsite power following the loss of one source. Engineering analysis determined that this action would ensure that the SLUPs relays either did not operate, or would reset within 13 seconds following the loss of one source of offsite power. If the CW bus transfer switch had been in manual for the July 29 event, the engineering calculations determined that the final 13 SPT secondary voltage would have been one percent higher and would have recovered in a shorter time.
Operators disabled the automatic transfer feature of the CW buses on Unit 1 and 2 at 2200 hours0.0255 days <br />0.611 hours <br />0.00364 weeks <br />8.371e-4 months <br /> on July 29 and returned Unit 1 to full power on August 4, 2003, at 0244 hours0.00282 days <br />0.0678 hours <br />4.034392e-4 weeks <br />9.2842e-5 months <br />. The NRC team subsequently determined that additional actions were necessary to ensure operability of both offsite power sources. These actions were evaluated and additional measures implemented by PSEG on August 22, 2003. (Refer to section 4OA2 for additional details.)
The team developed a detailed time line of the sequence of events, which is included as C of this report.
B.
CONTRIBUTING CAUSES
1. Equipment Performance/Failures
a. Inspection Scope
The team reviewed the adequacy of PSEGs investigations and root cause evaluations for the 1-5 500 kV circuit breaker ground fault relay actuation, for the unexpected loss of the second offsite power source to the three Unit 1 4.16 kV safety buses and for the position indication problem with 4T60 electrical disconnect switch. The team reviewed the adequacy of PSEGs planned corrective actions and extent of condition reviews for these items.
The team also reviewed the adequacy of the 4.16 kV safety bus power supply design, including transfer logic and setpoints, to assess conformance with the plant design and licensing basis requirements.
The team reviewed the adequacy of preventive and predictive maintenance of the Salem switchyard electrical equipment that was within the scope of the maintenance rule. Maintenance frequency and procedures were compared to vendor recommendations. System Health Reports for the 500 kV and 13 kV switchyard were reviewed and compared to the current switchyard status.
The troubleshooting and post maintenance testing associated with the 500 kV 1-5 breaker and breaker fault logic were reviewed. These consisted of reviewing condition reports, completed work orders, maintenance procedures, system drawings, vendor documents, interviews with the system manager and a system walkdown.
The maintenance rule scoping process was compared to the actual maintenance rule scope for the switchyard and 4.16 kV electrical vital bus components. Key personnel responsible for implementing the maintenance rule were interviewed. Additionally, equipment deficiencies and failures were reviewed to ensure they were properly dispositioned in accordance with the maintenance rule procedures.
The team reviewed completed surveillance testing procedures that test the offsite power transfer capability and the 4.16 kV vital bus feeder breaker undervoltage and degraded voltage trips. The team also reviewed completed surveillance testing procedures associated with sequenced loading and block loading of 4.16 kV vital bus components during any combination of LOCA and LOOP signals. These procedures were reviewed to ensure they were consistent with technical specifications, system drawings and the Updated Final Safety Analysis Report (UFSAR) and to ensure test results met the procedure acceptance criteria. The team also reviewed the surveillance procedure that performs periodic verification of the availability of two independent sources of offsite power.
b. Findings
The team found that the final PSEG investigations and root cause evaluations were generally adequate. However, during the initial week of the special inspection in August, the team noted that at that time there had been minimal progress in identifying the root cause of the event and the licensee did not have a comprehensive understanding of plant and operator response to the event. For example, the reasons for the #13 SPT tap changer failing to automatically restore bus voltage during the event were not understood by the root cause team, the operability evaluation performed to support plant restart was not adequate to preclude recurrence of the event, a detailed time line had not been developed and potential impacts on Hope Creek electrical calculations had not been properly assessed. Additionally, several examples of inadequate design controls were identified as discussed below.
b.1
Introduction.
A Green NCV was identified for the licensee failing to implement adequate design controls during plant modifications, setpoint changes, and calculation revisions associated with the offsite power supply to the 4.16 kV vital AC buses. As a result, the lower operating voltage limit established for the 4.16 kV safety buses was not adequate to ensure independence of the two offsite power sources. The loss of one offsite source could result in a complete loss of offsite power to the vital buses.
Description.
In 1992, PSEG initiated a significant redesign of the Salem switchyard to increase the reliability of the offsite power supply to the vital buses. This modification resulted in the circulating water pumps being powered by the same station power transformers as the 4.16 kV safety buses. The design also included bus transfer features such that on a loss of one offsite source the vital buses and the 3 circulating water pumps affected by the loss would automatically transfer to the remaining offsite source. The design activities included revisions of associated electrical calculations, including ES-15.012, Bus Transfer Calculation, to reflect the new system configuration.
However, design control measures, including determination of appropriate design inputs and independent reviews, were not adequate to ensure that the evaluations and calculation revisions included the most limiting transient conditions. During the redesign, the licensee computer calculation uncertainties, which had been determined from comparing calculation results with actual plant measurements, were not accounted for in the calculations and plant operating limits.
In 1994, the licensee identified that the setpoints for the degraded grid undervoltage relays needed to be increased to ensure adequate voltage to all components during a degraded grid condition. Again, associated calculations were revised and the changes implemented without identifying non-conservative assumptions that were previously introduced into the design calculations.
In 2001, the loop accuracy calculation for the control room 4.16 kV bus voltage indicators was revised and determined that the indicator loop accuracy was +/- 64 volts compared to the previously determined +/- 40 volts. The revised uncertainty values were not incorporated into plant operating procedures to ensure that the plant was operated within the established design basis.
Analysis.
In accordance with Inspection Manual Chapter (IMC) 0612, Appendix B, Issue Disposition Screening, the inspectors determined that the issue was more than minor because it was associated with the design control attribute of the initiating events cornerstone objective. Specifically, the licensees failure to implement adequate design control measures during plant modifications, setpoint changes, and calculation revisions resulted in an increase in the likelihood of a loss of offsite power event. In accordance with IMC 0609, Appendix A, Significance Determination of Reactor Inspection Findings for At-Power Situations, the inspectors conducted a SDP Phase 1 screening and determined that a SDP Phase 2 evaluation was required because the finding contributed to both the likelihood of a reactor trip and the likelihood that mitigation equipment would not be available.
The inspectors conducted a SDP Phase 2 evaluation of the risk significance of the performance deficiency and determined that the finding was of low to moderate safety significance (White). The inspectors used the following assumptions in the Phase 2 evaluation.
C The licensees failure to implement appropriate design control measures for plant modifications, setpoint changes, and calculation revisions associated with the electrical distribution system resulted in an increase in the likelihood of a loss of offsite power event by a factor of 10.
C The licensee conducted these plant modifications, setpoint changes, and calculation revisions in 1992, 1994, and 2001, respectively. Therefore, the inspectors used an exposure time of one year.
The inspectors reviewed the Phase 2 results and concluded that they were conservative because the SDP Phase 2 notebook did not credit the ability of the 13 SPT to automatically re-power the 4160 volt vital buses following the failure of an emergency diesel generator. Therefore, the inspectors determined that the finding should be evaluated using the SDP Phase 3 process.
The regional Senior Reactor Analyst (SRA) conducted the SDP Phase 3 analysis using the following assumptions.
C Due to the above performance deficiency, electrical faults on any one of six 500 kV breakers (e.g., breakers 1-5, 1-8, 1-9, 2-6, 2-8, and 2-10), two 500 kV buses (e.g., Bus 1 and 2), four 13 kV buses (e.g., 13 kV north bus, 13 kV south bus, 13 kV ring bus section 1, and 13 kV ring bus section 4), or eight station power transformers (e.g., #1, #2, #3, #4, #13, #14, #23, and #24) would have resulted in a loss of both offsite power sources to the 4160 volt vital buses when only one offsite power source should have been lost due to the fault. This resulted in an increase in the likelihood of a loss of offsite power event by 1.3841E-1 per year.
C Because this vulnerability existed since 1992, the analyst used the maximum exposure time of one year.
The analysts used the NRCs SPAR (Standardized Plant Analysis Risk) Model, Revision 3.02, to evaluate the significance of this finding. The analyst revised the model to reflect licensee procedures and operating experience as described in Section C of this report. The analyst determined the change in core damage frequency ( CDF) for this performance deficiency by multiplying the increase in the loss of offsite power initiating event frequency and the conditional core damage probability (CCDP) for this type of event (See Section C of this report). Therefore, the analyst determined that the CDF for this finding was approximately 4.15E-7 per year. The dominant accident sequence involved a loss of offsite power event with a failure of the reactor protection system to shutdown the reactor.
In addition, the analyst determined that the risk contribution due to fire events was not significant because the likelihood of a fire induced electrical fault on one of these components was approximately one order of magnitude less than the increase in the loss of offsite power initiating event frequency assumed in this analysis. The analyst also determined that the risk contribution due to seismic initiators was not significant because all consequential seismic events are assumed to result in a loss of offsite power. Furthermore, the analyst evaluated this finding using Inspection Manual Chapter 0609, Appendix H, Containment Integrity SDP. Because the facility has a large dry containment and the dominant accident sequences did not involve either a steam generator tube rupture or an inter-system loss of coolant accident, the finding did not significantly contribute to an increase in the large early release frequency for the facility.
As a result, the inspectors concluded that the safety significance of the performance deficiency was very low (Green).
Enforcement.
The failure of the licensee to implement adequate design control measures and to translate the plant design into plant procedures is considered a violation of 10 CFR 50 Appendix B - Criteria III, Design Control. Because the failure to implement adequate design control measures is of very low safety significance and has been entered into the corrective action program (Notifications 20153983, 20156551),this violation is being treated as an NCV consistent with Section VI.A of the NRC Enforcement Policy: NCV 05000272,05000311/2003008-01, Failure to Implement Adequate Design Control Measures.
2. Human Factor and Procedural Issues
a. Inspection Scope
The inspectors compiled a time line for the event based on data supplied by the plant computer and interviews with operations department personnel who were onsite during the event. The inspectors also reviewed the sequence of events log and overhead annunciator printout to verify the accuracy and thoroughness of PSEGs time line for the event.
The inspectors interviewed operators regarding the response to the event with particular emphasis on difficulties they encountered during the plant recovery. The inspectors reviewed the Transient Assessment Response Plan (TARP) report, which provided PSEGs initial assessment of the event and the justification for restoring the plant to full power. The inspectors also reviewed the post-trip review report associated with the reactor trip.
The inspectors interviewed emergency planning personnel associated with the Unusual Event declaration and reviewed the procedures utilized by the operators to verify the actions taken were consistent with the emergency operating and abnormal operating procedures.
b. Observations and Findings
b.1 The inspectors determined that PSEG operations overall response to the event was adequate. Emergency Action Level (EAL) declarations and notifications were timely and accurate, and operators placed the plant in a safe lineup using plant procedures following the event. However, the inspectors identified several equipment control and procedure adequacy issues.
Operating Procedure Deficiency The electric plant was restored following the loss of offsite power using abnormal operating procedure S1.OP-AB.LOOP-0003(Q), Partial Loss of Offsite Power. To restore offsite power to the 1B vital bus the procedure referred the operator to operating procedure S1.OP-SO.DG-0002(Q). Operators entered step 5.12 of S1.OP-SO.DG-0002(Q) to restore 1B vital bus to the 13 SPT. Operators placed redundant equipment powered from other vital buses in service to support plant operation and then stripped the 1B vital bus of all loads. Then, in accordance with the procedure, operators opened the 1B EDG output breaker. Immediately after operators opened the 1B EDG output breaker, the 13 SPT supply breaker for the 1B vital bus closed to reenergize the 1B vital bus.
Procedure S1.OP-SO.DG-0002(Q) did not reflect this sequence of events. Operators investigated this operation and determined that, for the conditions that the plant was in when the B EDG output breaker was opened, the automatic closure of the 13 SPT supply breaker for the 1B vital bus was in accordance with the design. When the 1B EDG output breaker was opened, the control circuitry for the 13 SPT supply breaker to the 1B vital bus sensed low voltage on the 1B vital bus and adequate voltage on the output of the 13 SPT and closed in the breaker from the 13 SPT to supply power to the bus. However, the operators did not document the unexpected breaker operations and associated procedure deficiencies in the corrective actions program, because after they analyzed the system response, they determined that it operated as designed. The team considered the failure of the operators to initiate a corrective action program notification to be a violation of 10 CFR 50 Criterion XVI, Corrective Actions. This finding was determined to be minor because there were no significant consequences and the system operated as designed. However, the decision to not document the unexpected breaker operations in the control room did not allow the corrective action process to disposition the importance of the issue, identify the potential causes and identify corrective actions for possible inadequate procedures or operator training.
Transformer Load Tap Changer Control The secondary side of the 13 SPT includes a LTC used to maintain output voltage for the transformer in the required operating band, 4.22 kV to 4.36 kV. In the automatic mode of operation, the LTC automatically maintains voltage in the required operating band during normal operating conditions when grid voltage may be changing very gradually. During transient conditions, the LTC was designed to adjust voltage back to within +/- 10% band of the setpoint (4.29 kV) following a 30 second time delay. During the July 29 event, the 13 SPT LTC did not automatically respond to a sustained low output voltage that existed following the bus transfers. Operators were required to manually adjust voltage back to its required operating band before returning the vital buses to the 13 SPT. Maintenance troubleshooting on July 30 identified that the auto/manual toggle switch on the LTC was out of position, and that this would have prevented the LTC from operating in the automatic mode. The maintenance technician did not document this condition until requested by the RCA team on October 23, 2003, two months after the event. Similar to the issue regarding the unexpected operations of the vital bus breakers during restoration from the event, the maintenance technicians decision to not document the switch out of position for the 13 SPT prevented the corrective action process from properly dispositioning the importance of the issue or identifying the potential causes or extent of condition for the mispositioning.
The inspection team discussed tap changer operations with several operators, and reviewed lessons learned from previous Salem events, in particular, the September 24, 2001, Salem Unit 1 trip. Based on these discussions, it was observed that the importance of operation of the LTC was under-emphasized because it was not credited in plant design bases. Operators were not familiar with the local tap changer controls and settings. The 4 kV system operating procedure did not contain directions for local operation of the LTC, nor did it provide a means of maintaining status for local LTC switch positions that could affect automatic operation. This issue was not considered to be a violation of NRC requirements because the equipment is not safety-related and, therefore, is not within the scope of 10 CFR 50 Appendix B, Criteria XVI, Corrective Action. This finding was determined to be minor because automatic tap changer operation is not relied on to ensure 4 kV bus voltage is maintained within design basis requirements. Operations within design basis limits is assured by verification on at least a once per shift basis by plant operators that the 4 kV bus voltage is within required operating limits. However, the team considered this to be an another example of a weakness in the implementation of the corrective action program. PSEG subsequently entered this issue into the corrective action program (Notification 20161499).
C. EVENT SIGNIFICANCE
a. Inspection Scope
The team conducted an initiating event assessment to determine the risk significance of the event. This risk assessment was based upon the assumptions stated below.
b. Findings and Observations
b.1 Event Risk Assessment The analysts used the NRCs SPAR model, Revision 3.02, to evaluate the significance of this event. The analyst revised the model to reflect licensee procedures and operating experience as follows.
C The loss of offsite power (LOOP) initiating event frequencies and offsite power recovery probabilities were updated to be consistent with NUREG/CR-5496, Evaluation of Loss of Offsite Power Events at Nuclear Power Plants: 1980 -
1996. These values are the NRCs current best estimate of both the likelihood of each of the LOOP classes (i.e., plant-centered, grid-related, and severe weather) and their recovery probabilities.
C The reactor coolant pump (RCP) seal loss of coolant accident probabilities were updated to be consistent with the Rhodes Model as documented in Appendix A of NUREG/CR-5167, Cost/Benefit Analysis for Generic Issue 23: Reactor Coolant Pump Seal Failure. The Salem Unit 1 RCP seals contain a mixture of both high and low temperature O-rings as follows.
RCP O-ring Type Installed 11 RCP All seals have high temperature O-rings installed.
First stage seal has high temperature O-rings installed while the 12 RCP remainder have low temperature O-rings installed.
First stage seal has high temperature O-rings installed while the 13 RCP remainder have low temperature O-rings installed.
First stage seal has high temperature O-rings installed while the 14 RCP remainder have low temperature O-rings installed.
In accordance with NUREG/CR-5167, Appendix A, the first stage seal is inherently stable; however, it is very susceptible to high leakage should the back pressure drop due to a failure of the second stage seal. In addition, no credit is given for the ability of the third stage seal to survive if subjected to a differential pressure greater than the normal operating differential pressure of greater than a few psid, which would occur given the failure of the first two seals. Therefore, the analyst used the Rhodes Model results for low temperature O-rings because in 3 of 4 RCPs the second stage seal would fail after 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> due to the failure of the low temperature O-rings, which would in turn result in failure of the first and third stage seals.
C The NRCs SPAR model success criterion for emergency AC power is 2 of 3 onsite emergency diesel generators (EDGs) or the gas turbine providing power to the 4160 volt AC buses. This criterion is consistent with the licensees probabilistic risk assessment (PRA) model. It is based upon the assumption that two service water pump trains are needed for safe shutdown and one EDG cannot supply enough AC power for more than one service water pump train.
The licensee completed an informal engineering analysis (NUTS Order 80058688), which the staff reviewed, that demonstrated only one service water pump train is needed to provide service water cooling following a LOOP provided that the non-essential service water loads are automatically isolated from the essential service water loads. The licensee determined that under these conditions a flow rate of approximately 13,935 gallons per minute (gpm) is needed to cool the essential service water loads. This flow rate is within the capacity of one service water pump, approximately 14,400 gpm. The non-essential service water loads are isolated by motor-operated valves that automatically close following a LOOP (i.e., 11SW20, 1SW26, and 13SW20 which are powered from the 1A, 1B, and 1C EDGs, respectively). In order to isolate the non-essential loads, either the 1SW26 valve or the 11SW20 and 13SW20 valves must close. Therefore, the analyst assumed that the success criteria for emergency AC power was either the 1B EDG or the 1A and 1C EDGs or the gas turbine providing power to the 4160 volt AC buses.
C The NRCs SPAR model required service water cooling to the motor-driven auxiliary feedwater (MDAFW) pump room coolers for success of the MDAFW pump trains. This criterion is consistent with the licensees probabilistic risk assessment (PRA) model. However, the licensee had completed Engineering Evaluation S-C-ABV-MEE-1472, Effect of the Loss of Auxiliary Building Ventilation on Appendix R Safe Shutdown Electrical Equipment and the Heat Stress Effect on the Capability to Perform Manual Actions, which the staff reviewed, that demonstrated the auxiliary building ventilation system would provide sufficient room cooling to support operation of the MDAFW pump trains following a loss of service water. Therefore, the analyst assumed that the MDAFW pump trains were dependent on either the service water system or the auxiliary building ventilation system for cooling.
C The human error probability for the operator failing to initiate feed and bleed cooling was updated to more realistically account for the time available to perform the action. The analyst determined that the revised failure probability was approximately 2.0E-3 using the Accident Sequence Precursor Human Reliability Analysis methodology.
C The model was revised to include operator recovery of the switchgear and penetration area ventilation system for the vital AC buses. The analyst determined that the recovery failure probability was approximately 1.1E-1 using the Accident Sequence Precursor Human Reliability Analysis methodology.
C The model was revised to reflect that had an emergency diesel generator failed during this event, the 13 station power transformer would have automatically re-powered the affected 4160 volt vital bus.
b.1 Risk Assessment Results The risk assessment determined that the dominant accident sequences for this event were as follows.
CCDP Core Damage Sequence Description 1.2E-6 C IE - Loss of offsite power (LOOP)
C Reactor protection system fails to shutdown the reactor 5.4E-7 C IE - Loss of offsite power (LOOP)
C Failure of the auxiliary feedwater system C Failure of feed and bleed core cooling 2.5E-7 C IE - Loss of offsite power (LOOP)
C Failure of emergency AC power C RCP seals fail without cooling and injection C Operators fail to recover AC power prior to core damage 2.5E-7 C IE - Loss of offsite power (LOOP)
C PORV opens and fails to reclose C Operators fail to recover AC power within 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> C Operators fail to initiate high pressure recirculation 2.1E-7 C IE - Loss of offsite power (LOOP)
C Failure of emergency AC power C Failure of the auxiliary feedwater system C Operators fail to recover AC power prior to core damage The team concluded that the conditional core damage probability (CCDP) for this event was approximately 3.0E-6. This indicates that the risk significance of the event was low.
b.2 Chemical and Volume Control System (CVCS) Modification
Introduction.
A Green NCV was identified for the licensee failing to properly translate a design change associated with a CVCS system cross connect modification into plant procedures.
Description.
The team identified that in November 2003, PSE&G failed to properly revise procedure S1.OP-AB.LOOP-0001(Q), Loss of Offsite Power, when implementing a plant modification that added the capability to cross-connect the Salem Units 1 & 2 chemical and volume control systems. If one unit was in a station blackout condition or had lost the CVCS system due to other causes, the opposite unit could supply reactor coolant make-up to the reactor coolant system and provide seal injection flow to the reactor coolant pump seals. The seal injection provides seal cooling and serves to prevent seal failure, and a subsequent seal LOCA, due to overheating. The modification consisted of physical changes to the CVCS systems and procedure changes that would direct operators when to use the cross-connect.
On November 11, 2003, the NRC identified that during a station blackout of one of the Salem Units, the procedure step to cross-connect CVCS between units would likely have been missed due to the step not being inserted at the proper place in the Loss of Offsite Power procedure. The purpose of the modification was to reduce the risk of core damage during a station blackout event (SBO) by providing additional mitigation capability to prevent a reactor coolant pump seal failure, which could result from a loss of seal injection.
Analysis.
The inspectors determined that this finding was more than minor because it affected the design control attribute of the Mitigating Systems cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable conditions. The issue was not a design or qualification deficiency that the licensee had evaluated in accordance with GL 91-18, and was determined to be of very low safety significance (Green) because it did not result in an actual loss of safety function of a single train for internal or external event initiated core damage sequences.
Enforcement.
10 CFR 50, Appendix B, Section III, Design Control, states, in part, that measures shall be established to assure that applicable regulatory requirements and the design basis, as defined in 10 CFR 50.2 and specified in the license application, for those structures, systems, and components to which this appendix applies are correctly translated into specifications, drawings, procedures, and instructions. Contrary to the above, it was identified on November 11, 2003, that the CVCS cross connect modification design was not correctly implemented into the Loss of Offsite Power Procedure, S1.OP-AB.LOOP-0001(Q), Revision 13. Because this finding is of very low safety significance and has been entered into the corrective action program (Notification 20165852), this violation is being treated as an NCV, consistent with Section V1.A of the Enforcement Policy: NCV 05000272,05000311/2003008-02, Failure to Properly Translate Design Into Plant Procedures.
OTHER ACTIVITIES (OA)
4OA2 Problem Identification and Resolution
a. Inspection Scope
In addition to performing detailed reviews of the corrective action aspects associated with the issues that were the focus of this inspection (1-5 breaker trip and loss of the second offsite source), the team reviewed the effectiveness of the licensee in addressing conditions adverse to quality that were identified prior to, during and after the event.
The team also reviewed a prior related event that involved the electrical system design and operation to evaluate the effectiveness of corrective actions associated with that event. Specifically, the inspectors reviewed the corrective actions associated with the September 24, 2001, 500 kV surge arrester failure on the 2 SPT.
b. Findings
b.1 Inadequate Operability Determination
Introduction.
A Green NCV was identified for the failure of the licensee to properly assess the design deficiencies with the electrical system and to implement adequate corrective actions to prevent recurrence before restarting Unit 1 following the July 29, 2003, event.
Description.
Following the July 29 event the licensee performed an operability evaluation to determine what actions were necessary to ensure operability of the offsite power system and thereby allow restart of Unit 1. The licensee concluded that inhibiting the automatic closure of the circulating water pump bus tie breaker would be sufficient action to prevent recurrence of the July 29 event, where the loss of one offsite source resulted in the loss of the second offsite source to the vital 4.16 kV buses. Based on this action, the plant was restarted on August 4, 2003. The special inspection team reviewed the operability determination and concluded that it was not adequate to ensure operability of the offsite sources as required by Technical Specification 3.8.1, A.C.
Sources. The team found that the net gain in post-transient voltage that would be realized by blocking auto transfer of circulating water pumps was not sufficient to prevent recurrence in all cases. Specifically, the evaluation assumed the bus would be at 4300 volts at the start of the transient when control room procedures allowed operation with the bus voltage as low as 4220 volts. Also, uncertainties between computer model results and actual plant test data were not accounted for in the operability evaluation. Finally, the team noted that the effects of block loading (simultaneous start of loads) on the electrical buses in the event of a LOCA without the loss of offsite power had not been evaluated. The licensee performed additional engineering evaluations and concluded that an additional action to increase the minimum operating bus voltage was necessary. Plant procedures were revised on August 22, 2003, to increase the lower limit on bus voltage by 55 volts.
Analysis.
In accordance with Inspection Manual Chapter (IMC) 0612, Appendix B, Issue Disposition Screening, the inspectors determined that the issue was more than minor because it was associated with the design control attribute of the initiating events cornerstone objective. Specifically, the failure to take adequate actions to correct a design control error resulted the lack of independence of the two offsite power sources and increased the likelihood of a loss of offsite power (LOOP) initiating event. In accordance with IMC 0609, Appendix A, Significance Determination of Reactor Inspection Findings for At-Power Situations, the inspectors conducted an SDP Phase 1 screening and determined that an SDP Phase 2 evaluation was required because the finding contributed to both the likelihood of a reactor trip and the likelihood that mitigation equipment would not be available.
The inspectors conducted an SDP Phase 2 evaluation of the risk significance of the performance deficiency and determined that the finding was of very low safety significance (Green). The inspectors used the following assumptions in the Phase 2 evaluation.
C The licensees failure to implement adequate corrective actions to correct design deficiencies associated with the electrical distribution system resulted in an increase in the likelihood of a loss of offsite power event by a factor of 10.
C The time that the plant was operated with the improper minimum bus voltage limit was from August 4 to August 22, 2003. Therefore, the inspectors used an exposure time of 18 days.
Because the SDP Phase 2 Worksheets do not include external initiating events, and the SDP Phase 2 results represented an increase in risk of greater than 1E-7, the Senior Reactor Analyst performed a Phase 3 analysis of the issue using the following assumptions.
C Due to the above performance deficiency, electrical faults on any one of six 500 kV breakers (e.g., breakers 1-5, 1-8, 1-9, 2-6, 2-8, and 2-10), two 500 kV buses (e.g., Bus 1 and 2), four 13 kV buses (e.g., 13 kV north bus, 13 kV south bus, 13 kV ring bus section 1, and 13 kV ring bus section 4), or eight station power transformers (e.g., #1, #2, #3, #4, #13, #14, #23, and #24) would have resulted in a loss of both offsite power sources to the 4160 volt vital buses when only one offsite power source should have been lost due to the fault. This resulted in an increase in the likelihood of a loss of offsite power event by 1.3841E-1 per year.
C Because this vulnerability existed from August 4 to August 22, 2003, the analyst used an exposure time of 18 days.
The analysts used the NRCs SPAR model, Revision 3.02, to evaluate the significance of this finding. The analyst revised the model to reflect licensee procedures and operating experience as described in Section C of this report. The analyst determined the change in core damage frequency ( CDF) for this performance deficiency by multiplying the increase in the loss of offsite power initiating event frequency, the exposure time, and the conditional core damage probability (CCDP) for this type of event (See Section C of this report). Therefore, the analyst determined that the CDF for this finding was approximately 2.05E-8 per year. The dominant accident sequence involved a loss of offsite power event with a failure of the reactor protection system to shutdown the reactor.
In addition, the analyst determined that the risk contribution due to fire events was not significant because the likelihood of a fire induced electrical fault on one of these components was approximately one order of magnitude less than the increase in the loss of offsite power initiating event frequency assumed in this analysis. The analyst also determined that the risk contribution due to seismic initiators was not significant because all consequential seismic events are assumed to result in a loss of offsite power. Furthermore, the analyst evaluated this finding using Inspection Manual Chapter 0609, Appendix H, Containment Integrity SDP. Because the facility has a large dry containment and the dominant accident sequences did not involve either a steam generator tube rupture or an inter-system loss of coolant accident, the finding did not significantly contribute to an increase in the large early release frequency for the facility.
As a result, the inspectors concluded that the safety significance of the performance deficiency was very low (Green).
Enforcement.
The failure of the licensee to implement adequate corrective actions to prevent recurrence of a significant condition adverse to quality is considered a violation of 10 CFR 50 Appendix B - Criteria XVI, Corrective Action. Because the finding was determined to be of very low safety significance and has been entered into the corrective action program (Notification 20157376) this violation is being treated as an NCV consistent with Section VI.A of the NRC Enforcement Policy: NCV 05000272,05000311/2003008-03, Failure to Implement Adequate Corrective Actions.
b.2 Corrective Action Program Weaknesses Cable Failures The root cause evaluation for the July 29, 2003, 1-5 breaker trip determined that the cause was shorted/grounded conductors in the ground fault protection circuit coincident with an unidentified transient that affected the ground bus. During this inspection, the team found that this was the third failure of a cable that runs from the station power transformers in the switchyard to the plant through penetrations in the turbine building wall. These failures have occurred within approximately a two year time frame and the licensee suspects they may have failed as a result of damage during work performed to install new seals where the cables pass through the turbine building wall. The team noted that the first two failures were treated as a broke/fix type failure and were not entered into the corrective action program. As a result, there was no apparent cause or extent review performed for the first two failures.
As a result of the significant impact of the third failure involved in the July 29 event, the licensee planned to inspect and test additional cables starting in the Unit 1 refueling outage planned for May 2004. The team found that this plan could have been more aggressive by taking advantage of a Unit 2 outage (which started in October 2003) to inspect similar cables on that unit. Also, at the end of the inspection, the licensee had not yet identified and prioritized cables on Unit 1 so that they could be inspected during an unplanned outage.
Following completion of the NRC team inspection, the cable inspection plans were revised and a sample of Unit 2 cables were checked during the outage. A plan was also being developed to be prepared to check Unit 1 cables during an unplanned outage of sufficient duration.
The team concluded that the licensee corrective actions for the cable failures should have been more aggressive in identifying the cause of the failures, evaluating the extent of condition of the problem and in implementing corrective actions to prevent recurrence. This issue was not considered a violation of NRC requirements because the cables affected were not safety-related components and, therefore, not subject to the requirements of 10 CFR 50 Appendix B, Criteria XVI, Corrective Action.
Failure to Initiate Condition Reports Section B.2.b.1 discusses an example where a deficiency with the electrical system operating procedure, S1.OP-SO.DG-0002(Q), was not entered into the corrective action program. This section discusses another example where an out of position switch for the 13 SPT LTC controls was not entered into the corrective action program until approximately two months after the event.
Corrective Action Timeliness The team noted that the corrective action plan for the bus transfer event included an item to review additional electrical calculations to further assess the extent of condition of design control issues. However, the team noted that the review effort was scheduled to continue until November 2004, eighteen months after the event. The team noted that this schedule could result in additional design issues remaining unidentified for a significant amount of time. The licensee subsequently revised the schedule and now expects to complete the reviews in March 2004.
Operating Experience Reviews The team found that the PSEG review of NRC Information Notice (IN) 95-37, Inadequate Offsite Power System Voltages During Design Basis Events, failed to identify the electrical system design issues (Section B.1.b1) even though the IN specifically discusses problems associated with changes to degraded grid relay setpoints as had been made at Salem in 1994, just one year prior to the notice.
Conclusion The team found examples of inconsistent application of your corrective action program that are similar to those identified in previous NRC inspections as discussed in both the last annual and mid-cycle performance review letters dated March 3 and August 27, 2003, respectively.
4OA3 Event Follow-Up
(Closed) LER 050000272/2003008, Reactor Trip due to turbine Trip Caused by a 500 kV Switchyard Breaker Trip On July 29, 2003, the Salem Unit 1 reactor tripped as a result of a ground fault relay actuation for 500 kV circuit breaker 1-5. This event was reviewed in detail by the special inspection team and its findings are discussed in the preceding sections of this report.
This LER is closed.
4OA6 Meetings, including Exit
The team met with Mr. P. Walsh and other members of PSEG management on August 22 and November 7, 2003, to debrief them on the preliminary results of the Special Inspection to date.
On December 16, 2003, the NRC team presented the inspection results to Mr. R.
Anderson and other members of the PSEG staff. The team asked the licensee whether any material examined during the inspection should be considered proprietary. No proprietary information was identified.
ATTACHMENT A
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee Personnel
J. Carlin Vice President - Engineering
K. Fleischer Design Engineering
C. Fricker Operations Manager
L. Hajos Consultant
C. Kapes Technical Support
S. Karimian Consultant
M. Khan Design Engineering
D. Lounsbury Operations
G. Modi Design Engineering
M. Mortarulo Consultant
J. Nagel Licensing Supervisor
G. Salamon Licensing Manager
C.Smyth Licensing Engineer
M. Tadjalli Design Engineering
P. Walsh Director of Engineering
NRC Personnel
- W. Lanning, Director, Division of Reactor Safety
- G. Malone, Resident Inspector, Salem
- D. Orr, Senior Resident Inspector, Salem
LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED
Opened and Closed During this Inspection
- 05000272,
- 05000311/2003008-01 NCV Failure to Implement Adequate Design Control Measures.
- 05000272,
- 05000311/2003008-02 NCV Failure to Properly Translate Design Into Plant Procedures
- 05000272,
- 05000311/2003008-03 NCV Failure to Implement Adequate Corrective Actions.
Closed During this Inspection LER
- 050000272/2003008 Reactor Trip due to turbine Trip Caused by a 500 kV Switchyard Breaker Trip