ML18059A541: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
 
(3 intermediate revisions by the same user not shown)
Line 3: Line 3:
| issue date = 11/24/1993
| issue date = 11/24/1993
| title = LER 93-012-00:on 931103,concluded That Seismic Qualification Did Not Exist for Relays Previously Installed in Chp & Chr Circuits Due to Undersized Relay Coils.Undersized Coils in Relays replaced.W/931124 Ltr
| title = LER 93-012-00:on 931103,concluded That Seismic Qualification Did Not Exist for Relays Previously Installed in Chp & Chr Circuits Due to Undersized Relay Coils.Undersized Coils in Relays replaced.W/931124 Ltr
| author name = KUEMIN J L, SLADE G B
| author name = Kuemin J, Slade G
| author affiliation = CONSUMERS ENERGY CO. (FORMERLY CONSUMERS POWER CO.),
| author affiliation = CONSUMERS ENERGY CO. (FORMERLY CONSUMERS POWER CO.),
| addressee name =  
| addressee name =  
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:Palisades Nuclear Plant: 27780 Slue Star Memorial Highway, Covert, Ml 49043 November 24, 1993 Nuclear Regulatory Commission Document Control Desk Washington, DC 20555 ' . DOCKET 50-255 LICENSE DPR-20 -PALISADES PLANT -GB Slade General Manager LICENSEE EVENT REPORT 93-012 -CONTAINMENT HIGH PRESSURE (CHP) AND CONTAINMENT HIGH RADIATION (CHR) RELAYS OPERABILITY Licensee Event Report (LER) 93-012 is attached.
{{#Wiki_filter:GB Slade General Manager Palisades Nuclear Plant:     27780 Slue Star Memorial Highway, Covert, Ml 49043 November 24, 1993 Nuclear Regulatory Commission Document Control Desk Washington, DC 20555 DOCKET 50-255 ~ LICENSE DPR PALISADES PLANT -
This event is reportable in accordance with 10CFR50.73(a)(2)(ii)(B) as a-condition outside the design basis of the plant.
LICENSEE EVENT REPORT 93-012 - CONTAINMENT HIGH PRESSURE (CHP) AND CONTAINMENT HIGH RADIATION (CHR) RELAYS OPERABILITY Licensee Event Report (LER) 93-012 is attached. This event is reportable in accordance with 10CFR50.73(a)(2)(ii)(B) as a-condition outside the design basis of the plant.
Gerald B Slade General Manager CC Administrator, Region III, USNRC NRC Resident Inspector  
          ~~'----
-Palisades Attachment n , ... 0 f\ > .. :, .. ,_ .. *. .. .... .. ...... 9312070325 ADOCK PDR A CiW5 ENERGY COMPANY NRC Form 388 U.S. NUCLEAR REGULATORY COMMISSION 18*831 APPROVED OMB NO. 3150-0104 EXPIRES: 8/31185 LICENSEE EVENT REPORT !LERI FACILITY NAME 111 DOCKET NUMBER 121 PAGE 131 Palisades Plant o I s I o I o I o I 2 I s I s 1 I OF o I 7 TITLE l'I (CHP) (CHR) CONTAINMENT HIGH PRESSURE AND CONTAINMENT HIGH RADIATION RELAYS OPERABILITY EVENT DATE 151 LER NUMBER 181 REPORT DATE 181 OTHER FACILITIES INVOLVED 181 SEQUENTIAi.
Gerald B Slade General Manager CC Administrator, Region III, USNRC NRC Resident Inspector - Palisades Attachment n.. ,. . 0 f\ .. >
REVISION FACILITY NAMES MONTH DAY YEAR YEAR NUMBER NUMBER MONTH DAY YEAR N/A ol51o!o!ol I --i I i 013 9 3 913 o 11 I 2 0 10 i I i 2 14 9 13 N/A ol51ololol I THIS REPORT IS SUBMITTED PURSUANT TO THE REQUIREMENTS OF 10 CFR l: /Ch<<:k one&deg;'"'&deg;"'
                    ~
of ttt. following/
                                  ,*~
1111 OPERATING N MODE 181 20.4021bl 20.4051cl 50.731all211ivl 73.711bl --._ POWER I 20.4051*111 llil 50.381cll11 50.731all21M 73.711cl LEVEL 01 01 o-....... ....... ._ 1101 20.4051*111 lliil 50.38Ccll21 50.73Call211viil OTHER ISpacify in Abatract ....... ._ ._ .. :. *,* 20.405Call1 lliiil 50.73Call21ftl 50.7 3Call211viiillAI below end in Text, .* .... ' .__ ....... ._ ... *. 20.4051all1 llivl x 50.731*1121Ciil
9312070325 6g665~55
: 50. 7 3 la112lCviiillBI NRC Form 388AI *-....... ._ 20.405Call11M 50.731*112lliiil 50.73Call21Cxl
~DR  ADOCK                 PDR                             A CiW5 ENERGY COMPANY
.*. LICENSEE CONTACT FOR THIS LER 1121 NAME TELEPHONE NUMBER James L. Kuemin, Licensing Administrator I 11sl41
 
* I a I s I , I 3 COMPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRIBED IN THIS REPORT 1131 MANUFAC* REPORTABLE MANUFAC* REPORTABLE CAUSE SYSTEM COMPONENT TUR ER TO NPRDS CAUSE SYSTEM COMPONENT TUR ER TO NPRDS I< .,, I I I I I I I I I I I I I I I I I I I I I , *** , **. -.,--*<< I I I I I I I SUPPLEMENT Al REPORT EXPECTED Cl 41 MONTH DAY YEAR hi NO EXPECTED ---, YES Vt Y*** campier. EXPECTED SUBMISSION DATE! SUBMISSION I I I DATE 1151 ABSTRACT UJmit to 1400 -*** I.e., -imetely fiftHn aifl(IM--*
NRC Form 388                                                                                                                                                       U.S. NUCLEAR REGULATORY COMMISSION 18*831                                                                                                                                                                     APPROVED OMB NO. 3150-0104 EXPIRES: 8/31185 LICENSEE EVENT REPORT !LERI FACILITY NAME 111                                                                                                                         DOCKET NUMBER 121                                     PAGE 131 Palisades Plant                                                                                                                           o   Is Io Io Io I2 I s Is                        1 I OF o   I 7 TITLE l'I CONTAINMENT HIGH PRESSURE (CHP) AND CONTAINMENT HIGH RADIATION (CHR) RELAYS OPERABILITY EVENT DATE 151                             LER NUMBER 181                           REPORT DATE 181                                           OTHER FACILITIES INVOLVED 181 SEQUENTIAi.         REVISION                                                         FACILITY NAMES MONTH       DAY     YEAR           YEAR             NUMBER           NUMBER         MONTH               DAY       YEAR N/A                                         ol51o!o!ol I
ry,,.writr.n line*I 1181 On November 3, 1993, with the pl ant in a hot standby condjtion, it was concluded that seismic qualification did not exist for relays that had been previously i nsta 11 ed in the CHP and CHR circuits.
i   I   i 013       9 3             913           o 11     I2        0   10         i I i         2   14     9 13       N/A                                         ol51ololol                   I THIS REPORT IS SUBMITTED PURSUANT TO THE REQUIREMENTS OF 10 CFR l: /Ch<<:k                     one&deg;'"'&deg;"' of ttt. following/ 1111 OPERATING POWER LEVEL 1101 MODE 181 I  01    01 N
During the refueling outage these relays were determined to have undersized coil s and were therefore modified.
o-20.4021bl 20.4051*111 llil 20.4051*111 lliil 20.405Call1 lliiil 20.4051cl 50.381cll11 50.38Ccll21 50.73Call21ftl 50.731all211ivl 50.731all21M 50.73Call211viil 50.7 3Call211viiillAI 73.711bl 73.711cl OTHER ISpacify in Abatract below end in Text,
Seismic testing of the relays in their original configuration did not prove they could operate following a design basis earthquake.
                          ... *.           20.4051all1 llivl                  x
Subsequent evaluation has also determined that no consideration was made relative to the relay circuit voltage drop, which when taken into account would result in a condition in which the pre-modified relays might not meet their design function.
                                                                              ....... 50.731*1121Ciil
The root cause of the event is the undersized relay coils. These are original pl ant design deficiencies.
                                                                                                                            ._     50. 7 3 la112lCviiillBI                    NRC Form 388AI 20.405Call11M                              50.731*112lliiil                            50.73Call21Cxl LICENSEE CONTACT FOR THIS LER 1121 NAME                                                                                                                                                                TELEPHONE NUMBER James L. Kuemin, Licensing Administrator                                                                                                        sARIEA1cl&deg;~    I  11sl41            *IaIsI ,            I  3 COMPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRIBED IN THIS REPORT 1131 MANUFAC*        REPORTABLE                                                                              MANUFAC*           REPORTABLE CAUSE    SYSTEM            COMPONENT                      TUR ER          TO NPRDS                            CAUSE SYSTEM          COMPONENT                  TUR ER            TO NPRDS I              I      I    I            I    I    I                        I<                            I            I      I        I      I    I    I I              I      I    I            I    I    I                                                      I            I      I         I       I     I     I SUPPLEMENT Al REPORT EXPECTED Cl 41                                                                                      MONTH        DAY      YEAR EXPECTED
Corrective action for this event includes seismically qualifying the relays consistent with the installed configuration, replacing the undersized coils in the relays, evaluating other similar relays, evaluating the effects of voltage drop in the relay circuits, updating equipment control documents, ensuring staff and procedures are knowledgeable and reflect the lessons learned from this event.
- - - , YES Vt Y*** campier. EXPECTED SUBMISSION DATE!                      hi        NO ABSTRACT UJmit to 1400 - * *
NRC Form 388A (9-831 FACILITY NAME 111 PALISADES PLANT EVENT DESCRIPTION U.S. NUCLEAR REGULATORY COMMISSION APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION DOCKET NUMBER 121 LER NUMBER 131 PAGE 141 SEQUENTIAL I I REVISION YEAR NUMBER NUMBER ol5lololol21515le 13 01 i I 2 I -I o I o 01 2 I OF I 0 I 7 On August 4, 1993 at 0350 hours, with the plant in cold shutdown for refueling, Technical Specifications Surveillance Test (TSSP) R0-12, "Containment High Pressure (CHP) and Spray System Tests," was being performed.
* I.e., -imetely fiftHn aifl(IM--* ry,,.writr.n line*I 1181 SUBMISSION DATE 1151          I           I         I On November 3, 1993, with the pl ant in a hot standby condjtion, it was concluded that seismic qualification did not exist for relays that had been previously i nsta 11 ed in the CHP and CHR circuits. During the refueling outage these relays were determined to have undersized coil s and were therefore modified. Seismic testing of the relays in their original configuration did not prove they could operate following a design basis earthquake. Subsequent evaluation has also determined that no consideration was made relative to the relay circuit voltage drop, which when taken into account would result in a condition in which the pre-modified relays might not meet their design function.
TSSP R0-12 verifies CHP circuitry initiates the Safety Injection, Containment Isolation, emergency operation of the Control Room Ventilation, Iodine Removal, and Containment Spray Systems. During the test of the left channel of the CHP circuitry, not all of the equipment required to actuate to the emergency mode of operation responded to the CHP signal. TSSP R0-12 was aborted and troubleshooting for the cause of the test failure was initiated.
The root cause of the event is the undersized relay coils.                                                                                    These are original pl ant design deficiencies.
It was subsequently determined that a relay [JE;RLY] (5P-1) in the channel of the CHP circuitry failed to fully actuate. Visual examination of the 5P-1 relay showed that the armature pin clip of the relay coil and the coil retaining clips were loose. Similar relays in the CHP and CHR circuits were inspected and no other loose armature pin clips or loose coil retaining clips were found. It was concluded that the loose armature pin clip and the loose coil retaining clips on the 5P-1 relay was an isolated case resulting from improper assembly of the relay. Concurrent with an evaluation of the failure, 5P-l relay was replaced with an in-stock spare on August 5, 1993 and maintenance testing of the left channel of CHP, using TSSP R0-12, was initiated on August 6, 1993. During the post-maintenance testing of the CHP circuit a left channel actuation was completed satisfactorily with all equipment responding to the emergency mode; however, during a second test of the left channel of CHP, the CHP relays again failed to seal-in. Troubleshooting determined that a problem existed with the replacement CHP relay (5P-1) and that the problem was inadequate clearance between the armature and the coil. Further investigation indicated that in certain orientations the armature struck the coil which stopped the travel of the armature thereby not allowing the relay to fully actuate. On August 11, 1993 mechanical loading information was received from the relay manufacturer indicating that a different coil was needed if the relay was configured with 9 or more normally closed contacts.
Corrective action for this event includes seismically qualifying the relays consistent with the installed configuration, replacing the undersized coils in the relays, evaluating other similar relays, evaluating the effects of voltage drop in the relay circuits, updating equipment control documents, ~nd ensuring staff and procedures are knowledgeable and reflect the lessons learned from this event.
An evaluation was then conducted of the operability of the CHP relays and it was determined that the relays might not be capable of performing their intended safety function.
 
On August 13, 1993, further investigation revealed that similar relays were also installed in the containment high radiation (CHR) circuitry.
NRC Form 388A                                                                      U.S. NUCLEAR REGULATORY COMMISSION (9-831                                                                                    APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111              DOCKET NUMBER 121                    LER NUMBER 131                        PAGE 141 SEQUENTIAL  I  I REVISION YEAR      NUMBER          NUMBER PALISADES PLANT            ol5lololol21515le 13                  01 i I 2 I - I o I o          01 2 I I 0I OF      7 EVENT DESCRIPTION On August 4, 1993 at 0350 hours, with the plant in cold shutdown for refueling, Technical Specifications Surveillance Test (TSSP) R0-12, "Containment High Pressure (CHP) and Spray System Tests," was being performed. TSSP R0-12 verifies CHP circuitry initiates the Safety Injection, Containment Isolation, emergency operation of the Control Room Ventilation, Iodine Removal, and Containment Spray Systems. During the test of the left channel of the CHP circuitry, not all of the equipment required to actuate to the emergency mode of operation responded to the CHP signal. TSSP R0-12 was aborted and troubleshooting for the cause of the test failure was initiated. It was subsequently determined that a relay [JE;RLY] (5P-1) in the l~ft channel of the CHP circuitry failed to fully actuate. Visual examination of the 5P-1 relay showed that the armature pin clip of the relay coil and the coil retaining clips were loose. Similar relays in the CHP and CHR circuits were inspected and no other loose armature pin clips or loose coil retaining clips were found. It was concluded that the loose armature pin clip and the loose coil retaining clips on the 5P-1 relay was an isolated case resulting from improper assembly of the relay. Concurrent with an evaluation of the failure, 5P-l relay was replaced with an identi~al in-stock spare on August 5, 1993 and post-maintenance testing of the left channel of CHP, using TSSP R0-12, was initiated on August 6, 1993.
Since CHR was required to be operational during refueling operations in accordance with Technical Specification 3.8.lc, the CHR system was declared inoperable.
During the post-maintenance testing of the CHP circuit a left channel actuation was completed satisfactorily with all equipment responding to the emergency mode; however, during a second test of the left channel of CHP, the CHP relays again failed to seal-in.
Refueling operations were halted until containment integrity was A four-hour notification was made to the NRC in accordance with 10CFR50.72(b)(2)(i).
Troubleshooting determined that a problem existed with the replacement CHP relay (5P-1) and that the problem was inadequate clearance between the armature and the coil.
During the period from August 14 to 26 new more powerful coils were seismically qualified, installed in the affected relays, and the relays were tested. This modification affected 2 relays in the CHP circuit and 4 relays in the CHR circuit. Eleven other relays of the same model in the CHP, CHR, and SIS circuits, with 8 or less normally closed contacts remained acceptable with their original coil size.
Further investigation indicated that in certain orientations the armature struck the coil which stopped the travel of the armature thereby not allowing the relay to fully actuate.
NRC Fann 388A 18-831 FACILITY NAME 111 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION DOCKET NUMBER 121 LER NUMBER 131 U.S. NUCLEAR REGULATORY COMMISSION APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER PALISADES PLANT olslololol2lsls 913 -01112 -010 Ol3 OF 017 Following further evaluation it was determined the event was not reportable and an LER in accordance with lOCFR 50.73 was therefore not made. The original notification was eventually retracted on October 18, 1993. The decision to not submit an LER was based on two factors. The first was that originally the event appeared to affect operability of the relays with undersized coils but subiequent voltage testing verified the relays would have operated below the minimum available voltage if no additional mechanical loading had existed. Secondly, seismic qualification test results were determined to have verified that the relays with the undersized coils would have operated following a design basis earthquake.
On August 11, 1993 mechanical loading information was received from the relay manufacturer indicating that a different coil was needed if the relay was configured with 9 or more normally closed contacts. An evaluation was then conducted of the operability of the CHP relays and it was determined that the relays might not be capable of performing their intended safety function.
The resultant conclusion of both of these issues was the determination that no issues remained that would have caused the relays to be inoperable.
On August 13, 1993, further investigation revealed that similar relays were also installed in the containment high radiation (CHR) circuitry. Since CHR was required to be operational during refueling operations in accordance with Technical Specification 3.8.lc, the CHR system was declared inoperable. Refueling operations were halted until containment integrity was establ~shed. A four-hour notification was made to the NRC in accordance with 10CFR50.72(b)(2)(i). During the period from August 14 to 26 new more powerful coils were seismically qualified, installed in the affected relays, and the relays were tested. This modification affected 2 relays in the CHP circuit and 4 relays in the CHR circuit. Eleven other relays of the same model in the CHP, CHR, and SIS circuits, with 8 or less normally closed contacts remained acceptable with their original coil size.
As is discussed below, both these reasons for retracting the notification later turned out to be invalid. During a subsequent NRC inspection the seismic qualification report was reexamined.
 
This review showed that the voltage required for the test relay to operate, with 11 normally closed contacts and the original undersized coil, was higher than the minimum available voltage. This information results in a condition where the seismic test report does not prove the equipment was qualified to perform its function following a design basis earthquake.
NRC Fann 388A U.S. NUCLEAR REGULATORY COMMISSION 18-831                                                                                      APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111              DOCKET NUMBER 121                    LER NUMBER 131                          PAGE 141 SEQUENTIAL        REVISION YEAR      NUMBER            NUMBER PALISADES PLANT            olslololol2lsls            913  -  01112        -     010        Ol3      OF    017 Following further evaluation it was determined the event was not reportable and an LER in accordance with 10CFR 50.73 was therefore not made. The original notification was eventually retracted on October 18, 1993. The decision to not submit an LER was based on two factors. The first was that originally the event appeared to affect operability of the relays with undersized coils but subiequent voltage testing verified the relays would have operated below the minimum available voltage if no additional mechanical loading had existed. Secondly, seismic qualification test results were determined to have verified that the relays with the undersized coils would have operated following a design basis earthquake. The resultant conclusion of both of these issues was the determination that no issues remained that would have caused the relays to be inoperable. As is discussed below, both these reasons for retracting the notification later turned out to be invalid.
This test report covered the configuration for the modified relays. A separate seismic qualification test, with a configuration of 8 normally closed contacts, provided acceptable results for the 11 relays that were not modified.
During a subsequent NRC inspection the seismic qualification report was reexamined.
The seismic qualification test of the modified relays with a configuration of 11 NC contacts remained acceptable.
This review showed that the voltage required for the test relay to operate, with 11 normally closed contacts and the original undersized coil, was higher than the minimum available voltage. This information results in a condition where the seismic test report does not prove the equipment was qualified to perform its function following a design basis earthquake. This test report covered the configuration for the pre-modified relays. A separate seismic qualification test, with a configuration of 8 normally closed contacts, provided acceptable results for the 11 relays that were not modified. The seismic qualification test of the modified relays with a configuration of 11 NC contacts remained acceptable. As a result, on November 3, 1993, with the plant in a hot standby condition, another notification was made to the NRC Operations Center to re-identify the issue as not being resolved.
As a result, on November 3, 1993, with the plant in a hot standby condition, another notification was made to the NRC Operations Center to re-identify the issue as not being resolved.
Following the November 3, 1993 NRC notification, additional engineering review has taken place with respect to voltage drop concerns with the original relay configuration of 9 or more normally closed contacts and an undersized coil. Calculations indicate that the voltage drop between the preferred AC bus inverter and the relays may have prevented the relays from performing their function without regard to a seismic event. Operating voltages in 5 of the 6 CHP and CHR relays, in the pre-modified condition, were within one volt of the minimum available (specified) voltage from the inverter. With this effect of voltage drop considered, the first reason provided above for retracting the original notification to the NRC is also shown to be incorrect. The calculated voltage drop is 1.37 volts in the right channel and .65 volts in the left channel.
Following the November 3, 1993 NRC notification, additional engineering review has taken place with respect to voltage drop concerns with the original relay configuration of 9 or more normally closed contacts and an undersized coil. Calculations indicate that the voltage drop between the preferred AC bus inverter and the relays may have prevented the relays from performing their function without regard to a seismic event. Operating voltages in 5 of the 6 CHP and CHR relays, in the pre-modified condition, were within one volt of the minimum available (specified) voltage from the inverter.
This event is reportable to the NRC in accordance with 10CFR50.73(a)(2)(ii)(B) as a condition outside the design basis of the plant.
With this effect of voltage drop considered, the first reason provided above for retracting the original notification to the NRC is also shown to be incorrect.
CAUSE OF THE EVENT The root cause of the event is the undersized reiay coils. Other causes of the event that led to the inoperable condition are: The mechanical loading factor information was unavailable in the plant vendor files; post seismic functional testing did not specify minimum acceptable voltage; and with the undersized coils any margin between the minimum available voltage and the minimum pick-up voltage was eliminated by the circuit voltage drop.             '
The calculated voltage drop is 1.37 volts in the right channel and .65 volts in the left channel. This event is reportable to the NRC in accordance with 10CFR50.73(a)(2)(ii)(B) as a condition outside the design basis of the plant. CAUSE OF THE EVENT The root cause of the event is the undersized reiay coils. Other causes of the event that led to the inoperable condition are: The mechanical loading factor information was unavailable in the plant vendor files; post seismic functional testing did not specify minimum acceptable voltage; and with the undersized coils any margin between the minimum available voltage and the minimum pick-up voltage was eliminated by the circuit voltage drop. '
 
NRC Form 31111A 19-831 FACILITY NAME 111 PALISADES PLANT ANALYSIS OF THE EVENT LICENSEE EVENT REPORT (LERI TEXT CONTINUATION DOCKET NUMBER 121 LER NUMBER 131 U.S. NUCLEAR REGULATORY COMMISSION APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER 0 I 5 I 0 I 0 I 0 I 2 I 5 I 5 9 I 3 -0 I 1 I 2 -0 I 0 0 I 4 OF 01 7 During the performance of TSSP RO-I2 on August 4, I993, the left channel CHP test signal was initiated.
NRC Form 31111A                                                                    U.S. NUCLEAR REGULATORY COMMISSION 19-831                                                                                    APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111              DOCKET NUMBER 121                    LER NUMBER 131                        PAGE 141 SEQUENTIAL      REVISION YEAR      NUMBER          NUMBER PALISADES PLANT            0 I5 I0 I0 I0 I2 I5 I5    9  I3 -  0  I 1I  2 -      0 I  0    0 I  4    OF    01 7 ANALYSIS OF THE EVENT During the performance of TSSP RO-I2 on August 4, I993, the left channel CHP test signal was initiated. Most, but not all of the equipment went into the emergency mode of operation upon receipt of the CHP signal and remained in the emergency mode of operation.
Most, but not all of the equipment went into the emergency mode of operation upon receipt of the CHP signal and remained in the emergency mode of operation.
Contacts from two components make up the left channel seal-in circuit. These components are relay SP-I and reset push button CHP-L. Since the seal-in circuit did not function properly, the CHP-L contact was verified closed. The left channel test was then re-initiated to determine whether the equipment would operate as designed. All of the equipment changed into and remained in the emergency mode of operation upon receipt of the CHP signal.
Contacts from two components make up the left channel seal-in circuit. These components are relay SP-I and reset push button CHP-L. Since the seal-in circuit did not function properly, the CHP-L contact was verified closed. The left channel test was then initiated to determine whether the equipment would operate as designed.
The results of the failed test were reviewed and it was determined that there was only one equipment failure mode that could cause the results observed. This single equipment failure mode was the lack of relay SP-I to fully actuate. Therefore, relay SP-I was removed for inspection and bench testing. The failure was determined to be caused by two loose coil retaining clips and one loose armature pin clip that allowed coil movement and allowed the armature pin to slide partially out of the armature.
All of the equipment changed into and remained in the emergency mode of operation upon receipt of the CHP signal. The results of the failed test were reviewed and it was determined that there was only one equipment failure mode that could cause the results observed.
Concurrent with the inspection and bench testing of the originally installed SP-I relay, a relay was obtained from stock and installed in the CHP circuit.
This single equipment failure mode was the lack of relay SP-I to fully actuate. Therefore, relay SP-I was removed for inspection and bench testing. The failure was determined to be caused by two loose coil retaining clips and one loose armature pin clip that allowed coil movement and allowed the armature pin to slide partially out of the armature.
On August 6, I993, post-maintenance testing was performed on the replacement relay. The first left channel CHP actuation was completed with all equipment operating as designed.
Concurrent with the inspection and bench testing of the originally installed SP-I relay, a relay was obtained from stock and installed in the CHP circuit. On August 6, I993, post-maintenance testing was performed on the replacement relay. The first left channel CHP actuation was completed with all equipment operating as designed.
However, during the second actuation the CHP relays failed to seal-in. Relay SP-I was inspected and it was determined that the coil retaining clips and armature pin clip were properly installed, therefore, the failure mode of the replacement SP-1 relay was not the same as the failure mode of the original SP-1 relay. CHP was actuated several more times and relay SP-1 operated, albeit intermittently. All other left channel CHP relays operated properly and consistently.
However, during the second actuation the CHP relays failed to seal-in. Relay SP-I was inspected and it was determined that the coil retaining clips and armature pin clip were properly installed, therefore, the failure mode of the replacement SP-1 relay was not the same as the failure mode of the original SP-1 relay. CHP was actuated several more times and relay SP-1 operated, albeit intermittently.
The relays are Joslyn Clark model number SU-.12-76 utilizing TB113-3 coils. These relays were purchased to stock in 1987 and were bought at that time as identical replacements to the original design.
All other left channel CHP relays operated properly and consistently.
During troubleshooting of the replacement SP-1 relay, it continued to fail to fully actuate and it was observed that the armature moved only half the full travel distance and that the relay was buzzing loudly. Inspection of the relay indicated that there was inadequate clearance between the armature and the coil. This interference was found to be associated only with the replacement relay and is the apparent result of manufacturing deficiencies.
The relays are Joslyn Clark model number SU-.12-76 utilizing TB113-3 coils. These relays were purchased to stock in 1987 and were bought at that time as identical replacements to the original design. During troubleshooting of the replacement SP-1 relay, it continued to fail to fully actuate and it was observed that the armature moved only half the full travel distance and that the relay was buzzing loudly. Inspection of the relay indicated that there was inadequate clearance between the armature and the coil. This interference was found to be associated only with the replacement relay and is the apparent result of manufacturing deficiencies.
The original equipment manufacturer (Joslyn Clark) was notified of the relay failures and was requested to review the application and configuration of the aforementioned relays at Palisades. The man&#xb5;facturer supplied information that indicated that the coils .(18113-3) used in the relays at Palisades, in which the relays were configured
The original equipment manufacturer (Joslyn Clark) was notified of the relay failures and was requested to review the application and configuration of the aforementioned relays at Palisades.
 
The man&#xb5;facturer supplied information that indicated that the coils .(18113-3) used in the relays at Palisades, in which the relays were configured
NFIC Form 388A U.S. NUCLEAR FIEGULATOFIY COMMISSION 18*831                                                                                      APPROVED OMB NO. 3160-0104 EXPtFIES: 8/31 /86 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION
* NFIC Form 388A 18*831 FACILITY NAME 111 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION DOCKET NUMBER 121 LEFI NUMBER 131 U.S. NUCLEAR FIEGULATOFIY COMMISSION APPROVED OMB NO. 3160-0104 EXPtFIES:
* FACILITY NAME 111             DOCKET NUMBER 121                   LEFI NUMBER 131                           PAGE 141 SEQUENTIAL         FIEVISION YEAR       NUMBER           NUMBER PALISADES PLANT           o Is IoIoIoI2 Is Is        e 13 -     q i 12       - o 10           o 1s       oF     oI7 with greater than eight normally closed contacts, were undersized. The manufacturer recommended that when the relay model was configured with greater than eight normally closed (NC) contacts, a more powerful coil (TB113-61) should be used. The more powerful coil is necessary to overcome the increased mechanical loading introduced by the greater spring resistance opposing normally closed contact actuation. During the refueling outage, more powerful coils were replaced in the 6 relays having 9 or more NC contacts.
8/31 /86 PAGE 141 SEQUENTIAL FIEVISION YEAR NUMBER NUMBER PALISADES PLANT o I s I o I o I o I 2 I s I s e 13 -q i 12 -o 10 o 1s oF o I 7 with greater than eight normally closed contacts, were undersized.
The manufacturer recommended that when the relay model was configured with greater than eight normally closed (NC) contacts, a more powerful coil (TB113-61) should be used. The more powerful coil is necessary to overcome the increased mechanical loading introduced by the greater spring resistance opposing normally closed contact actuation.
During the refueling outage, more powerful coils were replaced in the 6 relays having 9 or more NC contacts.
All 6 relays were in the CHP or CHR circuits.
All 6 relays were in the CHP or CHR circuits.
Eleven of the same model relays that had 8 or less NC contacts retained their original TB113-3 coils. Seismic Qualification The originally installed and replacement 5P-1 relays were configured with eleven normally closed contacts and one normally open (NO) contact and a coil rated at 60 Hz, 110-120 volts AC. During the investigation of this event, it was that seismic qualification for the 5P-1 relays, and all similar type relays installed in the CHP and CHR circuits at Palisades, were seismically qualified, via testing, when purchased to stock, in 1987. After purchase, the relays replaced, on an identical replacement basis, the originally installed relays. Relay replacement occurred on a preventive maintenance activity to replace aged equipment and not as a modification to the plant. Seismic testing at that time qualified the relays in a configuration with 12 normally open contacts only. However, these relays were installed in the CHP and CHR circuits with various combinations of normally open and normally closed contacts.
Eleven of the same model relays that had 8 or less NC contacts retained their original TB113-3 coils.
Therefore, the relays were installed in the CHP and CHR circuits in a configuration which had not been seismically qualified, thus rendering the relays technically inoperable.
Seismic Qualification The originally installed and replacement 5P-1 relays were configured with eleven normally closed contacts and one normally open (NO) contact and a coil rated at 60 Hz, 110-120 volts AC. During the investigation of this event, it was det~rmined that seismic qualification for the 5P-1 relays, and all similar type relays installed in the CHP and CHR circuits at Palisades, were seismically qualified, via testing, when purchased to stock, in 1987. After purchase, the relays replaced, on an identical replacement basis, the originally installed relays. Relay replacement occurred on a preventive maintenance activity to replace aged equipment and not as a modification to the plant. Seismic testing at that time qualified the relays in a configuration with 12 normally open contacts only. However, these relays were installed in the CHP and CHR circuits with various combinations of normally open and normally closed contacts.
The relay supplier was unaware of the mechanical loading factor associated with the relay contact configuration until that information was provided to Consumers Power Co. by the relay manufacturer in August of this year. The seismic qualification testing was provided by the equipment supplier, Farwell and Hendricks, which purchased the relays as commercial grade equipment from the manufacturer, Joslyn Clark. Seismic testing was conducted in August 1993 for the relay model with 8 NC and 4 NO contacts with the original coil. To determine past operability of the relay model with the original coil size and 11 NC and 1 NO contacts additional seismic test was conducted.
Therefore, the relays were installed in the CHP and CHR circuits in a configuration which had not been seismically qualified, thus rendering the relays technically inoperable. The relay supplier was unaware of the mechanical loading factor associated with the relay contact configuration until that information was provided to Consumers Power Co. by the relay manufacturer in August of this year. The seismic qualification testing was provided by the equipment supplier, Farwell and Hendricks, which purchased the relays as commercial grade equipment from the manufacturer, Joslyn Clark.
Although the relay operated in the post test condition, the test report indicates that the measured voltage, where it picked up ten consecutive times, exceeded the minimum voltage available; 115.3 volts measured versus 114.5 volts available.
Seismic testing was conducted in August 1993 for the relay model with 8 NC and 4 NO contacts with the original coil. To determine past operability of the relay model with the original coil size and 11 NC and 1 NO contacts a~ additional seismic test was conducted. Although the relay operated in the post test condition, the test report indicates that the measured voltage, where it picked up ten consecutive times, exceeded the minimum voltage available; 115.3 volts measured versus 114.5 volts available. This minimum acceptable voltage limit was not specified in the test as an acceptance criteria and the discrepancy was not identified until an NRC inspector reviewed the report.
This minimum acceptable voltage limit was not specified in the test as an acceptance criteria and the discrepancy was not identified until an NRC inspector reviewed the report. Seismic test results for the tested Joslyn Clark 5U12-76 relay with the TB113-3 coil indicates a tendency for the minimum required pick-up voltage to increase after the relay has experienced a series of seismic tests. Minimum required pick-up voltage for 5 of the 6 relays configured with 9 or more NC contacts was within one volt of the minimum voltage available.
Seismic test results for the tested Joslyn Clark 5U12-76 relay with the TB113-3 coil indicates a tendency for the minimum required pick-up voltage to increase after the relay has experienced a series of seismic tests. Minimum required pick-up voltage for 5 of the 6 relays configured with 9 or more NC contacts was within one volt of the minimum voltage available. A seismic event could effect (raise) the minimum required pick-up voltage such that it was abov~ the minimum available voltage.
A seismic event could effect (raise) the minimum required pick-up voltage such that it was the minimum available voltage.
 
NRC Form 3118A 19*831 FACILITY NAME 111 PALISADES PLANT Voltage Drop LICENSEE EVENT REPORT (LERI TEXT CONTINUATION DOCKET NUMBER 121 YEAR LER NUMBER 131 SEQUENTIAL NUMBER U.S. NUCLEAR REGULATORY COMMISSION APPllOVED OMB NO. 3160-0104 EXPIRES: 8/31186 REVISION NUMBER PAGE 141 0 I 6 OF 01 7 Recent evaluation of the operability of the 6 relays with the original undersized TB113-3 relays and 9 or more normally closed contacts led to consideration of the circuit voltage drop from the preferred AC bus inverter to the relays. The minimum specified voltage from the inverter is 114.5 volts. This is based on the manufacturer's specified voltage output of 118 volts +/-3%. In determining the past operability of these relays the voltages required to operate the relays were in a range of 113.5 to 114.3 volts. The remaining one relay operated at 104.5 volts. Voltage drop calculations indicate worst case minimum available relay voltages between 113.1 and 113.8 volts. Based on the calculated voltage drop, the minimum pick-up voltage of 5 relays exceeded the minimum available voltage by .15 to .97 volts (.13 to .85%}. Safety Significance During a CHP event 2 of the 8 CHP relays may not have fully actuated.
NRC Form 3118A U.S. NUCLEAR REGULATORY COMMISSION 19*831                                                                                    APPllOVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111              DOCKET NUMBER 121                    LER NUMBER 131                        PAGE 141 SEQUENTIAL      REVISION YEAR      NUMBER          NUMBER PALISADES PLANT                                                                                    0 I 6   OF     01 7 Voltage Drop Recent evaluation of the operability of the 6 relays with the original undersized TB113-3 relays and 9 or more normally closed contacts led to consideration of the circuit voltage drop from the preferred AC bus inverter to the relays. The minimum specified voltage from the inverter is 114.5 volts. This is based on the manufacturer's specified voltage output of 118 volts +/-3%. In determining the past operability of these relays the voltages required to operate the relays were in a range of 113.5 to 114.3 volts.
During a CHR event 3 of the 8 CHR relays may not have fully actuated.
The remaining one relay operated at 104.5 volts. Voltage drop calculations indicate worst case minimum available relay voltages between 113.1 and 113.8 volts. Based on the calculated voltage drop, the minimum pick-up voltage of 5 relays exceeded the minimum available voltage by .15 to .97 volts (.13 to .85%}.
This would have potentially caused some equipment required to mitigate an accident to not actuate to their accident position (refer to Attachment 1}. Specifically, had the CHP relays not actuated some redundant containment isolation valves (CV-1002 and CV-1007 primary system drain tank (PSDT) outlet valves, and CV-1064 and CV-1065 clean waste receiver tank (CWST) vent valves) would not have actuated; several other containment isolation valves would not have actuated although their redundant valves would have actuated; or in some cases redundant signals to some containment isolation valves would not have actuated.
Safety Significance During a CHP event 2 of the 8 CHP relays may not have fully actuated. During a CHR event 3 of the 8 CHR relays may not have fully actuated. This would have potentially caused some equipment required to mitigate an accident to not actuate to their accident position (refer to Attachment 1}. Specifically, had the CHP relays not actuated some redundant containment isolation valves (CV-1002 and CV-1007 primary system drain tank (PSDT) outlet valves, and CV-1064 and CV-1065 clean waste receiver tank (CWST) vent valves) would not have actuated; several other containment isolation valves would not have actuated although their redundant valves would have actuated; or in some cases redundant signals to some containment isolation valves would not have actuated. The manual containment isolation actuation feature would have remained operable and the valves would have been closed early in an event by operator action as prompted by the emergency procedures for the containment isolation status check.
The manual containment isolation actuation feature would have remained operable and the valves would have been closed early in an event by operator action as prompted by the emergency procedures for the containment isolation status check. The seal in contact for the left channel CHP signal also would not have actuated (and nitrogen addition to the hydrazine tank would have not have been available.
The seal in contact for the left channel CHP signal also would not have actuated (and nitrogen addition to the hydrazine tank would have not have been available. No credit was taken for nitrogen addition which isolated during accidents. A recent amendment to the plant Technical Specifications deleted the hydrazine as being a necessary addition to the containment spray.}.
No credit was taken for nitrogen addition which isolated during accidents.
During a CHR event some redundant containment isolation valves would not have actuated had the 3 of the 8 CHR relays not fully actuated (CV-1103 and CV-1104 containment sump drain valves) (refer to Attachment 1). A prevent signal for the safeguards rooms' sump pumps would have also been disabled. Emergency Operating Procedures would direct stopping these pumps following receipt of a Recirculation Actuation Signal.
A recent amendment to the plant Technical Specifications deleted the hydrazine as being a necessary addition to the containment spray.}. During a CHR event some redundant containment isolation valves would not have actuated had the 3 of the 8 CHR relays not fully actuated (CV-1103 and CV-1104 containment sump drain valves) (refer to Attachment 1). A prevent signal for the safeguards rooms' sump pumps would have also been disabled.
Additionally the seal-in signal for the left channel CHR would not pick up.
Emergency Operating Procedures would direct stopping these pumps following receipt of a Recirculation Actuation Signal. Additionally the seal-in signal for the left channel CHR would not pick up. The result of the loss of the left channel seal-in contacts would be as follows. Three containment isolation valves would revert to their original state if the CHP signal drops out due to pressure inside containment dropping to below the setpoint.
The result of the loss of the left channel seal-in contacts would be as follows. Three containment isolation valves would revert to their original state if the CHP signal drops out due to pressure inside containment dropping to below the setpoint. The two affected flow paths would remain isolated by a check valve or a redundant valve. One of the two containment spray valves would close .. For both CHR and CHP events one train of the control room HVAC would revert to its pre-event mode, and the other train would remain in its emergency mode. The containment isolation annunciator window would indicate that a containment isolation signal no longer existed.
The two affected flow paths would remain isolated by a check valve or a redundant valve. One of the two containment spray valves would close .. For both CHR and CHP events one train of the control room HVAC would revert to its pre-event mode, and the other train would remain in its emergency mode. The containment isolation annunciator window would indicate that a containment isolation signal no longer existed.
 
* NRC Form 388A 19*831 FACILITY NAME 11! LICENSEE EVENT REPORT (LERI TEXT CONTINUATION DOCKET NUMBER 12! LER NUMBER 131 U.S. NUCLEAR REGULATORY COMMISSION APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER PALISADES PLANT 0 I 5 I 0 I 0 I 0 I 2 I 5 I 5 9 13 -0 I 1 I 2 -0 I 0 0 I 7 OF 01 7 A more likely scenario, than one in which no relay actuation occurs, would be one in which mqst of the relay actuation would occur and would result in few critical equipment failures.
NRC Form 388A                                                                       U.S. NUCLEAR REGULATORY COMMISSION 19*831                                                                                      APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION
In this scenario the relays' normally closed contacts would open when the relays were energized and the normally open contacts would not close due to the insufficient pick up voltage. This would result in operation similar to the original condition which occurred on August 4, 1993 when the seal in contact on the SP-1 relay failed to pick up but where the other relay caused actuation occurred.
* FACILITY NAME 11!              DOCKET NUMBER 12!                    LER NUMBER 131                          PAGE 141 SEQUENTIAL        REVISION YEAR      NUMBER           NUMBER PALISADES PLANT             0 I5 I0 I0 I0 I2 I5 I5    9  13 -  0 I1I    2  -      0 I 0   0 I OF     01 7 A more likely scenario, than one in which no relay actuation occurs, would be one in which mqst of the relay actuation would occur and would result in few critical equipment failures. In this scenario the relays' normally closed contacts would open when the relays were energized and the normally open contacts would not close due to the insufficient pick up voltage. This would result in operation similar to the original condition which occurred on August 4, 1993 when the seal in contact on the SP-1 relay failed to pick up but where the other relay caused actuation occurred.
CORRECTIVE ACTION Based on the information provided by the manufacturer, the coils in all the relays in the CHP and CHR circuits configured with nine or more normally closed contacts were replaced with a more powerful coils. Seismic qualification of the .relay with the more powerful coil, and with a bounding contact configuration of eleven normally closed contacts, was obtained.
CORRECTIVE ACTION Based on the information provided by the manufacturer, the coils in all the relays in the CHP and CHR circuits configured with nine or more normally closed contacts were replaced with a more powerful coils. Seismic qualification of the .relay with the more powerful coil, and with a bounding contact configuration of eleven normally closed contacts, was obtained. In addition, seismic qualification of the relay with the origin~lly installed coil, and with a contact configuration of eight normally closed contacts, was also obtained. Testing was successfully completed following the modification of the relays utilizing the more powerful coil.
In addition, seismic qualification of the relay with the installed coil, and with a contact configuration of eight normally closed contacts, was also obtained.
All Joslyn Clark relays installed in the plant have been reviewed to verify the seismic testing was representative of the installed relay condition. Existing qualification reports encompass all of these relays in safety related applications. A comparison of testing performed to the installed configuration was made and all relays are included within the bounds of the qualification testing.
Testing was successfully completed following the modification of the relays utilizing the more powerful coil. All Joslyn Clark relays installed in the plant have been reviewed to verify the seismic testing was representative of the installed relay condition.
Systems engineers and procurement engineers have been trained on the lessons learned from this event.
Existing qualification reports encompass all of these relays in safety related applications.
A Nuclear Utilities Procurement Issues Committee (NUPIC) performance based audit of Farwell and Hendricks has been on-going and will assess the generic applications of the event.
A comparison of testing performed to the installed configuration was made and all relays are included within the bounds of the qualification testing. Systems engineers and procurement engineers have been trained on the lessons learned from this event. A Nuclear Utilities Procurement Issues Committee (NUPIC) performance based audit of Farwell and Hendricks has been on-going and will assess the generic applications of the event. Design control, maintenance and procurement procedures are being reviewed for enhancements to formalize the determination or review of critical characteristics.
Design control, maintenance and procurement procedures are being reviewed for enhancements to formalize the determination or review of critical characteristics.
ADDITIONAL INFORMATION None
ADDITIONAL INFORMATION None
" ATTACHMENT 1 Consumers Power Company Palisades.Plant Docket 50-255 LICENSEE EVENT REPORT 93-012 CONTAINMENT HIGH PRESSURE {CHP) AND CONTAINMENT HIGH RADIATION
 
{CHR) RELAYS OPERABILITY November 24, 1993 4 Pages
ATTACHMENT 1 Consumers Power Company Palisades.Plant Docket 50-255 LICENSEE EVENT REPORT 93-012 CONTAINMENT HIGH PRESSURE {CHP) AND CONTAINMENT HIGH RADIATION {CHR) RELAYS OPERABILITY November 24, 1993 4 Pages
..
 
* SP-I Component CV-0155 CV-2083 CV-2009 CV-0738 CV-0771 CV-0770 CV-0939 CV-1001 CV-1002
CHP/CHR IAvs wnH 9 OR MORE NO.RMALLY cLol  CONTACTS SP-I Component     Description                Redundant/Alternate Action CV-0155       Water to quench tank      Redundant signal from 5P-2 CV-2083       PCP bleed off              CV-2099 isolates (5P-2}
* Sea 1-i n CV-1064 SV-0436A 5P-4 Component CV-0739
CV-2009      PCS letdown isol          Redundant signal from SP-2 CV-0738      S/G B recirc              Redundant signal from 5P-2 CV-0771      S/G A bottom blow          CV-0767 isolates (SP-8} ,
* CV-0501
CV-0770      S/G A bottom blow          CV-0768 isolates (SP-8}
* CV-0510 CV-1007 CV-1004 CV-1037 CV-1038 CR HVAC CV-1045 CV-1065 SV-04368 SR-1 Component CV-0155 CV-2083 CV-2009 CV-0738 CV-0771 CV-0770 CV-0939 CV-1001 CV-1002
CV-0939      Shield cool surge tank    Redundant signal from 5P-2 CV-1001      PSDT recirc                Redundant signal from SP-2 CV-1002      PSDT outlet                Closed by Cl status check
* Sea 1-i n CV-1064 CHP/CHR IAvs wnH 9 OR MORE NO.RMALLY cLol CONTACTS Description Water to quench tank PCP bleed off PCS letdown isol S/G B recirc S/G A bottom blow S/G A bottom blow Shield cool surge tank PSDT recirc PSDT outlet Seals-in left channel CWRT vent N2 to hydrazine tank Description S/G A recirc S/G B MSIV S/G A MSIV PSDT outlet CWRT inlet CWRT recirc CWRT recirc B activation CWRT pump suction CWRT vent N2 to hydrazine tank Description Water to quench tank PCP bleed off PCS letdown isol S/G B recirc S/G A bottom blow S/G A bottom blow Shield cool surge tank PSDT recirc PSDT outlet Seals-in left channel CWRT vent Redundant/Alternate Action Redundant signal from 5P-2 CV-2099 isolates (5P-2} Redundant signal from SP-2 Redundant signal from 5P-2 CV-0767 isolates (SP-8} , CV-0768 isolates (SP-8} Redundant signal from 5P-2 Redundant signal from SP-2 Closed by Cl status check CHP remains if press > 4 psig Closed by Cl status check Only opened to add N2 to tank Redundant/Alternate Action Redundant signal from SP-3 Redundant signal from SP-3 Redundant signal from SP-3 Closed by Cl status check Redundant signal from SP-3 Redundant signal from SP-3 CV-1036 isolates (SP-3} Redundant signal from 5P-7 CV-1044 isolates (5P-3) Closed by Cl status check Only opened to add N2 to tank Redundant/Alternate Action Redundant signal from 5R-2 CV-2099 isolates (5R-2} Redundant signal from SR-2 Redundant signal from 5R-2 CV-0767 isolates (5R-8} CV-0768 isolates (5R-8} Redundant signal from 5R-2 Redundant signal from 5R-2 CV-1007 isolates (5R-4} CHR remains if rad > lOR/hr CV-1065 isolates (5R-4} * = Normally Open contact 5R-4 (Low minimum pickup !tage -relay still functions)
* Sea 1-i n    Seals-in left channel      CHP remains if press > 4 psig CV-1064      CWRT vent                  Closed by Cl status check SV-0436A      N2 to hydrazine tank      Only opened to add N2 to tank 5P-4 Component    Description                Redundant/Alternate Action CV-0739      S/G A recirc              Redundant signal from SP-3
* Component CV-0739 CV-1007 CV-1004 CV-1037 CV-1038 CR HVAC CV-1045 CV-1065 5R-5 P-728 CV-1358 CV-1501 CV-1503 SV-2413A SV-24138 SV-2415A SV-24158 CV-1103 CV-1910 V-46 P-738 5R-6 CV-1104 CV-1358 SV-2412A SV-24128 SV-2414A SV-24148 CV-1502 CV-1503 CV-1102 P-72A CR HVAC
* CV-0501      S/G B MSIV                Redundant signal from SP-3
* CFMS V-46 P-73A Description S/G A recirc PSDT outlet CWRT inlet CWRT recirc CWRT recirc Train 8 activation CWRT pump suction CWRT vent East ESF rm sump pump N2 to containment Htg drain -contain Htg steam to contain Containment H2 monitor Containment H2 monitor Containment H2 monitor Containment H2 monitor Containment sump drain PCS sample Air room purge fan West ESF rm sump pump Containment sump drain N2 to containment Containment H2 monitor Containment H2 monitor Containment H2 monitor Containment H2 monitor Htg drain -contain Htg steam to contain Vent header isolation East ESF rm sump pump Train A activation Input signal Air room purge fan West ESF rm sump pump * = Normally Open contact Redundant/Alternate Action N/A . N/A N/A N/A N/A N/A N/A N/A Secured by LOCA EOP Check valve is 2nd isolation Always closed Line blanked off Redundant signal from 5R-2 Redundant signal from 5R-2 Redundant signal from SR-2 Redundant signal from SR-2 Closed by CI status check CV-1911 isolates (SR-8) Dampers isolate (SR-7, SR-8) Secured by LOCA EOP Closed by CI status check Check valve is 2nd isolation Redundant signal from SR-7 Redundant signal from SR-7 Redundant signal from SR-7 Redundant signal from SR-7 Always closed Line blanked off CV-1101 isolates (SR-3) Secured by LOCA EOP Redundant signal from SR-3 Redundant signal from SR-3 Dampers isolate (SR-7, SR-8) Secured by LOCA EOP 
* CV-0510      S/G A MSIV                Redundant signal from SP-3 CV-1007      PSDT outlet                Closed by Cl status check CV-1004      CWRT inlet                Redundant signal from SP-3 CV-1037      CWRT recirc                Redundant signal from SP-3 CV-1038      CWRT recirc               CV-1036 isolates (SP-3}
"
CR HVAC      Tra~n B activation        Redundant signal from 5P-7 CV-1045      CWRT pump suction          CV-1044 isolates (5P-3)
* Areas of Concern I. CV-1002 (SP-1) and CV-1007 (SP-4) both fail to receive an isolation signal. These valves are in the suction line to the Primary System Drain Tank pumps. The valves are only opened when necessary to recirculate the tank or drain it. This is an infrequent evolution.
CV-1065      CWRT vent                  Closed by Cl status check SV-04368      N2 to hydrazine tank      Only opened to add N2 to tank SR-1 Component    Description               Redundant/Alternate Action CV-0155      Water to quench tank       Redundant signal from 5R-2 CV-2083      PCP bleed off             CV-2099 isolates (5R-2}
The valves would be closed by the operator when he performs the Containment Isolation checklist per the Emergency Operating Procedures.
CV-2009      PCS letdown isol           Redundant signal from SR-2 CV-0738      S/G B recirc               Redundant signal from 5R-2 CV-0771      S/G A bottom blow         CV-0767 isolates (5R-8}
: 2. Seal-in signal for left channel of Containment High Pressure (SP-I) or left channel of Containment High Radiation (SR-I) would not pick up. The right channel signals would seal-in. The actuation signals would remain as long as the initiating condition existed (pressure>
CV-0770      S/G A bottom blow         CV-0768 isolates (5R-8}
4 psig, radiation
CV-0939      Shield cool surge tank     Redundant signal from 5R-2 CV-1001      PSDT recirc                Redundant signal from 5R-2 CV-1002      PSDT outlet                CV-1007 isolates (5R-4}
> IO R/hr). Also, most components actuated by CHP or CHR require an individual reset in addition to resetting the CHP/CHR signal. 3. CV-I064 (SP-I) and CV-I06S (SP-4) fail to isolate. These valves vent the Clean Waste Receiver Tanks to the main exhaust stack. In addition, the rupture disk on one of the CWRT's is removed to allow venting the containment building.
* Sea 1-i n    Seals-in left channel      CHR remains if rad > lOR/hr CV-1064      CWRT vent                  CV-1065 isolates (5R-4}
The valves would be closed by the operator while performing the Containment Isolation checklist per the Emergency Operating Procedures.
    * =  Normally Open contact
: 4. SV-0436A (SP-I) and SV-04368 (SP-4) fail to close. These valves add nitrogen to the hydrazine tank (T-I02). They are not containment isolation valves and are very rarely opened. 5. East and West Engineered Safeguards room sump pumps would remain in automatic following a Containment High Radiation (5R-5 & 5R-6). This creates a concern if leakage from Low Pressure Safety Injection pumps causes the sumps to fill after their suction has shifted to the containment sump. The handswitches are placed in OFF by EOP-4 following receipt of a Recirculation Actuation Signal (RAS). 6. CV-II03 (5R-5) and CV-II04 (5R-6) fail to isolate. These valves drain the containment sump and are infrequently opened. The valves would be closed by the operator while performing, the Containment Isolation checklist per the Emergency Operating Procedures.
 
Components which left channel seal-in is
~  5R-4 (Low minimum pickup !tage - relay still functions)  ~
: 1. CV-3001 -(CHP only) Containment spray valve will close. Containment spray is no longer needed if containment pressure drops below -4 psig. 2. CV-0701 & CV-0735 -(CHP only) Feed Regulating Valve and its bypass valve to the A Steam Generator will go to the positions that are directed by their controllers.
Component    Description                Redundant/Alternate Action CV-0739      S/G A recirc                N/A CV-1007      PSDT outlet              . N/A CV-1004      CWRT inlet                  N/A CV-1037      CWRT recirc                N/A CV-1038      CWRT recirc                N/A CR HVAC      Train 8 activation          N/A CV-1045      CWRT pump suction          N/A CV-1065      CWRT vent                  N/A 5R-5 P-728        East ESF rm sump pump      Secured by LOCA EOP CV-1358      N2 to containment          Check valve is 2nd isolation CV-1501      Htg drain - contain        Always closed CV-1503      Htg steam to contain        Line blanked off SV-2413A      Containment H2 monitor      Redundant signal from 5R-2 SV-24138      Containment H2 monitor      Redundant signal from 5R-2 SV-2415A      Containment H2 monitor      Redundant signal from SR-2 SV-24158      Containment H2 monitor      Redundant signal from SR-2 CV-1103      Containment sump drain      Closed by CI status check CV-1910      PCS sample                  CV-1911 isolates (SR-8)
Containment isolation is maintained by a check valve if still needed. 3. CV-0910 -(CHP only) Component Cooling Water valve to containment is maintained closed by redundant signal from right channel relay (5P-8). 4. CV-0911 -(CHP only) Component Cooling Water from containment is still isolated by CV-0940. 5. CV-0437A -(CHP only) Sodium Hydroxide valve to suction of left channel safeguards pumps will close. This valve is normally isolated and sodium hydroxide is only used under certain conditions as directed by the Emergency Operating Procedure.
V-46          Air room purge fan          Dampers isolate (SR-7, SR-8)
: 6. SV-0436A -(CHP only) Nitrogen supply valve to the hydrazine tank will open if its handswitch is in open. Since this valve is only opened when necessary to establish nitrogen overpressure in the tank, it is rarely open. 7. Control Room HVAC -For Containment High Radiation, 5R-4 will maintain Control Room HVAC Train B in the emergency mode of operation.
P-738        West ESF rm sump pump      Secured by LOCA EOP 5R-6 CV-1104      Containment sump drain      Closed by CI status check CV-1358      N2 to containment          Check valve is 2nd isolation SV-2412A      Containment H2 monitor      Redundant signal from SR-7 SV-24128      Containment H2 monitor      Redundant signal from SR-7 SV-2414A      Containment H2 monitor      Redundant signal from SR-7 SV-24148      Containment H2 monitor      Redundant signal from SR-7 CV-1502      Htg drain - contain        Always closed CV-1503      Htg steam to contain        Line blanked off CV-1102      Vent header isolation      CV-1101 isolates (SR-3)
Train A will revert to its pre-event mode. For Containment High Pressure, the right channel signal still remains and will maintain Control Room HVAC train A in the emergency mode, and train B will revert to its pre-event mode. 8. EK-1126 -Containment Isolation annunciator window changes from solid to slow flashing alerting the operators to the fact that a containment isolation signal no longer exists.}}
P-72A         East ESF rm sump pump      Secured by LOCA EOP CR HVAC       Train A activation          Redundant signal from SR-3
* CFMS         Input signal                Redundant signal from SR-3 V-46         Air room purge fan          Dampers isolate (SR-7, SR-8)
P-73A         West ESF rm sump pump       Secured by LOCA EOP
    * =  Normally Open contact
 
" Areas of Concern I. CV-1002 (SP-1) and CV-1007 (SP-4) both fail to receive an isolation signal. These
* valves are in the suction line to the Primary System Drain Tank pumps. The valves are only opened when necessary to recirculate the tank or drain it. This is an infrequent evolution. The valves would be closed by the operator when he performs the Containment Isolation checklist per the Emergency Operating Procedures.
: 2. Seal-in signal for left channel of Containment High Pressure (SP-I) or left channel of Containment High Radiation (SR-I) would not pick up. The right channel signals would seal-in. The actuation signals would remain as long as the initiating condition existed (pressure> 4 psig, radiation > IO R/hr). Also, most components actuated by CHP or CHR require an individual reset in addition to resetting the CHP/CHR signal.
: 3. CV-I064 (SP-I) and CV-I06S (SP-4) fail to isolate. These valves vent the Clean Waste Receiver Tanks to the main exhaust stack. In addition, the rupture disk on one of the CWRT's is removed to allow venting the containment building. The valves would be closed by the operator while performing the Containment Isolation checklist per the Emergency Operating Procedures.
: 4. SV-0436A (SP-I) and SV-04368 (SP-4) fail to close. These valves add nitrogen to the hydrazine tank (T-I02). They are not containment isolation valves and are very rarely opened.
: 5. East and West Engineered Safeguards room sump pumps would remain in automatic following a Containment High Radiation (5R-5 & 5R-6). This creates a concern if leakage from Low Pressure Safety Injection pumps causes the sumps to fill after their suction has shifted to the containment sump. The handswitches are placed in OFF by EOP-4 following receipt of a Recirculation Actuation Signal (RAS).
: 6. CV-II03 (5R-5) and CV-II04 (5R-6) fail to isolate. These valves drain the containment sump and are infrequently opened. The valves would be closed by the operator while performing, the Containment Isolation checklist per the Emergency Operating Procedures.
 
~ Components which repositi~if left channel  seal-in is lost~
: 1. CV-3001 - (CHP only) Containment spray valve will close. Containment spray is no longer needed if containment pressure drops below - 4 psig.
: 2. CV-0701 & CV-0735 - (CHP only) Feed Regulating Valve and its bypass valve to the A Steam Generator will go to the positions that are directed by their controllers.
Containment isolation is maintained by a check valve if still needed.
: 3. CV-0910 - (CHP only) Component Cooling Water valve to containment is maintained closed by redundant signal from right channel relay (5P-8).
: 4. CV-0911 - (CHP only) Component Cooling Water from containment is still isolated by CV-0940.
: 5. CV-0437A - (CHP only) Sodium Hydroxide valve to suction of left channel safeguards pumps will close. This valve is normally isolated and sodium hydroxide is only used under certain conditions as directed by the Emergency Operating Procedure.
: 6. SV-0436A - (CHP only) Nitrogen supply valve to the hydrazine tank will open if its handswitch is in open. Since this valve is only opened when necessary to establish nitrogen overpressure in the tank, it is rarely open.
: 7. Control Room HVAC - For Containment High Radiation, 5R-4 will maintain Control Room HVAC Train B in the emergency mode of operation. Train A will revert to its pre-event mode. For Containment High Pressure, the right channel signal still remains and will maintain Control Room HVAC train A in the emergency mode, and train B will revert to its pre-event mode.
: 8. EK-1126 - Containment Isolation annunciator window changes from solid to slow flashing alerting the operators to the fact that a containment isolation signal no longer exists.}}

Latest revision as of 11:59, 3 February 2020

LER 93-012-00:on 931103,concluded That Seismic Qualification Did Not Exist for Relays Previously Installed in Chp & Chr Circuits Due to Undersized Relay Coils.Undersized Coils in Relays replaced.W/931124 Ltr
ML18059A541
Person / Time
Site: Palisades Entergy icon.png
Issue date: 11/24/1993
From: Kuemin J, Slade G
CONSUMERS ENERGY CO. (FORMERLY CONSUMERS POWER CO.)
To:
NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
LER-93-012, LER-93-12, NUDOCS 9312070325
Download: ML18059A541 (13)


Text

GB Slade General Manager Palisades Nuclear Plant: 27780 Slue Star Memorial Highway, Covert, Ml 49043 November 24, 1993 Nuclear Regulatory Commission Document Control Desk Washington, DC 20555 DOCKET 50-255 ~ LICENSE DPR PALISADES PLANT -

LICENSEE EVENT REPORT 93-012 - CONTAINMENT HIGH PRESSURE (CHP) AND CONTAINMENT HIGH RADIATION (CHR) RELAYS OPERABILITY Licensee Event Report (LER)93-012 is attached. This event is reportable in accordance with 10CFR50.73(a)(2)(ii)(B) as a-condition outside the design basis of the plant.

~~'----

Gerald B Slade General Manager CC Administrator, Region III, USNRC NRC Resident Inspector - Palisades Attachment n.. ,. . 0 f\ .. >

~

,*~

9312070325 6g665~55

~DR ADOCK PDR A CiW5 ENERGY COMPANY

NRC Form 388 U.S. NUCLEAR REGULATORY COMMISSION 18*831 APPROVED OMB NO. 3150-0104 EXPIRES: 8/31185 LICENSEE EVENT REPORT !LERI FACILITY NAME 111 DOCKET NUMBER 121 PAGE 131 Palisades Plant o Is Io Io Io I2 I s Is 1 I OF o I 7 TITLE l'I CONTAINMENT HIGH PRESSURE (CHP) AND CONTAINMENT HIGH RADIATION (CHR) RELAYS OPERABILITY EVENT DATE 151 LER NUMBER 181 REPORT DATE 181 OTHER FACILITIES INVOLVED 181 SEQUENTIAi. REVISION FACILITY NAMES MONTH DAY YEAR YEAR NUMBER NUMBER MONTH DAY YEAR N/A ol51o!o!ol I

i I i 013 9 3 913 o 11 I2 0 10 i I i 2 14 9 13 N/A ol51ololol I THIS REPORT IS SUBMITTED PURSUANT TO THE REQUIREMENTS OF 10 CFR l: /Ch<<:k one°'"'°"' of ttt. following/ 1111 OPERATING POWER LEVEL 1101 MODE 181 I 01 01 N

o-20.4021bl 20.4051*111 llil 20.4051*111 lliil 20.405Call1 lliiil 20.4051cl 50.381cll11 50.38Ccll21 50.73Call21ftl 50.731all211ivl 50.731all21M 50.73Call211viil 50.7 3Call211viiillAI 73.711bl 73.711cl OTHER ISpacify in Abatract below end in Text,

... *. 20.4051all1 llivl x

....... 50.731*1121Ciil

._ 50. 7 3 la112lCviiillBI NRC Form 388AI 20.405Call11M 50.731*112lliiil 50.73Call21Cxl LICENSEE CONTACT FOR THIS LER 1121 NAME TELEPHONE NUMBER James L. Kuemin, Licensing Administrator sARIEA1cl°~ I 11sl41 *IaIsI , I 3 COMPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRIBED IN THIS REPORT 1131 MANUFAC* REPORTABLE MANUFAC* REPORTABLE CAUSE SYSTEM COMPONENT TUR ER TO NPRDS CAUSE SYSTEM COMPONENT TUR ER TO NPRDS I I I I I I I I< I I I I I I I I I I I I I I I I I I I I I SUPPLEMENT Al REPORT EXPECTED Cl 41 MONTH DAY YEAR EXPECTED

- - - , YES Vt Y*** campier. EXPECTED SUBMISSION DATE! hi NO ABSTRACT UJmit to 1400 - * *

  • I.e., -imetely fiftHn aifl(IM--* ry,,.writr.n line*I 1181 SUBMISSION DATE 1151 I I I On November 3, 1993, with the pl ant in a hot standby condjtion, it was concluded that seismic qualification did not exist for relays that had been previously i nsta 11 ed in the CHP and CHR circuits. During the refueling outage these relays were determined to have undersized coil s and were therefore modified. Seismic testing of the relays in their original configuration did not prove they could operate following a design basis earthquake. Subsequent evaluation has also determined that no consideration was made relative to the relay circuit voltage drop, which when taken into account would result in a condition in which the pre-modified relays might not meet their design function.

The root cause of the event is the undersized relay coils. These are original pl ant design deficiencies.

Corrective action for this event includes seismically qualifying the relays consistent with the installed configuration, replacing the undersized coils in the relays, evaluating other similar relays, evaluating the effects of voltage drop in the relay circuits, updating equipment control documents, ~nd ensuring staff and procedures are knowledgeable and reflect the lessons learned from this event.

NRC Form 388A U.S. NUCLEAR REGULATORY COMMISSION (9-831 APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111 DOCKET NUMBER 121 LER NUMBER 131 PAGE 141 SEQUENTIAL I I REVISION YEAR NUMBER NUMBER PALISADES PLANT ol5lololol21515le 13 01 i I 2 I - I o I o 01 2 I I 0I OF 7 EVENT DESCRIPTION On August 4, 1993 at 0350 hours0.00405 days <br />0.0972 hours <br />5.787037e-4 weeks <br />1.33175e-4 months <br />, with the plant in cold shutdown for refueling, Technical Specifications Surveillance Test (TSSP) R0-12, "Containment High Pressure (CHP) and Spray System Tests," was being performed. TSSP R0-12 verifies CHP circuitry initiates the Safety Injection, Containment Isolation, emergency operation of the Control Room Ventilation, Iodine Removal, and Containment Spray Systems. During the test of the left channel of the CHP circuitry, not all of the equipment required to actuate to the emergency mode of operation responded to the CHP signal. TSSP R0-12 was aborted and troubleshooting for the cause of the test failure was initiated. It was subsequently determined that a relay [JE;RLY] (5P-1) in the l~ft channel of the CHP circuitry failed to fully actuate. Visual examination of the 5P-1 relay showed that the armature pin clip of the relay coil and the coil retaining clips were loose. Similar relays in the CHP and CHR circuits were inspected and no other loose armature pin clips or loose coil retaining clips were found. It was concluded that the loose armature pin clip and the loose coil retaining clips on the 5P-1 relay was an isolated case resulting from improper assembly of the relay. Concurrent with an evaluation of the failure, 5P-l relay was replaced with an identi~al in-stock spare on August 5, 1993 and post-maintenance testing of the left channel of CHP, using TSSP R0-12, was initiated on August 6, 1993.

During the post-maintenance testing of the CHP circuit a left channel actuation was completed satisfactorily with all equipment responding to the emergency mode; however, during a second test of the left channel of CHP, the CHP relays again failed to seal-in.

Troubleshooting determined that a problem existed with the replacement CHP relay (5P-1) and that the problem was inadequate clearance between the armature and the coil.

Further investigation indicated that in certain orientations the armature struck the coil which stopped the travel of the armature thereby not allowing the relay to fully actuate.

On August 11, 1993 mechanical loading information was received from the relay manufacturer indicating that a different coil was needed if the relay was configured with 9 or more normally closed contacts. An evaluation was then conducted of the operability of the CHP relays and it was determined that the relays might not be capable of performing their intended safety function.

On August 13, 1993, further investigation revealed that similar relays were also installed in the containment high radiation (CHR) circuitry. Since CHR was required to be operational during refueling operations in accordance with Technical Specification 3.8.lc, the CHR system was declared inoperable. Refueling operations were halted until containment integrity was establ~shed. A four-hour notification was made to the NRC in accordance with 10CFR50.72(b)(2)(i). During the period from August 14 to 26 new more powerful coils were seismically qualified, installed in the affected relays, and the relays were tested. This modification affected 2 relays in the CHP circuit and 4 relays in the CHR circuit. Eleven other relays of the same model in the CHP, CHR, and SIS circuits, with 8 or less normally closed contacts remained acceptable with their original coil size.

NRC Fann 388A U.S. NUCLEAR REGULATORY COMMISSION 18-831 APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111 DOCKET NUMBER 121 LER NUMBER 131 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER PALISADES PLANT olslololol2lsls 913 - 01112 - 010 Ol3 OF 017 Following further evaluation it was determined the event was not reportable and an LER in accordance with 10CFR 50.73 was therefore not made. The original notification was eventually retracted on October 18, 1993. The decision to not submit an LER was based on two factors. The first was that originally the event appeared to affect operability of the relays with undersized coils but subiequent voltage testing verified the relays would have operated below the minimum available voltage if no additional mechanical loading had existed. Secondly, seismic qualification test results were determined to have verified that the relays with the undersized coils would have operated following a design basis earthquake. The resultant conclusion of both of these issues was the determination that no issues remained that would have caused the relays to be inoperable. As is discussed below, both these reasons for retracting the notification later turned out to be invalid.

During a subsequent NRC inspection the seismic qualification report was reexamined.

This review showed that the voltage required for the test relay to operate, with 11 normally closed contacts and the original undersized coil, was higher than the minimum available voltage. This information results in a condition where the seismic test report does not prove the equipment was qualified to perform its function following a design basis earthquake. This test report covered the configuration for the pre-modified relays. A separate seismic qualification test, with a configuration of 8 normally closed contacts, provided acceptable results for the 11 relays that were not modified. The seismic qualification test of the modified relays with a configuration of 11 NC contacts remained acceptable. As a result, on November 3, 1993, with the plant in a hot standby condition, another notification was made to the NRC Operations Center to re-identify the issue as not being resolved.

Following the November 3, 1993 NRC notification, additional engineering review has taken place with respect to voltage drop concerns with the original relay configuration of 9 or more normally closed contacts and an undersized coil. Calculations indicate that the voltage drop between the preferred AC bus inverter and the relays may have prevented the relays from performing their function without regard to a seismic event. Operating voltages in 5 of the 6 CHP and CHR relays, in the pre-modified condition, were within one volt of the minimum available (specified) voltage from the inverter. With this effect of voltage drop considered, the first reason provided above for retracting the original notification to the NRC is also shown to be incorrect. The calculated voltage drop is 1.37 volts in the right channel and .65 volts in the left channel.

This event is reportable to the NRC in accordance with 10CFR50.73(a)(2)(ii)(B) as a condition outside the design basis of the plant.

CAUSE OF THE EVENT The root cause of the event is the undersized reiay coils. Other causes of the event that led to the inoperable condition are: The mechanical loading factor information was unavailable in the plant vendor files; post seismic functional testing did not specify minimum acceptable voltage; and with the undersized coils any margin between the minimum available voltage and the minimum pick-up voltage was eliminated by the circuit voltage drop. '

NRC Form 31111A U.S. NUCLEAR REGULATORY COMMISSION 19-831 APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111 DOCKET NUMBER 121 LER NUMBER 131 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER PALISADES PLANT 0 I5 I0 I0 I0 I2 I5 I5 9 I3 - 0 I 1I 2 - 0 I 0 0 I 4 OF 01 7 ANALYSIS OF THE EVENT During the performance of TSSP RO-I2 on August 4, I993, the left channel CHP test signal was initiated. Most, but not all of the equipment went into the emergency mode of operation upon receipt of the CHP signal and remained in the emergency mode of operation.

Contacts from two components make up the left channel seal-in circuit. These components are relay SP-I and reset push button CHP-L. Since the seal-in circuit did not function properly, the CHP-L contact was verified closed. The left channel test was then re-initiated to determine whether the equipment would operate as designed. All of the equipment changed into and remained in the emergency mode of operation upon receipt of the CHP signal.

The results of the failed test were reviewed and it was determined that there was only one equipment failure mode that could cause the results observed. This single equipment failure mode was the lack of relay SP-I to fully actuate. Therefore, relay SP-I was removed for inspection and bench testing. The failure was determined to be caused by two loose coil retaining clips and one loose armature pin clip that allowed coil movement and allowed the armature pin to slide partially out of the armature.

Concurrent with the inspection and bench testing of the originally installed SP-I relay, a relay was obtained from stock and installed in the CHP circuit.

On August 6, I993, post-maintenance testing was performed on the replacement relay. The first left channel CHP actuation was completed with all equipment operating as designed.

However, during the second actuation the CHP relays failed to seal-in. Relay SP-I was inspected and it was determined that the coil retaining clips and armature pin clip were properly installed, therefore, the failure mode of the replacement SP-1 relay was not the same as the failure mode of the original SP-1 relay. CHP was actuated several more times and relay SP-1 operated, albeit intermittently. All other left channel CHP relays operated properly and consistently.

The relays are Joslyn Clark model number SU-.12-76 utilizing TB113-3 coils. These relays were purchased to stock in 1987 and were bought at that time as identical replacements to the original design.

During troubleshooting of the replacement SP-1 relay, it continued to fail to fully actuate and it was observed that the armature moved only half the full travel distance and that the relay was buzzing loudly. Inspection of the relay indicated that there was inadequate clearance between the armature and the coil. This interference was found to be associated only with the replacement relay and is the apparent result of manufacturing deficiencies.

The original equipment manufacturer (Joslyn Clark) was notified of the relay failures and was requested to review the application and configuration of the aforementioned relays at Palisades. The manµfacturer supplied information that indicated that the coils .(18113-3) used in the relays at Palisades, in which the relays were configured

NFIC Form 388A U.S. NUCLEAR FIEGULATOFIY COMMISSION 18*831 APPROVED OMB NO. 3160-0104 EXPtFIES: 8/31 /86 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION

  • FACILITY NAME 111 DOCKET NUMBER 121 LEFI NUMBER 131 PAGE 141 SEQUENTIAL FIEVISION YEAR NUMBER NUMBER PALISADES PLANT o Is IoIoIoI2 Is Is e 13 - q i 12 - o 10 o 1s oF oI7 with greater than eight normally closed contacts, were undersized. The manufacturer recommended that when the relay model was configured with greater than eight normally closed (NC) contacts, a more powerful coil (TB113-61) should be used. The more powerful coil is necessary to overcome the increased mechanical loading introduced by the greater spring resistance opposing normally closed contact actuation. During the refueling outage, more powerful coils were replaced in the 6 relays having 9 or more NC contacts.

All 6 relays were in the CHP or CHR circuits.

Eleven of the same model relays that had 8 or less NC contacts retained their original TB113-3 coils.

Seismic Qualification The originally installed and replacement 5P-1 relays were configured with eleven normally closed contacts and one normally open (NO) contact and a coil rated at 60 Hz, 110-120 volts AC. During the investigation of this event, it was det~rmined that seismic qualification for the 5P-1 relays, and all similar type relays installed in the CHP and CHR circuits at Palisades, were seismically qualified, via testing, when purchased to stock, in 1987. After purchase, the relays replaced, on an identical replacement basis, the originally installed relays. Relay replacement occurred on a preventive maintenance activity to replace aged equipment and not as a modification to the plant. Seismic testing at that time qualified the relays in a configuration with 12 normally open contacts only. However, these relays were installed in the CHP and CHR circuits with various combinations of normally open and normally closed contacts.

Therefore, the relays were installed in the CHP and CHR circuits in a configuration which had not been seismically qualified, thus rendering the relays technically inoperable. The relay supplier was unaware of the mechanical loading factor associated with the relay contact configuration until that information was provided to Consumers Power Co. by the relay manufacturer in August of this year. The seismic qualification testing was provided by the equipment supplier, Farwell and Hendricks, which purchased the relays as commercial grade equipment from the manufacturer, Joslyn Clark.

Seismic testing was conducted in August 1993 for the relay model with 8 NC and 4 NO contacts with the original coil. To determine past operability of the relay model with the original coil size and 11 NC and 1 NO contacts a~ additional seismic test was conducted. Although the relay operated in the post test condition, the test report indicates that the measured voltage, where it picked up ten consecutive times, exceeded the minimum voltage available; 115.3 volts measured versus 114.5 volts available. This minimum acceptable voltage limit was not specified in the test as an acceptance criteria and the discrepancy was not identified until an NRC inspector reviewed the report.

Seismic test results for the tested Joslyn Clark 5U12-76 relay with the TB113-3 coil indicates a tendency for the minimum required pick-up voltage to increase after the relay has experienced a series of seismic tests. Minimum required pick-up voltage for 5 of the 6 relays configured with 9 or more NC contacts was within one volt of the minimum voltage available. A seismic event could effect (raise) the minimum required pick-up voltage such that it was abov~ the minimum available voltage.

NRC Form 3118A U.S. NUCLEAR REGULATORY COMMISSION 19*831 APPllOVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION FACILITY NAME 111 DOCKET NUMBER 121 LER NUMBER 131 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER PALISADES PLANT 0 I 6 OF 01 7 Voltage Drop Recent evaluation of the operability of the 6 relays with the original undersized TB113-3 relays and 9 or more normally closed contacts led to consideration of the circuit voltage drop from the preferred AC bus inverter to the relays. The minimum specified voltage from the inverter is 114.5 volts. This is based on the manufacturer's specified voltage output of 118 volts +/-3%. In determining the past operability of these relays the voltages required to operate the relays were in a range of 113.5 to 114.3 volts.

The remaining one relay operated at 104.5 volts. Voltage drop calculations indicate worst case minimum available relay voltages between 113.1 and 113.8 volts. Based on the calculated voltage drop, the minimum pick-up voltage of 5 relays exceeded the minimum available voltage by .15 to .97 volts (.13 to .85%}.

Safety Significance During a CHP event 2 of the 8 CHP relays may not have fully actuated. During a CHR event 3 of the 8 CHR relays may not have fully actuated. This would have potentially caused some equipment required to mitigate an accident to not actuate to their accident position (refer to Attachment 1}. Specifically, had the CHP relays not actuated some redundant containment isolation valves (CV-1002 and CV-1007 primary system drain tank (PSDT) outlet valves, and CV-1064 and CV-1065 clean waste receiver tank (CWST) vent valves) would not have actuated; several other containment isolation valves would not have actuated although their redundant valves would have actuated; or in some cases redundant signals to some containment isolation valves would not have actuated. The manual containment isolation actuation feature would have remained operable and the valves would have been closed early in an event by operator action as prompted by the emergency procedures for the containment isolation status check.

The seal in contact for the left channel CHP signal also would not have actuated (and nitrogen addition to the hydrazine tank would have not have been available. No credit was taken for nitrogen addition which isolated during accidents. A recent amendment to the plant Technical Specifications deleted the hydrazine as being a necessary addition to the containment spray.}.

During a CHR event some redundant containment isolation valves would not have actuated had the 3 of the 8 CHR relays not fully actuated (CV-1103 and CV-1104 containment sump drain valves) (refer to Attachment 1). A prevent signal for the safeguards rooms' sump pumps would have also been disabled. Emergency Operating Procedures would direct stopping these pumps following receipt of a Recirculation Actuation Signal.

Additionally the seal-in signal for the left channel CHR would not pick up.

The result of the loss of the left channel seal-in contacts would be as follows. Three containment isolation valves would revert to their original state if the CHP signal drops out due to pressure inside containment dropping to below the setpoint. The two affected flow paths would remain isolated by a check valve or a redundant valve. One of the two containment spray valves would close .. For both CHR and CHP events one train of the control room HVAC would revert to its pre-event mode, and the other train would remain in its emergency mode. The containment isolation annunciator window would indicate that a containment isolation signal no longer existed.

NRC Form 388A U.S. NUCLEAR REGULATORY COMMISSION 19*831 APPROVED OMB NO. 3160-0104 EXPIRES: 8/31186 LICENSEE EVENT REPORT (LERI TEXT CONTINUATION

  • FACILITY NAME 11! DOCKET NUMBER 12! LER NUMBER 131 PAGE 141 SEQUENTIAL REVISION YEAR NUMBER NUMBER PALISADES PLANT 0 I5 I0 I0 I0 I2 I5 I5 9 13 - 0 I1I 2 - 0 I 0 0 I 7 OF 01 7 A more likely scenario, than one in which no relay actuation occurs, would be one in which mqst of the relay actuation would occur and would result in few critical equipment failures. In this scenario the relays' normally closed contacts would open when the relays were energized and the normally open contacts would not close due to the insufficient pick up voltage. This would result in operation similar to the original condition which occurred on August 4, 1993 when the seal in contact on the SP-1 relay failed to pick up but where the other relay caused actuation occurred.

CORRECTIVE ACTION Based on the information provided by the manufacturer, the coils in all the relays in the CHP and CHR circuits configured with nine or more normally closed contacts were replaced with a more powerful coils. Seismic qualification of the .relay with the more powerful coil, and with a bounding contact configuration of eleven normally closed contacts, was obtained. In addition, seismic qualification of the relay with the origin~lly installed coil, and with a contact configuration of eight normally closed contacts, was also obtained. Testing was successfully completed following the modification of the relays utilizing the more powerful coil.

All Joslyn Clark relays installed in the plant have been reviewed to verify the seismic testing was representative of the installed relay condition. Existing qualification reports encompass all of these relays in safety related applications. A comparison of testing performed to the installed configuration was made and all relays are included within the bounds of the qualification testing.

Systems engineers and procurement engineers have been trained on the lessons learned from this event.

A Nuclear Utilities Procurement Issues Committee (NUPIC) performance based audit of Farwell and Hendricks has been on-going and will assess the generic applications of the event.

Design control, maintenance and procurement procedures are being reviewed for enhancements to formalize the determination or review of critical characteristics.

ADDITIONAL INFORMATION None

ATTACHMENT 1 Consumers Power Company Palisades.Plant Docket 50-255 LICENSEE EVENT REPORT 93-012 CONTAINMENT HIGH PRESSURE {CHP) AND CONTAINMENT HIGH RADIATION {CHR) RELAYS OPERABILITY November 24, 1993 4 Pages

CHP/CHR IAvs wnH 9 OR MORE NO.RMALLY cLol CONTACTS SP-I Component Description Redundant/Alternate Action CV-0155 Water to quench tank Redundant signal from 5P-2 CV-2083 PCP bleed off CV-2099 isolates (5P-2}

CV-2009 PCS letdown isol Redundant signal from SP-2 CV-0738 S/G B recirc Redundant signal from 5P-2 CV-0771 S/G A bottom blow CV-0767 isolates (SP-8} ,

CV-0770 S/G A bottom blow CV-0768 isolates (SP-8}

CV-0939 Shield cool surge tank Redundant signal from 5P-2 CV-1001 PSDT recirc Redundant signal from SP-2 CV-1002 PSDT outlet Closed by Cl status check

  • Sea 1-i n Seals-in left channel CHP remains if press > 4 psig CV-1064 CWRT vent Closed by Cl status check SV-0436A N2 to hydrazine tank Only opened to add N2 to tank 5P-4 Component Description Redundant/Alternate Action CV-0739 S/G A recirc Redundant signal from SP-3
  • CV-0501 S/G B MSIV Redundant signal from SP-3
  • CV-0510 S/G A MSIV Redundant signal from SP-3 CV-1007 PSDT outlet Closed by Cl status check CV-1004 CWRT inlet Redundant signal from SP-3 CV-1037 CWRT recirc Redundant signal from SP-3 CV-1038 CWRT recirc CV-1036 isolates (SP-3}

CR HVAC Tra~n B activation Redundant signal from 5P-7 CV-1045 CWRT pump suction CV-1044 isolates (5P-3)

CV-1065 CWRT vent Closed by Cl status check SV-04368 N2 to hydrazine tank Only opened to add N2 to tank SR-1 Component Description Redundant/Alternate Action CV-0155 Water to quench tank Redundant signal from 5R-2 CV-2083 PCP bleed off CV-2099 isolates (5R-2}

CV-2009 PCS letdown isol Redundant signal from SR-2 CV-0738 S/G B recirc Redundant signal from 5R-2 CV-0771 S/G A bottom blow CV-0767 isolates (5R-8}

CV-0770 S/G A bottom blow CV-0768 isolates (5R-8}

CV-0939 Shield cool surge tank Redundant signal from 5R-2 CV-1001 PSDT recirc Redundant signal from 5R-2 CV-1002 PSDT outlet CV-1007 isolates (5R-4}

  • Sea 1-i n Seals-in left channel CHR remains if rad > lOR/hr CV-1064 CWRT vent CV-1065 isolates (5R-4}
  • = Normally Open contact

~ 5R-4 (Low minimum pickup !tage - relay still functions) ~

Component Description Redundant/Alternate Action CV-0739 S/G A recirc N/A CV-1007 PSDT outlet . N/A CV-1004 CWRT inlet N/A CV-1037 CWRT recirc N/A CV-1038 CWRT recirc N/A CR HVAC Train 8 activation N/A CV-1045 CWRT pump suction N/A CV-1065 CWRT vent N/A 5R-5 P-728 East ESF rm sump pump Secured by LOCA EOP CV-1358 N2 to containment Check valve is 2nd isolation CV-1501 Htg drain - contain Always closed CV-1503 Htg steam to contain Line blanked off SV-2413A Containment H2 monitor Redundant signal from 5R-2 SV-24138 Containment H2 monitor Redundant signal from 5R-2 SV-2415A Containment H2 monitor Redundant signal from SR-2 SV-24158 Containment H2 monitor Redundant signal from SR-2 CV-1103 Containment sump drain Closed by CI status check CV-1910 PCS sample CV-1911 isolates (SR-8)

V-46 Air room purge fan Dampers isolate (SR-7, SR-8)

P-738 West ESF rm sump pump Secured by LOCA EOP 5R-6 CV-1104 Containment sump drain Closed by CI status check CV-1358 N2 to containment Check valve is 2nd isolation SV-2412A Containment H2 monitor Redundant signal from SR-7 SV-24128 Containment H2 monitor Redundant signal from SR-7 SV-2414A Containment H2 monitor Redundant signal from SR-7 SV-24148 Containment H2 monitor Redundant signal from SR-7 CV-1502 Htg drain - contain Always closed CV-1503 Htg steam to contain Line blanked off CV-1102 Vent header isolation CV-1101 isolates (SR-3)

P-72A East ESF rm sump pump Secured by LOCA EOP CR HVAC Train A activation Redundant signal from SR-3

  • CFMS Input signal Redundant signal from SR-3 V-46 Air room purge fan Dampers isolate (SR-7, SR-8)

P-73A West ESF rm sump pump Secured by LOCA EOP

  • = Normally Open contact

" Areas of Concern I. CV-1002 (SP-1) and CV-1007 (SP-4) both fail to receive an isolation signal. These

  • valves are in the suction line to the Primary System Drain Tank pumps. The valves are only opened when necessary to recirculate the tank or drain it. This is an infrequent evolution. The valves would be closed by the operator when he performs the Containment Isolation checklist per the Emergency Operating Procedures.
2. Seal-in signal for left channel of Containment High Pressure (SP-I) or left channel of Containment High Radiation (SR-I) would not pick up. The right channel signals would seal-in. The actuation signals would remain as long as the initiating condition existed (pressure> 4 psig, radiation > IO R/hr). Also, most components actuated by CHP or CHR require an individual reset in addition to resetting the CHP/CHR signal.
3. CV-I064 (SP-I) and CV-I06S (SP-4) fail to isolate. These valves vent the Clean Waste Receiver Tanks to the main exhaust stack. In addition, the rupture disk on one of the CWRT's is removed to allow venting the containment building. The valves would be closed by the operator while performing the Containment Isolation checklist per the Emergency Operating Procedures.
4. SV-0436A (SP-I) and SV-04368 (SP-4) fail to close. These valves add nitrogen to the hydrazine tank (T-I02). They are not containment isolation valves and are very rarely opened.
5. East and West Engineered Safeguards room sump pumps would remain in automatic following a Containment High Radiation (5R-5 & 5R-6). This creates a concern if leakage from Low Pressure Safety Injection pumps causes the sumps to fill after their suction has shifted to the containment sump. The handswitches are placed in OFF by EOP-4 following receipt of a Recirculation Actuation Signal (RAS).
6. CV-II03 (5R-5) and CV-II04 (5R-6) fail to isolate. These valves drain the containment sump and are infrequently opened. The valves would be closed by the operator while performing, the Containment Isolation checklist per the Emergency Operating Procedures.

~ Components which repositi~if left channel seal-in is lost~

1. CV-3001 - (CHP only) Containment spray valve will close. Containment spray is no longer needed if containment pressure drops below - 4 psig.
2. CV-0701 & CV-0735 - (CHP only) Feed Regulating Valve and its bypass valve to the A Steam Generator will go to the positions that are directed by their controllers.

Containment isolation is maintained by a check valve if still needed.

3. CV-0910 - (CHP only) Component Cooling Water valve to containment is maintained closed by redundant signal from right channel relay (5P-8).
4. CV-0911 - (CHP only) Component Cooling Water from containment is still isolated by CV-0940.
5. CV-0437A - (CHP only) Sodium Hydroxide valve to suction of left channel safeguards pumps will close. This valve is normally isolated and sodium hydroxide is only used under certain conditions as directed by the Emergency Operating Procedure.
6. SV-0436A - (CHP only) Nitrogen supply valve to the hydrazine tank will open if its handswitch is in open. Since this valve is only opened when necessary to establish nitrogen overpressure in the tank, it is rarely open.
7. Control Room HVAC - For Containment High Radiation, 5R-4 will maintain Control Room HVAC Train B in the emergency mode of operation. Train A will revert to its pre-event mode. For Containment High Pressure, the right channel signal still remains and will maintain Control Room HVAC train A in the emergency mode, and train B will revert to its pre-event mode.
8. EK-1126 - Containment Isolation annunciator window changes from solid to slow flashing alerting the operators to the fact that a containment isolation signal no longer exists.