ML20092J445
| ML20092J445 | |
| Person / Time | |
|---|---|
| Site: | Peach Bottom  |
| Issue date: | 09/18/1995 |
| From: | Rainey G PECO ENERGY CO., (FORMERLY PHILADELPHIA ELECTRIC |
| To: | NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM) |
| References | |
| NUDOCS 9509220136 | |
| Download: ML20092J445 (7) | |
Text
.
Pe ch o Atomic Power Station 1
A g-1 PECO ENERGY
";"L"T "**""
Delta. PA 17314-9739 i-'
717 456 7014 September 18, 1995 Docket Nos. 50-277 i
50-278 License Nos. DPR-44 DPR-56 j
s U. S. Nuclear Regulatory Commission Attn: ' Document Control Desk -
Washington, DC 20555 Subjecti Peach Bottom Atomic Power Station Units 2 & 3 Response to Notice of Violation (Combined Inspection Report No.
4 50-277/95-11 & 50-278/95-11) i t
Gentlemen:
in response to your letter dated August 17,1995, which transmitted the Notice of i
Violation concerning the referenced inspection report, we submit the attached response. The subject report concerned a Routine Resident Safety inspection that was conducted April 23 through June 24, 1995, and a Predecisional Enforcement Conference that was conducted August 2,1995.
If you have any questions or desire additional information, do not hesitate to contact us.
1 t
/ W i-raid R. Rainey Vice President Peach Bottom Atomic Power Station
. Attachment cc:
R. A. Burricelli, Public Service Electric & Gas R. R. Janati, Commonwealth of Pennsylania T. T. Martin, US NRC, Administrator, Region 1 W. L. Schmidt, US NRC, Senior Resident inspector H. C.- Schwemm, VP - Atlantic Electric R.1.' McLean, State of Maryland
. A. F. Kirby lil, DelMarVa Power j
MOU2d 9509220136 950918 So PDR ADOCK
./.
}l a _ _ _'05000277 eDn_
l i
(
RESPONSE TO NOTICE OF VIOLATION
- Restatement of Violation 10 CFR Part 50, Appendix B, Criterion lil, Design Control, requires in part, that measures
- shall be' established to assure that applicable regulatory requirements and the design-basis, as defined in 10 CFR 50.2 and as specified in the license application, for those 1
structures, systems, and components to which this appendix applies, are correctly
)
translated into specifications, drawings, procedures and instructions. The design control l
' measures shall provide for verifying and checking the adequacy of design, such as by
.the performance of design reviews, by the use of alternate or simplified calculational i
~ methods, or by the performance of a suitable testing program.
j Technical Specification (TS) 3.9.B.3 allows one Emergency Diesel Generator (EDG) to l
Lbe inoperable for a maximum of 7 days. TS Limiting Conditions for Operation (LCOs)
. 3.O.C and 3.O.D require that both units be placed in hot shutdown within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> when '
.2 EDGs are not available in order to demonstrate adequate core cooling following a j
a design basis accident.
l Contrary to the above, in June 1995, design controls for modification to two of the EDGs did not assure that the appropriate design basis was appropriately translated into drawings, nor provide for adequate verification and checking of the design, in that (1) j there were inaccurate drawings of the diesel control circuit which 'ed to a design that i
i would not perform its function in that the output breakers would not close automatically; l
(2) design reviews and post installation testing did not identify and correct the design l
error or fully verify the design adequacy; and (3) modification testing for the E-2 EDG l
was insufficient in that it did not identify the design error and also led to a loss of a 4kV bus and a small reactor power excursion. As a result of this error, the E-2 EDG was inoperable from June 4,1995 to June 21,1995, (16.5 days), and the E-2 EDG and E-4 l
EDG were both inoperable from June 18,1995 to June 21,1995 (2.5 days), which is in
~
excess of TS 3.9 B.3., and TS LCOs 3.O.C. and 3.O.D. respectively.
1 i
This is a Severity Level Ill violation (Supplement 1).
1-a Y:
i i
i f
i
+
n i
i
I Backaround i
On June 4,1995, the E-2 Diesel Generator (D/G) was removed from service for a i
planned maintenance outage and installation of Modification (Mod) P-00231, Diesel
' Generator Enhancements. While performing the modification acceptance test (MAT) on June 10,1995, the E-2 D/G outlet breaker unexpectedly tripped open which resulted in the loss of the E-22 4kV bus. It was determined that the revision of the routine test (RT) l l
associated with the modification was inadequate because the specified testing sequence resulted in the inadvertent energization of the newly added D/G output breaker trip relay, t
The E-2 D/G modification was re-designed and the RT was revised so that routine testing would not result in a D/G output breaker trip. Following successful completion of the MAT, the E-2 D/G was declared operable on June 11,1995.
{
i On June 18,1995, the E-4 D/G was removed from service for a planned maintenance outage and installation of Mod P-00231. During the performance of a relay calibration i
i on the E-4 D/G on June 20,1995, an Instrument and Control (l&C) technician questioned i
. the wiring configuration on the 59G relay. It was determined that the E4 59G relay wiring had been changed during Mod P-00231 installation and the wiring configuration would prevent the E-4 D/G Output Breaker from automatically closing on a dead bus condition.
Because the E-2 D/G had recently been modified, plant staff questioned whether the I
wiring configuration for the E-2 D/G 59G relay could also be incorrect. The modification had not been installed on the E-1 or E-3 D/Gs.
A Troubleshooting / Minor Maintenance Test (TMT) was developed and initiated to determine the status of the E-2 D/G. At 01:38 a.m. on June 21,1995, the results of the TMT indicated that the same wiring discrepancy existed on the E-2 D/G and that it would also prevent the D/G Output Breaker from closing on a dead bus condition. The E-2 D/G was declared ' inoperable and TS Limiting Conditions for Operation (LCO) action j
statements 3.O.C and 3.O.D were entered for having two D/Gs inoperable (E-2 and E-4 j
j D/Gs). TS required that both units be shutdown in six hours. During this time, if a Loss j
of Offsite Power would have occurred, the E-2 D/G Output Breaker would have been i
I manually closed by Operations personnel to energize the dead bus as required by approved plant procedures. In addition, adequate core cooling would also have been j
available from core spray pumps powered by the E-1 and E-3 D/Gs in the event of a l
l design basis loss of cooling accident (LOCA).
j A contingency repair plan was implemented to correct the 59G relay wiring configuration and at 2:50 a.m., Shift Management was notified that the E-2 D/G wiring discrepancy had been corrected and performance of a post maintenance test had been completed j
. satisfactorily. The E-2 D/G was declared operable and the applicable TS LCOs were j
then exited.
l i
i f
i
}
)
i e
l
'.3$
i Reasons For The Violation A discrepancy existed on the pre-modification wiring diagram for the configuration of the 59G relays associated with each D/G. The representation of the 59G relay on the pre-modification diagram showed a set of contacts between terminals 1 and 2 and another i
set of contacts between terminals 3 and 4. In reality, the 59G relay contacts were i
located between terminals 1-3 and 2-4. Prior to the modification, the relay was i
appropriately wired with the correct contact configuration. During the installation of the modification, however, the E-2 and E-4 59G relays were wired in accordance with the i
59G wiring diagram, which was incorrect. The wiring diagram error was fcund to have been in existence since 1976, but the actual cause of the error could not be determined.
Prior to the design and installation the modification, field walkdowns were conducted to ensure consistency between the modification design and the actual plant wiring i
configuration. This is a proceduralized requirement for all plant modifications. A Drawing j
Change' Request (DCR) was created to correct various drawing deficiencies identified during the original modification walkdown and subsequent detailed walkdowns. Prior to the original walkdown, the modification designer had identified a potential problem due I
to a discrepancy between the Conceptual Design (CD) and the wiring diagram for the 59G relay. The schematic diagram used in the CD indicated a ten terminal relay whereas the 59G relay diagram showed a six terminal relay. The Engineer of Choice (EOC) i Responsible Engineer (RE) was aware of the modification designer's concern and i
expected to see a six terminal relay as numbered on the wiring diagram to confirm that the point numbering in the conceptual design was incorrect. The RE verified six terminals j
as laid out on the wiring diagram during the walkdown, but failed to identify that the wires landed on the 59G relay as noted in the walkdown package, did not agree with the wiring diagram, The panel wiring information including all the 59G relay connections were i
shown on the wiring diagram in the walkdown package. The RE also did not refer to the 3
vendor manual for the relay to verify the relay contact configuration. The RE was overly focused on the terminal numbering issue and assumed the relay contact configuration lj was correct once he identified the correct numbering and layout. The RE was not properly focused to verify the details inside the relay.
i.
The independent review of the Document Change Request (DCR) was less than t
adequate. The independent reviewer (IR) did not appropriately review the information obtained as a result of the walkdown to verify the design wiring of the 59G relay. The IR was aware of the potential discrepancy with the CD and the 59G relay wiring diagram, however, the IR only reviewed the information obtained from the walkdown against the i
j
.known as-built discrepancies identified in the DCR for the modification. The IR accepted i
the RE's judgement that the remaining information, including the configuration of the 59G relay, was correctly shown on the drawings without performing an independent review of the relay against the available reference information.
The test planning for Mod P-00231 was not well coordinated between the Lead Station i
Responsible (LSR) engineer, the installation test engineer, and the l&C planner.
b i
j Discussion of the overall testing did not identify the division of testing responsibilities clearly enough to ensure adequate testing overlap. Communication between the
' installation test planner and the I&C planner did not clearly establish a requirement for l&C to functionally test the 59G relay when it was calibrated, l&C calibration procedures i
i do not require functional testing of components being calibrated. As a result, wires to the 4
relay terminals were lifted prior to the calibration and were relanded following the calibration without functionally testing the 59G relay. Since it was thought that l&C would e
be functionally testing the 59G relay, installations did not perform testing on this portion l
of the circuit and jumpered around the 59G relay terminal-points. In addition, the l
~ installation test planner did not document the credit taken for tiie I&C calibration testing in the Installation Test Record (ITR) per MOD-CG-10 "Installaron Testing", Figure 3.
The LSR assumed that the installation testing would functiona? test a majority of the f
modification installation, including the 59G wiring and contacts, but did not document the credit taken for installation testing in the Acceptance Test Plan (ATP) as required by MOD-C-5, " Mod Process Acceptance Testing, Station Checklist and Mod Training 4
Bulletin". The LSR wrote the ATP to address only the MATS written to verify various functions of the modification. During verbal communication between the LSR and the l
COC installation engineer, it was not specifically identified that the installation tests l
would verify the wiring and contacts for the 59G relay. This also was not documented as a prerequisite in the MAT. As a result, the 59G relay was not functionally tested. The 4
overall scope of the modification testing verified the function of the components impacted by the modification, with the exception of the 59G relay contacts.
During the calibration of the E-2 D/G 59G relay per procedure IC-C-11-04004,
" Calibration Testing of Instantaneous Voltage Relays", a Nuclear Maintenance Division 3
(NMD) technician supporting Peach Bottom I&C personnel, initiated calibrations required 1
in the panel. The technician lifted wires for the relay coil in the terminal block. He then lifted the relay wires at terminal points 1 and 2. Because he knew the contact configuration for the relay, he tested to ensure the contacts closed as required by procedure between terminal points 1 and 3. The technician then relanded the relay wires to terminal points 1 and 2. The technician failed to exhibit the proper questioning attitude that would have identified the incorrect wiring configuration.
During the installation of the modification on the E-4 D/G, craft personnel identified i
discrepancies between the wiring diagram in the installation package and the installed wiring on the 59G relay. The installation engineer, on location during this work, stopped work on this relay and involved the installation test planner. They decided that the wiring error was probably just f!ip-flopped wires since the field configuration had been walked down during the design phase. Since the modification was being tested, the installation engineer and test planner believed that any functional problems would be determined J
at that time. They resumed removal activities and did not communicate the non-conforming condition to the LRE or the EOC RE.
L
_.___=_ _ _ _
~ _ _ _
H The Corrective Steos That Have Been Taken and the Results Achieved i
The. 59G relay wiring discrepancies on the E-2 and E-4 D/Gs were expeditiously l
evaluated.
The E-2 D/G 59G relay wiring discrepancy was corrected and post maintenance testing 1
was performed satisfactorily. Other modification related components were evaluated and confirmed to have been functionally tested. The E-2 D/G was returned to an operable status at 2:50 a.m. on June 21,1995.
I' i
A Performance Enhancement Program (PEP) issue was initiated June 21,1995, to l.
determine the root cause and causal factors of this event and to develop appropriate
[
corrective actions to prevent recurrence.
The E-4 D/G wiring was corrected and satisfactorily tested prior to exiting the TS LCO that was entered for maintenance and modification activities.
l
)
An extensive review of the E-2 and E-4 D/G MATS was completed with no additional l
findings.
i Installation'of all other plant configuration changes was suspended. A site stand-down on configuration changes was initiated and meetings were conducted with appropriate Peach Bottom and Chesterbrook personnel to discuss the significance of the diesel
[
events, the importance of properly conducting Engineering activities, and the need to conduct complete modification testing. A 17 member panel was formed to develop a plan l
[
to verify the adequacy of the design and testing of plant configuration changes and to review and evaluate possible improvements to the plant configuration control process.
I As a result of this panel review, Administrative Guideline, AG-123 " Maintaining Configuration Control of Design Changes"was developed. AG-123 establishes a process checklist to ensure that the design review, walkdown and testing of plant configuration changes is performed to an adequate level of detail. All pending configuration changes were evaluated using AG-123 to confirm their acceptability.
New modification procedures developed prior to this event were reviewed and revised to ensure the lessons learned from this event and expectations of AG-123 were appropriately incorporated.
Individuals were held accountable for their actions which contributed to this event.
Individuals were counselled by the Director of Site Engineering and appropriate l
disciplinary action was taken for supervisors and individuals.
Management All-hands meetings were conducted where AG-123 was discussed with plant personnel. The importance of a quality design, complete testing and effective communication was emphasized.
I
~. _ ___
i
)
Mbification team members were counselled by line supervision to clarify management expectations concerning configuration changes. Also,' an accountability meeting was conducted by the Director of Engineering to discuss the issues with the modification team.-
The Corrective Steos that Will Be Taken to Avoid Further Violations The lessons learned from this event will be communicated to appropriate engineering personnel. The appropriateness of actions taken during this event will be discussed as well as management's expectations regarding the use of self-check principles. The role -
of independent reviewers and the appropriate methods required to verify the accuracy.
of work will be emphasized. These actions will be completed by September 30,1995.
The responsibility of Engineering Branch Managers and additional potential LSRs outside of Engineering to ensure that ATPs cover the entire scope of testing activities including 4-installation, I&C activities, applicable tests and testing requirements, and MAT testing
. prior to approval of the ATP will be clarified. This will be completed by September 30, i
t 1995.
An overall assessment of AG-123 reviews that have been performed will be conducted to determine and evaluate any generic performance issues as a result of the work process. The results of this assessment will be distributed to personnel to develop appropriate process indicators and action plans. This assessment will be completed by October 1,1995.
j Date When Full Comoliance Was Achieved I
Full compliance was achieved on June 21,1995, when the E-2 D/G 59G relay wiring discrepancy was corrected and the E-2 D/G was declared operable, exiting TS LCOs 3.O.C and 3.O.D.
)
4 i
9 i
)
i 4
i 3
,