IR 05000352/1999004
| ML20210U243 | |
| Person / Time | |
|---|---|
| Site: | Limerick  |
| Issue date: | 08/10/1999 |
| From: | NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML20210U226 | List: |
| References | |
| 50-352-99-04, 50-353-99-04, NUDOCS 9908200043 | |
| Download: ML20210U243 (20) | |
Text
.i i
,
h U.S. NUCLEAR REGULATORY COMMISSION
REGION I
Docket No License No NPF-39 NPF-85 Report No Licensee: PECO Energy Correspondence Control Desk P.O. Box 195 Wayne, PA 19087-0195 Facilities: Limerick Generating Station, Units 1 and 2 Location: Wayne, PA 19087-0195 Dates: May 25,1999 through July 12,1999
'
Inspectors: A. L. Burritt, Senior Resident inspector F. P. Bonnett, Resident inspector D. J. Florek, Project Engineer, DRP D. Cpilison, Reactor Engineer, DRP C. G. Cahill, Reactor Engineer, DRS L. M. James, Reactor Engineer, DRS Approved by: Curtis Cowgill, Chief Projects Branch 4 Division of Reactor Projects 9908200043 990810 PDR ADOCK 05000352 G PDR ,
j
.y i
.
'
EXECUTIVE SUMMARY Limerick Generating Station, Units 1 & 2 NRC Inspection Report 50-352/99-04,50-353/99-04 This integrated inspection included aspects cf PECO Energy operations, engineering, and maintenance. The report covers a seven-weck period of resident inspectio Operations !
e Prior to the Unit 1 automatic scram on June 11,1999, PECO had not included a long standing instrumentation issue (momentary reactor water level spike indication following a turbine trip from high power) in PECO's program for operator work-arounds. PECO's corrective actions for this issue were adequate. (Section O2.2)
e Although PECO's post-scram review of the June 11,1999, automatic scram on Unit 1 was adequate, the review missed several documentation deficiencies regarding event details and unified control room log entries. (Section O3.1)
e Operators responded appropriately to the June 11,1999, automatic scram on Unit The scram was caused by a personnel error during performance of a procedure for a routine turbine overspeed test. Two other personnel errors involving procedure use were noted during the event recovery and subsequent startup which resulted in missed technical specification surveillance activities. (Section O4.1)
e LER 1-99-006 reported the late performance of an off-gas grab sample when the off-gas hydrogen analyzers were inoperable and the steam Jet air ejectors and off-gas systems were in-service The event occurred when an individual inappropriately signed-off an integrated startup procedure step indicating the hydrogen analyzers were in-servic This incorrect procedure notation led to the late performance of an off-gas grab sample and consequently the failure to compiy with TS 3.3.7.12 and TS 3.11.2.5. This Severity IV violation is being treated as a Non-Cited Violation, consistent with Appendix C of the NRC Enforcement Policy. This violation is in the PECO's corrective action program as l PEP 10009941. (Section 08.2) {
Maintenance e The Unit 1 high pressure coolant injection (HPCI) failure to start on June 23,1999 was a 1 maintenance preventable functional failure. PECO missed the opportunity to prevent the HPCI failure by not adequately reviewing prior industry experience, which recommended replacement of a component in the turbine's govemor electro-hydraulic regulator following water intrusion into the HPCI oil system. Water intrusion had occurred during the extended HPCI operation on April 20,1999. Additionally, a personnel error during the corrective maintenance extended the HPCI system outage time. (Section M1.3)
l li
.i .
,-
e The Nuclear Maintenance Department (NMD) management had recognized that performance had degraded in 1998 in the execution of refueling activities at Limerick and Peach Bottom. NMD management evaluated the performance deficiencies in aggregate and implemented appropriate corrective actions. The associated actions implemented prior to the recent Limerick refueling outage resulted in improved performance. (Section M4.1)
Enaineerina
- The PECO evaluations of the causes for the equipment that malfunctioned during the April 20,1999 reactor scram were adequate. The equipment malfunctions were unrelated to each other. (Section E1.1)
,
iii
r
- 4 ..
..
,9
'
TABLE OF CONTENTS Summary of Plant Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1. Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 O1 Conduct of Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 01.1 General Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 02 Operational Status of Facilities and Equipment . . . . . . . . . . . . . . . . . . . . . . . . 1 02.1 Facility Tours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 O2.2 Reactor Water Level Instrumentation - Momentary Level Spike - Unit 1 2 03 Operations Procedures and Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 03.1 Post-Scram Procedure (GP-18) Review - Unit 1. . . . . . . . . . . . . . . . . . 3 04 Operator Knowledge and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 04.1 Unplanned Reactor Shutdown - Unit 1 . . . . . . . . . . . . . . . . . . . . . . . . . 4 08 Miscellaneous Operations issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 08.1 (Closed) LER 1-99-005: RPS and ESF Actuation Caused by Operator Error During Routine Turbine Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 5 08.2 (Closed) LER 1-99-006: Late Performance of a TS Required Off-gas Grab Sam ple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 11. M aintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 M1 Conduct of Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 M1.1 General Comments on Maintenance Activities . . . . . . . . , . . . . . . . . . . 6 M1.2 General Comments on Surveillance Activities . . . . . . . . . . . . . . . . . . . 6 M1.3 High Pressure Coolant injection Failure to Start - Unit 1 . . . . . . . . . . . 7 M4 Maintenance Staff Knowledge and Performance . . . . . . . . . . . . . . . . . . . . . . . 8 M4.1 Review of Nuclear Maintenance Division Performance . . . . . . . . . . . . 8 111. Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 E1 Conduct of Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 E Equipment Malfunctions During Unit 1 Loss of Feedwater and Scram on April 20. 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 E8 Miscellaneous Engineering issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 E8.1 (Closed) VIO 50.352, 353/98-05-01: Ineffective Corrective Action on Agastat Relays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 V. Management Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 X1 Exit Meeting Summary . . . . , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 X2 Review of Year 2000 Readiness of Computer Systems . . . . . . . . . . . . . . . . . 13 X3 NRC/PECO Public Meeting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 INSPECTION PROCEDURES USED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 ITEMS OPENED, CLOSED, AND DISCUSSED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 LIST OF ACRONYMS USED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 iv e
Ll-
E
., ,
.
Report Details Summary of Plant Status Unit i began this inspection period operating at 100% power. The unit remained at full power throughout the inspection period with minor exceptions for testing, rod pattern adjustments, and the following plant event.-
e June 11 An operator error during a weekly main turbine surveillance test caused the main turbine stop valves to shut automatically, shutting down the turbine and producing an automatic reactor shutdown (scram) from 100%
powe * June 13-15 Control room operators made the reactor critical, synchronized the unit to the grid, and increased reactor power to 100%.
Unit 2 began this inspection period with control room operators restarting the reactor following the recent refueling outage. The operators made the reactor critical on May 24. From May 25 through May 31, operators synchronized the unit generator to the grid and increased reactor power to 100%. The unit remained at full power throughout the inspection period with minor exceptions for testing and rod pattom adjustment . Operations 01 Conduct of Operations 01.1 General Comments (71707)
During this inspection period, PECO Energy (PECO) conducted at power activities at Limerick Units 1 and 2 safely. Routine operations, surveillance, and other plant-related activities were generally performed per station procedures, in a deliberate manner with clear communications, and with effective oversight by shift supervision. Shift tumovers were comprehensive. Operators implemented effective controls for work activities using
' conservative decision makin Operational Status of Facilities and Equipment O2.1 Facility Tours (71707)
The inspectors routinely conducted independent plant tours and walkdowns of selected portions of safety-related systems during the inspection period. These activities consisted of the verification that system configurations, power supplies, process parameters, support system availability, and current system operational status were consistent with Technical Specdication (TS) requirements and Updated Final Safety Analysis Report (UFSAR) descriptions. System operability and material conditions were noted to be acceptable in all cases. The inspectors did not identify any substantive concems or deficiencies as a result of these walkdown .
.
.. >
.
~
2 l 02.2 Reactor Water Level Instrumentation - Momentary Level Soike - Unit 1 (71707) . Insoection Scope (37551RRf The inspector reviewed a reactor water level instrumentation momentary level spike that caused a reactor core isolation cooling (RCIC) initiation and a number of system isolations during the June 11,1999, Unit 1 automatic shutdown (scram). The inspector evaluated plant parameter information from this event as well as documentation of previous occurrences. The inspector also assessed PECO's evaluation and corrective actions for this issu Observations and Findings On June 11,1999, Unit 1 experienced a generator load reject causing a turbine trip from 100% power and an automatic scram. Several primary containment system isolations and an automatic start of the reactor core isolation cooling (RCIC) system occurred due to a momentary low level spike (less than the level 2 setpoint of -38") on some of the reactor water level instrument channels. PECO determined that actual reactor water level was well above the level 2 (-38") safety system actuation setpoint. The momentary level spike in indicated reactor water level was due to a known phenomenon that occurs during rapid reactor pressurization transients such as turbine trips from high power The rapid cessation of steam flow and the associated interaction with reactor vessel intemals result in localized pressure changes within the reactor vessel. Since reactor water level indication is derived from pressure in specific locations within the reactor vessel, these localized pressure changes may cause high amplitude short duration changes in indicated level on some reactor water level instrument PECO engineering determined that the instrumentation system was operable based on the instrument response being consistent with historic transients and that the instrument response had been previously evaluated and found to be acceptable. PECO engineering also determined that the equipment subject to the momentary low level spike functioned correctl The inspector determined PECO's operability assessment was correct but was concerned that operators were being unnecessarily challenged by the momentary level spike. The operators were faced with conflicting information that increased the complexity of the actions required to confirm appropriate safety system response Since only some of the level instruments experienced the momentary level spike, the operators had to determine if the low level indication was an actual condition and why some equipment responded and some did not. For example, RCIC and the high pressure coolant injection (HPCl) system start on the same level 2 setpoint, but use
' different level indications. In this transient, the HPCI level instruments were not affected as a result of a previous modification and therefore HPCI did not start, bat the RCIC level
' instruments were affected and did start. Additionally, since the momentary level spike caused the reactor er closure ventilation system to isolate, building temperatures increased causing operators to have to implement additional emergency operating procedures (EOP).'
... ,.
.
.
Following a review of the momentary reactor water level spike indication and discussions with the inspector, PECO determined that this issue was an operator work-around. An operator work-around is an equipment problem that causes compensatory measures that impact an operators ability to control the plant during a transient. PECO added this reactor water level instrumentation issue to their operator concems/ work-arounds program and planned to modify the affected instrumentation to prevent unnecessary safety function actuations. A similar modification had been implemented for HPCI reactor water level instrumentation in response to an event documented in LER 2-93-005. PECO also planned to incorporate additional reviews to assure operator work-arounds and operator challenges are appropriately identified following unintended plant shutdowns. The inspector determined that PECO's corrective actions for this issue were adequat The inspector also identified that PECO's power re-rate analysis had not specifically assessed whether the momentary reactor water level spike would be more severe under the higher steam flows. PECO subsequently evaluated the impact of the higher power and determined that no additional adverse impacts from momentary reactor vessel level spikes were created by the higher steam flo Conclusion Prior to the Unit 1 automatic scram on June 11,1999, PECO had not included a long standing instrumentation issue (momentary reactor water level spike indication following a turbine trip from high power) in PECO's program for operator work-arounds. PECO's corrective actions for this issue were adequat Operations Procedures and Documentation 03.1 Post-Scram Procedure (GP-18) Review - Unit 1 (71707) Lnspection Scope The inspector reviewed the evaluation of the June 11,1999, Unit i scram documented in procedure GP-18, " Scram Event Review" to determine if PECO properly evaluated the i plant response and corrected the identified deficiencies prior to restart. The inieector discussed the evaluation with members of the operations staf Observations and Findinas The inspector determined that the post scram event review was adequate with all significant deficiencies identified and appropriately addressed prior to the restart of the unit. Nevertheless, the inspector identified some deficiencies in PECO's documentation of the event. For example, the GP-18 event data sheets indicated that the 13 kV electrical buses did not transfer when, in fact, the buses transferred automaticall During the event an anomalous (-38") reactor water level indication occurred (discussed in section O2.2 of this report) which caused some, but not all, of the safety systems to actuate. A comment associated with the system response data sheets indicated that the
i
.. -,
.
L,
I safety system actuations were a result of a " valid" water level. This differed from the
)
post-scram parameter traces, part of the GP-18 data package, which indicated that actual reactor level was never less than -7". This section of GP-18 did not explain the apparent discrepancies.
l The inspector determined that operators did not document in the unified control room log l (UCRL) some of the actions that were specified in Operations Manual section (OM)-L-l 8.2, Narrative Log / Scope of Entry. The operators did not document the use of emergency operating procedure T-101, RPV Control, the configuration changes of the recirculation pumps and reactor water clean-up system, and the missed chemistry surveillance. Additionally while the operators documented the initiation of the RCIC system, they did not indicate that the initiation signal was due to an anomalous reactor
'
_ water level indication. The inspector noted that PECO's post-scram review did not i identify these deficiencie f Conclusions Although PECO's post-scram review of the June 11,1999, automatic scram on Unit 1
, was adequate, the review missed several documentation deficiencies regarding event i
'
details and unified control room log entries.
l 04 Operator Knowledge and Performance l
04.1 Unolanned Reactor Shutdown - Unit 1 (93702/71707) Inspection Scope The inspector reviewed the unplanned scram of Unit 1 including operator response to the event, the cause of the event, and corrective actions implemented to prevent a
'
recurrence. The inspector evaluated the routine test procedure which led to the transient and discussed the procedure and the event with operators and other members of PECO's staf Observations and Findinos On June 11,1999, Unit 1 automatically shutdown from 100% power due to personnel error. An equipment operator, while performing a routine backup overspeed test for the main turbine, failed to bypass the trip function under test as required by the surveillance l . procedure. When the operator depressed the overspeed test pushbutton, the main turbine stop valves shut causing a turbine trip and reactor scram. The main control room staff responded appropriately to stabilize the unit. All systems necessary to mitigate the event responded as required.
l PECO determined that the operator did not use good self-check and attention-to-detail i skills. Also, the procedure (RT-6-031-312-1, Backup Overspeed and Power / Load Unbalance Test) lacked specific detail conceming the potential consequences of the tes PECO enhanced the procedure by adding cautions that wamed the operator of the
., .
.
potential to shutdown the turbine and printed across the cover page * Peer-Check Required." The operations staff further reviewed and revised other surveillance procedures to include the peer-check requirement. The inspector independently )
'
cor.'irmed the cause of the event and determined that the corrective actions for the oparator performance and procedural deficiencies were adequate. The inspector determined that although the operator did not appropriately follow the procedure as written, this does not constitute a violation of NRC requirements since the equipment being tested was a non-safety related componen PECO identified that two other personnel errors occurred during the event recovery and subsequent reactor startup. These two personnel errors resulted in missed technical specification requirements. During the event recovery, the control room supervisor did not request a grab sample of the reactor water for conductivity monitoring within the required time specified in technical specifications following the isolation of the continuous reactor water conductivity monitors. PECO determined that guidance contained in procedure GP-3, Normal Plant Shutdown, contributed to the missed sample. PECO promptly modified the guidance. The inspector determined that although the TS surveillance requirements were not met, since the necessary actions were taken per TS 4.0.3 a violation of TS did not occur. The personnel error during startup is discussed Section 0 Conclusions Operators responded appropriately to the June 11,1999, automatic scram on Unit The scram was caused by a personnel error during performance of a procedure for a routine turbine overspeed test. Two other personnel errors involving procedure use were noted during the event recovery and subsequent startup which resulted in missed technical specification surveillance activitie Miscellaneous Operatioris issues (90712)
l 08.1 (Closed) LER 1-99-005: RPS and ESF Actuation Caused by Operator Error During i Routine Turbine Testing. The inspector performed an "in-field" review of this event I discussed in Section O4.1 of this report. No new issues were revealed by the LE .2 (Closed) LER 1-99-006: Late Performance of a TS Required Off-gas Grab Sample. The >
inspector performed an "in-field" review of this event. On June 14, during startup '
activities being performed at Unit 1, the control room supervisor (CRS) discovered that the steam jet air ejectors (SJAE) and off-gas system had been in-service for about 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> without an off-gas hydrogen analyzer being operable. All three analyzers were in the purge mode of operation. Without the necessary hydrogen analyzers in-service during SJAE operation, grab samples of the off-gas system are required every four-hours in accordance with TS 3.3.7.12 and TS 3.11.2.5. The CRS immediately notified chemistry to obtain and analyze an off-gas grab sample and to place the analyzers in-servic .. ,
.
.
PECO determined that a personnel error caused the event. An individual did not perform a procedure step as written to place the analyzers in-service, but mistakenly concluded, using control room indications, that the analyzers had already been placed in-servic PECO further determined that the procedural step lacked specific detail for verifying that the analyzers were in-service and corrected this procedural deficiency. Nevertheless, the inspector determined that the procedure step, had it been properly performed, would have resulted in the analyzers being placed in-servic The inspector determined that, at the time of the event, the individual did not ,
appropriately follow the step in procedure GP-2 as written. This resulted in the failure to i comply with TS 3.3.7.12 and TS 3.11.2.5 which is a violation of NRC requirements. This Severity IV violation is being treated as a Non-Cited Violation (NCV 50-352/99-04-01),
consistent with Appendix C of the NRC Enforcement Policy. This violation is in the PECO's corrective action program as PEP 1000994 . Maintenance M1 Conduct t.,f Maintenance M1.1 General Comments on Maintenance Activities (62707)
The inspectors observed selected maintenance activities to determine whether approve procedures were in use, technical specifications were satisfied, maintenance was i performed by knowledgeable personnel, and post-maintenance testing was appropriately j complete '
The inspectors observed portions of the following work activities:
- Unit 1, Division 1 Safeguards Battery (1 AD101) replacement - June 8-10; e Unit 2, D2318-month Overhaul- June 21 - 25; e Unit 1, HPCI Minimum Flow Valve (HV-F012) VOTES - June 22; )
- Unit 1, Division 2 Safeguards Battery (1BD101) replacement - June 22 - 2 Observed maintenance activities were conducted well using approved procedures and were completed with satisfactory results. Communications between the various work and support groups were good and supervisor oversight was goo M1.2 General Comments on Surveillance Activities (61726)
The inspectors observed selected surveillance tests to determine whether approved procedures were in use, test instrumentation was properly calibrated and used, technical specifications were satisfied, testing was performed by knowledgeable personnel, and test results either satisfied the acceptance criteria or were properly dispositione The inspectors observed portions of the fol!owing surveillance activities:
r, .
e
'
.
'
e Unit 1 - ST-3-107-790-1, Control Rod Scram Testing, - June 15; e Unit 1 - ST-6-055-230-1, HPCI Pump, Valve and Flow - June 24; e Unit 2 - IC-11-02042, D23 Diesel Generator Bearing insulation Test - June 17; e Unit 2 - RT-6-092-323-2, D23 Diesel Generator Overspeed Trip Test - June 1 !
'
Observed surveillance tests were conducted well using approved procedures and were completed with satisfactory results. Communications among the various work and support groups were good and supervisor oversight was goo M1.3 Hioh Pressure Coolant Iniection Failure to Start - Unit 1 (62707) Insoection Scope The inspector reviewed the circumstances of the high pressure injection system (HPCI)
failing to start on demand and subsequent automatic shutdown following corrective maintenance. The inspector discussed the event with members of the plant staff and reviewed PEP's 10009964,10009965, and 1000997 Observations and Findinos On June 23,1999, during post-maintenance testing, the Unit 1 HPCI turbine did not start on demand because of a failed component in the turbine's govemor electro-hydraulic ,
regulator called an EG-R. l&C technicians replaced the failed EG-R and the failed EG-R I was sent to the corporate laboratory for failure analysis. The root cause of the failed EG-R was mechanical binding caused by rust and the accumulation of grit in the servo are The cause of the rust was due to water intrusion into the HPCI lube oil system during the prolonged system operation during the April 20,1999, unplanned shutdown. The cause of the water intrusion is discussed in section E1,1 of this repor PECO determined that the June failure was a maintenance preventable functional failure. PECO determined that they had not adequately reviewed previous industry experience for the April event. Three similar events at other facilities had occurred that resulted in EG-R failures. PECO determined that, based on review of the industry information, they should have replaced the EG-R following the water intrusion on April 20,1999. Following the April event, the EG-R was not replaced but flushed with oil and exercised to ensure operatio On June 24, during the system retest following the EG-R replacement, the HPCI turbine automatically shutdown due to excessive turbine speed PECO determined that the new EG-R had been improperly wired during installation with the polarity of the control signal to the govemor control valve being reversed. The cause of this error was due to a configuration change on the EG-R component and inadequate documentation of wire terminations in the lifted lead log. Prior to installation, PECO verified that the replacement EG-R was the same model and stock code as the original; however, did not identify that the manufacturer had reversed the configuration of the 'A' and 'B' wire terminals on the new component. The l&C technicians wiring the EG-R did not
.= ,
.
recognize the configuration change and documented the lifted leads as ' top' and ' bottom'
verses 'A' and 'B' as labele PECO is revising their procedures to ensure terminal markings are verified prior to connecting electrical leads. Engineers are also expanding the scope of this corrective l
action to include other systems that utilize this type of component. PECO did not j consider the reversed wire terminals a procurement receipt discrepancy because the )
exact orientation of the terminals is not described in the vendor manual PECO determined that personnel error was the root cause of the HPCI overspeed even l The technicians did not verify the correct polarity when installing the new EG-R and did not accurately document the removal of the leads. The error resulted in increased HPCI system unavailability tim Conclusion The Unit i high pressure coolant injection (HPCI) failure to start on June 23,1999 was a maintenance preventable functional failure. PECO missed the opportunity to prevent the HPCI failure by not adequately reviewing prior industry experience, which recommended replacement of a component in the turbine's govemor electro-hydraulic regulator ,
following water intrusion into the HPCI oil system. Water intrusion had occurred during l the extended HPCI operation on April 20,1999. Additionally, a personnel error during the corrective maintenance extended the HPCI system outage tim M4 Maintenance Staff Knowledge and Performance M4.1 Review of Nuclear Maintenance Division Performance (62707) Insoection Scope The inspector reviewed PECO's evaluation of recent fuel handling events at the Limerick Generating Station (LGS) and the Peach Bottom Atomic Power Station (PBAPS). The inspector discussed these issues with several senior reactor operators that are limited to fuel handling (LSRO) and members of the Nuclear Maintenance Division (NMD)
managemen Observations and Findinos A new Director of NMD and manager of the reactor group were recently put in plac Both individuals were previously licensed and brought an operational perspective to the division. Following the refueling outage at PBAPS in November 1998, this management recognized that several performance deficiencies existed and performance was declining. NMD management identified issues involving the authority of the LSRO, experience level of platform operators, and communications to and from the refueling l
'
platfor ,
-
,, *
.
r
NMD management considered that the authority of the LSROs had diminished. Although the LSROs are the licensed authority in charge of activities on the refueling floor, too many other work activities were being scheduled so that they were becoming unable to maintain cognizance of refueling activities and the other work. Further, the LSROs were not being given the opportunity to input to the work planning process or to review the work schedule. NMD management also identified that the experience level of the operators was also declining. Staff experience had decreased so that just a limited number of qua!Wied people existed that could perform certain tasks and procedure This resulted in schedule delays and low personnel morale. Because the group has had an excellent performance record in past years, NMD management believed that they were becoming complacent. Standards and expectations were declining to the point that the staff had been accommodating small deficiencies as acceptable. Further, performance data (i.e. mis-positioning events) was indicating declining performanc Lastly, communications to and from the refueling platform had become lax and distracting. The LSRO was frequently being interrupted and distracted by low priority communications during fuel movement The new management assessed all NMD PEP issues at both reactor facilities, involving the reactor group, in aggregate. Corrective actions were developed and implemented as a " step-up plan." The performance issues were discussed at an all day staff training seminar prior to performance of the December 1998 Unit 1 maintenance outage to replace a leaking fuel bundle. Corrective actions included:
e standardization of the fuel handling process, procedures, and practices; e improved operator formality of activities on the refueling platform, in concert with improved coordination of fuel floor activities during times of fuel handling to reduce distractions; e improved communications guidelines and restrictions to reduce distractions; e painting a compass rose on each refueling platform for directional orientatio These and other corrective actions were implemented prior to the recent refueling outage at LGS Unit 2. They focused on restoring the responsibility and authority level of the LSRO on the refuel floor and to holding the staff accountable for their performance. The inspector observed these corrective actions during the outage and found them to be effectiv NMD management indicated that errors and near-misses were unacceptable performance. A program to capture and track near-misses was established for errors during double verification. Each near miss is logged in LSRO log and discussed with the on-coming shift. If two near misses occur, the crew stops refueling activities and holds a small critique. A PEP is initiated if a third near miss occurs. Management has communicated the expectation that speed is not the issue if the job can not be performed right the first tim The inspector determined that the new NMD management recognized that problems existed for some time and that they have actively taken steps to improve staff performance. PEP issues that occur at PBAPS and LGS have been assessed in
!
.. ,, .
.
.
aggregate and the assessments were used to formulate corrective actions. Special training sessions were held to heighten LSRO and platform operator awareness and to improve and standardize performance. Corrective actions that have been developed as a " step-up plan" were implemented prior to LGS maintenance outage in December 1998 and reemphasized prior to the recent LGS refueling outage. Notable improvement in fuel handling performance was previously identified by the inspector and was a result of the NMD review and associated corrective action Conclusions The Nuclear Maintenance Department (NMD) management had recognized that performance had degraded in 1998 in the execution of refueling activities at Limerick and Peach Bottom. NMD management evaluated the performance deficiencies in aggregate and implemented appropriate corrective actions. The associated actions implemented prior to the recent Limerick refueling outage resulted in improved performanc IIL Engineering E Conduct of Engineering (92903)
E Eauioment Malfunctions Durina Unit 1 Loss of Feedwater and Scram on April 20.1999 inspection Scope (92903)
The inspector reviewed the root cause evaluation and corrective actions for several of the equipment malfunctions that occurred during the loss of feedwater transient and subsequent scram at Unit 1 on April 20,1999. The scram was discussed in inspection report 50-352,353/99-03 Specifically the inspector assessed the trip of the breaker that supplied power to the deep bed domineralizer valve position indication, the failure to trip of RPT breaker 10-A201-01, the failure of the mechanical vacuum pump Agastat relay and the HPCI vacuum pump tri Observations and Findinas Trio of Breaker that Shiad Power to Deep Bed Demineralizer Valve Position Indication On April 20,1999, at 1830 hours0.0212 days <br />0.508 hours <br />0.00303 weeks <br />6.96315e-4 months <br />, breaker 104C-T-G-28, "Non-Regulating XFMR Disconnect 10-Y818" on panel 10-Y818, "120 VAC Non-Regulating Power Distribution j
'
Panel," tripped open. This feed provided power to various valve position indication circuits on the deep bed domineralizer system. The loss of valve position indication to the programmable logic controller (PLC) resulted in all eight of the condensate deep bed
' effluent flow control valves closing in accordance with the PLC programming. The resultant loss of feedwater flow caused the reactor feed pump to trip on low sutan pressure and a decreasing reactor vessel water level. The reactor subsequently scrammed on low level as designe e
. . ,, .
..
'
PECO initiated PEP 10009727 in response to this event. PECO determined that the L domineralizer digital upgrade design modification in 1992 evaluated a complete loss of power but did not consider the effects of a partial loss of power. PECO initiated PEP 10009762 to evaluate the engineering practices for failure modes and effects analysis for digital upgrades. The plant response to this event was within the bounds of the loss of feedwater analysi PECO investigated the cause of the 104C-T-G-28 breaker trip and found the most likely cause of the breaker trip was an anomaly in the electrical power system or a spurious breaker trip. PECO replaced circuit breaker 104C-T-G28 with a new breaker and initiated power system quality monitoring. The monitoring, as described in Evaluation 23 of PEP 10009727, revealed no abnormalities. The inspector observed engineering root cause meetings, interviewed the system engineer and reviewed the results in PEP 10009727 and found PECO's conclusions to be reasonabl PECO initiated a change to the condensate deep bed domineralizer system operating procedure to open the breakers for the vessel outlet flow control valves after placing the system in service. This change was evaluated in NCR 99-00981. The inspector walked down portions of the system, interviewed the system operator, reviewed the ECR 99-00981 and found PECO's actions to be acceptabl Failure of 10-A201-01 RPT Breaker to Trio The 10-A201-01 recirculation pump trip (RPT) circuit breaker failed to trip on a valid low reactor water level signal during the April 20,1999, Unit 1 scram. The remaining RPT breakers,10-A201-02,10-A202-01 and 10-A202-02, tripped automatically on the low reactor water signal securing the recirculation pumps as designe PECO's initial visual investigation of the 10-A201-01 breaker identified a damaged # 1 anticipated transient without scram (A1WS) trip coil. The breaker was an ABB model 5HK250 breaker. The trip coil showed signs of overheating and the coil plunger was stuck in place. The faulty breaker was removed for inspection and a spare breaker was installe PECO assembled a team consisting of personnel from engineering, maintenance, Valley Forge Corporate Labs, and an ABB representative to investigate the cause of the failur The team determined that the breaker did not have indications of grease hardening and venfied the proper freedom of movement of the trip bar, trip latch and trip relle Additionally, the auxiliary contacts did not show signs abnormal wear. Tests conducted by Valley Forge Corporate Labs concluded that most likely cause of the coil failure was mechanical binding, although no specific root cause was determine ; PECO evaluated the extent of condition by testing the three remaining unit 1 RPT breakers. The trip circuits, including the sensors and the pump breakers, are Class 1 The three trip coils in each breaker were energized to a degraded voltage to evaluate the condition of each coil and checked for mechanical binding in the breaker or grease hardening. All of the coils successfully tripped the breakers with the exception of the 10-
i
- .'.. .
.
,
A201-02 #1 ATWS trip coil. Tne coil was replaced under work order C0188250. Due to preliminary troubleshooting onsite, the fault conditions could not be reproduced in the lab.= PECO has initiated PEP 10010033 to further evaluate the condition of the 10-A201-02 #1 ATWS trip coil. Since the unit 2 RPT breakers were overhauled during the refueling outage under the most current maintenance procedure (M-092-004) which performed the degraded voltage actuation test of the coils, no additional testing was required on Unit Unit 1 Mechanical Vacuum Pumo Aaastat Relav Failure PECO determined that mechanical vacuum pump failed to start because the coil for relay B21H-K128, " Main Steam Line Hi Rad" relay, located in panel 10-C611 was ope PECO replaced the relay under work order (W/O) C0188232 and successfully operated the pump. PECO sent the failed relay to the PECO Nuclear Corporate Laboratories for evaluation. The relay was evaluated under project number 1999 0548 LT. The lab found that the magnet wire had burned ope Based on lab evaluation, discussions with the manufacturer (Thomas & Betts), and the fact that the relay had only been in service for three months, PECO concluded that this I was failure was a rare early life failure and not age relate The inspector concluded that PECO conducted a thorough root cause evaluation and that PECO's conclusion that the relay failure was a rare early life failure was reasonabl HPCI Vacuum Pumo Trio and Intrusion of Water into the HPCI Lube Oil System Following the scram on April 20,1999, the Unit i HPCI was operated continuously for either vessel injection or reactor pressure control for about 16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br />. The HPCI vacuum
- pump tripped and was restarted an estimated 15 times while HPCI was in servic PECO also identified that water had entered the HPCI lube oil syste PECO determined that the vacuum pump tripped because of a high water level in the HPCI barometric condenser caused by higher than expected water inputs into the barometric condenser. PECO also determined that the water intrusion into the HPCI lube oil system was caused by the high barometric water level. PECO determined that pressure control valve PCV-056-1F035, which controls cooling flow to the lube oil cooler and barometric condenser, was controlling pressure at significantly higher value than required which then allowed the higher water input into the barometric condense PECO re-calibrated the PCV, and monitored the PCV performance during a subsequent successful test. Although PECO removed the oil from the water and cleaned affected portions of the oil systems, section M1.3 of this report discusses a subsequent failure caused by the water intrusio The HPCI system remained operable during the event. PECO determined that the amount of water in the lube oil did not affect pump performance. The barometric condenser and vacuum pump are not safety related and are not required for HPCI operatio ,= .. .
.
.
13 Conclusion The PECO evaluations of the causes for the equipment that malfunctioned during the April 20,1999 reactor scram were adequate. The equipment malfunctions were unrelated to each othe E8 Miscellaneous Engineering issues (92902)
E (Closed) VIO 50-352. 353/98-05-01: Ineffective Corrective Action on Agastat Relays PECO had permitted safety-related Agastat relays in higher temperature (95'F)
conditions to remain in service beyond their intended service life and to experience
~
corresponding higher failure rates than expecte This Severity Level IV violation was issued in a Notice of Violation prior to the March 11, 1999, implementation of the NRC's new policy for treatment of Severity Level IV violations (Appendix C of the Enforcement Policy). Because this violation would have been treated as a Non-Cited Violation, in accordance with Appendix C, this violation is being closed out in this report. This violation is in the PECO corrective action program as PEP 10008117. This violation is close V. Management Meetings X1 Exit Meeting Summary i
The inspector presented the inspection results to members of plant management at the i conclusion of the inspection on July 28,1999. The plant manager acknowledged the inspectors'
findings. The inspectors asked whether any materials examined during the inspection should be considered proprietary. No proprietary information was identifie X2 Review of Year 2000 Readiness of Computer Systems ;
The staff conducted an review of Y2K activities and documentation using Temporary Instruction (TI) 2515/141, " Review of Year 2000 (Y2K) Readiness of Computer Systems at Nuclear Power Plants." The review addressed aspects of Y2K management planning, documentation, l implementation planning, initial assessment, detailed assessment, remediation activities, Y2K J testing and validation, notification activities, and contingency planning. The reviewers used 1 NEl/NUSMG 97-07, " Nuclear Utility Year 2000 Readiness," and NEl/NUSMG 98-07, " Nuclear l'
Utility Year 2000 Readiness Contingency Planning," as the basis for this review. The results of this review will be combined with the results of other reviews in a summary report to be issued by July 31,199 ...-*
.
.
4'
'
X3 NRC/PECO Public Meeting On June 9,1999, the NRC conducted a Public Meeting to review the results of the Plant Performance Review (PPR) issued in our letter dated April 9,1999. Senior managers from the NRC and PECO were in attendanc .
I
,
- . . ,
e
- I
INSPECTION PROCEDURES USED l l
l lP 61726: Surveillance Observation i IP 62707: Maintenance Observation
'
IP 71707: Plant Operations IP 90712: In-office Review of Written Reports 1 IP 93702: Prompt Onsite Response to Events at Operating Power Reactors
]
IP 92902: Follow-up Maintenance IP 92903: Follow-up Engineering ITEMS OPENED, CLOSED, AND DISCUSSED Opened / Closed 50-352/99-04-01 iJCV Late Performance of a TS Required Off-gas Grab Sample. (Section 08.2) l Closed 50-352,353/98-05-01 NOV ineffective Corrective Action on Agastat Relays (Section E8.1)
50-352/1-99-005 LER RPS and ESF Actuation Caused by Operator Error During Routine Turbine Testing. (Section 08.1)
50-352/1-99-006 LER Late Performance of a TS Required Off-gas Grab )
Sample. (Section 08.2)
50-353/2-99-001 LER Unavailability of Safe Shutdown Equipment in the Event of a Fire. (Section E8.2)
Discussed
)
l None
,
n --
.+-.. *
,
LIST OF ACRONYMS USED ATWS Anticipated Transient Wdhout Scram CFR Code of Federal Regulations CRS Control Room Supervisor EOP Emergency Operating Procedure ESF Engineered Safety Feature HPCI High Pressure Coolant injection IR inspection Report LER Licensee Event Report LG Limerick Generating Station LSRO Senior Licensed Operator Limited to Fuel Handling MCC Motor Control Center NCV Non-Cited Violation NMD Nuclear Maintenance Division NRC Nuclear Regulatory Commission
' OM Operations Manual PBAPS: Peach Bottom Atomic Power Station PCRVICS ~ Primary Containment and Reactor Vessel isolation Control System PCV Pressure Control Valve PECO PECO Energy PEP Performance Enhancement Process PLC- Programmable Logic Controller RECW Reactor Enclosure Cooling Water RCIC Reactor Core Isolation Cooling
- RPS Reactor Protection System RPT Recirculation Pump Trip RPV Reactor Pressure Vessel SJAE Steam Jet Air Ejectors TS Technical Specification UCRL Unified Control Room Log '
UFSAR Updated Final Safety Analysis Report URl' Unresolved item VIO Violation W/O Work Order i
e l