ML060580162
| ML060580162 | |
| Person / Time | |
|---|---|
| Site: | Salem, Hope Creek  |
| Issue date: | 02/01/2006 |
| From: | - No Known Affiliation |
| To: | Office of Nuclear Reactor Regulation |
| References | |
| FOIA/PA-2005-0194 | |
| Download: ML060580162 (10) | |
Text
Performance Themes for Use after Inspection Exits
- 1.
POOR EQUIPMENT RELIABILITY.
Numerous plant level events and shutdowns [review expanded results over last 12 months; Pls]
Salem 1 - Equipment problems necessitating unplanned significant downpowers or reactor trips.
January 21 - Grassing/Circulating water system challenges (CW)
Feb 25 - Station Air Transient (not a significant downpower, but a near loss of station air to both Salem units)
March 3 - Grassing/CW March 24 Grassing/CW, necessitated two significant downpowers July 29 - Rx Trip due to 500kV 1-5 Breaker failure, complicated by CW SWGR modification Sep 19 - Grassing/CW Sep 20 - Switchyard salting and arcing (Actually can be considered a conservative operational decision)
Oct 15 - TS required shutdown due to stuck 14 SG FRV (FME)
Nov 25 - Grassing/CW Salem 2-Equipment problems; necessitating unplanned significant downpowers or reactortrips. -
March 29 - Manual Rx Trip for grassing/CW April 17 - Continuous pzr spraydown due to operator error (not a significant downpower, but a near required manual rx trip)
May 10 - Stuck open moisture separator reheat drain tank relief valve Sep 20 - Switchyard salting and arcing (Actually can be considered a conservative operational decision)
Nov.23 - Manual Rx trip due to dropped rod during physics testing Hope Creek - Equipment problems necessitating unplanned significant downpowers or reactor trips.
March 17 - Unplanned maintenance outage due to stuck open TBV.
June 16 - TS required downpower for EDG intercooler pump seal leak.
June 25 - Downpower after electrical transient. Transient tripped operating offgas train and SOV failure prevented remotely starting redundant train requiring downpower to maintain vacuum.
September 19 - Electrical transient due to lsabeVsalting/500kv switchyard fault. Automatic scram on low level resulted from unreliable RFP aux oil
system that resulted in 213 RFPs tripping. Failure of 7.2 kv non-safety breaker to trip.
October 4 - Manual scram due to EHC oil leak. Operators had problems controlling level due to long standing work around in feedwater system and design problem with setdown setpoint function. Pi hit.
December 5 - Planned forced shutdown to repair steam leak. Required to proceed to cold shutdown due to repetitive RWCU flange steam leak.
Frequent component failures and unplanned LCO entries Salem:
14BF19 (14 SG FRV), fine issue CFCU's, numerous occurrences related to bearing issues, sw valve issues, & controller issues 12AF1 1 (SG AFW FCV}), non-thorough corrective actions did not id out of position control air isolation valve 12SW380 (12 CCW. HX outlet), less than complete questioning attitude did not consider actuator air leak as a symptom of a bigger problem EDG PT Drawer Fuses, slow to implement DCP for a known recurring problem, less than complete interim corrective actions, poor questioning attitude while implementing interim corrective actions SW1 7 valves, significant leakby on header crosstie valves has often challenged ability to perform IST and affected results creating emergent conditions 1 SW pump, non-aggressive IST performance trending did not trend failure, contingency plants were not established 12SW 17, SW header cross-tie MOV, deficient maintenance practices caused repeat failure with replacement torque switch Unit 3 Gas turbine, 800 hours0.00926 days <br />0.222 hours <br />0.00132 weeks <br />3.044e-4 months <br /> unavailability, various, starting air, operator attentiveness, relay issues.
Chillers, various, control air, sw valve performance, operator inattentiveness caused a chiller to trip on freeze protection One occurrence of a EDG inoperability due to starting air compressor performance CCW heat exchanger grassing CCW pump oilers, poor DCP design change, required reversing back to original oiler design, engineering did not understand bearing design yet implemented change Charging pump biofouling, slow to implement positive displacement pump operation which will almost eliminate ECCS unavailability due to grassing 1 3AFWPP (turbine driven) steam admission valve, near luck that this was not risk significant, non-aggressive IST performance trending, non-questioning attitude on surveillance results, did not follow IST program
requirements, maintenance practices were root cause for valve/turbine inoperability RHR waterhammer, repeated occurrences, NRC id'd loose pipe hangers that should have been id'd by ops/engineers, corrective actions were slow, initial engineering response was not complete, less than complete corrective action implementation caused a containment spray waterhammer during the U2 outage and an additional RHR waterhammer Auxiliary building ventilation charcoal testing, repeated testing failures, availability of charcoal has impacted testing schedule, test failure mechanism initially not understood Salem Unit 2 ECCS leakage during startup, 22RHR hx leak not early identified, 22RH19 packing leakage not identified during 22RH18 pmt Hope Creek A SW traveling screen failures due to inadequate maintenance/procedure adherence. (July 2003)
Repeat SLC pump failures due to inadequate IST trending (October 2002)
Repetitive intercooler and jacket water seal leaks on EDGs Examples of missedlextended PMslcorrective maintenance/CA's/DCP affecting component performance Salem:
Redundant air panels, three examples Station air trarisient in February caused letdown line relief to lift, affected air panel had never been properly pm'ed 23 Chiller inop due to fouling or redundant air panel line, no pm exists to blowdown air lines with known water intrusion events Redundant air panels are not clearly assigned to system engineers EDG PT drawer fuse DG#P, slow to implement DCP caused multiple EDG inoperabilities Unit 2 Nov 23 manual reactor trip for dropped rod, CRDM fuse DCP deferred and risk accepted resulting in startup challenges and ultimate manual reactor trip to dropped rod during physics testing
- 2.
CORRECTIVE ACTION INEFFIECTIVENESS.
Substantive cross-cutting issue for PI&R likely to continue Hope Creek Biennial PI&R Inspection - Nov/Dec'03 3 Findings + 1 licensee id Finding Green self-revealing finding: Failure to identify and correct a deficiency associated w/ CIV.
Green self-revealing: failure to identify and correct condition that rendered B control room/CREF unit chiller inoperable.
Green NRC ID NCV: Failure to take timely CA for repeat cycling of RHR min flow valve cycling during pump start and multiple trip unit alarms.
Green licensee id NCV: failure to take adequate CA to preclude repetitive missed TS surveillance requirements Overall:
Team identified several examples of where PSEG failed to identify discrepant conditions and initiate notification (minor issues)
Weaknesses in evaluation of degraded conditions, documentation of actions, and completion of identified CA.
Weaknesses in PSEG's effectiveness in precluding recurrence of conditions adverse quality. (From review of past events)
Ineffective problem resolution such that repetitive problems occur Salem:
See above under 1, numerous plant equipment issues have ineffective corrective actions as a root and contributing cause.
Hope Creek 10/03 - CREF repeat failures due to lack of follow through on float check.
10/03 - Repeat problems with 1 and 2 FWH isolation on full power scram Day-to-day evidence that problems are unreported Salem:
EDG material condition, ops and system engineer have not set high standards for EDG and EDG room material condition Safety-related structures with roof leaks, organization has been slow to fix, EDG has been impacted twice, U3 gas turbine on at least one occasion
Unit 2 PRT quench water relief valve operation, operators accepted as normal system operation 11 and 22 RHR HX leaks, leaks existed for some time before identified PZR doghouse and U2 containment FME condition, the numerous examples suggest that frequent entries by station personnel were not identifying and removing FME that can impact sump screen performance U2 RHR HX temperature computer points reversed, over 1 month in outage before NRC had to identify, operators almost continuously monitored Hope Creek:
EDG material condition Evidence of poor self-critical behaviors and questioning attitudes Salem:
12AI 1 (SG AFW FCV), non-thorough corrective actions did not id out of position control air isolation valve 12SW380 (12 CCW HX outlet), less than complete questioning attitude
- did not consider actuator air leak as a symptom of a bigger problem Unit 2 pressurizer continuous spray down, control.room operators did not question multiple indications that operators in the field were not performing operations as pre-briefed RHR waterhammer, repeated occurrences, NRC id'd loose pipe hangers tAhat should have been id'd byops/engineers, corrective actions were slow,
- initial engineering response was not complete, stress analysis was not being expedited, less than complete corrective action implementation.
caused a containment spray waterhammer during the U2 outage and an additional RHR waterhammer Unit 2 SFP HX leak and chromates, operators and engineers did not initially question impact of chromates from CC water on spent fuel and SFP, very old data was used to support emergent question when chromate levels were already very high, OD focused on loss of inventory from CC system Hope Creek:
3/03 - Reassembly of #4CIV during RFI 1 with unmatched cap screws and uneven flange gap not identified or assessed, resulting in 10/4 manual scram.
3/03 - Drywell pipe insulation standards and drywell cooler cleanliness from RF1 1 identified as unacceptable during 12/03 drywell entry. Potentially impacts torus strainer performance.
3/03 - During non-routine shutdown unexpected EHC response when 2 TBVs opened/shut. Operator proceeded and 2nd transient occurred. Eq problem.
3/03 - Rl 1 leak checks for EHC not completed with system pressurized due to inadequate understanding of system.
7/03 - Operator identified irregularity with EDG fire impairment. No follow-up until inspectors questioned it later. Identified unnecessary fire tagout and missed fire watch.
8/03 - Significant EHC oil additions in 8/03 and 10/03 not questioned/trended to identify #4CIV EHC leak.
10/03 - EDG lube oil bolted joint leaks treated as housekeeping without identifying as repetitive and questioning why different hardware visible between EDGs.
10/03 - C EDG intercooler leak. Assumed housekeeping w/o assessing history.
12/03 - Operators did not adequately monitor steam leak or map out likely plant response and procedure usage until asked by inspectors. Leak assumed to be pinhole (assume the best) and not E/C.
12/03 - Maintenance re-installed stud in #1 TCV that was visually different than studs in other TCVs with engineering to evaluate for later correction in RF12.
12/03 - For two years B EDG load oscillates in droop during monthly ST load.
IDR mod in 11/03 ineffective. CR operability assessment did not re-address after new information to question cause. Also applicable TS requirement in droop mode not discussed.
9 Ineffective post-event reviews and root cause analyses Salem:
July 29 trip and electrical transient, 3 Weeks after event minimal progress was made on root cause, operability eval was not comprehensive, lowest operational bus voltage was not considered for net gain in voltage by blocking auto transfer of CW busses - non-conservative, root cause for switchyard breaker failure was not extensive (3 cable failures in one year time from that appear to be associated with cable penetration seals).
1 CC1 7 MOV valve (component cooling crosstie valve), root cause attributed hardened grease yet did not go one step further to id cause of hardened grease, NRC inspectors id'd poor maintenance practices that probably led to hardened grease Actions and evaluations uncoordinated between departments Salem:
144BF1 9, 14 SG Main feed reg valve troubleshooting, fine had bound valve, troubleshooting was being redefined late in the game by l&C after operators had already made plans to reduce power and shutdown the plan. Initial observations by personnel on the mid-night shift were not clearly understood.
ORGANIZATIONAL ISSUES (For possible use during management discussions)
A.
ENGINEERING Evaluations of events and degraded conditions have been inaccurate and less than thorough Salem:
July 29 trip and electrical transient, 3 weeks after event minimal progress was made on root cause, operability eval was not comprehensive, lowest operational bus voltage was not considered for net gain in voltage by blocking auto transfer o0 CW busses - non-conservative, root cause for switchyard breaker failure was not extensive (3 cable failures in one year time from that appear to be associated with cable penetration seals). July 29 reactor trip and LOOP RHR waterhammer, repeated occurrences, NRC id'd loose pipe hangers that should have been' id'd by ops/engineers, corrective actions were slow, initial engineering response was not complete, less than complete corrective action implementation caused a containment spray waterhammer du ring the U2 outage and an additional RHR waterhammer 12SW380 (12 CCW HX outlet), less than complete questioning attitude did not consider actuator air leak as-a symptom of a bigger problem 12AF1 1 (SG AFW FCVI, non-thorough corrective actions did not id out of position control air isolation valve-Hope Creek:
1 and 2 string FWH isolation from full power scrams cause evaluated correctly but corrective actions viewed as enhancement rather than design problem. Condition continues since 1998. (10/03)
A SW traveling screen evaluation missed procedure adherence' problems and deltas between VTIM and maintenance procedure. (7/03)
HPCI lube oil pressures; out of required range during ST but not evaluated or corrected until inspector inquired. (3103)
System (or component) engineers have lacked understanding and have been ineffective in addressing problems [qualification, stability, higher standards, etc]h
Salem:
EDG banjo bolts and turbocharger issues, system engineers were not familiar with historic issues on banjo bolts and turbocharger failures Station air, a significantly challenged system, has had three system engineers over two years, weak corrective actions exist for water intrusion, Salem is confronted with a large DCP feasibility to replace both units compressors 11 SW pump, non-aggressive IST performance trending did not trend failure, contingency plans were not established, valve issues (leak by) and flow monitoring have often challenged repeatable results from other service water pump tests 13AFWPP (turbine driven) steam admission valve, near luck that this was not risk significant, non-aggressive IST performance trending, non-questioning attitude on surveillance results, did not follow IST program requirements, maintenance practices were root cause for valve/turbine inoperability Recent reorg has challenged system engineering stability, yet no identified problems related to inexperience with assigned system have occurred Hope Creek PMT and engineering walkdowns identified EHC oil leaks from turbine valves, but engineering did not understand that with turbine tripped these leaks were not under normal pressure, so leaks were accepted without adequate technical justification. Causal factor in 10/4/03 scram.
Engineers did not properly evaluate shaft thrust during troubleshooting of A EDG intercooler pump seal leak even through this information was available in work packages used 3 months prior. (6/03)
Some design changes did not solve problems and had implementation issues Salem:
CRDM control power D('P. Westinghouse had recommended a control power DCP and later recommended that larger fuses be installed due to increased susceptibility for dropped rods. PSEG deferred fuse DCP and a manual rx trip was necessitated due to a dropped rod during physics testing.
CC Pump bearing oilers. PSEG had to reverse a DCP back to the original design on six pumps because the bearing design and oil requirements was not initially well understood Unit 2 turbine upgrade. PSEG is in the initial development of a DCP to raise Tave. The unit 2 1turbine upgrade assumed a higher steam
generator pressure that is not currently achievable. Salem has experienced degradation in s/g performance after trips yet the impact of this was not considered in the turbine replacements.
ECCS Accumulator MOV stroke time DCP. PSEG developed a DCP to change the stroke time of ECCS accumulator discharge MOVs yet did not consider the impact on check valve testing. This resulted in an urgent IST relief request to NRR.
Hope Creek 10/03 - B EDG IDR relay mod disturbed governor. Jumper detached and not identified in bottom of cabinet. Test failure. Resulted in un-installation of mod and accumulation of significant LCO time w/o safety benefit.
Maintenance & Work Control Weak work packages hamper planning and work execution Standards for planning and implementation are often low Supervision at work site has not supported standards and accountability Hope Creek Wrong procedure used to install A EDG intercooler pump seal first time (6/03)
A TS screen key cut wfo procedure direction contributing to failure.
Unmatched belts installed to MG set fan contributing to downpower. Also prior broken belt adjustment bolt not reported and repaired.
C.
Operations Operations procedural adherence inconsistent at times Hope Creek:
Level not maintained wilthin EOP band after RF1 1 scram to shutdown.
(3/03)
Abnormal procedure for RCP vibrations directed pump speed be reduced, however operators assumed indication was inaccurate w/o technical basis. (11/03)
D.
Licensing / Regulatory Credibility.
Poor working relationships during inspections & defensiveness to NRC concerns Slow, ineffective evaluations and corrective actions on NRC issues Ineffective NRC/PSEG interactions on NOEDs, PRA assessments, etc Tendency to blame assessments on communication Overly optimistic approaches (most challenging site)
G:\\Br.3\\Assessment\\Dec 16 03 talking points.wpd