ML20147B287
| ML20147B287 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 06/19/1995 |
| From: | Jason White NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | Barber G NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| Shared Package | |
| ML20147B009 | List: |
| References | |
| FOIA-96-351 NUDOCS 9701300132 | |
| Download: ML20147B287 (59) | |
Text
..
h1
/
From:
G.
S ott Barber (GSB) /
To:
JRW Date:
Monday, June 19, 1995 3:46 pm
,r
Subject:
SALEM ENF CONF SCHEDULE CHANGE i
THE 6/23 SALEM ENFORCEMENT CONFERENCE ON MULTIPLE EXAMPLES OF INEFFECTIVE CORRECTIVE ACTIONS HAS BEEN TENATIVELY RESCHEDULED FOR 10:00 AM, THURSDAY, JULY 20.
OE CONCURRED IN THE CHANGE.
IN ADDITION TO THE PREVIOUS EXAMPLES, THE NEW CONFERENCE WILL 4
INCLUDE THE RHR MINIMUM FLOW VALVE, EMERGENCY SWITCHGEAR FAN, AND CONTAINMENT AIRLOCK SEAL OPERABILITY ISSUES.
THESE ISSUES WERE PANELED ON 6/13/95.
SEE PANEL BRIEFING PACKAGE FROM THAT DATE
[
FOR ADDITIONAL BACKGROUND.
CC:
RWC,WDL,DJH,JAJ,JFS3,LNO,CSM,JGS,THF
't i
(
be e
L f' bf j
t' 9701300132 970124 i
a m
wp.s,,
as s-w+,am.
4 b
osak-+0**ta-A~"M4L s e'Im md-4&
4-1LA 4
-'-*'^A'"
-^A
""'"'='M^'"#'--'A'
- ^ " " ^ " "
"'""*"'~#""
b 7/2Wu-f: s I'
SA2EM
- ~6 fx e - & w =
4 l
4 3
MdF'74a3 4/ d M # d / ~ C d 4/.$ M v f CAL /A/ IM MsMI.W
~
s w evo w r u c ~ ~ vsa c w w - ~ a m a I
or as aar m cuze n n, m a io mr
' m oo o4 1/4 m coes' cA::ra~,
1 A-te uso der dev*eA7 CAs' cur 3rier 4c daars.s o,.sA.j-g
[
I
~
/
1 l.W
}
1>te.p sq &.s/Qu
)n'*n0 0/
DOF.3/Dc~
a M.$
MMM M[
C'4.
oae stoepany af l
0 x a 'r s a 4 w s & E 9 AUrc AJon m ad gl t
o m ioJ
& w ac.
Dw'r - meoues I
h
/
/
h[.m [.
1 h { Yp fb$
4Ad/* fW 47A*97A) VA3* VM3 4 W.f * /-
Le sa2saaes A^t?~~o/o mo sa>
asavcse JrN Vamb/6. Thf l
$&)
J I
9 f
fe 1 4 1. v e e 4 6. & -a avw s w severw a,
i Am L ee x)A
/ 4-6 xworr-ser-e pC;" J/H/4st pql'
~ 0$
WMV y
gr- ?*>v n A
/t2L E 3*WS cr4;E//tpt o*>F44At.4)/*
fY /,k,',vnO YW
.l a
^ l l f
1 Doc, 7M'sta r r& WMY ftAd5&mJT~ s921/4 //Y
/D 1
t MbJrow 9~? s/*9se.rzv23r'
/f4Croc c./Das.)
S j
t
I SALEM PREDECISIONAL ENFORCEMENT CONFERENCE s
IR No.
Date Discussion of Issues / Apparent Violations 94-32 3/30/94 The original POPS setpoint analysis (SER date 2/21/80) supported a 375 psig setpoint. A 3/15/93 Westinghouse
= -
NSAL informed the licensee of nonconservatisms in the setpoint methodology for POPS for low temperature overpressure transient conditions. After 9 months of analysis, the licensee concluded that the corrected peak Sect.
transient pressure would exceed P/T limits (450 psig, 10 2.0 unit 1; and 475 psig, unit 2),
i.e., 485 psig. On 12/3/93 the licensee dispositioned the matter by administratively limiting operation to 2 RCPs when less Sig.
than 200* and increasing each unit's P/T limit by 10%
Issue based on unapproved ASME code case N-514 (10CFR50.60).
15 Though the licensee knew that the design bases was exceeded, the condition was not reported (10CFR50.72/73).
In early January 1994, the licensee recognized the 1
inappropriateness of using an unaooroved code case and subsequently elected to take credit for the capacity 20 provide by RPo. system suction relief valve RH3 to augment POPS relief. The licensee's analysis indicated that with RH3 available, the transient peak pressure would remain below the App G limits. The licensee took credit for, j
but continued to analyze the ability of, RH3 until April 25 94, though no 50.59 evaluation or approval from NRC for TS amendment was initiated.
In April 94, a DEF was
[l generated which identified that RH3 was not credited in the POPS analysis or in the design bases.
[ Note: For over a year, the licensee had failed to effect any 30 corrective action or come to any resolution of the POPS setpoint issue (10CFR50 App. B, Criterion XVI)].
Though the issue was finally entered in the DEF process, the fact that Salem was outside design bases was still not reported. To addrcas immediate safety concerns, 35 procedure revision was made to assure that only one RCP tl would be available in Mode 5 (to limit the dynamic head error affecting P/T limits), and efforts were taken to assure the availability of RH3 (though no TS amendment was initiated).
[ Note: Throughout this period, the 40 licensee's effort was directed toward developing a rationale to support that the nonconservatism expressed in the NSAL did not apply to Salem.)
In a final attempt, the licensee elected to change the bases upon which the original POPS analysis was founded by relying on 45 procedural controls to limit possible injection sources.
By limiting magnitude of mass addition, and revising the limiting transient upon which the setpoint was based, the licensee was able to predict a peak transient of 438 psig (i.e., below the P/T limits) and therefore considered 50 that the existing POPS setpoints continued to be valid.
However, as of 12/94 the change in the POPS design basis was not reviewed to determined if an USQ existed (10CFR50.59).
G: SALEM.PEC 1
9 "
IR No.
Date Discussion of Issues / Apparent Violations 95-02 4/7/95 A.
In refueling outage 2R7 (May 93) the licensee installed a design change to accommodate the results of their analysis of pressurizer safety valve performance concerns discussed in 0737, Item i
Sect.
II.D.1.
The DCP involved system modifications to 4.4 remove the loop seals associated with 2PR3, 4, and 5 by establishing a drain system for the loop seals Sig.
(with associated drain valves and header; and Issue change of the valve internals to materials designed 10 to operate in a steam only environment. Upon completion of the modification, the drain valves were added to the valve lineup scheme, including the common isolation valve 2PR66 in the common drain header. The required lineup specified 2PR66 15 to be open, however the lineup was not accomplished. Further, no evidence could be I'l determined that a post modification test had been accomplished relative to the DCP. Consequently, the 2PR66 remained closed causing water due to 20 condensation to remain within the loopseal throughout the operating cycle to 2R8 (October 1994). As a result, a safety related system existed in an unanalyzed configuration (10CFR50, App. B, Criterion V).
B.
The following are issues involve 10CFR50 App. B, criterion XVI.
B.1 On June 7, 1994, material management documentation for limit switches associated with head vent valves erroneously identified the material as NSR. A DEF identified that a switch short circuit could render two head vent valves inop since they were powered Sect.
from the same common circuit. However, the DEF did
[,
35 4.5 not identify any operability or safety concerns based on the reviewers conclusion that, whether NSR Agg.
or SR, the switches were the same and were Issue differentiated only by test certification for the SR component.
In February 1995, the licensee determined that NSR switches were actually 40 installed in the Unit 1 vent valves. However no safety evaluation or analysis was performed to demonstrate the acceptability of NSR parts installed in a SR application or the bases for continued operability of the system and unit.
45 50 G: SALEM.PEC 2
r i
i IR No.
Date Discussion of Issues / Apparent Violations 95-02 4/7/95 B.2 On Feb. 24, 1991 Unit operators put the control of a PORV in manual mode, rendering it inop, but Sect.
failed to adhere to the TS 3.4.3 AS to closed the 2.3 block valve within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. The condition was identified and corrected 23 hours2.662037e-4 days <br />0.00639 hours <br />3.80291e-5 weeks <br />8.7515e-6 months <br /> later. This is a Agg.
repeat of a similar occurrence involving Unf t 2 on
==-
Issue March 24, 1994 B.3 On July 6,1994 head vent valve 2RC40 failed to 10 operate during testing while Unit 2 was in cold shutdown. The licensee speculated that the low RCS temperature caused boric acid crystallization that i
Sect.
prevented the valve from functioning.
Later when 4.6 RCS temperature was increased, the valve stroke i
15 test satisfactorily, convincing the licensee of the Agg.
accuracy of their speculation. The valve was Issue returned to service on July 10, 1994 with no further review, evaluation, or assessmert iaw normal work control process procedures.
20 Consequently no actions were init16 tad to address maintenance, operability, corrective action
. issues, or generic implications.
25 i
1 I
f 1
l 1
G: SALEM.PEC 3
I
IR No.
Date Discussion of Issues / Apparent Violations 95-07 5/24/95 The following issues involve 10CFR50, App. 8, Criterion XVI.
Sect.
1.
An oil sample lab raport dated 8/4/94 recommended 4.3 resampling and changing oil on the 21 high-head SI Agg.
pump due to significant increase in wear particle Issue concentration.
4 Sect.
2.
An oil analysis dated 11/28/94 identified high wear 10 4.3 particle concentration in the 22 high head SI pump Agg.
speed increaser oil.
Issue 4
In both these cases, the system engineer, though aware of the findings of the lab reports, did not initiate any 15 followup, evaluation, or corrective measure; or establish a bases for operability or reliability in view of the apparent degraded condition of the equipment. The a
degraded nature of the equipment was not entered into the Equipment Malfunction Identification System (EMIS) until 20 March 20, 1995.
Sect.
3.
A lab report, dated 10/6/94, recommended resampling 4.3 the 23 AFW turbine lube oil due to some small Agg.
amount of water found and an increase in wear 25 Issue particle concentration.
The degraded nature of the equipment was not entered into i
the EMIS by the system engineer until 3/27/95. The system engineer did not initiate any review, evaluation, or establish any basis for equipment operability or reliability.
]7g, A r !
~
Sect.
4.
In May 1994, a systems engineer initiated a work I
4.3 request to inspect the 2Al 28 VDC battery charger
)DefM 35 Agg.
due to configuration concerns involving the ground Issue detection circuit. The work order to accomplish yg/
the task was not issued until April 1995.
/
Sect.
5.
LER 95-05 identified seven instances, between 40 4.3 5/8/90 and 1/14/95, of PSVs being beyond the 1%
Agg.
tolerance required by TS 4.0.5 for Unit 1.
Four Issue instances were identified between 11/14/94 and 1/14/95 involved 2 or the 3 installed PSVs. In all instances, the vendor notified the appropriate 45 system engineer by telephone, with written followup reports. Notwithstanding, the responsible system engineer never initiated an Incident Report, and consequently, root cause, operability, and reportability actions were not accomplished.
G: SALEM.PEC 4
IR No.
Date Discussion of Issues / Apparent Violations 95-10 7/14/95 The following issues involve 10CFR50, App. B, Criterien XVI.
Sect.
1.
Though aware of degraded equipment condition 2.4 affecting the 22 RHR and the 21 RHR pump minimum recirculation flow valves since 1/26/95 and 2/9/95, Sig.
respectively, the licensee did not initiate any Issue action to determine and correct the cause of the condition or establish a basis for operability of 10 the affected RHR systems until 6/7/95 when Unit 2 was shutdown law TSAS.
Sect.
2.
Though aware of degraded equipment condition 2.5 affecting the 12 control area switchgear supply fan 15 motor (CASSF) since 12/11/94, the licensee Sig.
initiated no action to correct the condition, Issue evaluate operability with regard to design basis, or acquire replacement equipment (which was known to be obsolete. The design basis expressed in 20 UFSAR 9.4.6 describes a design consisting of three 50% capacity supply fans-two operating; one in standby. On 5/12/95 the 13 CASSF motor trip on overload.
In response the licensee attempted to develop a justification for continued operation 25 with two of the three fan motors inoperable based i:
on a dubious rationale. Finally, after being unable to justify operability, the licensee shut the unit down iaw TSAS on 5/17/96.
9 Sect.
3.
On 3/6/95 the Unit 1 perscanel airlock failed a i
3.6 leak test. No root cause assessment was accomplished. PSE&G wiped the seal with a masolin Agg.
cloth (oil impregnated) to remove any dirt which Issue was presumed to be the cause. Retest was 35 satisfactory. On 5/3/95, the airlock again failed the leak test. The same solution was applied.
Retest was satisfactory. On 5/8/95, the leak recurred. The licensee was prepared to apply the same corrective action until root cause was I
40 challenged by the SRI. Subsequently, a thorough I
root cause assessment was performed which l
identified that the airlock seal was deformed and was the cause of the recurrent leak. The previous wipedowns with masolin cloth merely applied an oil film which temocrarily masked the leakage.
l G: SALEM.PEC 5
0-IR No.
Date Discussion of Issues / Apparent Violations 95-10 7/14/95 4.
Since 2/29/92, several instances occurred involving 4
Cont.
failure (cracks) of the threaded portion of the a pressure switch instrument pipe nipples associated
- Sect, with emergency diesel generator jack water cooling 6.1.B system.
Previously, the failed component was merely rethreaded and/or replaced without any root Agg.
cause effort being applied. On 6/7/95, the Issue licensee finally determined that the cause was due to resonance frequency that could be treated by modification of the size and mounting of the pipe.
10 Sect.
5.
On 7/11/92, the 21RH10 (21 RHR pump discharge 6.1.A isolation valve) was observed to be " clanking."
on
'4/16/93 the valve was opened and inspected to 15 Agg.
investigate the cause of the noise. Two deep wear Issue marks were discovered on the disc, but engineering
[,
concluded that they did not affect operability of the valve since seat damage was not observed. The marks vere buffed out; the valve reassembled, and 20 placed in service. On 6/10/95, the loud noise was once again observed. While inspection of the valve was planned, the 21 pump was not considered to be affected even though the licensee had no basis to support the conclusion in view of the possibility 25 (as described by the SRI) that if the disk
(
separated from the stem (due to whatever mechanism was causing the loud impact noises), RHR flow would be lost or restricted. Subsequently, the licensee performed a more thorough assessment and developed a reasonable rationale (based on input from the l
valve vendor and a search of industry experience t
I with the valve type) that support that separation I
of the disk from the stem was not a likely failure.
Subsequently, the licensee committed to examine the i
35 valve and determine root cause once the system could be removed for service.
s l
f h
7 n
l u
G: SALEM.PEC 6
r L
.s e, w -
i possd. S*d S1"/
g ACTION ITEM NO.95-222 do e t ' # d'/'b 7 jf /so th) + f" '
DUE DATE,
O I
deme *, b '
p,- pRf ridsM N '
pf ggpw 4 W FROM: REGIONAL ADMINISTRATOR
//r/f,
y ACTION COPY INFORMATION COPY T. T. MARTIN
/
r W. F. KANE
/
R.W. COOPER
/
C.W.HEHL
/
J. T. WIGGINS J.J.McOSCAR
[
D. SCRENCI / V. DRICKS
/
C.Z.GORDON D. J. HOLODY
/(2.u,5)
D. J. CHAWAGA K. D. SMITH
(
ACTION REQUESTED:
1.
If corrective actions are acceptable and no special NRC response is needed, inform the Enforcement Officer who in turn will notify the Director, OE. (OE will acknowledge g receipt of the check, and inform the licensee that implementation of the corrective actions will be reviewed during a subsequent inspection.)
2.
If corrective actions are unacceptable, or a special NRC response is needed because of such things as a denied violation, or other licensee disagreement, such as with the application of escalation / mitigation factors, prepare response for concurrence of EO, RC, DRA, and RA, and for the signature of the Director, OE.
DATE ACTION COMPLETED:
(Return this form to Regional Administrator's secretary after action is completed.)
Region I Form 190 March 1994
(
- \\
-(
v Pche Service 0*ctnc and Gas Company L cn R. Elleson Pubhc Service Electne and Gas Company P.O. Box 236. Hancocks Bndge, NJ 08038 609-3391100 ce soci.., ome., a pr...o."'
Nuclear Business Un t NOV 151995 LR-N95196 United States Nuclear Regulatory Commission l
Document Control Desk Washington, DC 20555 Attn:
Mr. James Lieberman e
Director - Office of Enforcement I
Gentlemen:
RESPONSE TO NRC NOTICE OF VIOLATION INSPECTION REPORT NOS. 50-272/311/94-32, 50-272/311/95-02, 50-272/311/95-07 AND 50-272/311/95-10 SALEM GENERATING STATION UNIT NOS. 1 AND 2 DOCKET NOS. 50-272 AND 50-311 On October 16, 1995, the Nuclear Regulatory Commission (NRC) issued a Notice of Violation (NOV) and proposed a $600,000 civil penalty for violations identified by the NRC during four inspections that occurred between December 5, 1994 and June 23,
{
1995.
The NRC issued to Public Service Electric & Gas (PSE&G) reports for these inspections on March 30, April 7, May 24, and July 14, 1995.
A predecisional enforcement conference was held on July 28, 1995.
PSE&G does not dispute the violations cited in the October 16, 1995 NOV.
Therefore, pursuant to 10CFR2.201, PSE&G submits its reply to the October 16, 1995 NOV.
An electronic transfer of funds payable to the Treasurer of the United States in the amount of the proposed civil penalty will be made on November 15, 1995.
As the NRC is aware, PSE&G management realized that significant steps were necessary to reverse the performance decline at Salem.
Therefore, on June 7, 1995, a decision was made to maintain Salem Unit Nos. 1 and 2 shutdown - until performance improves to acceptable levels.
The self-imposed shut down sent a significant message to PSE&G employees.
PSE&G management is i
il Documant Control Desk ND; 151995 LR-N95196 serious about the changes necessary for plant safety, personnel performance, and process improvement.
PSE&G evaluated the apparent violations'and broader concerns identified in the four inspection reports.
Based on this i
evaluation, our July enforcement conference presentation focused on three critical broad areas that had to be improved before acceptable and long-lasting changes at Salem could occur.
These areas are: (1) establishment of a culture that will facilitate improvement, (2) improvement of self-assessment capabilities, and I
(3) ensuring timely and thorough problem assessment and resolution.
These focus areas and their underlying problems are a subset of concerns being addressed in the Salem Restart Plan.
The details of the Restart Plan will be formally submitted on the docket and discussed with you during the public meeting presently scheduled for December 1995.
(
In addition to our response contained in Attachments 1 through 5, we provide below a discussion of our progress in addressing the three focus areas.
Culture change Improved personnel and organizational performance is currently and
{'
will continue to be a focal point for the new management team and is considered essential in establishing the proper safety culture within the Nuclear Business Unit (NBU).
To aggressively change the culture of the NBU, most of its top management has been replaced. This change signals the most important factor that distinguishes present activities from those of the past.
One of the key characteristics of the new managers is the ability to lead by example.
Personnel selected for this team have demonstrated the necessary leadership capabilities as well as the high standards necessary to develop a quality organization.
Most of the individuals come from nuclear units which have had successful performance turn-arounds and operate at an excellent level.
NBU management has placed an emphasis on the development and communication of roles and responsibilities to the organization, as well as establishing expectations for individual performance.
The following are examples of initiatives which have been established to drive the process of change.
l o,
Documtnt Control Desk
-3 NOV 151995 l
LR-N95196 l
First, several of the action plans developed to support restart recognize the need fer improved definitions of organizational and individual roles and responsibilities.
For example, a Conduct of Operations document is now being finalized which communicates management expectations and, as importantly, establishes the ethic of the Operations organization.
Roles and responsibilities for system engineers have already been defined and communicated to the System Readiness Review Process and system engineer support improvement initiatives.
The goal of these communications is to establish the necessary standards against which personnel and t 1 organizational performance can be measured and held accountable.
Secondly, a Performance Ranking process has been instituted to assess individual performance in the following behavioral areas; Teamwork and Leadership, Initiative and Results Achievement, Job Knowledge, Communication, and Adaptability and Flexibility.
Individuals will develop improvement plans appropriate to their l
overall standing.
Tais process is designed to identify and confront substandard performance that has gone undetected or unchallenged to date.
In addition, personnel who fail to make prescribed improvements will be held accountable, up to and including discharge.
This ranking process represents the first of four performance review efforts to be conducted within the NBU over the next 18 months.
This focus on performance is intended to i
re-emphasize the responsibility of managers and supervisors to set and enforce proper performance standards and revise substantially the quality and productivity of the workforce.
Finally, managers and supervisors are being provided training to assist them in identifying, confronting and correcting performance issues.
The process being utilized has been implemented successfully at other nuclear plants, as well as non-nuclear
~
companies.
NBU management has established the expectation that line managers and supervisors attend this training and utilize this process.
Two protocol groups have been established to ensure the process is being implemented uniformly and consistently.
The Managers protocol group has recently developed the course content and identified significant issues to be addressed.
The Executive protocol group has evaluated the course content and training to ensure that expectations for this process have been satisfied.
In the longer term, the Managers protocol group will evaluate implementation of the process to promote consistency and make e..
t i.
Document Control Desk NOV 151995
-4 LR-N95196 appropriate recommendations on policy issues to the Executive I
r protocol group.
l These actions, effectively implemented, are expected to improve individual and organizational performance and will provide the infrastructure for the proper safety culture within the NBU.
As l
the impacts of these actions are measured, appropriate changes in i
approach and method will be made to achieve the lasting and profound changes being targeted.
I Self-Assessment Improvement The long-term objective for this focus area is to develop an organization which instinctively takes necessary steps to improve performance through effective self-assessment and timely corrective action.
A program defining expectations for self-(
assessment during routine operations has been developed.
Each Salem department has identified specific representatives to support this program.
These representatives have been trained on the program and its expectations.
To date, all but one Salem Station department has performed a self-assessment using this program.
The remaining departmental self-assessment will be completed in the near future.
Issues identified during these assessments will be reviewed and incorporated into the Salem
{
Restart Plan, as appropriate.
A second program, which provides guidance on conducting self-assessments for readiness to return to operation following refueling outages, is being developed.
Salem personnel are demonstrating their willingness to identify deficiencies and to initiate actions necessary for correction.
Indications of this can be seen in the March 24, 1995, d,
7
" Organizational Effectiveness Assessment Report for Salem Nuclear Generating Station," and our presentation during the July 28, 1995 enforcement conference.
This continues to be shown by system walkdown results, backlog review, and most notably, the number of condition reports being generated on a daily basis.
NBU management has and will continue to monitor, and to the extent necessary intervene, when self-assessment related expectations are not met.
L i
i
[
l Document Control Desk
-s-NOV 151995 LR-N95196 Timely / Appropriate Resolution A consolidated Corrective Action Program (CAP) has been implemented to communicate NBU management expectations on timely problem identification and resolution and provides clear i
definition of roles and responsibilities.
The CAP was designed using input from other utilities which have effectively managed program consolidations as measured by improved program and station
{
performance.
The consolidated program includes a low threshold
'l for reporting problems, provides aggressive problem assessment / root cause determination expectations and places
-management in charge of root cause and corrective action completion times.
Results to-date indicate that personnel are not hesitant to raise issues through the process.
The Director - Quality Assurance / Nuclear Safety Review has oversight responsibility for the CAP.
He has dedicated resources, under the Manager - Corrective Action and Quality Services, to fulfill that responsibility.
Measures have been established to monitor the performance of the corrective action process.
Recent data indicate overall improvement in evaluation completion times and a reduction in overdue corrective actions.
Station management receives daily reports on overdue evaluations - most of which have resulted from the volume of issues generated by system walkdowns.
i Accountability for CAP implementation rests with station line management.
As such, station managers review root cause evaluations for completeness and adequacy.
A Corrective Action j
Review Board (CARB) has been established at Salem and the General Manager - Salem Operations is its chairman.
Completed root cause assessments for significant issues are presented to the CARB where the adequacy of the cause determination and selected corrective
~
actions are evaluated.
A performance measure has been established which tracks the acceptance / rejection rate for CARB presentations.
This indicator is included in the monthly report to senior management.
O
i
~
DocumGnt Control DQsk NOV 151995 LR-N95196 A new element, being incorporated under the CAP improvement area, is the Operational Experience Feedback (OEF) Program.
This program is under review to identify needed improvements in the processing of internal and external OEF information.
This review includes a validation of actions taken in response to past OEF items.
Improvements to the OEF process itself will include the establishment of well defined roles and responsibilities, and standards of performance for implementing organizations.
Performance measures will also be established to allow NBU management to monitor program effectiveness and assign accountability if performance standards are not satisfied.
These changes are being made in order to better integrate the OEF program into the operation of the stations.
NBU management recognizes that, in addition to the changes already described, culture improvements and self-assessment capability improvements are essential to anchoring the CAP as an integral
[
part of sustained performance improvement.
We will establish and achieve appropriate performance standards for the CAP at Salem prior to restart.
Summagv We agree with the NRC that performance within the NBU must improve.
Our commitment to maintain the Salem Units shutdown until required performance improvements are demonstrated, changes to the NBU management team, and our aggressive actions to strengthen the safety culture within the NBU, illustrate the fundamental differences between our present actions and those of the past.
We will not restart the Salem Units until the hardware, important processes and programs, and organizational and individual performance reach acceptable levels.
Changes will continue, as needed, to ensure that expectations continue to be met after resumption of power operation.
s
NOV 151995 Docum:nt Control Desk
-7 LR-N95196 If you have any questions regarding this submittal, please do net hesitate to contact me.
Sincerely, Attachments se
i Document Control Desk 8
LR-N95196 C
Mr. T. T. Martin, Administrator - Region I
[
U. S. Nuclear Regulatory Commission 475 Allendale Road King of Prussia, PA 19406 Mr. L. N. Olshan, Licensing Project Manager - Salem U. S. Nuclear Regulatory Commission One White Flint North 11555 Rockville Pike Mail Stop 14E21 Rockville, MD 20852 Mr.
C. Marschall - Salem (SO9)
USNRC Senior Resident Inspector Mr. K. Tosch, Manager, IV
'NJ Department of Environmental Protection Division of Environmental Quality Bureau of Nuclear Engineering CN 415 Trenton, NJ 03625 (b)
.l
REF: LR-N95196 STATE OF NEW JERSEY
)
)
SS.
COUNTY OF SALEM
)
L. Eliason, being duly sworn according to law-deposes and says:
I am Chief Nuclear Officer & President - Nuclear Business Unit of Public Service Electric and Gas Company, and as such, I find the matters set forth in the above referenced letter, concerning the Salem Generating Station, Unit Nos. 1 and 2, are true to the best of my knowledge, information and belief.
{,
~
i Subscribed and Sworn te before me g
this._/d fh day of / h>1,mO/A_., 199 5 I
d_ta n h tf OluA
('
' Notary Publih c(f New Jersey KIMBERLY JO BROWN NOTARY PUBLIC OF NEW JERSEY My Commission expires on
~
I 0
1
Document Control Desk
-1 LR-N95196 ATTACHMENT 1 VIOLATION I.
10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action,
- requires, in part, that conditions adverse to quality are promptly identified and corrected; and in the case of significant conditions adverse to quality, the cause of the condition shall be documented, appropriately reported to levels of management, and corrective action taken to preclude repetition.
A.
Contrary to the above, a significant condition adverse to quality existed at the Salem Unit 2 facility frora January 26, 1995, until June 7, 1995, in that the Licensee was aware that the No. 22 Residual Heat Removal (RHR) pump minimum recirculation flow valve would not open on low RHR flow as required to prevent pump failure. Similarly, the Licensee was aware that the same significant condition adverse to quality existed at the facility from February 9, 1995, until June 7, 1995, for the No. 21 RHR pump minimum recirculation flow valve. However, prior to June 7, 1995, the Licensee f ailed to determine the cause of the valve failures or initiate corrective measures. (01013)
This is a Severity Level III Violation (Supplement 1)
Civil Penalty - $100,000 s
RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
On January 26, 1995, and February 9, 1995, different operating crews identified failure of the automatic open feature for the Residual Heat Removal (RHR) pumps minimum flow recirculation va3ves 21RH29 and 22RH29.
Both failures occurred as Salem Unit 2 was nearing completion of its eighth refueling outage (2R8).
Each failure was observed while the console operator (a licensed Reactor Operator (RO)) was reducing RHR flow in preparation to align RHR as an Emergency Core Cooling System (ECCS) flowpath.
In both cases, the operating crew initiated an( Action Request (AR).
Troubleshooting for these valves was subsequently scheduled for August 2, 1995 and June 27, 1995, respectively.
Although Operations personnel recognized that valve operability was Mode-dependent, they did not establish mode change constraints when the failure of the automatic open feature was recognized.
a Documant Control Desk
-2
- (cont'd)
LR-N95196 1
In June, 1995, following Operations department identification of 54 open work orders with potential operability concerns, these valves were targeted for immediate operability assessment.
Once valve operability was questioned, the RHR system was operated to test and evaluate valve response.
Valve 21RH29 failed to operate and was declared inoperable.
When tested, valve 22RH29 opened on pump start.
The Engineering Analysis Group (EAG) was tasked with performing a follow-up operability assessment.
The results of follow-up engineering evaluations did not provide sufficient basis to confirm 22 RHR loop operability.
As a result, with both RHR loops inoperable, at 18:27 hours on June 7, 1995, the operating crew entered Technical Specification 3.0.3 and commenced shutdown of Salem Unit 2.
I the time of the initial valve misoperation events, At an Operations Standing Order and Operability Determination (OD)
Flowchart were in place to guide Operations personnel in making Operability Determinations.
Licensed operators had received training on the use of the OD flowchart during the 1994 fall training segment.
Although the Standing Order and OD Flowchart e
were available on January 26, 1995, and February 9, 1995, the
[
operating crews did not perform an Operability Determination when the operation of the RH29 valves came into question.
ROOT CAUSE ASSESSMENT The RH29 valve control relays were tested and the most probable cause for valve misoperation was attributed to failure of the Struthers-Dunn low flow interlock relay.
1 PSE&G has determined that the root cause of the failure to identify and correct this condition adverse to quality was inadequate management commitment to the operability Determination process.
This was demonstrated by the following:
1.
The implementation of NRC Generic Letter (GL) 91-18 operating philosophy was not timely and effective in improving Operability Determinations.
2.
The implementation of Operations Department procedures (Operability Flowchart and Operations Department Directive SC. OP-DD.ZZ-OD02 (Q) (OD-2), " Operability Determinations") to improve Operability Determinations was ineffective.
3.
Less-than-adequate safety culture within the Operations, Technical Engineering, and Station Planning organizations, which was manifested by a tolerance for equipment problems and insufficient follow-through to correct these problems.
fa L
t Document Control Desk 3 - (cont'd) 1 LR-N95196 CORRECTIVE STEPS THAT HAVE BEEN TAKEN The Struthers-Dunn valve control relays for valve 21RH29 were replaced.
The 22RH29 valve control relays passed in situ functional testing and will be replaced prior to Unit restart.
Salem Unit 2 was shutdown to comply with Technical Specification requirem'ents.
To address the less-than-adequate safety culture issues, PSE&G management decided that Salem Units 1 and 2 will cemain shutdown until performance improves.
The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
OD-2, " Operability Determinations" has been revised to provide better guidance and expectations for performance of Operability Dete rminations.
Operator' awareness of NRC GL 91-18 is being reinforced during Salem licensed operator training.
These actions assure that management expectations regarding roles and responsibilities in the Operability Determination process are clearly understood and consistently applied
[
As an interim measure, the Operations department reviews active operability Determinations (OD's) periodically to ensure that actions and contingencies are progressing and/or completed.
The review process is directed by Operations procedure SC.OP-DD.ZZ-OD4 0 (Q), " Shift Routines."
To assess the effectiveness of the OD process, the Safety Review Group (SRG) is, on an interim basis, independently evaluating the OD's and providing feedback to
(
Operations management.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE The Operability Determination process, including the OD-2 procedure, is being further enhanced to: 1) improve the Engineering and Operations departmental interface; 2) ensure consistency between OD-2 and NC.NA-AP.ZZ-0006 (Q) (NAP-6)
" Corrective Action Program"; and 3) ensure tracking of Operability Determination status.
These improvements to the process will be completed by March 1,
- 1996, i
n-L
1 i
Document Control Desk
-4 Attachment i fcont'd)
LR-N95196 DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED PSE&G has identified and corrected the cause of the valve failures.
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner.
PSE&G will not 1
restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
l1 l
\\
i Document Control Desk LR-N95196 ATTACRMENT 2 VIOLATION B.
Contrary to the above, a significant condition adverse to quality existed at the Salem Unit 1 facility from l
December 12, 1994, until May 16, 1995, in that the No. 12 safety related switchgear ventilation supply fan failed on December 12, 1994, and the Licensee did not initiate resolution of the condition or effect any corrective measures to resolve the condition promptly.(02013)
This is a Severity Level III Violation (Supplement 1).
Civil Penalty - $100,000 RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
In December, 1994, the No. 12 Switchgear Penetration Area Ventilation System (SPAVS) supply fan tripped on overload
{i protection.
Further investigation revealed that the fan motor bearings had failed.
Repair of the fan motor bearings necessitated that the fan motor assembly be removed from the I
system.
A Temporary Modification (T-Mod) was required to maintain system / plenum integrity with the fan motor assembly removed.
As a result of poor planning and lack of communication, corrective actions had not been taken to repair the No. 12 SPAVS supply fan when the No. 13 SPAVS supply 5 ailed on May 12, 1995.
At the time of these failures, no spare supply fan motors were available.
Troubleshooting revealed that the second fan motor had developed an internal short to ground.
In accordance with the Salem Updated Final.3afety Analysis Report (UFSAR), normal system operation requires 1:wo of the three 50%
capacity SPAVS supply fans to be in service, with the third fan available in a standby mode to accommodate failures.
With the failure of No. 12 SPAVS supply fan motor in December, 1994, station personnel failed to recognize that SPAVS was operating outside the UFSAR assumptions.
On May 12, 1995, two of the three supply fans became unavailable and System Engineering personnel were unable to clearly establish the system's ability to fulfill its intended safety function.
A shutdown of Salem Unit 1 was initiated on May 16, 1995.
l l
l
I il Document Control Desk Attachment 2 (cont'd)
LR-N95196 ROOT CAUSE ASSESSMENT PSE&G has determined that the root cause of this event was ineffective corrective action.
Involved personnel failed to
)
l recognize the significance of losing redundant, important to i
safety components.
Due to a less-than adequate safety culture, prompt corrective actions, consistent with the safety significance of the equipment, were not initiated as evidenced i
l by:
1.
Failure to repair the first failed SPAVS supply fan motor in a timely manner.
2.
Lack of communication in the System Engineering organization.
3.
Failure to complete the work planning for repair by the issuance of a T-Mod which was not accomplished prior to the second SPAVS supply fan motor failure.
The Corrective Action Program (CAP), in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be identified and resolved in a timely manner.
That same program did not provide clear guidance on the need to perform nor the required content of assessments to support continued assurance of equipment operability.
The following contributing factors were also identified:
1.
Adequate Preventative Maintenance (PM) program tasks were not established for these fan motors.
Opportunities to establish appropriate PM's were missed due to lack of follow-through with regard to industry experience notifications and a previous SPAVS fan motor failure.
2.
Lack of clear understanding by Operations and Engineering personnel of the SPAVS design basis.
3.
Operations did not have a tracking system to assure that inoperable Technical Specification systems or support systems would be corrected in a timely manner.
CORRECTIVE STEPS THAT HAVE BEEN TAKEN On May 16, 1995, Salem Unit 1 was shutdown to comply with Technical Specification requirements when reasonable assurance of system operability could not be established.
l l
l
i Document Control Desk s
3 LR-N95196 (cont'd) i The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
j Preventive Maintenance Change Requests (PMCR's) were generated to create new PM Recurring Tasks to replace the SPAVS fan motor bearings on a regular basis.
All three Salem Unit 1 SPAVS supply fans were inspected and the fan motors replaced.
Operations Department procedure SC.OP-DD.ZZ-OD10 (Q) " Removal and Return of Nuclear Safety Equipment" has been issued.
This procedure provides guidelines for removal and return to service of all Technical Specification related equipment.
OD-2, " Operability Determinations" has been revised to provide better guidance and expectations for performance of Operability Determinations.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE All SPAVS supply fans on Salem Unit 2 will be inspected and the fan motor bearings will be replaced, on an as-needed basis, prior to unit restart.
Process improvements for the Operating Experience Feedback Program (OEF) are presently under evaluation.
This activity is being managed under the Corrective Action Program element of the Salem Restart Plan.
The Technical Specification Action Tracking procedure has been revised to require the NSS to verify and initial for completed Technical Specification acti;ns and allow for tracking of potential Technical Specification entries.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The No. 12 and 13 SPAVS supply fans were repaired.
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner.
PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
1 i,
Document Control Desk
- 1 LR-N95196 L
ATTACHMENT 3 VIOLATION l
C.
The Licensee was informed by Westinghouse on March 15, 1993, of a significant condition adverse to quality i
involving nonconservatisms in the setpoint methodology i
for the Pressurizer Overpressure Protection System (POPS) for low temperature overpressure transient conditions.
1.
Contrary to Criterion XVI, the Licensee took nine
{
months of analysis, from March 1993 to December 1993, to conclude that the corrected peak transient pressure would exceed pressure / temperature (P/T) limits as described in each unit's technical specifications limits.
After completing the analysis, from December 30, 1993, and continuing for approximately one month, the Licensee dispositioned the matter of the nonconservatism in the setpoint methodology for
[
the POPS by 1) administratively limiting RCS operation to two reactor coolant pumps when the RCS was less than 200 F and 2) increasing each unit's P/T limit by 10%; the latter corrective action was inadequate because it utilized as a basis an unauthorized ASME Code Case (N-514),
)
which the Licensee was aware was not acceptable j
pursuant to 10 CFR 50.55(a).
(03013)
This is a Severity Level III Violation (Supplement 1)
Civil Penalty -
$100,000 t
2.
Contrary to Criterion XVI, in January 1994, i
following the Licensee recognizing the unacceptability of using unauthorized Code Case N-514 as a corrective action to disposition the POPS setpoint methodology, the Licensee elected to implement corrective action by taking credit for the relief capacity provided by RER system suction l
relief valve RH3 to augment POPS relief capacity.
6 i
Document Control Desk Attachment 3 (cedt'd)
LR-N95196 However, as the Salem FSAR (Section 7. 6. 3. 2) describes the POPS system to include two Power Operated Relief Valves (PORVs) and does not describe Valve RH3, this corrective action was inadequate because an evaluation was not performed to determine the acceptability of the use of Valve 4
RH3 as part of the POPS system.
In addition, the Licensee failed to identify that on the receipt of a safety injection (SI) signal, a previously operating positive displacement charging pump's discharge, combined with the discharge from the high head safety injection pump that starts on i
1 receipt of the SI signal, could have injected water mass into the RCS at a rate that could have prevented POPS from performing its function.
-(04013)
This is a Severity Level III Violation (Supplement I)
Civil Penalty - $100,000 RESPONSE - DESCRIPTION OF CIRCUMSTANCES
(
PSE&G does not dispute the violation.
On March 15, 1993, Public Service Electric & Gas (PSE&G) was advised by Westinghouse of a generic issue involving a non-conservative setpoint calculation in the analysis of the i.
Pressurizer Overpressure Protection System (POPS).
The Low Temperature Overpressure Protection System (LTOPS) protects the Reactor Pressure Vessel (RPV) against pressurized thermal shock ri events as required to comply with 10CFR50 Appendix G criteria
(" Fracture Toughness Requirements").
PSE&G requested Westinghouse to perform a Salem plant-specific analysis for the cases of one, two or four reactor coolant pumps running.
On September 29, 1993, PSE&G received the plant-specific Westinghouse results and had evidence that a non-conservative setpoint (375 psig) could lead to violating the Technical Specifications and Appendix G pressure / temperature limits.
Over a period of three months (September to December, 1993), Nuclear Engineering personnel performed calculations to address this concern.
1 i
I 4
A
I Document Control Desk
-3 LR-N95196 (cont'd)
On December 30, 1993, the Nuclear Engineering department issued an evaluation (MEC-93-917) which restricted operations in Mode 5 to two reactor coolant pumps.
The recommended restrictions were implemented via revisions to the plant's Integrated Operating Procedures (IOP's).
Nuclear Engineering personnel improperly tool: credit for American Society of Mechanical Engineers (ASME) Code Case N-514 (which had not yet received NRC approval) as part of dispositioning this issue.
Despite the involvement of multiple departments during this evaluation process, numerous opportunities to recognize the reportability requirements for this issue were missed and, as a consequence,
'i j
the condition was not reported to the NRC.
i
^
On May 26, 1994, another evaluation (MEC-94-630) was issued which further restricted the number of operating Reactor Coolant pumps from two to one pump in Mode 5.
The new calculated transient values showed that Salem Unit 2 pressure did not exceed specified limits.
However, it was recognized that Salem Unit 1 could i
exceed its pressure limit during a mass addition transient below 200 degrees F.
Involved personnel failed to evaluate the calculated deviation from the specified limit against l
reportability requirements.
Likewise, there was a failure to recognize the need to establish justification for continued operation while this condition existed and the need to report that justification to the NRC.
i i
On June 13, 1994, Nuclear Engineering issued calculation S-C-RC-MDC-1358.
This calculation inappropriately took credit for use of a relief valve (RH3) in the Residual Heat Removal (RHR) system.
On November 17, 1994, it was determined that Salem Unit 1 could operate outside of the design / licensing basis for the POPS analysis if the following conditions existed: 1) a safety Injection (SI) signal was initiated; 2) Reactor Coolant System (RCS) temperature was below 200 degrees F; 3) a Reactor Coolant pump was in service; 4) a Positive Displacement Charging Pump was i
in service; and 5) power remained available to a maximum of one Centrifugal Charging Pump.
This discovery resulted in the issuance of Licensee Event Report (LER) 272/94-017.
l l
j l
I Document Control Desk 4 - (cont'd)
LR-N95196 On February 7, 1995, the NRC approved PSE&G's use of ASME Code Case N-514.
At that time, appropriate 10CFR50.59 Safety Evaluations were parformed for the resultant changes to the Updated Final Safety Analysis Report (UFSAR).
Implementation of the ASME Code Case provided additional margin (10%) and higher pressure / temperature limits for POPS during the LTOP conditions and re-established plant operation within its design and licensing bases.
In April, 1995, PSE&G issued Incident Reports to identify and evaluate the organization's inappropriate actions and their causal factors.
[
ROOT CAUSE ASSESSMENT PSE&G has determined that the root causes of this event were:
1.
Lack of understanding of the regulatory significance and reportability implications of the Westinghouse analysis results.
Specifically, the organization became too focused on the technical resolution aspects of the issue without I
adequate consideration of regulatory requirements.
2.
Lack of supervisor / management sensitivity to the need to implement existing procedures and processes which require timely entry of issues into the Corrective Action Program (CAP).
Monitoring of the Corrective Action process by management was insufficient.
3.
Inadequate training of engineering personnel on the use of ASME Code Cases, requirements of 10CFR50.59 and requirements for regulatory reporting.
l L i CORRECTIVE STEPS THAT HAVE BEEN TAKEN The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
)
1 l
l l
I l
i l
i
Document Control Desk
-5 (cont'd)
LR-N95196 NBU Management has re-emphasized the expectation that supervisory personnel must assess issues objectively.
Specifically, supervisory personnel must maintain their oversight role.
The i
Manager - Nuclear Engineering Design (NED) has verbally reinforced this expectation to the engineering design organization.
The Nuclear Engineering Design organization was surveyed relative to any past reliance on unapproved ASME Code Cases.
Based on this survey, no other instances of unapproved ASME Code Case use were identified.
Personnel involved in this occurrence have received appropriate reinforcement on procedure compliance, their responsibility for compliance with regulatory requirements, and problem reporting.
Management has re-emphasized by internal memorandum and follow-up review with engineering personnel that the potential impact on the UFSAR must be considered whenever design basis calculations, evaluations or assumptions are revised.
Departmental procedures provide clear guidance on these requirements.
The expectation for procedural adherence was also reinforced.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE
]
Engineering Design and Licensing & Regulation management will reinforce expectations for organizational interface to their i
personnel.
This will be completed by March 15, 1996.
Lessons learned from this issue will be disseminated to Engineering Support personnel during 4th quarter Operating Experience Feedback (OEF) training. This will be completed by January 15, 1996.
Process improvements for the Operating Experience Feedback Program (OEF) are presently under evaluation.
This activity is being managed under the Corrective Action Program element of the Salem Restart Plan.
Specific training on the ASME Code and NRC restrictions on its use will be provided to appropriate engineering support personnel.
This will be completed by January 31, 1996.
I Documsnt Control Desk Attachment 3 (cont'd)
LR-N95196
+
The Engineering Qualification training program is being revised to assure that job qualifications are consistent with job requirements and that Engineering personnel are trained consistently.
Required personnel training in' Code Job Packages will be incorporated into the Engineering Qualification Guide.
This training will include ASME Code Cases, NC.NA-AP.ZZ-0028(Q)
" Code Job Packages" procedure requirements and regulatory reporting requirements.
The revised Engineering Qualification Guides will be completed by January 31, 1996.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The request to use ASME Code Case N-514 at Salem station was approved by the NRC.
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifyin conditions adverse to quality in a timely manner. g and resolving PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
t O
i Document Control Desk 1
LR-N95196 ATTACRMENT 4 - IST EXAMPLE VIOLATION D.
Contrary to the above, on several occasions, conditions adverse to quality existed, but were not identified and promptly corrected,,as evidenced by the following examples:
1.
On June 7, 1994, the Licensee identified that material management documentation for limit switches related to the reactor head vent valves, j
improperly classified the components as non-safety related. A nuclear design discrepancy evaluation form (DEF) identified that a switch short circuit could render two head vent valves inoperable since the components were powered from the same common circuit. Notwithstanding, the DEF did not identify any concern relative to operability or safety. In February 1995, the Licensee determined that
[
non-safety related limit switches were actually installed in reactor head vent valves 1RC41 and 1RC43 at Salem Unit 1. Subsequently, the Licensee failed to perform and document an engineering evaluation to demonstrate the acceptability of continued Salem Unit 1 operation with non-safety-related parts installed in a safety-related application.
RESPONSE - DESCRIPTION OF CIRCUMSTANCES l
PSE&G does not dispute the violation.
On June 7, 1994, a Discrepancy Evaluation Form (DEF) was written to resolve an apparent conflict in saf te y classification between the Reactor Head vent valves and their corresponding position indicating limit switches for Salem Units 1 and 2.
On March 3, 1995, it was determined that non-safety related limit switches were installed in two Salem Unit 1 Reactor Head vent valves.
Investigation into this occurrence indicates that, in April, 1992, an opportunity to resolve the noted discrepancy was missed when a different DEF on the same subject was dispositioned.
The identified corrective actions in that DEF were not carried through to completion.
l i
4
,,.4m
-~
,.m
i Document Centrol Desk Attachment 4 (cont'd)
LR-N95196 ROOT CAUSE ASSESSMENT PSE&G has determined that the root cause of this occurrence was the erroneous classification of the Reactor Head vent valve limit switches as non-safety related.
Due to personnel error, these switches wer'e incorrectly assigned a non-safety related purchase class during a spare part Folio Classification initiative in 1986.
This error in classification initiated a sequence of events which resulted in the installation of non-safety related limit switches in an application originally designed to use safety related components.
4 The root cause of the failure to resolve this condition adverse to quality in a timely manner is attributed to an inadequate Corrective Action Program (CAP).
The CAP, in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be identified and resolved in a timely manner.
That same program lacked centralized oversight of the various mechanisms to identify and resolve discrepancies, i
CORRECTIVE STEPS THAT HAVE BEEN TAKEN In March, 1995, Nuclear Engineering Design issued an assessment to resolve the outstanding DEF.
This assessment concluded that the non-qualified switches did not affect the operability of the Reactor Head vent valves.
)
The following changes were made in the Nuclear Procurement and Material Management (NP&MM) system:
The Purchase Class 4 (PC4) Limit Switch Folio parts were put "On Hold" and were re-classified as " obsolete."
A New Purchase Class 1 Limit Switch Folio was created.
The Reactor Head vent valve limit switch component ID's were removed from the computerized Managed Maintenance Information System (MMIS).
Separate component ID's were determined to be unnecessary as the Bill of Materials (BOM) for the valves contains the Folio information for the limit switches.
e
' )
Document Control Desk Attachment 4 (cont'd)
LR-N95196 The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT R;.CURRENCE Non-safety related limit switches in the Reactor Head vent valves will be replaced prior to restart of Salem Unit 1.
Outstanding DEF's are being reviewed for impact on plant
- systems, including operability issues.
This will be completed prior to restart of Salem Units 1 and 2.
PSE&G is currently conducting a review of the MMIS database to determine if there have been other occurrences of safety related components being purchased as non-safety related.
The scope of this review will include components acquired under purchase class "PC4" (non-safety related).
Any additional occurrence (s) of non-safety related parts in safety related applications, discovered during this review, will be dispositioned under the current CAP guidelines which include documentation of operability Determination and evaluation for reportability, when appropriate.
il This review will be completed prior to restart of either Salem Unit 1 or 2.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The Engineering department has dispositioned the outstanding DEF.
i i
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner.
PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
l l
l
. _ ~ - - -.... _ -. -. -.
4 j
Document Control Desk LR-N95196 4
ATTACHMENT 4 - 2ND EXAMPLE f
VIOLATION i
2.
On February 24, 1995, Unit No. 1 operators placed 1
control of a PORV in the manual mode, rendering it i
inoperable, and failed to adhere to the Technical Specification 3.4.3 action statement which required operators to close the block valve within i
one hour. A shift supervisor discovered that the i
PORV had been erroneously placed in the manual mode and corrected it on February 25, 1995, about 23 hours2.662037e-4 days <br />0.00639 hours <br />3.80291e-5 weeks <br />8.7515e-6 months <br /> later, i
RESPONSE - DESCRIPTION OF CIRCUMSTANCES j
PSE&G does not dispute the violation.
j On February 24, 1995, Salem Unit 1 was in the process of raising j
Reactor Coolant System (RCS) pressure using Integrated Operating j
Procedure 2 (IOP-2).
To support a controller inspection, the Pressurizer pressure master controller was removed and pressure i
i control was placed in manual.
This action rendered Power.-
1 Operated Relief Valve (PORV) 1PR2 inoperable and required closing i
of PORV block valve 1PR7.
The operator did not close valve 1PR7 j
and the oversight went unnoticed for approximately 22 hours2.546296e-4 days <br />0.00611 hours <br />3.637566e-5 weeks <br />8.371e-6 months <br />.
4 l
Although a pre-job brief was performed prior to this evolution, j
it did not cover all TS required actions.
Specifically, the j
briefing did not discuss closing valve 1PR7.
The Nuclear Control Operator (NCO) and the Nuclear Shift Supervisor (NSS) failed to conduct adequate self-checking.
The NSS failed to maintain the l
proper supervisory overview to insure that the Technical
{
Specification action was completed.
l ROOT CAUSE ASSESSMENT The root cause of this event has been attributed to personnel error on the part of the supervisor (NSS) and the control operator (NCO).
A contributing cause to this event was inadequate guidance in the Technical Specification Action Tracking Log.
This log did not prompt operators to verify that TSAS are completed when the action. statement is entered.
[.
I i
Document Control Desk 2
, -2ND example LR-N95196 (cont'd)
CORRECTIVE STEPS THAT HAVE BEEN TAKEN Appropriate disciplinary actions were taken for the individuals involved.
j The NSS primary work location has been moved into the respective Control Room area as of March 3, 1995, to improve oversight and i
management of control room activities.
l The Technical. Specification Action Tracking procedure has been revised to require the NSS to verify and initial for completed Technical Specification actions and allow for tracking of j
potential Technical Specification entries.
2 J
An Information Directive 95-017 and two separate shift briefings 4
were completed for each of the Operations crews.
l j
The operations Department has re-emphasized the use of self-i i
checking techniques, peer verification and expectations for NSS i.
oversight.
1 1
Operations management re-emphasized the conditions under which j
the PORV's should b.e declared inoperable during Licensed Operator Requalification (LOR) training in segment 4, 1995.
Understanding i
of Technical Specification actions by Operations personnel were j
verified through LOR examinations.
.l CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE 4
I Ahe Operations Department is developing an Operations Standards document which will reference appropriate procedure guidance for i
conducting pre-job briefings.
The Operations Standards document will be implemented by November 21, 1995.
{
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED
{
The PORV block valve was closed to comply with TS requirements.
4 PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifyin l
conditions adverse to quality in a timely manner. g and resolving PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
i 4
1 4
4 a
rw
k, i
Document Control Desk
-1 -3RD example LR-N95196 (cont'd)
ATTACHMENT 4 - 3RD EXAMPLE VIOLATION 3.
On July 6, 1994, safety-related reactor head vent valve 2RC40 failed to operate (stroke open) during testing while Unit No. 2 was in cold shutdown.
Subsequently, the valve was returned to normal service on July 10, 1994, without any review or assessment in accordance with established procedures; that is, the Licensee failed to process this occurrence in accordance with the applicable " Work Control Process" procedure.
Consequently, this failure of a safety related component was never documented and formally assessed relative to preventive maintenance, operability, actions to prevent recurrence, or generic implications.
RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
1 On July 6, 1994, the 2RC40 valve failed its post-maintenance testing due to indications of reduced flow and dual position indication problems.
Subsequent investigation, including consultation with the vendor, indicated that the most probab3e cause of the valve failing to stroke open was due to boric acid solidification around the pilot plug.
The boron solidification was suspected to be the result of valve seat leakage.
The i
r Maintenance Engineer recommended backflushing of the valve with demineralized water and increasing the Reactor Coolant System (RCS) temperature to 180 F to dissolve the boron.
This resulted in proper valve operation and supported the original root cause supposition.
Therefore, it was concluded that a temporary condition could develop at low RCS temperatures and pressures that could result in boric acid binding of the valve.
On~ July 8, 1994, the valve was placed back into service with a recommendation to evaluate the need for additional valve preventive maintenance.
4 1
h i
i
j l
l Document Control Desk Attachment 4-3RD example j
LR-N9519f (cont'd)
In Decem'oer, 1994, the Salem Unit 2 head vent valves were replace 6 as a result of excessive seat leakage.
Similar 3
1 conditions had been previously observed on the Salem Unit 1 valves 1RC40 and IRC42 and prompted their replacement in May, j
i 1994.
In May, 1995, the vendor disassembled and inspected valve I
2RC40, which had been removed in December, 1994, to identify any
)
}
material condition that could have caused the valve's failure to j
The test results on valve binding were inconclusive but open.
indicated that the reported leaking of the valve could be i
attributed to steam cutting between the valve and the pilot valve j
dise due to normal wear.
i On April 5, 1995, an Incident Report was initiated and a root I
l cause analysis undertaken which arrived at much the same l
conclusions as that of the vendor.
The root cause of the failure j
of valve 2RC40 to stroke was indeterminate.
The type of 1
i degradation experienced by the head vent valve would not have i
alone prevented it from stroking.
Degradation of the valve j
internals, however, was identified as a causal factor in both the j
valve leakage and failure to stroke and was attributed to a lack
~
of preventive maintenance.
PSE&G has determined that this 1i failure mode is applicable only to the Reactor Head vent valves.
L j
In April, 1995, a re-analysis of the Preventive Maintenance requirements for these valve internals was completed and a 54-month inspection was recommended.
j i
i ROOT CAUSE ASSESSMENT i
l The final root cause for the failure of valve 2RC40 to open was j
inconclusive.
Probable causal factors include:
1.
Lack of preventive maintenance on valve internal components.
l 2.
Accumulation of boric acid precipitate on valve pilot plug.
)
the root causes for the failure to identify and correct this j
condition adverse to quality were:
4-1.
An inadequate-Corrective Action Program (CAP).
The CAP, in t
effect at that time, failed to establish sufficiently low reporting thresholds to ensure that conditions adverse to L
quality would be identified and resolved in a timely manner.
2.
Management failure to establish and enforce high i
expectations for equipment and personnel performance.
i a
5 A
-l i
I I
L i
i i
I
t) i i
i ij*
Document Control Desk l
LR-N95196 Attachment 4-3RD example f
(cont'd)
CORRECTIVE STEPS THAT HAVE BEEN TAKEN 1
i The Reactor Head vent valves 1RC40/42 and 2RC40/41/42/43 have i
been replaced in May, 1994, and December, 1994, respectively.
j Valves 1RC41 ard IRC43 are being replaced during the current 4
outage.
l The Corrective Action Program (CAP)
]
in the cover' letter to this Attachment.has been revised as described 1
Appropriate Operations Department procedures have been revised.
i
.These revisions include guidance to preclude boric acid accumulation in the valve body.
i l'
An Action Request to identify any solenoid operated valves other j
than the reactor head vent valves that serve as a Reactor Coolant System (RCS) pressure boundary and could potentially be subject to the same or a similar failure mode, such as boric acid binding j
due to seat leakage, has been completed.
PSE&G has determined that this failure mode is applicable only to the Reactor Head l
l valves.
vent NC.NA-BP. ZZ- 0 0 02 ( Z), " Root Cause Analysis Guidelines," has been j
developed to provide additional information and guidance in the use of various root cause analysis techniques which have been 1
proven effective in resolving both human and equipment i
performance problems.
i Within the Salem Maintenance Department, PSE&G has established dedicated resources to conduct required root cause analyses, develop and recommend appropriate corrective actions, and assure their proper implementation and overall effectiveness through followup assessments.
i CORRECTIVE STEPS TO B3 TAKEN TO PREVENT RECURRENCE New PM Recurring Tasks (RT's) have been initiated to implement 54-month PM to open and inspect the Reactor Head vent valve a
i internals and to repair as needed.
A new Maintenance Department procedure has been issued to provide guidance on the disassembly, inspection and refurbishment of the Reactor Head vent valves.
These corrective actions will be completed prior to restart of the affected unit.
6
~
I
\\
Document Control Desk
-4
- -3RD example LR-N95196 (cont'd)
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED This condition was documented and a root cause analysis was completed.
PSE&G will have acnieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner.
PSE&G will not either Salem Unit 1 or 2 until performance in this and restart other areas has improved.
I e
S
5
\\
Documsnt Control Dask
-1 LR-N95196 ATTACHMENT 4 - 4TH & STH EXAMPLES VIOLATION 4.
An oil sample laboratory report, dated August 4, 1994, recommended resampling and changing the oil-on the No. 21 high-head safety injection pump based upon a ten-fold increase in wear particle j
concentration. ka oil analysis, dated November 28, 1994, identified high wear particle concentration in the No. 22 high-head safety injection speed increaser oil. In both these cases, pump the system engineer, though aware of the findings of the lab reports, did not initiate any follow-up evaluation or corrective measure, nor establish a bases for operability or reliability in view of the apparent degraded condition of the equipment.
The degraded nature of the equipment was not entered into the Equipment Malfunction Identification System (EMIS) until March 20, 1995.
5.
A lab report, dated October 6, 1994, recommended resampling the No. 23 Auxiliary Feedwater (AFW) turbine lube oil due to a detectable amount of water contamination and an increase in wear i
particle concentration. However, the degraded nature of the equipment was not entered into the EMIS until March 27, 1995, and the system engineer did not initiate review, and evaluation, or establish any basis for equipment operability or reliability.
RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
PSE&G acknowledges that the issues identified in this violation addressed in a timely fashion.
Documentation of were not equipment status was deficient and inadequately maintained.
Document Control Desk
- 2
- -4TH & STH LR-N95196 (cont'd)
ROOT CAUSE ASSESSMENT PSE&G attributes the root cause of these occurrences to:
1.
Management's failure to enforce expectations regarding individual's responsibilities for the Performance Monitoring program.
2.
The lengthy turnaround time for laboratory analyses (including radioactive material handling) challenged the ability of the System Engineer to make timely decisions.
The root cause for the failure to identify and correct these conditions adverse to quality is:
1.
An inadequate Corrective Action Program (CAP).
The CAP, in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be resolved in a timely manner.
That same program did not provide clear guidance on the need to perform nor the required content of assessments to support continued assurance of equipment operability.
CORRECTIVE STEPS THAT HAVE BEEN TAKEN The 23 AFW Pump was declared inoperable.
This action was completed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of when PSE&G received notification from the laboratory that the follow-up oil sample had been confirmed to be the wrong grade for the component.
The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
Roles and responsibilities within System Engineering have been defined and communicated as described in the cover letter to this Attachment.
i Within System Engineering, a component reliability group was i
established to provide improved focus on equipment performance i
and reliability issues.
The Manager - Component Reliability will define and communicate roles and responsibilities for tracking
\\
i l
i
1 l
Document Control Desk
-3 -4TH & STH i
LR-N95196 1
(cont'd) and trending of performance monitoring data.
This will be completed by January 15, 1996.
Lube oil abnormalities from this occurrence have been documented i
by an Abnormal Condition Report to the System Manager from the Lube Oil Analysis Program Manager.
This process will remain in place to document future reports of abnormal indications.
PSE&G has contracted with a lube oil analysis laboratory capable 4
t I
of handling radioactively-contaminated lube oil samples.
The laboratory's ability to handle contaminated material will reduce the time from sample collection to condition determination by l
reducing the' required count time per sample.
1 l
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE The Lubricating 011 Program is being assessed to identify recommendations on a comprehensive lube oil program. The program recommendations are due by the end of the fourth quarter, 1995.
These recommendations will ?-
valuated and an Implementation Plan for approved recommend i,ns will be established by the end of the first quarter, 1996.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The abnormal Lube oil conditions were documented, reviewed and evaluated for operability impact.
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner.
PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
I J
1 Document Control Desk 1 -
LR-N95196 4
).
e ATTACHMENT 4 6TH EXAMPLE i
VIOLATION
?
d -
6.
LER 95-05 identified seven instances, between May 8, 1990 and January 14, 1995, of Pressurizer i'
safety valves (PSVS) being beyond the 1% tolerance required by TS 4.0.5 for Unit 1.
Four instances were identified between November 14, 1994, and j
January'14, 1995, which involved 2 of the 3 1
' installed PSVS. In all instances, the vendor notified the appropriate system engineer by j
ii telephone and written follow-up reports. However, the responsible system engineer never initiated an Incident Report. Consequently, root cause, i
operability, and reportability actions were not accomplished.
i RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
(
Beginning in May 8, 1990, eight (8) occurrences (total for both l
Salem Units) of Pressurizer code safety valves (PSV's) exceeding the 2485 psig +/- 1% lift set pressure were identified.
Seven of i
l those instances were cited within the Notice of Violation.
An eighth occurrence was self-identified and reported to the NRC via i
LER Supplement 272/95-05-01, dated October 31, 1995.
These occurrences were identified during testing required by Technical i
Specification (TS) 4.0.5.
Failure to report these anomalies I;
i.
resulted from personnel error in that Incident Reports (IR's) were' not written in accordance with NC.NA-AP.ZZ-0006 (Q) requirements.
procedure i
j ROOT CAUSE ASSESSMENT t
i The causes of the lift setpoint variances are a combination of 1
variability due to individual valve performance characteristics and random test variations which are common for these valves.
j The specific causes for Salem Station's variation are:
~
1.
Minor test loop instrument error.
i 2.
Valve design limitations.
3.
Applied loads from the discharge piping.
i l
6 i
N Document Centrol Desk 2 - - 6th example LR-N95196 (cont'd) i PSE&G has determined that the programmatic root cause of this violation was management's failure to clearly and adequately communicate expectations regarding when an IR was required.
Specifically, the System Engineers did not recognize the requirement to initiate IR's for these lift setpoint anomalies, in accordance with Nuclear Administrative Procedure NC.NA-AP.ZZ-
}
0006 (Q), " Corrective Action Program" in effect at the time.
They also failed to recognize the reportability implications for the out-of-tolerance valve performance data.
Consequently, the testing anomalies were not reviewed against the 10CFR50.73 i
" Licensee Event Report" (LER) reporting criteria and required LER j
reporting did not occur.
1 CORRECTIVE STEPS THAT HAVE BEEN TAKEN I
PSE&G has performed engineering evaluations as part of the Fuel Upgrade Margin Recovery program.
The thermal-hydraulic analysis indicates that PSV lift setpoint variances up to 3% are s
acceptable.
The structural analysis is more limiting, indicating that variances of +2.2 to -3.0% are acceptable.
Analyses within the Fuel Upgrade Margin Recovery program are continuing.
(l i
i The Corrective Action Program (CAP) has been revised as described in the cover let er to this Attachment.
Lessons learned from this violation were incorporated into the third quarter Operating Experience Feedback (OEF) training for Engineering Support personnel.
Appropriate discipline was taken with personnel involved in the i
failure to initiate IR's.
i Applied loading on the PSV's from the discharge piping has been reduced.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE A single point of contact within the PSE&G organization will be established to ensure coordination of activities associated with PSV testing.
This will be completed by March 29, 1996.
l i
1
3 Document Control Desk
-3 LR-N95196 6th example (cont'd)
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The lift setpoint variance conditions were documented, reviewed and assessed to demonstrate acceptability.
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at conditions adverse to quality in a timely manner. identifying and resolving PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has impreved.
r O
i 1
i
)
Document Control Desk
-1 j
LR-N95196 ATTACEMENT 4 - 7TH EXAMPLE
}
{
VIOLATION 1
7.
On March 6, 1995, May 3, 1995, and May 8,
- 1995, l
the Salem Unit i staff failed to determine the cause, correct, or prevent recurrence of failure of the Containment 100 foot elevation personnel l
airlock to pass its local leak rate test.
RESPONSE - DESCRIPTION OF CIRCUMSTANCES F
l PSE&G does not dispute the violation.
s' i
On March 6, 1995, the Salem Unit 1 Containment personnel airlock i
on the 100 foot elevation failed its local leak rate test (LLRT).
1 A work request was initiated for the Maintenance Department to investigate and correct the problem.
Maintenance technicians inspected the door seals and identified no obvious seal damage i
(
but noted that dirt had accumulated on the seal surface near the bottom of the door.
The door seal was wi rag and the LLRT was successfully rerun. ped clean with a damp i
i Following the incident, i
Maintenance and Operations agreed to have Operations personnel i
wipe down the door seal and retest the airlock in the event of another airlock failure prior to contacting the Maintenance l
Department.
This was noted as the corrective action in Incident Report (IR) #95-204.
On May 3, 1995, the airlock again failed the local leak rate l
test.
Operations personnel wiped down the door seal and
.i satisfactorily retested the airlock.
The Operations Department initiated IR #95-518 and Action Request (AR) #950503088 to
{
evaluate and document the occurrence.
On May 5, 1995, an LLRT i
was satisfactorily conducted but indicated an elevated leakrate.
i i
On May 8, 1995, the airlock failed its LLRT for the third time.
The door seal was wiped down and the LLRT was successfully rerun.
Operations initiated IR #95-551 and AR #950508110 to troubleshoot j
and correct the recurring condition.
Subsequent investigation revealed a significant buildup of dirt and hardened grease in the grooveson the seal surface which was caused by the gasket set.
Seal surface wipedown would not have been effective in removing 4
this buildup.
I k
i-i j
i i
i 3
a
II s.
i Document Control Desk 2 - - 7TH example
.LR-N95196 (cont'd)
J ROOT CAUSE ASSESSMENT l
The root cause of this event has been attributed to less-than-adequate management j
demonstrated by:
~ expectations of system performance as 1.
Inexperienced personnel were assigned to perform the initial inspection and corrective actions.
2.
Inexperienced personnel were assigned to perform the initial root cause evaluation.
j j
Deficiencies in both the preventive maintenance and surveillance test procedures also contributed to this event.
i j
CORRECTIVE STEPS THAT HAVE BEEN TAKEN l
The gaskets have been replaced on the Salem Unit 1 containment t
personnel airlock on the 100 foot elevation and the leakage test was satisfactorily performed.
The Salem Unit 1 containment personnel airlock (130 foot
[
elevation) and equipment hatch gaskets will be replaced prior to of Salem Unit 1.
restart The Salem Unit 2 giskets on the containment personnel airlocks and equipment hatch will be replaced prior to restart of Salem Unit 2.
The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
Procedure NC.NA-BP.ZZ-0002 (Z), " Root Cause Analysis Guidelines" has been developed to provide additional information and guidance in the use of various root cause analysis techniqyes proven effective in resolving both human and equipment performance problems.
Within the Salem Maintenance Department, PSE&G has dedicated resources to conduct required root cause analyses, develop and recommend appropriate corrective actions, and assure their proper implementation and overall effectiveness through followup assessments.
Appropriate Salem Maintenance procedures have been revised to include specific guidance on seal inspection, cleaning, and maintenance to assist in troubleshooting of leakage problems.
i J
Document Control Desk 3
7TH example LR-N95196 (cont'd)
Appropriate Maintenance procedures will be revised to change the airlock seal lubricant specification from Dow Corning ill to Dew Corning 3451 in accordance with Nuclear Engineering recommendations.
3 1
The above procedure revisions will be completed prior to restart.
4 of either Salem Unit 1 or 2.
a CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE i
A Preventive Maintenance Change Request (PMCR) has been initiated to evaluate the need for additional Preventive Maintenance (PM) i a
tasks for the containment airlock gaskets.
The PMCR recommends PM's following the six-month Structural Integrity Test and gasket replacement at the end of each refueling cycle.
j Appropriate Operations Department procedures will be revised to provide guidance on maintaining seal surface cleanliness and for performing leak rate testing, including discrete leakage criteria for determining when additional corrective action is required.
[
These corrective actions will be completed prior to restart of the affected unit.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The cause of the airlock soal failure occurrences was documented and evaluated, and the condition was corrected.
r PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifyin conditions adverse to quality in a timely manner. g and resolving PSE&G will not restart either Salem Unit 1 or 2 until performance in this and other areas has improved.
d i
I I
a 4
1 t
b
l.
Document Control Desk -
4 LR-N95196 1
ATTACHMENT 4 - 8TH EXAMPLE l
VIOLATION 4
i 8.
From February 29, 1992 until June 7, 1995, Salem Unit 1 staff failed to correctly determine the j
cause or take action to preclude recurrence of f ailures of instrument lines connected to the i
jacket water cooling system for the No. 1B and No.
l 1C emergency diesel generators.
i RESPONSE - DESCRIPTION OF CIRCUMSTANCES i
On June 1, 1995, during a 1B Emergency Diesel Generator (EDG)
{
Surveillance Test, a jacket water leak was identified at the 4
threaded connection of a 1/4" pipe nipple to an elbow upstream of
)
The failed component was instrument root valve 1DA46B.
subsequently replaced in kind.
As part of the root cause i
analysis for the 1/4" nipple failure, natural vibrational i
frequency tests were performed on all EDG's (at both Salem Unit 1 j
and 2) at locations congruent to this failure.
f
{
The' test results showed that piping at specific locations on this 1
and other EDG units could potentially experience damage or fail i
in response to induced vibrational stresses.
The testing indicated locations with natural vibration resonance frequencies very close to an integer multiple of the frequency which corresponds to the EDG shaft operating speed.
The affected EDG's were declared inoperable pending further analysis.
i A review of the past failure and maintenance history of the Salem Unit 1 and 2 EDG's was performed to identify occurrences of similar failures.
PSE&G's analysis indicates that there have been repeated failures due to vibration-induced fatigue and the nature of these failures was not recognized.
Failure.
recurrent to recognize this repetitive problem was due to inadequate root cause analyses and the fact that the failures were attributed to a wide variety of causes.
Recommendations stemming'from this analysis included design change activities to create more vibration-tolerant configurations and maintaining failed components for subsequent laboratory analysis.
Corrective actions taken in the past were ineffective at resolving the vibration-induced component failures, as evidenced by the recurring nature of these problems.
l l
a i
l*
Document Control Desk Attachment 4 LR-N95196 8TH example l
(cont'd)
ROOT CAUSE ASSESSMENT PSE&G attributes the root cause of the piping nipple failure to a design which did not adequately include tolerance for vibrational stresses.
A contributing cause was a lack of specifications for dimensions potentially critical to vibration tolerance in the manufacturer's documentation.
The root cause of the failure to identify and correct these component failures was an inadequate Corrective Action Program (CAP).
The CAP, in effect at that time, had numerous program elements which lacked adequate capacity for integration and i
oversight.
The CAP did not facilitate detection of common failure elements nor did it ensure that conditions adverse to quality were assessed for impact on operability in a timely manner.
CORRECTIVE STEPS THAT HAVE BEEN TAKEN i
All affected EDG's were declared inoperable but available, pending resolution of the potential for vibration-induced
{l, failure.
Interim contingency plan guidance was provided to the operations' Department.
This guidance established requirements to maximize the availability of the demineralized water supply to t
fill the EDG jacket water system in the event of a postulated failure.
A short-term adjustment to the cantilever length of the affected piping was made.
This action reduced the potential for resonance between this piping and the engine / header.
A vibration tolerance design review of the EDG's and peripheral equipment was conducted.
This review resulted in recommendations for appropriate enhancement modifications to harden the diesel engines against vibration-related concerns.
The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
A " lessons-learned" memorandum relative to this issue was issued by the Manager - Nuclear Engineering Design (NED) to all appropriate NED personnel.
Subsequent rolldowns to NED personnel have communicated the " lessons learned."
-. ~, -
I' Document Control Desk 3 - - 8TH example LR-N95196 (cont'd)
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Jacket Water Pressure transmitting tubing runs will be redesigned to eliminate the piping nipples and associated piping isolation valves.
This work has been completed for 1B EDG.
Modification packages have been prepared for the remaining five EDG's.
The Design Change Process (DCP) checklists will be revised to include specialty engineering review for vibration-induced failure issues.
These changes will be incorporated at the next revision to the appropriate procedures.
Maintenance and Planning Department training programs will be revised to include specific information regarding the general nature of fatigue failure and system vibratory response.
These corrective actions will be completed prior to restart of either Salem Unit 1 or 2.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED The cause of the instrument line failures was identified and 1
actions were taken to reduce their susceptibility to vibration-induced failures.
PSE&G will have achieved compliance with 10CFR50 Appendix B, criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifyin conditions adverse to quality in a timely manner. g and resolving PSE&G will not Salem Units 1 and 2 until performance in this and other restart i
areas has improved.
l
.l 4
~~-
i Document Control Desk 1 -
LR-N95196 ATTACEMENT 4 - 97H EXAMPLE VIOLATION 9.
From July 11, 1992 until June 10, 1995, Salem staff failed to determine the cause, evaluate the potential safety consequences, and establish corrective action for an abnormal condition affecting the No. 21 Residual Heat Removal discharge manual isolation valve (21RH10) associated with impact noise from the interior of the valve. (05013)
RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
On July 11, 1992, during mode 5 operation, unusual noise was identified coming from Residual Heat Removal (RHR) system valve 21RH10.
A slightly lesser noise was heard from valve 22RH10 and from the Unit 1 operating RHR loop No. 21.
A Salem Technical Department Memo (92-138) was issued to inform the Operations Department that this noise may be caused by flow-induced vibrations from existing play in the male / female discs and/or disc arm.
On April 21, 1993, a maintenance activity to open and inspect the valve was completed and wear marks were found on the downstream seat in two locations.
The cause of the wear marks was attributed to " wedge banging against seat ring."
No internal parts were found in need of replacement with the exception of packing and a gasket.
On June 30, 1995, valve 21RH10 was again reported making a metallic banging noise internally.
A maintenance supervisor did an in-field observation of the valve and concluded that the noise j
was abnormal.
An Action Request (JG) was written documenting the j
noise; however, a formal Operability Determination to assess the impact of the noise on system functional capability was not documented prior to June, 1995.
l
n Document Control Desk 2 - 9TH example LR-N95196 (cont'd)
ROOT CAUSE ASSESSMENT PSE&G has determined that the root causes of this event were:
1.
Inadequate-performance by the System Engineer regarding record keeping and tracking / trending of equipment malfunctions.
2.
Inadequate Corrective Action Program (CAP) as indicated by:
Inadoquate management and supervisory oversight of equipm?nt failure follow-up.
- Lack of documented engineering analysis of the physical condition of the valve.
'I The Corrective Action Program (CAP), in effect at that time, lacked sufficiently low thresholds to ensure that conditions adverse to quality would be identified and resolved in a timely That same program did not provide clear guidance on the manner.
need to perform nor the required content of assessments to support continued assurance of equipment operability.
CORRECTIVE STEPS THAT HAVE BEEN TAKEN On June 15, 1995, Salem System Engineering completed a Follow-up Assessment of Operability and determined that the valve noise did not adversely affect the functional capability of the RHR system.
Work Orders have been issued to open and inspect valve 21RH10 to determine the reasons for the noise currently being experienced.
Valve 21RH10 is scheduled to be opened and inspected after Unit 2 core off-load.
l For the purpose of trending, vibration data on valve 21RH10 is being taken periodically and reviewed by the System Manager.
This will continue until the loop is taken out of service following core off-load.
The vendor for the valve was contacted to obtain recommendations on actions to be taken.
The vendor stated that because of.the valve design and its location in a turbulent flow area, impact noises can be expected.
The vendor did not recommend any periodic preventive measures.
The Corrective Action Program (CAP) has been revised as described in the cover letter to this Attachment.
Documsnt Control Desk Attachment 4 LR-N95196 9TH example (cont'd)
System Engineering Department roles and responsibilities have l
been identified and clearly communicated to all System Engineering personnel.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Additional corrective actions, if any, will be identified after valve disassembly and inspection, as stated above.
System readiness reviews are currently underway and include assessment of.the readiness of plant systems to support unit restart.
Self-assessments of the effectiveness of the system engineering organization to carry out its roles and responsibilities will be conducted.
These corrective steps will be completed prior to restart of either Salem Unit 1 or 2.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Salem System Engineering issued their Followup Assessment of Operab'3ity for this valve condition on June 15, 1995.
PSE&G will have achieved compliance with 10CFR50 Appendix B, Criterion XVI, when the Corrective Action Program and related processes have been proven effective at identifying and resolving conditions adverse to quality in a timely manner.
PSE&G will not either Salem Unit 1 or 2 until performance in this and restart other areas has improved.
.____._....___m..___m...
Documant Control Desk
-1 LR-N95196 ATTACEMENT 5 VIOLATION II.
10 CFR Part 50, Appendix B, Criterion V,
" Instructions, Procedures, and Drawings", requires that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings of a type appropriate to the circumstances, and shall be accomplished in accordance with these instructions, procedures and drawings.
Instructions, procedures, or drawings shall include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily. accomplished.
Contrary to the above, following a modification in May 1993, that installed a drain system for the Salem Unit 2 Pressurizer code safety loop seals, the Licensee did not ensure that an activity affecting quality was satisfactorily accomplished in that the procedure that directed the installation of the modification to the Pressurizer code safety loop seals drains did not adequately ensure that the drain valves were properly positioned prior to plant startup after the modification. Specifically, valve 2PR66, a valve in a common drain line for the 2PR3, 2PR4, and 2PR5, Pressurizer safety valves, was left closed throughout the operating cycle between May 1993 and October 1994. (06013) j This is a Severity Level III Violation. (Supplement I)
Civil Penalty - $100,000 RESPONSE - DESCRIPTION OF CIRCUMSTANCES PSE&G does not dispute the violation.
During the 2R7 outage, a design change package (DCP) was implemented to add drain lines and drain valve 2PR66 to the Pressurizer Overpressure Protection system.
Valve ~2PR66 was installed to drain the line downstream of the Pressurizer Safety valve loop seals in order to prevent potential water hammer.
s-Document Control Desk
-2
- (cont'd)
LR-N95196 Final testing of the newly installed drain lines was completed on April 27, 1993.
On October 19, 1994, in preparation for the Salem Unit 2 eighth refueling outage (2R8), valve 2PR66 was discovered to be closed.
Valve 2PR66 being left in the closed position prevented drainage of the Power-Operated Relief Valve (PORV) and Pressurizer Safety Valve loop seal lines and re-established the loop seals, thus defeating the purpose of the design change.
After valve 2PR66 was discovered closed, the computer-based Tagging Request and Inquiry System (TRIS) was checked to confirm the expected valve position.
The normal position for this valve is "open" in accordance with TRIS.
The exact time when valve 2PR66 was closed and why this occurred is indeterminate.
The most probable period when valve 2PR66 was manipulated and left closed was dete'rmined to be after flushing activities were performed as part of DCP testing.
The DCP included verification of the valve positions during and the end of the testing portion of the modification.
Valve at 2PR66 was documented to be open after the testing.
Subsequent to the testing, there was a final acceptance walkdown of the system prior to turnover to operations.
The DCP did not require a written component list which documented valve positions during the walkdown.
As a result, valve 2PR66 was not verified to be open after the DCP, when the system was turned over to Operations.
The Operations DCP Coordinator understood that, in order to approve the design package turnover to operations for TRIS revision, it was only necessary to verify that the component change had been made in the computer database.
The Operations DCP Coordinator signed off the " Change Package Turnover to Operations" checklist for DCP 2EC-3190, without ensuring that a
temporary valve position lineup (referred to as an " auxiliary lineup") had been or would be performed prior to returning the plant to power operation.
The Operations DCP Coordinator had not received any training related to expected roles and responsibilities for providing or receiving the final component configurations after modification.
e.
I Documsnt Control Dask LR-N95196 cont'd)
At the time of this event, there existed an excessive TRIS backlog of 6000 changes waiting to be processed.
The Operations Staff supervision failed to take prompt action when the TRIS backlog became unmanageable.
an inappropriately low priority. TRIS database maintenance received This was compounded by the fact that the TRIS Coordinator was assigned other collateral duties.
The TRIS Coordinator did not create an auxiliary lineup in accordance with SC.OP-DD.ZZ-OD16, " TRIS Operations."
Procedure SC.OP-DD.ZZ-OD16 does not specify a time limit for performing an auxiliary lineup.
However, an auxiliary lineup was expected to have been performed prior to declaring the TRIS database complete.
"RC-MECH-001" is a standard valve lineup used to restore affected systems to a ready condition in preparation for plant startup.
The revision of SC.OP-DD.ZZ-OD16, in Offect at the time valve 2PR66 was added to the database, specified that the auxiliary lineup be completed and confirmed in TRIS before the component'is added to its applicable standard lineups.
The auxiliary lineup for valve 2PR66 was delayed and eventually never performed.
As a timely manner.
added to the RC-MECH-001 lineup in a l
result, valve 2PR66 was not ROOT CAUSE ASSESSMENT PSE&G has determined that the root cause of this event was inadequate commitment to the DCP turnover process and TRIS maintenance program by Operations Management as demonstrated by the following:.
1.
Operations had less-than-adequate turnover acceptance of DCP's.
Roles and responsibilities were not clearly defined.
Supervision failed to communicate expectations effectively.
2.
The Operations department allowed the TRIS database to become unmanageable.
The backlog was accepted.
The safety significance of the backlog on system design and operability was not adequately evaluated.
e.
Documsnt Control Desk
-4 Attachmont 5 (cont'd)
LR-N95196 CORRECTIVE STEPS THAT HAVE BEEN TAKEN The Operations Department reviewed TRIS database change requests initiated from DCP's completed during the time period from the beginning of 2R7 to the present.
This review encompassed 485 DCP's which have gone to "Part A" closure since the beginning of 2R7.
Part A closure signifies that the activity has been field-installed and the DCP has been turned over to Operations.
PSE&G has evaluated the elements of the DCP process which ensure that Operations procedures and the TRIS database are updated.
This evaluation has determined that the process is adequate.
The TRIS backlog was reduced to zero in May of 1995.
The backlog is being maintained at zero.
Operations has assigned additional personnel as TRIS coordinators.
The Coordinators are responsible for all TRIS interfaces including procedure SC.OP-DD.ZZ-OD16.
Operations Senior Reactor Operators (SRO's) have been assigned ownership of plant systems.
The SRO interacts with the project managers and System Managers associated with the DCP from conception.
The SRO accepts responsibility for system turnover to Operations.
CORRECTIVE STEPS TO BE TAKEN TO PREVENT RECURRENCE Operations procedure SC.OP-DD.ZZ-OD16 is being revised. The revision will emphasize Operations management's expectations and incorporate the auditing process for TRIS revision requests.
This will be completed by December 1, 1995.
DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Valve 2PR66 was correctly positioned for existing plant conditions.
PSE&G has evaluated the DCP process relative to accomplishing appropriate valve positioning after modification activities are complete.
This evaluation indicates that the process is adequate.
E r.e., 55 I C ? 8 m
]