ML18102A190
| ML18102A190 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 06/18/1996 |
| From: | Larry Nicholson NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML18102A189 | List: |
| References | |
| 50-272-96-06, 50-272-96-6, 50-311-96-06, 50-311-96-6, NUDOCS 9606240241 | |
| Download: ML18102A190 (60) | |
See also: IR 05000272/1996006
Text
Docket Nos:
License Nos:
Report No.
Licensee:
Facility:
Location:
Dates:
Inspectors:
- Approved by:
U. S. NUCLEAR REGULATORY COMMISSION
REGION I
50-272' 50-311
50-272/96-06, 50-311/96-06
Public Service Electric and Gas Company
Salem Nuclear Generating Station, Units 1 & 2
P.O. Box 236
Hancocks Bridge, New Jersey 08038
April 7, 1996 - May 18, 1996
C. S. Marschall, Senior Resident Inspector
J. G. Schoppy, Resident Inspector
T. H. Fish, Resident Inspector
Larry E. Nicholson, Chief, Projects Branch 3
Division of Reactor Projects
9606240241 960618
ADOCK 05000272
G
EXECUTIVE SUMMARY
Salem Nuclear Generating Station
NRC Inspection Report 50-272&311/96-06
This integrat~d inspection included aspects of licensee operations,
engineering, maintenance, and plant support. The report covers a 6-week
period of resident inspection; in addition, it includes the results of
announced inspections by a regional engineering inspector, attached as a stand
alone feeder to this report.
Operations
The operating shift demonstrated good outage risk management and effective
control of complex evolutions in the successful execution of a component
cooling water (CCW} system outage (Section 01.2}.
Operators repeatedly displayed good awareness of the potential safety impact
of a no. 23 service water (SW} pump strainer trip. They appropriately took
aggressive acti-0ns to insure availability of a heat sink (Section 01.3).
Reactor engineering thoroughly evaluated and maintenance effectively
controlled a transf~r of new fuel from the Unit 1 to the Unit 2 new fuel
storage pit (Section 01.4).
Operators quickly detected a loss of overhead annunciators (OHA} for Salem
Unit- 2.
Technicians caused the problem through use of a faulty probe. * The
technicians should have identified the problem with the probe before using it.
The symptoms of the loss of OHA differed from symptoms previously identified
during system failures. Operators considered the OHA problem resolved when
technicians addressed the symptoms of the failure. Operator failure to insure
that technicians had identified and corrected the underlying cause indicated
poor understanding and application of effective corrective action (Section
01.5).
Operators discovered that river water temperature had exceeded the temperature
limit assumed in a temporary modification for the no. 12 component cooling
water heat exchanger (CCWHX}.
The operations staff appropriately determined
operability based on engineering judgement, and initiated further engineering
analysis. Engineering did not effectively communicate to the control room
operators that the temporary modification restricted operation of the no. 11
CCWHX, nor that the staff had made changes to the CCW operating procedure to
implement those restrictions (Section 02.2}.
Operators made timely and appropriate notification to the. NRC for discovery of
a suspected controlled substance (Section 04.1}.
The management review committee (MRC} appropriately concluded that plans and
actions to address weak root cause skills were acceptable.
The MRC also
concluded that insufficient objective evidence existed to accept that
corrective action effectiveness had sufficiently improved.
As a result, the
MRC deferred approval of this problem statement. The MRC appropriately
ii
questioned the presenter; their decision to defer the approval of corrective
action effectiveness reflected a strong safety ethic (Section 07.1).
Although the MRC frequently added to the quality of efforts by the Salem
staff, inspectors noted inconsistency in the quality of MRC reviews.
The MRC
charter lacks guidance for review of the Restart Action Plans, the Startup and
Power Ascension Plan, the affirmations of NRC Restart Issue Completions, or
the closure of other items affecting startup (Section 07.2).
The Quality Assurance and Nuclear Safety Review (QA/NSR) department reports
for January/February 1996 and March 1996 provided substantive assessment of
Sal em performance.
The reports reflect significant improvement in the
organizations ability to identify performance strengths and weaknesses, and
communicate them to PSE&G management.
The QA/NSR reports, however, missed a
number of opportunities to provide information on performance trends from one
period of assessment to the next (Section 07.3).
The inspectors concluded that Salem did not initiate and complete license
change requests in a timely manner after they identified Technical
Specification discrepancies (Section 08.1).
Maintenance
Salem engineers corrected a licensee-identified omission in the surveillance
procedure for auxiliary feedwater response time (Section Ml.2) .
Maintenance technicians' poor foreign material exclusion practices resulted in
introduction of material from grinding onto safety injection pump internals.
Maintenance technicians provided less than adequate documentation of safety
injection pump repair activities (Section M3.l).
Engineering
Salem had not updated the Final Safety Analysis Report to reflect license
amendments dated May 4, 1994, pertaining to changes to spent fuel pool design.
Since the licensee did not complete the design changes until December 1995,
however, the inspectors concluded that PSE&G met the requirements of 10 CFR
50.7l(e).
(Section El.I)
A recent discovery of a licensee operating their facility in a manner contrary
to the Updated Final Safety Analysis Report (UFSAR) description highlighted
the need for a special focused review that compares plant practices,
procedures, and parameters to the UFSAR description. While performing the
inspections discussed in this report, the inspectors reviewed applicable
portions of the UFSAR that related to the areas inspected.
The following
inconsistencies were noted between wording of the UFSAR and the plant
practices, procedures, and parameters observed by the inspectors: auxiliary
feedwater surveillances (Section Ml.2}, service water strainer malfunctions
(Section E2.l), a degraded jacket water after-cooler heater (Section E2.2),
iii
fuel handling area ventilation discrepancies (Section E7.l), and spent fuel
pool cooling.
(Section El.I)
Service water pump strainer malfunctions represent nonconformance with the
automatic, self-cleaning features specified in the Final Safety Analysis
Report (FSAR section 9.2.1.2) and a ~hallenge to SW system reliability
(Section E2.1).
Engineering did not evaluate a degraded emergency diesel generator jacket
water after~cooler heater with regard to UFSAR requirements.
Failure to
perform a 10 CFR 50.59 safety evaluation is unresolved pending resolution of
similar nonconformances with the UFSAR and licensing basis (control air and
fuel handling building ventilation) (Section E2.2).
Engineering failure to insure that FHAV maintained the required differential
pressure during normal operation is unresolved pending completion of
inspection of related ques:tions concerning compliance with regulatory,
licensing basis, and design basis requirements* (Section E7.1).
Between April 8 and May 14, four station air compressor trips and two
additional compressor failures required operators to take contingency actions ..
Inspectors concluded that maintenance and system engineering staff did not
effectively ensure station air system reliability (Section ES.I).
The inspector's review of five previously identified issues indicated
acceptable resolution in most cases. Calculations and analyses were usually
- detailed and thorough and the evaluations indicated, in general, good
understanding of the identified issues.
The evaluation of the other two
issues, pertaining to onsite fuel oil availability and EOG load fluctuations
analysis, was less than thorough.
In the first case, the evaluation failed to
ensure the availability of the emergency connection to the diesel fuel oil
storage tank.
In the second case, the root cause analysis failed to address
the changes made to the system while troubleshooting, thereby potentially
invalidating the analysis (Stand alone feeder).
The inspectors concluded the 10 CFR 50.59 evaluations performed for the
advanced feedwater control system (ADFCS) modifications may not have been
performed correctly. This remains an unresolved item.
It appears that the
possibility for a different type of initiating event or a malfunction of a
different type than any evaluated previously in the USFAR may be created, even
though the licerisee stated that the modified analyses show the effects are
bounded in some instances (Section EB.4).
The inspectors determined through interviews that engineering judgement based
on operational experience at other ADFCS plants and computer simulations most
likely formed the bases for the Salem values of the sampling rates and
processing delays (Section EB.4).
The inspectors concluded that the setpoints and control constants were based
on empirical data and analyses from other ADFCS plants, with the necessary
particularization for Salem (Section EB.4) .
iv
The inspectors concluded. that the licensee considered EMI suscepti-bility.
(Section EB.4).
The inspectors concluded, based on their limited scope of V&V audit, that the.
retrofit of a V&V program to the ADFCS increased the accuracy and consistency
of the software with the system drawings (Sectioh E.7).
In the Salem ADFCS, the inspectors concluded that most of the cross".'"connected
data are used in MSS algorithms and the "last .data val~es" are used on loss of
data. These factors tend to moderate any functional problems caused by cross-
connected ~ata corruption or loss {Section E.7).
The inspectors concluded that, prior to DCP closeout, the configurat~on
control process for the ADFCS was successful due to knowledgeable software
engineers, in the absence of software-specific guidance.
The inspectors also
concluded that the software configuration control procedures provided for
post-DCP closeout were adequate for controlling software changes {Section
E. 7) ~
The inspectors concluded that the ADFCS access control provided sufficient
security for the system (Section E.7).
The inspectors concluded that the licensee met their HMI design objective of
minimizing impact on plant operations personnel (Section E.7) .
-
-
~he inspectors concluded that the use of the ADFCS trainer was an effective
method for enhancing the technical quality of the I&C maintenance activities.
{Section E.7).
-
Over-all, the engineering .performed for the ADFCS modification* generally met
design objectives.
The inspectors need to confirm that the feedwater control
reliability problems noted in inspecti-0n report 94-13 were completed.
Therefore, restaft action item II.4 remains open (Section E.7).
Plant Support
Overall, during requalification_training, operators demonstrated significant
improvement in .command and control, technical competence, and teamwork.
In
one case, however, a Senior Nuclear Shift Supervisor {SNSS) incorrectly
classified a postulated event. The lesson plah fci~ the event contained the
incorrect ~lassification, the instructor did not detect the error and did not
take appropriate corrective action until the NRC questioned the corrective
action and the operations manager became involved (Section P4.l).
During an unannounced call out Emergencj Preparedness {EP) exercise, the
Emergency Response Organization {ERO) met the goal of manning the Technical
Support Center (TSC) and the Emergehcy Operations Facility (EOF) within 90
minutes. Overall, the ERO adequately discharged its duties required to protect
the health and safety of the public, however, the EP staff reduced the
effectiveness of the training by not allowing the ERO staff, in some cases, to
make errors (Section PS.I).
v
Salem management responded aggressively and quickly to the report of discovery
of an apparently illegal substance in the protected are~. Their actions were
comprehensive and notifications were timely.
By including a search of Hope
Creek facilities, management showed sensitivity to the generic implications of
discovering an apparently illegal substance in the protected area. The
inspectors concluded Salem management responded appropriately. Laboratory
tests later concluded the substance was a powdered coffee creamer (Section
Sl.1) .
vi
..
EXECUTIVE SUMMARY
TABLE OF CONTENTS
I. Operations ..
II. Maintenance ..... .
III. Engineering
'IV. Pl ant Support .
V. Management Meetings
TABLE OF CONTENTS
0
vii
ii
vi
1
10
13
20
23
Report Details
Summary of Plant Status
Unit 1 and Unit 2 remained defueled for the duration of the inspection period.
I. Operations
01
Conduct of Operations
01.1 General Comments (71707)
Using Inspection Procedure 71707, the inspectors conducted frequent
reviews of ongoing plant operations.
In general, plant staff conducted
professional and safety-conscious operations. Specific events and
noteworthy observations are detailed in the sections below.
01.2 Unit 2 Component Cooling Water Outage. NRC Restart Inspection Item III.7
(71707)
The inspector observed the operating shifts' preparation and execution
of a Unit 2 component cooling (CC) water maintenance outage. Operations
management planned the activity to minimize shutdown risk. This
included a comprehensive contingency plan involving increased spent fuel
pool temperature monitoring and backup cooling strategies.
On April 25,
1996, operators implemented the plan and maintained a good safety
conscious focus throughout the CC system outage.
The operating shift
prioritized and controlled the related activities to effectively shorten
the outage duration (25.6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> actual/46 hours planned) resulting in a
shutdown risk reduction.
In conclusion, the operating shift
demonstrated good outage risk management and effective control of
complex evolutions in the successful execution of a component cooling
water outage.
01.3 Service Water Strainer Trip. NRC Restart Inspection Item III.7
a.
Inspection Scope (71707)
b.
The inspectors assessed operator response to a service water (SW) pump
strainer trip.
Observations and Findings
On April 18, at 9:19 a.m., no. 23 SW pump strainer tripped on thermal
overload. Operators reset the overloads, then restarted the strainer so
that electricians could troubleshoot it. Electricians reported normal
strainer amperes and operators returned the strainer to automatic
operation.
The strainer tripped again at 10:55 a.m .. At the time of
the malfunctions operators had only the nos. 22 and 23 SW pumps
available as the heat sink for Unit 2 Spent Fuel Pit (SFP) cooling.
In
2
response to the second strainer trip, operators established security at
no.- 2 service water bay door, suspended de-silting operations at the
service water intake structure, and expedited returning no. 21 SW pump
to service. Operators started no. 21 SW pump at 12:19 p.m .* The
strainer malfunction did not affect SW header pressure or SFP
temperature.
Operators recognized that the trip of no. 23 SW pump strainer rendered
no. 23 SW pump inoperable and left them with only one operable SW pump
(no. 22) to provide a heat sink for SFP cooling. They also recognized
that when they returned no. 21 pump to service, they could not consider
it operable because technicians had not co~pleted their post maintenance
tests. Finally, operators recognized that since the same vital bus
powers nos. 21 and 22 SW pumps, SFP cooling was vulnerable to any
adverse conditions that could challenge bus reliability.
c. *Conclusions
Operators repeatedly displayed good awareness of the potential safety
impact of a* no. 23 SW pump strainer trip. They appropriately took
aggressive actions to insure availability of a heat sink.
01.4 Transport of New Fuel
a.
Inspection Scopa (60705)
The inspector reviewed reactor engineering's 10 CFR 50.59 safety*
evaluation for the transfer of new fuel from the Unit 1 new fuel storage
pit to the Unit 2 new fuel storage pit.
In addition, the inspector
observed the h*ndling, transfer and storage of the new fuel.
b.
Obse~vations and Findings
On April 21, 1996, reactor engineering completed a thorough 10 CFR 50.59
safety evaluation to address the movement of new fuel between units.
Reactor engineering determined that there was no expected adverse
radiological effects to the station personnel or public.
In addition,
there was no expected adverse effects to the fuel assemblies, plant
systems or structures.
During the week of April 29, maintenance moved 16 fuel assemblies from
Unit 1 to Unit 2. Maintenance used existing procedures and equipment to
support this move.
Reactor engineering conducted a new fuel receipt
inspection of the transferred assemblies ~s if the new fuel arrived
directly from the vendor.
Mechanical maintenance, reactor engineering,
operations, and radiation protection conducted the fuel movements with
methodical precision and tedious attention~to-detail. The mechanical
maintenance supervisor effectively controlled the activity.
In
addition, Quality Assurance provided good oversight of the process .
c.
3
Conclusions
Reactor engineering thoroughly evaluated and maintenance effectively
controlled a transfer of new fuel from the Unit 1 to the Unit 2 new fuel
storage pit.
01.5 Loss of Overhead Annunciators. NRC Restart Item 111.40
a.
Inspection Scope (71707)
b.
The inspectors observed operator and maintenance staff response to a
loss of overhead annunciators.
Observations and Findings
During an 18 month surveillance of the Salem Unit 2 control room
overhead annunciators (OHA) on May 8, operators determined that the OHA
stopped re~ponding to plant conditions. The operators initiated the
steps of abnormal procedure S2.0P-AB.ANN-0001(Q), Loss of Overhead
Annunciator System.
The response directed by the procedure included
increased monitoring of critical plant functions, including service
water, component cooling water and spent fuel pool
co~ling. The
operators also implemented measures to protect these systems from
activities with potential to adversely affect their operation.
As
directed by procedure, operators attempted to shift from Sequence of
Events Recorder (SER) 'A' to SER 'B' with no change in OHA response.
(The SERs are microprocessors that receive input from the plant through
eleven scanner cards, process the information, and pass it through to
control room annunciators, printers, and alarm display monitors.)
Instrumentation and controls (l&C) technicians, with close supervisor
involvement, postulated that the technicians had caused the OHA loss of
function.
They postulated that the technicians used a defective test
instrument probe that momentarily grounded a power supply, causing the
scanner cards to stop communicating to the SERs.
After the technicians
briefly interrupted power to the scanner cards, the cards resumed
communication with the SERs and restored OHA function.
The technicians
continued trouble shooting by demonstrating that the faulty probe caused
comparable symptoms in a partial OHA system set up for training.
Maintenance supervisors concluded that technicians inappropriately used
the faulty probe, since they should have detected the potential for
shorting a power supply through visual inspection of the probe.
After technicians restored OHA function the operators demonstrated that
the OHA system functioned properly, exited the abnormal procedure, and
returned to the previously implemented measures for the degraded OHA
with the reactor defueled.
The inspectors noted that the operators
relaxed their compensatory measures after the technicians restored
functionality to the OHAs, but prior to verifying the suspected cause.
The inspectors concluded that the operators considered corrective action
complete for the immediate problem based on correcting the symptoms
-.
4
without insuring that technicians had identified and corrected the
underlying cause.
c. Conclusions
Operators quickly detected a loss of OHA for Salem Unit 2. Technicians
caused the problem through use of a faulty probe.
The technicians -
should have identified the problem with the probe before using it. The
symptoms of the loss of OHA differed from symptoms previously identified
during system failures. Operators considered the OHA problem resolved
when technicians addressed the symptoms of the fa-i lure. Operator
failure to insure that technicians had identified and corrected the
underlying cause indicated poor understanding and application of
effective corrective action.
02
Operational Status of* Facilities and Equipment
02.1 Control of Plant Configuration. NRC Restart Item 111.2
a.
Inspection Scope (71707)
Inspectors reviewed temporary modification 95-073, revision 0, to insure
that the licensee mainfained a safe plant configuration for the existing
mode of operation.
b.
Observations and Findings
During the current Salem Unit l outage, plant staff removed valves in
the service water system supplying the no. 12- component cooling water
heat exchanger (CCWHX).
The plate type no. 12 CCWHX consists of two
sections, no. 12A and no. 128.
Plant workers installed flanges to
isolate service water to the no. 12A section of the heat exchanger.
Engineering based the acceptability of the temporary modification on an
assumption that the no. 11 CCWHX remained continuously available while
the flanges remained in place. The modification also assumed that the
Delaware River temperature would remain below 58.3 degrees F for the
duration of the modification.
On May 2, the inspectors noted that river water temperature exceeded 60
degrees F.
Operators had previously identified that river temperature
exceeded 58.3 degrees F and initiated a Condition Report.
The operators
58.3 degrees F on the ability of CC to dissipate the heat generated
during plant startup. Operators realized that worst case heat loads for
the defueled unit were substantially less than the assumed conditions.
Engineering subsequently calculated a river water limit of 90 degrees F
for current plant conditions.
The inspectors questioned the controls for insuring no. 11 CCHX remained
in service .. Control room operators did not know that engineering had
initiated a change to the operating procedure requiring that no. 11 CCHX
c.
5
remain in service for the duration of the modification.
In response to
the inspectors' questions, operators initiated a CR to document
ineffective controls to insure the plant remained within analyzed
conditions resulting from a temporary modification, and a CR to document
ineffective communication of procedure changes to the operating staff.
The inspectors noted that, as a result of the operators' concern over
river water temperature, the no. II CCWHX remained available or in
service throughout the duration of the temporary modification.
Conclusions
Operators discovered that river water temperature had exceeded the
temperature limit assumed in a temporary modification for the no. I2
CCWHX.
They appropriately determined operability based on engineering
judgement, and initiated further engineering analysis. Engineering did
not effectively communicate to the control room operators that the
temporary modification restricted operation of the no. II CCWHX, nor
that the staff had made changes to the CC operating procedure to
implement those restrictions.
04 Operator Knowledge and Performance
04.I Event Notification
On April II, the inspectors observed operator response to the report of
a suspicious substance found in a locker room in the Protected Area.
Details are in Section SI.I of this report ..
The inspectors concluded that the operators made timely and appropriate
notification to the NRC, in accordance with* IO CFR 26.73 and 10 CFR
50.72(b)(2)(vi).
07
Quality Assurance in Operations
07.1
NRC Restart Action Plan Item III.IO. Corrective Action Plan COpen)
a.
Scope
The NRC issued its Restart Action Plan (RAP) on February 23, I996 that
defined the scope of activities that the NRC will address prior to
restart.
RAP Sections II and III list 43 technical and 2I programmatic
items that the NRC plans to inspect prior to restart. This inspection
addressed one of those items: the Corrective Action Plan.
b. Observations and Findings
On April I9, I996, the Manager - Corrective Action and Quality Services
(CA & QS), forwarded the closure package for problem statement No. 5 of
their Corrective Action Plan (CAP) which states that, "Root cause
analysis skills and procedures are weak.
Corrective actions are often
6
not effective at preventing recurrence." Licensee policy requires that
this closure plan prdvides the "objective evidence" to ensure that this
item has been satisfactorily addressed.
The plan sponsor, the
Corrective Action Group (CAG), combined each of the problem statement
No. 5 sub-statements into three major initiatives: (1) Improve the
Nuclear Business Units (NBU) overall knowledge of cause analysis, (2)
Improve the procedure for cause analysis, (3) Develop and publish
expectations for root c~use evaluations.
Improve, monitor, and assess
the adequacy of evaluations and effectiveness of corrective actions.
Regarding the first item, the licensee has developed training using
input from industry root cause experts. The licensee's new focus on
root cause (RC) is to train a smaller number of specialists that can be
dedicated to RC on more-or-less a full time basis versus the previous
approach of training every engineer on-site and giving them little or no
experience with actual plant problems or events. This was an
improvement over previous practice.
Regarding the second item, the licensee developed a new Root Cause
Manual (RCM) that incorporated the best ideas from a sample of other
nuclear utilities, including: its layout, use of:illustrations and
diagrams, and overall simplicity. The RCM provides very specific
direction on the scope and content of Root Cause Analysis Reports. This
. was also an improvement over previous practice .
For the third item, the licensee published expectations for level 1 and
2 evaluations in NAP-06, the corrective action program procedure, and
the RC manual.
Regarding improving corrective action effectiveness, the
licensee initiated a Corrective Action Review Board (CARB) to evaluate
the quality of level 1 condition reports (CRs).
The board consists of
the Station General Manager and other key managers and has been
monitoring the quality of level 1 CRs.
Based on a sampling of the
performance indicators and the CARB minutes, the inspector noted that
the quality of these evaluations has improved over the past three
months.
However, ultimately, corrective action effectiveness will be
verified when continued uneventful power operation is achieved.
On April 25, the inspector observed the presentation of Problem
Statement No. 5 of Corrective Action Program to the Management Review
Committee (MRC).
The April 19 closeout package provided the underlying*
basis for the presentation. The p~esenter stated that the root cause
training had been completed and that the RC manual was being actively
used for Level 1*and 2 CRs.
The MRC questioned him at length and
determined that sufficient improvements had been made to accept the
closeout for the root cause problem statement.
Regarding the effectiveness of corrective action, the presenter was
unable to provide firm "objective evidence" that corrective action
effectiveness was sufficiently improved.
The inspector noted that
details contained in the package alternately strengthened and weakened
the presenter's argument.
However, these details were not referred to
during the presentation. For example, the performance indicators on
c.
7
corrective action quality showed a generally improving trend on the
quality of level 1 and 2 CRs, while, in another case, 26 of 34 level 1
CRs were missing effectiveness reviews by department managers. These
details were not discussed by MRC.
Conclusions
Based on a detailed review of the licensee's actions for weak root cause
skills, the inspector concluded that their plans and actions to date
were acceptable. *
The MRC was unable to conclude that corrective action effectiveness was
sufficiently improved.
Thus, MRC deferred approval of this problem
state~ent. The inspector agreed with MRC's conclusion.
MRC questioning
of the presenter was appropriate and their decision to defer the
approval of corrective action effectiveness reflected a strong safety
ethic.
07.2 Management Review Committee Performance. NRC Restart Item 111.21
a.
Inspection Scope (71707).
During the inspection period, inspectors observed MRC activities to
assess the effectiveness of their review of closeout packages for NRC
. manual chapter 0350 inspection items~
b. Observations and Findings
The MRC frequently added to the quality of efforts by the Salem staff.
For example, .the MRC found several packages documenting corrective
actions unacceptable.
The MRC challenged the conclusions presented.with
several of the issues.
As an example, in a summary of valves reviewed
for pressure locking and thermal binding concerns, the MRC noted that
the staff had excluded valves based on their normally open position *.
The MRC noted that emergency operating procedures required operators to
change the valve position. The MRC did, however~ demonstrate
inconsistency in .the quality of NRC review.
For. example, the MRC
members occasionally lost independence by defending the position of a
staff member presenting information.
In addition, the MRC charter does
not provide guidance as to the purpose or basis for MRC review of the
. Restart Action Plans, the Startup and Power Ascension Plan, the
affirmations of NRC Restart Issue Completions, tir the closure of other
items affecting startup.
By contrast, the MRC charter contains very
detailed screening criteria for MRC use in classifying proposed
maintenance or modifications as required for restart, required after
restart, or not required. Without guidance for review of plans and
completion packages, the MRC cannot provide consistently effective
reviews .
- tt
'
8
c. Conclusions
Although the MRC frequently added to the quality of efforts by the Salem
staff, inspectors noted inconsistency in the quality of MRC reviews.
The MRC charter lacks guidance for review of the Restart Action Plans,
the Startup and Power Ascension Plan, the affirmations of NRC Restart
Issue Completions, or the closure of other items affecting startup.
07.3 Effectiveness of Quality Assurance and Nuclear Safety Review. NRC
Restart Item 111.20
a.
Inspection Scope (71707)
The inspectors reviewed the results of QA and NSR activities documented
in monthly reports for January/February 1996 and for March 1996 to
assess the effectiveness of their contribution to improving Salem
performance.
b. Observations and Findings
The QA/NSR monthly reports indicated substantial improvement in the
number and quality of performance observations.
For example, the
reports identified operator problems with tagging, poor operator
Technical Specification tracking, and maintenance procedure non-
compliances.
The reports identified generic problems with temporary
modifications, radiation worker inattention to detail, and problems with
ERO response timeliness. The QA/NSR staff formatted the reports to
support the electronic performance indicator system.
As a result, the
reports display performance in operations, maintenance, engineering and
plant support using the colors green (excellent), yellow (meets
standard), red (needs improvement), and blue (insufficient data). The
format, including the clearly written executive summary, aids the reader
in gaining perspective on performance strengths and weaknesses in each
functional area.
Based on a comparison of the information, the inspector noted several
performance observations in both reports.
For example, both reports
noted that the backlog of Salem'operator work around remained greater
than 200, yet the March report did not identify that the
January/February report contained the same observation.
Both reports
noted that tagging problems continued in operations and maintenance, and
they noted maintenance training deficiencies, but the March report did
not indicate that the same observation appeared in the previous report.
The reports both noted problems with temporary modifications, overdue
engineering corrective action evaluations, and concern with ERO manning
timeliness. The reports did not assess or document corrective action
for these identified problems.
Both reports concluded that QA/NSR staff
had gathered insufficient data to rate engineering performance, but the
March report did not indicate whether the QA/NSR department had
implemented actions intended to provide engineering assessment in a
future report.
~
9
c. Conclusions
The QA/NSR reports for January/February 1996 and March 1996 provided
substantive assessment of Salem performance.
The reports reflect
significant improvement in the organization's ability to identify
performance strengths and weaknesses, and communicate them to PSE&G
management.
The QA/NSR reports, however, missed a number of
opportunities to provide information on performance trends from one
period of assessment to the next.
08
n;scellaneous Operat;ons Issues
08.1 Timeliness of licensing Submittals
The following are several examples of untimely licensing submittals:
1.
On February 1, 1996, the licensee told NRR that a one,...time change,
to the Technical Specifications would be submitted to allow entry
into Mode 6 with the Control Room Ventilation System inoperable.
On April 2, 1996, the licensee said that this request would *be
submitted by April 12, 1996.
The submittal was made May 7, 1996,
with a requested completion date of June 21, 1996, to support
entry into Mode 6 on July 3, 1996.
2 .
On February 20, 1996, the licensee informed NRR that it was
planning to request an exemption to 10CFR55.31(a)(5).
The
regulation requires that applicants for an operator license
perform five significant control manipulations which effect
reactivity or power level on the facility for which the license is
sought~ The request for the exemption was submitted on May 10,
1996.
The licensee requested that the NRC grant the exemption by
July 3, 1996, to support entry into Mode 6.
3.
An internal licensee memorandum dated October 24, 1995, stated
th~t Unit 2 Technical Specification 4.9.12.d.2 "is inconsistent
with the' system design basis and needs to be changed".
The
Technical Specification required verification, at least once per
18 months, that the system automatically starts on a high
radiation test signal. The actual plant configuration did not
have an automatic start provision.
In May 1996, the licensee,
. because of this discrepancy between the Technical Specification
and the system design, declared the Fuel Handling Building (FHB)
ventilation system inoperable.
An operable FHB ventilation system
is required before any fuel movement in the FHB is allowed.
The
licensee had been planning to move fuel in May and was considering
asking the NRC for some expedited licensing action so that the FHB
ventilation system could be declared operable. The NRC indicated
that it probably would not take expedited license action because
the licensee had ample time to submit the Technical Specification
change prior to actually needing it. By letter dated May 29, 1996,
10
the licensee committed to install the auto-start feature prior to
moving fuel, thus making the system design consistent with the
existing Technical Specification.
The inspectors concluded that Salem did not initiate and complete
license change requests in a timely manner after they identified
Technical Specification discrepancies.
II. Maintenance
Ml
Conduct of Maintenance
Ml.I General Comments
a.
Inspection Scope (62703)
The inspectors observed all or po_rtions of the following activities:
960221159:'
2C diesel generator starting air and turbo boost
system upgrade
960214251:
2C diesel generator motor operating potentiometer
replacement .
960329013:
2C diesel generator/generator control panel/clean
inspect
960228238:
no. 4 service water bay pipe replacement
960228146:
no. 22 service water header piping replacement
and
960320176:
no. 21 safety injection pump suction piping inspection
960516056: Artificial Island Meteorological Monitoring Program
Calibration and Maintenance Procedure
- The inspectors observed that the pl ant staff performed the maintenance
effectively within the requirements of the station maintenance program.
b.
Inspection Scripe {61726}
The i nspecto.rs observed a 11 or portions of the fo 11 owing survei 11 ances:
S2.0P-ST.DG-0014:
2C diesel generator endurance run
The inspectors observed that plant staff did the surveillance safely,
effectively proving operability of the associated system
Ml. 2 Auxiliary Feedwater Pump Surveillance. NRC Restart Item I I. 42
a.
Inspection Scope (71707) .
The inspectors reviewed the surveillance procedures for the auxiliary
feed water (AFW) system, to determine whether they satisfied the
associated Technical Specification (TS) requi~ements .
11
b. Observations and Findings
On April 3, during the plant status meeting, the inspectors learned that
. Condition Report (CR) 960306246 questioned the adequacy of the AFW TS
surveillances. The inspector reviewed the condition report and
following surveillances:
S2.0P-ST.SSP-00ll{Q), Engineered Safety Features-Response Time
Testing
S2.0P-ST.AS-0004(Q), Inservice Testing Auxiliary Feed Water Valves
Hodes 1-6
S2.IC-TR.ZZ-0002{Q), Unit 2 Master Time Response Procedure.
Salem Unit 2 TS 3.3.2.1, Engineered Safety Feature Actuation System
Instrumentation, states, in part, that the Engineered Safety Feature
Actuation System (ESFAS) shall be operable with response times as shown
in Table 3.3-5. The table requires the AFW pumps to respond to an
initiating signal within 60 seconds.
The CR 960306246 stated that the
steam generator AFW discharge valves to the steam generators (AF21)
response times, when incorporated in the Unit 2 Master Time Response
Procedure, would not meet the 60 second criteria established by TS.
Salem staff had not previously considered opening of the AF21 valves in
meeting the TS 3.3.2.1 requirement.
As a result of CR 960306246, the Nuclear Fuels group performed the
calculation (DSl.6-0145), AFW Response Time Evaluation.
The calculation
determined that the AF21 valve would only open 45% within 60 seconds.
The calculation also determined that although the AF21 valves would not
fully open within 60 seconds, the AFW system would establish sufficient
flow to maintain the steam generators as a heat sink, as described in
Chapter 15 of the Final Safety Analysis Report (FSAR). The inspector
reviewed the calculation and determined that it adequately demonstrated
the ability of the AFW system to deliver required flow in 60 seconds, as
described in the FSAR.
Technical Specification Bases 4.3.1 & 4.3.2 states that the surveillance
requirements specified for the AFW system ensure that the overall system
functional capability is maintained comparable to the original design
standards.
One of the original design standards, the Accident Analysis,
Chapter 15 of the FSAR, requires that during a loss of normal feedwater
and loss of offsite power, two steam generators begin to receive
auxiliary feedwater from one motor-driven auxiliary feedwater pump
(MDAFWP) within 60 seconds.
Prior to the development of calculation DSl.6-0145, the AFW response
time surveillances did not demonstrate the ability to meet the
requirements of TS 3.3.2.1. This licensee identified and corrected
violation is being treated as a Non-Cited Violation consistent with
Section VII.B.1. of the NRC Enforcement Policy .
.
I
12
c; Conclusions
Salem engineers corrected a licensee-identified omission in the
surveillance procedure for auxiliary feedwater response time.
M3
Maintenance Procedures and Documentation
- M3.1 Safety Injection Pump Inspection and Repair. NRC Restart Items 111.3 and
111.5
a.
Inspection Scope (62703)
b.
The inspector reviewed the.work package and associated procedures for
safety injection (SI) pump maintenance conducte~ under work order 950725243.
The inspector discussed the activity with maintenance
technicians, supervisors, and managers.
Observations and Findings
On May 3, 1996, quality assurance (QA) inspectors performed a
.
cleanliness inspection and maintenance technicians lowered the no. 22 SI
pump upper casing.
On May 6, maintenance technicians cut out 22SJ103,
no. 22 SI pump casing vent valve, under work order 950523070.
This
activity left grinding chips in the pump.
Maintenance technicians,
involved with SI pump assembly, removed the pump casing, cleaned the
pump internals, and wrote a condition report (CR 960507291) to document
the occurrence~ Quality assurance reverified internal cleanliness in
accordance with NC.NA-AP.ZZ-0021 (NAP 21), System Cleanliness Program.
On May 7, maintenance technicians helium arc welded the 22SJ103 valve
back in place and left various grinding shavings on the casing's
external surfaces. The maint~ance technician, involved with pump
assembly, questioned the internal condition of the pump.
The
maintenance supervisor determined that no possibility of internal debris
intrusion existed based on the weld type (no slag) and post-weld
grinding only. Maintenance technicians did not reinspect the pump
internals. *
On May 9, the .inspector reviewed the work package and procedure
documentation.
Based on the above information, the inspector expressed
a foreign material exclusion*(FME) concern, regarding the no. 22 SI
pump, to the Unit 2 Senior Maintenance Supervisor.
On May 10, the
maintenance supervisor noted a shaft rub on the no~ 22 SI pump and
decided to remove tbe upper casing to investigate. The maintenance
supervisor determined that grinding chips internal to the casing wear
'ring, located beneath the 22SJ103 vent valve, caused the shaft rub.
The
maintenance supervisor stated that the foreign material could cause
increased wear on the ring, but did not threaten pump operability. The
maintenance supervisor determined that the grinding chips were most
likely the result of initially cutting out the vent valve. Maintenance
staff disa~sembled the pump and removed the chips. The senior
maintenance supervisor initiated actions to communicate lessons learned
-
I
c .
13
to all maintenance personnel. The inspector noted that poor planning
and coordination of maintenance resulted in poor FME practices on an
assembled component.
Ample opportunity existed to perform the vent
valve replacement with the upper casing removed from the pump.
In
addition, failure to properly restore cleanlines~ is a violation of NAP-
21 requirements. This licensee identified and corrected violation is
being treated as a Non-Cited Violation, consistent wtth Section VII.B.l
of the NRC Enforcement Policy.
Additionally, on May 9, the inspector identified a number of procedure
usage and documentation deficiencies. Contrary to management
expec~ation, maintenance technicians used SC.MD-CM.SJ-0001 (Revision 5),
Safety Injection Pump Disassembly,_Inspection, Repair and Reassembly, in
the field 14 days after the issue date. Technicians validated the
procedure on April 18, and used it to record information until May 6 {18
days). Maintenance technicians repeatedly marked steps "N/A" {non-
applicable) without providing justification in the comments section of
the procedure. Technicians did not initial all steps completed in the
procedure. Technicians did not completely enter all required
-
information on the procedure data sheet {attachment 8). This is a
'violation of NC.NA-AP.ZZ-0001 {Revision 7), Nuclear Procedure System,
requirements.(VIO 50-272&311/96-06-01)
Conclusions
Maintenance technicians' poor foreign material exclusion practices
resulted in introduction of material from grinding onto safety injection
pump internals. Maintenance technicians provided less than adequate
,documentation of safety injection pump repair activities. These
activities resulted in two violations.
III. Engineering
El
Conduct of Engineering
El.I Updates to the Final Safety Analysis Report CFSAR)
a.
Inspection Scope
b.
During a recent evaluation of spent fuel pool decay heat removal and
refueling practices, the inspectors reviewed licensing basis documents
for Salem Units 1 and 2.
The documents included the UFSAR {Updated
Final Safety Analysis Report) and documents related to Amendments 151
and 131 of the Salem license dated May 4, 1994.
Observations and Findings
In a report to the Nuclear Regulatory Commission, the staff noted that
Salem was included in a category of plants where:
c.
14
.... plant-specific FSARs did not reflect information associated
with spent fuel pool decay heat removal from applicable license
amendments.
An NRC regulation, specifically 10 CFR 50.7l(e),
requires that the FSAR be periodically updated to reflect such
i nforlTiat ion.
NRC regulation 10 CFR 50.7l(e) requires that licensees periodically
revise the FSAR to include the effects of: all changes made in the
facility or procedures as described in the FSAR; all safety evaluations
performed by the licensee in support of requested license amendments .*..
The regulation requires that the periodic updates .be submitted annually
or within six months after each refueling outage provided the interval
between successive updates does not exceed 24 months.
During the staff review, the inspectors noted that the Salem FSAR did
not reflect information submitted by the licensee in support of the
rerack amendment.
Specifically, in a letter dated April 28, 1993, the
licensee listed the heatload for a partial core offload as being 23.8
BTU/hr and the resulting maximum spent fuel pool temperature as being
148.94 F.
For the full core offload the heatload was listed as 38.57
BTU/hr and the pool temperature as 179.93 F.
However, FSAR Section
9.1.3.l states that, the fuel pool water temperature is limited to 120 F
for a partial core offload and limited to 150 F for a full core offload.
The FSAR should include the most recent applicable rerack amendment
design information .
The inspectors noted however, that the staff approval of the rerack
amendment was issued in May 1994 and the modification was only completed
in December 1995.
Conclusions
The inspectors concluded that the information regarding the rerack
project need only be included in FSAR revisions subsequent to the
completion of the modification. Thus, the inspectors concluded that
Salem had not exceeded the time period allowed by 10 CFR 50.7l(e) for
periodic revisions to the FSAR.
This item raised in the May 21, 1996
report to the Commission regarding periodic FSAR updates at Salem is
closed.
E2
Engineering Support of Facilities and Equipment
E2.l Service Water Pump Strainer Trips (71707)
During the inspection period, the inspector noted five Unit 2 service
water (SW) pump strainer trips while in service. Continued problems
with no. 23 SW pump strainer accounted for four of the above trips.
These trips resulted in SW pressure reductions and SW pump auto starts,
however, no significant reduction in safety margin due to plant
condition (shutdown and defueled). Engineering worked with maintenance
in an attempt to determine the root cause. At the end of the period,
engineering had not completed their root cause.determination. Service
..
I
'15
water pump strainer malfunctions represent nonconformance with the
automatic, self-cleaning features specified in the Updated Final Safety
Analysis Report (UFSAR section 9.2.1.2) and a challenge to SW system
reliability.
E2.2 Diesel Generator Jacket Water After-Cooler Heater Inoperability. NRC
Restart Item III.19
a.
Inspection Scope C71707l
b.
On May 13, 1996, during a tour of the plant, the inspector noted that
the no. 2A emergency diesel generator (EDG) jacket water (JW) after-
cooler heater breaker was tagged open.
The inspector reviewed the UFSAR
and discussed the UFSAR requirements with the Senior Nuclear Shift
Supervisor.
Observatioris and Findings
The inspector noted that UFSAR section 9.5.5 states that each EDG JW
system is supplied with an after-cooler heater. This 2kw
.
thermostatically controlled heater maintains water temperature in the
after-cooler piping when the engines are not in operation. Contrary to
this UFSAR requirement, the no. 2A EDG JW after-cooler heaters were
inoperable since April 20, 1995.
Engineering did not perform a 10 CFR
50.59 safety evaluation of this change to the UFSAR .
On May 4, 1995, engineering performed an operability determination and
concluded that the EDG was operable, given that the lube oil pre-lube
pump remained operable.
On June 8, 1995, engineering completed a Review
and Assessment of this operability determination.
Engineering concluded
that the 14 days between work order initiation and operability
determination was not consistent with Generic Letter 91-18 requirements .
. In addition~ engineering stated that the JW heater condition (WO 950420252) should be corrected prior to ambient temperature dropping to
50°F to avoid having an inoperable EOG if the pre-lube pump failed.
On March 11, 1996, outage management noted that WO 950420252 was listed
on the tagout as "schedule-hold."
On March 15, 1996, an electrical
supervisor determined that a faulty thermostat and heater required
draining JW to replace. Outage management scheduled this work activity
for the next no. 2A EDG outage window.
On May
14~ the Senior Nuclear Shift Supervisor initiated a condition.
report (960514306) to evaluate the 2kw heater UFSAR requirement with
respect to timeline~s of corrective actions.
c. Conclusions
Engineering did not evaluate a degraded emergency diesel generator
jacket water after-cooler heater condition with regard to UFSAR *
requirements.
Failure to perform a 10 CFR 50.59 safety evaluation is a
violation. However, this item is unresolved pending resolution of
I
E7.
-~
16
similar apparent nonconformances with the UFSAR and licensing bas fs
(control air and fuel handling building ventilation).
(URI 50-
2721311/96-06-02)
ADFCS Modification Engineering Quality Assurance CIP 520021
Software Verification and Validation CV&V)
.The V&V effort was initiated by the Digital Systems Group and was
implemented as a retrofit to the ADFCS design change package.
The V&V
was performed by the Digital Systems Group and independent consultants.
The V&V effort consisted of *five phases~
indep~ndent review of WDPF base (Phase I);
requirements verification (Phase II);
application software verification (Phase Ill);
validation testing (Phase IV);
final integrated report (Phase V).
The inspectors reviewed the V&V plan for the ADFCS and audited results
of the V&V for Phase I, Phase II, and Phase III. Time constraints
prevented the review Phases IV and V and two additional independent
consultant reviews .
During the implementation of each phase, discrepancy reports were
generated for any issues discovered during the revi~w~ At the
completion of each task, a comment/discrepancy review cycle was entered
to resolve any issues raised in that phase.
The next phase was not
initiated until the previous review cycle was satisfactorily completed.
All problems identified during the V&V phases were tracked in the V&V
discrepancy log.
The independent review of WDPF base software (Phase*I) was performed by
Data Refining Technologies. Data Refining Technologies reviewed the
design hardware and process utilized by Westinghouse to deliver the
ADFCS~ with specific emphasis on issues related to the use of
microprocessors and other hardware containing stored programs in control
equipment.
The conclusions of this report found that Westinghouse had
some weak points in the area of software life cycle, but no significant
issues that would affect the functionality of the ADFCS Project. The
inspectors concluded that the review was thorough and arrived at sound
conclusions based on valid reasoning.
The requirements verification (Phase II) resulted in the generation of a
system requirements document (SRO), the detailed design document (DOD),
and the requirements traceability matrix (RTM).
The RTM links the
Salem-specific requirements from the requirements documentation to the
applicable sections in the SRO, ODD, factory acceptance test (FAT), and
site acceptance test (SAT).
Generic requirements were.addressed in the
design analysis section of the Salem ADFCS design change package.
The
inspectors concluded that the RTM provided added assurance that the
17
requirements were properly translated into the design and tested when
necessary.
The inspectors reviewed the discrepancy reports and, with the help of
the licensee, categorized the 128 discrepancies as:
89% drawing
discrepancies; 8% documentation problems; 2% design discrepancies; and
1% software errors. The inspectors noted that the drawing
inconsistencies were discovered in Phase III, the application software
review.
The inspectors noted that the discrepancy log was under
informal control, which could impair accurate tracking of all issues,
although no instances of impaired.tracking were identified.
The application software verification (Phase Ill) resulted in the
generation of several discrepancy reports addressing various drawing and
software discrepancies.
However, there were no major functional or
translation errors found.
The inspectors concluded that this phase
proved to be a beneficial part in resolving inconsistencies in the
documentation and drawings, resulting in a more correct representation
of the as-built system.
The inspectors concluded, based on their limited scope V&V audit, that
the retrofit of a V&V program to the ADFCS increased the accuracy and
consistency of th.e software with the system drawings.
Cross-Connected Data Points
A certain number of software variables is generated in one DPU and
transmitted over the redundant communication bus to the other DPU.
According to the licensee, these cross-connected soft~are variables are
unique to the fleet of ADFCS plants. The inspectors reviewed a portion
of these cross-connected data points to determine the importance of
these cross-connected data points with respect to the correctness and
potential failure modes of the application.
The trace was facilitated by a matrix in the source code that referenced
the appropriate loop/ladder/algorithm and DPU for each cross-connected
data point.
In the event of a bus failure, there was a backup bus.
In
the event that data values were to get corrupted or lost during
transmission, most of the software process blocks were designed to use
the previ6us data values.
The trace to the software block diagram showed that when a cross-
connected data point was for a MSS involved with an important algorithm,
as compared to being used for recording, only one out of the three
inputs involved a cross-connected data point.
By design, if one data
value passed i~ bad, the MSS will function properlj.
The inspectors noted that the use of cross-connected data has the
potential to disrupt proper functioning.
In the Salem ADFCS, the
inspectors concluded that most of the cross~connected data are used in
MSS algorithms and the "last data values" are used on loss of data.
I
18
These factors tend to moderate any functional problems caused by cross-
connected data corruption or loss.
Software Configuration Control
The inspectors reviewed the licensee software configuration control
procedures covering the ADFCS.
Prior to shipment of the ADFCS to PSE&G,
Westinghouse maintained and implemented all software changes.
However,
the inspectors noted that the ADFCS project team also tracked these
changes~ which included changes-initiated as a result of the testing and
the FAT.
Once shipped to PSE&G, the ADFCS design was controlled under PSE&G
design change package (DCP} workbook control procedures.
The associated
software was maintained by the Digital Systems Group (DSG}.
Changes to
the DCP, including software, were controlled using a modification change
request (MCR} form,_ which had to be approved by the ADFCS project team.
.
.
.
The inspectors noted that none of the procedures used prior to DCP
closeout, including the MCR form, were specific to software. Although
this did not appear to be an*issue for this modification, the inspectors*
considered this to be a potential area for diminished control in the
site procedures, especially if the DSG was not involved.
Once the DCP reaches closeout, the ADFCS design descri p.t i.on and the
master system disks are kept in the Document Control Center and
controlled under a software-specific procedure, NC.NA-AP.ZZ-0064(Q},
"Software and Micro-Processor Based Systems (Digital Systems}." The
Digital Systems Group interpreted and expanded this procedure into their
own version; NA.DE-AP.ZZ-0054{Q}, "Process Computer Maintenance and
Modification Control Program."
The Document Control Center maintains a list of engineers in the Digital
Systems Group that have the authority to make changes to the ADFCS
software.
The design change package, including the source code listing,
is stored in a different location in the Document Control Center than
the master disks.
To facilitate a software change, an "approved" engineer will obtain .a
copy of the master disks from the Document Control Center. After
updat i og the ma.ster software disks, the engineer wi 11 download the
updated software to the ADFCS via the engineering workstation or using a
laptop computer.
The engineer will follow a Westinghouse WDPF loading
procedure.
As the data is downloading to the database, a line-by-line
verification is performed to ensure the proper transfer. Once the ..
download is complete, the program-is recompiled and a message is
displayed on the screen indicating that the download was a success/
failure and the number of lines compiled.
The new revised master disks
will replace the master disks in the Document Control Center. Only the
pages in the source code listing affected by the changes will be stored
with the change package .
I
I
19
The licensee stated that there was no contractual provision to ensure
that Westinghouse would notify PSE&G if a problem was found after system
shipment and training.
The inspectors concluded that, prior to DCP closeout, the configuration
control process for the ADFCS was successful due to knowledgeable
software engineers, in the absence of software-specific guidance.
The
inspectors also concluded that the software configuration control
procedures provided for post-DCP closeout were adequate for controlling
software changes.
The inspectors noted that, since previous versions of
7the disks and associated source code listings were not maintained, it
would be difficult to reconstruct and verify a previous version of
software, if the need arose.
Software Access Control
To ensure the integrity of the ADFCS, the ADFCS project team defined
four separate levels of software access.
The Salem Operations
Department (Level 1) is limited to the ability to monitor system status
screens and system parameters.
The Salem I&C Department (Level 2) is
given the ability to remove points from scan and input dummy values for
calibration purposes, to trend points for troubleshooting purposes, to
monitor system status, and to acknowledge and clear system alarms.
In
addition to the Level 2 functions, the Salem Technical Department (Level
3) is given the ability to use the control tuning functions for the
steam flow drift compensation and for entering values and forcing logic
ladders. However, if the system software is reloaded, all bias
adjustments made to the steam flow transmitters will revert to their
original values. All functions will be enabled for the Digital Systems
Group (Level 4).
Each level is password protected. A standard computer keyboard will not
be maintained at the engineering workstation.
Instead, users will
access the system via a keyboard with defined function keys.
The
functions will only be accessible from the operate position of the
keyboard keyswitch.
The keyboard key is maintained by the Digital
Systems Group and Salem Operations.
The inspectors reviewed a memo documenting the Salem ADFCS access
control procedure and observed a hands-on demonstration of the access
limitations on the ADFCS engineering workstation installed in Salem
Unit I. The inspectors verified that there was no system response to
function keys, unless the keyswitch was in the operate position. The
inspectors also verified that an attempt to access a function key not
permitted in a given access level resulted in an error message.
The
inspectors concluded that the ADFCS access control provided sufficient
security for the system.
Conclusions
The inspectors concluded, based on their limited scope V&V audit, that
the retrofit of a V&V program to the ADFCS increased the accuracy and
--,-
I
20
consistency of the software with the system drawings.
In the Salem AOFCS, the inspectors concluded that most of the cross-
connected data are used in MSS algorithms and the "last data values" are
used on loss of data. These factors tend to moderate any functional
problems caused by cross-connected data corruption or loss.
The inspectors concluded that prior to OCP closeout, the configuration
control process for the AOFCS was successful due to knowledgeable
software engineers, in the absence of software-specific guidance.
The
inspectors also concluded that the software configuration control
procedures provided for post-OCP closeout were adequate for controlling
software changes.
The inspectors concluded that the AOFCS access control provided adequate
security for the system.
The inspectors concluded that the licensee met their HMI design
objective of minimizing impact on plant operations personnel.
The inspectors concluded that the use of the AOFCS trainer was an
effective method for enhancing the technical quality of the l&C
maiptenance activities .
E7.1 Operations Safety Review COSRl Group Review Summary Records CRSRl. NRC
Restart Items 111.11 and III.67
a.
Inspection Scope (37550)
The inspector reviewed the following RSRs and associated documentation
to.determine the adequacy of the reviews:
RSR 343, Review of SORC meeting minutes
b.
Observations and Findings
Section 1.0, General Information, of Nuclear Administrative Procedures
NC.NA-AP.ZZ-0059(Q), 10.CFR 50.59 Applicability Reviews and Safety
Evaluations requires that the preparer should address any mode or
operating condition restrictions, if applicable. The inspector reviewed
the RSR 662, 22 Component Cooling Heat Exchanger System Upgrade, and
noted that the 10 CFR 50.59 evaluation did not address any mode or
operating condition restrictions for installing the flow measurement
The inspector determined that no mode or operating condition
restrictions applied, since the upgrade consisted of installation of a
gauge that could be done without affecting system operation .
c.
21
Salem Unit 2 TS 4.9.12.d.3, requires that the fuel handling area
ventilation (FHAV) system maintain a negative building pressure of 0.125
inch water gauge or greater negative pressure.
The UFSAR section
9.4.3.2.2 requires that the differential pressure controller maintains a
0.1 inch water gauge negative pressure in the building. The inspector
noted, while reviewing RSR 667, Fuel Handling Building System Upgrades,
dated 10/25/94, that a change to the differential pressure control
systems setpoint from a -0.1 inch to a -0.2 inch water gauge was
identified in Design Change Request (DCR) 2EC-3242, Rev.3.
The
inspector also noted that 50.59 Review and Safety Evaluation, Rev.I, for
the DCR did not address the change to the setpoint. Although the Salem
staff changed the differential pressure controller setpoint to meet the
requirements of TS 4.9.12.d.3, they did not consider possibility for
adverse consequences of increasing differential pressure on the FHAV
components prior to making the change, as required by 10 CFR 50.59.
Also, the original RSR reviewed identified a change to the setpoint;
however, the second level reviewer inappropriately determined that the
conservative nature of the setpoint change did not necessitate a more
extensive review.
As a result of FHAV problems discovered in August 1995, engineers
completed an analysis that concluded FHAV components could withstand 8.3
inches of water gauge negative pressure. This licensee identified and
corrected violation is being treated as a Non-Cited Violation,
consistent with Section VII.B.l of the NRC Enforcement Policy.
In
Licensee Event Report 95-24 for both Salem units, the licensee
identified several FHAV problems, including that the surveillance
procedure for TS 4.9.12.d.3 did not insure that FHAV maintained the
required differential pressure during normal operation. At the close of
the inspection period, the licensee and the inspectors continued to
pursue additional questions about FHAV compliance with regulatory,
design basis, and licensing basis requirements. These issues will
remain unresolved pending completion of the inspection, and will be
addressed as part of the Unresolved Item discussed in section E2.2,
above.
The inspector reviewed RSR 343, Maintenance Management Information
System Discrep~ncies for Emergency Diesel Generator air receiver valves,
and noted that the RSR documented discrepancies in the MMIS system such
that system functional descriptions and locations for all of the valves
in that system were missing.
The inspector verified that the MMIS
system had been revised for all of the valves identified in the RSR.
The inspector determined that these changes were made in accordance with
the applicable Department Administrative Procedure, NC.DE-AP.ZZ-0015(Q),
MMIS Resource Data Module.
Conclusions
Engineering failure to insure that FHAV maintained the required
differential pressure during normal operation is unresolved pending
completion of inspection of related questions concerning compliance with
regulatory, licensing basis, and design basis requirements.
EB
22 -
Miscellaneous Engineering Issues (92903)
ES.I Reliability of Station Air Compressors. NRC Restart Item II.2
-a.
Inspection Scope C92~03)
During the period, the inspector noted continued problems involving
station air compressors.
b. Observations
On April 8; 1996, no~ 2 station air compressor (SAC) tripped on high
vibration. The system manager recommended a detailed investigation
after plant staff returned the no. 3 SAC to service.
On April 25, no. 2
SAC tripped on high vibration. * The system manager determined that the
multiple trips _warranted a detailed investigation. The operating shift
maintained a station air contingency plan that relied on installed
temporary air compressors.
On May 1, no. 1 SAC tripped due to excessive water level in the moisture
separator. The Unit 1 and Unit 2 emergency control air compressors
(ECACs) started automatically and maintained control air header pressure
at -approximately 85 psig as designed. Control room operators entered
Sl(2).0P-AB.CA~ooo1, Loss of Control Air. Nuclear equipment operators
started and loaded the temporary air compressors.
The system manager
determined that rust particles from corroded carbon steel piping blocked
the ~oisture separator drain piping, causing the compressor trip.
On May 5, operators placed no. 3 SAC in service following maintenance.
Later in the day, operators removed no. 3 SAC-from service_ due to air_
leakage at the aftercooler inlet.
On May 6, operators placed no. 3 SAC
back in service. , On May 11, operators prepared to unload no. 3 SAC to
investigate high compressor current swings. While loading ho. 1 SAC,
operators noticed water spraying from air piping. The system manager
determined that condensation in the discharge piping and leaking flanges
caused this effect.
On May 14, operators returned no. 1 SAC to service
following maintenance.
As operators unloaded no. 3 SAC~ however, it
tiipped on high vibration.
c. Conclusions
Between April 8 and May 14, four station air compressor trips and two
additional compressor failures required operators to take contingency
actions. Inspectors concluded that maintenance and system engineering
staff did not effectively ensure station air system reliability.
(Open) Inspector Follow-up Item 50-272&311/95-21-02: Service water (SWi
reliability issues. This issue was open pending NRC review of licensee
corrective actions stemming from condition reports addressing challenges
to SW.
23
The first concern involved SW bay desilting.
In the recent past,
maintenance closed out a work order for multiple SW pump silt
inspections prior to performing all activities. -Engineering determined
that a separate work order for each SW pump bay si.lt inspection would
preclude recurrence (PR 00960113315).
The inspector noted that the corrective action addressed the particular
case of missed silting inspections, however, did not fully answer silt
build-up concerns.
The system engineer stated that 92-day silt
inspections routinely found more than 3 feet of silt build-up. The
system engineer attributed this to the present shutdown condition .
(usually only one SW pump operating per unit). The inspector observed
that operations does not implement a SW pump rotation plan to preclude a
similar condition during normal power operation. Silt build-up
increases shutdown risk when defueled and may prevent the SW system from
fulfilling its design basis safety function at power (see NRC inspection
report 95-21).
The second concern involved the ability of the SW system to perform its
design function under worst case conditions. This concern evolved from
challenges to SW from grass, debris, silt, and ice given the non-safety
related and seismic class III construction of the instruments and
controls associated with the SW traveling screens.
In addressing the
above concern, engineering determined that operator error and
inattention, not material deficiency, caused the January 7, 1996, SW
pressure perturbation (PR 00960107144).
The inspector determined that the Service Water System (SWS) functioned
as designed on January 7, 1996.
However, PR 00960107144 did not address
or allay inspector concern regarding potential SWS vulnerabilities.
In
particular, the ability of SW to function as designed with traveling
screens in place, but not *rotating; given worst case river grass loading
as experienced in the l~st two years.
Inspector assumed traveling
screens not rotating based on non-safety related, seismic class Ill,
screen permissive pressure switches and seismic class III screen motors.
In addition, SW screen differential pressure switches are non-safety
- related and cannot be relied upon to provide control room operators
sufficient warning of a blocked screen. A control room operator's first
indication would be a low SW pressure alarm.
The inspector determined that this item remains open pending NRC review
of '(l) licensee's siltation control program, (2) susceptibility of SW
traveling screens to debris clogging, (3) licensee's interpretation of
UFSAR {section 9.2.1.2) statement "The SWS is designed for class I
(seismic) conditions except for the turbine area service water piping
outside of the service water intake structure" relative to seismic
classification of SW components in the intake structure.'
I
24
E8.3
Hagan Modules R~furbishment and Replacement
a.
Inspection Scope
On February 23, 1996, the NRC issued their Restart Action Plan for the
- Salem Units. This plan contains the programs and corrective actions
that the NRC.will inspect prior to .the restart of the Salem plants. The
reliability and configuration control' of the Westinghouse Model 7100
process instruments, al so known as Hagan modules, are i terns I 1.16 *and
111.2 of the NRC restart action plan.
The NRC concerns with the Hagan modules stemmed from a number of
previous inspection observations and findings and several plant events,
including: (1) the automatic reactor shutdown and actuation of the
safety injection system at Salem Unit 1, on April 7, 1994 (augmented
team inspection report 50-272 and 50-311/94-80); and (2) recurring
problems with the main steam atmospheric relief (MSlO) valves of both
units, in February 1995 (resident inspection report 50-272 and
50-311/95-02). The NRC concerns included component reliability,
configuration control, and application issues,. such as cabinet
temperature and susceptibility to electromagnetic radio frequency
interference (EMl/RFI).
The application concerns were identified in
inspection report No. 50-272 and 50-311/96-01.
The pu~pose of this inspection ~as to review PSE&G's resolution of the
various NRC concerns and to eva 1 uate their acceptability.
b.
Observations and Findings
PSE&G Program for the Hagan Modules
The Hagan module reliability and configuration control were the subject
of numerous problem reports.
To address all concerns in these areas,
PSE&G established a project team.
The objectives of this team were to
evaluate the identified anomalies and failures and to develop a program
for resolving them.
PSE&G resolution of the Hagan module concerns
involved the replacement of some modules and the refurbishment of
others. The replacement modules were furnished by Nuclear Utility
Services (NUS).
The replacement and refurbishment decisions were based
primarily on availability of spare parts and components and their
recognition of the decreased reliability of the existing Hagan modules
due to aging.
The results of the NRC review of the licensee program are
described in the sections below.
Design & Procurement of Replacement Modules
During the current outage the licensee will *replace three types of
modules: signal summators, signal isolators, and RTD low.level
amplifiers. The req*uirements for these and other modules scheduled to
be replaced in the future were described in Purchase Specification No.
S-C-RCP-EDS-0308, Revision 2 ..
I
25
In general, the technical and performance requirements for all NUS
- modules were derived from those specified by Westinghouse for the Hagan
modules.
Qualification requirements were based on applicable industry
standards and Regulatory Guides.
The inspector's review of this
document identified no discrepancies, except for the maximum specified
ambient temperature (110 °F)-which was less than the one specified for
.the-Hagan modules {120 °F).
Further review determined that the
furnished NUS modules have a maximum temperature rating of 122 °F for
normal operation and 135 °F for abnormal {200 hr) operation. Therefore,
the discrepancy was not a concern.
The environmental and seismic qualification of the above modules had
been addressed.
For instance, reports EIP-QR-MBA800, Revision 0, and
EIP-QR-800, Revision 1, addressed qualification of the summators and
.isolators, respectively. - The inspector reviewed these documents and
identified no disc~epancies.
Change Package Process for Replaced Modules
The replacement of the Hagan modules with NUS modules was addressed by
design change 2EC-3450, package No. 1.
The inspector's review of this
document determined that the technical aspects of the change had been
evaluated in sufficient detail to ensure the acceptability of the new
replacement modules.
The changes had undergone multi-disciplinary
review and had been evaluated in accordance with lOCFR 50.59 to ensure
that the changes did not constitute an unreviewed safety question.
The
licensee had addressed the impact of the change on instrument loop
accuracy (engineering evaluation No. s~c-RCP-CEE-1037) a~d on the vital
bus total harmonic distortion. Regarding the instrument loop accuracy
calculation, the inspector observed that, in assessing the loop error
due to*calibration temperature, the licensee had assumed the normal
rather than the minimum ambient temperature.
The inspector's review of the safety evaluation, NC.NA-AP.ZZ-0059-3,
Revision 0, determined that the licensee based its conclusion that the
change did not constitute an unreviewed safety question on a detailed
review of the replacement components and their difference from the
replaced components.
The inspector identified no areas of concern with
the change package.
However, the instrument loop accuracy is unresolved
pending the licensee's revision of calculation S-C-RCP-CEE-1037 to
include minimum calibration temperature and maximum operating
temperature.
(50-272;311/96-06-03)
Module Refurbishment
~he scape and process for the refurbishment of fhe Hagan modules were
described in the Salem maintenance procedure, SC.IC-PM.ZZ-0023{Q),
Revision 10, "Hagan Module Refurbishment."
The method for conducting
the work was described in Procedure SC.IC-TI.ZZ-OOOl(Q), Revisio~ 3,
"Soldering/Desoldering."
-1
I
26
As described below, problems with configuration control resulted in the
existence at Salem of modules with different revision level. The intent
of the refurbishment was not only to perform needed repairs, replace
aged components, and bring the modules to a like-new status, but also to
upgrade all modules to their latest revision level. The procedure did
that in great detail and included drawings as well as check list tables
to identify the scope of work.
Fourteen types of modules in various
configurations were addressed by the procedure.
Subsequent to its repair*and upgrading, the licensee planned to subject
each module to bench testing, calibration, and a 50-hour burn-in period.
The licensee developed bench testing requirements for each type of
module and generated the required response to the various inputs. The
module bench testing process and the test acceptance criteria were
described in a series of procedures developed purposely for the task.
For instance, procedure SC.IC-GP.ZZ-0123(Q), Revision 4, was prepared*
for the bench testing of the Hagan loop power supplies and SC.1C-GP.ZZ-
0125(Q), Revision 1, was used for the Hagan Model 118-MVI amplifiers.
The inspector's review of the refurbishment procedures and of a sample
of bench testing procedures identified no areas of concern.
Change Package Process for Upgraded Modules
The licensee developed generic design change packages (DCPs) for each
type of module using the equivalent replacement process. For each type
of module, the licensee provided a table describing the module
specifications in both the present and upgraded versions.
Sine~ no
changes were identified in the table, the licensee concluded that the
upgrading would not affect the form, fit and function of the module.
The design change package also included a list of upgraded parts and
changes.
For instance, the upgrades of manual/automatic setpoint
station modules type 6627007-GOl and -G02, involved the addition of two
capacitors and the replacement of three transistors, three relays, and
one diode with Westinghouse components.
The DCP, however, did not
describe either the function of the two capacitors or the bases and
impact for using different piece parts.
Because the licensee considered the modified modules as equivalent
replacements, they did not perform an evaluation of the changes in
accordance with lOCFR 50.59.
Instead, in a note on the first page of
the DCP, they simply referred to a generic safety evaluation, No. A-O-
VARX-NSE-0727-1, as applicable. This document, titled, "Equivalent
Replacement and Document Update Generic Evaluation," generically
addressed and answered "no" to each question regarding whether or not
the change constituted an unreviewed safety question.
The document
never referred to the Hagan module refurbishment.
The inspector disagreed that the upgraded modules were equivalent
replacements in that the upgrading constituted in itself a functional
change of the module.
In addition, the design change package had not
I
27
described either the functions of the added components or the impact of
the replaced components on the performance of the module.
Therefore,
the licensee had not proven either that the upgraded module was an
equivalent replacement or that a safety evaluation was not warranted.
The acceptability of the design change packages for upgraded Hagan
modules is unresolved pending the licensee's revision and NRC's review
of the applicable documents to:
(1) identify the function of all module
upgrades, (2) describe the impact of these upgrades on the performance
of the modules, (3) clarify why the changes do not constitute all
unreviewed safety question, and (4) show the applicability of the
equivalent replacement process.
(50-272;311/96-06-04)
(Updated) Unreso~ved Item 50-272 and 50-311/96-01-01: Operating
Temperature of the Modules
During a January 1996 review of the Hagan upgrade program (NRC report
50-272 and 50-311/96-01), the inspector discovered that some modules
might be "normally" operated close to their design temperature limit of
120 °F.
Therefore, he expressed a concern that, under extreme ambient
conditions, the same modules might be used beyond such limit.
As a followup to the above observation, the inspector reviewed the
licensee evaluation of the operating temperature of the modules and
actions to address any discrepancies found.
The inspector determined
that, in December 1995, the licensee had conducted a temperature mapping
of a rack to determine the need for door and module housing
modifications.
They conducted six tests during which they measured the
ambient temperature both inside and outside the rack with three types of
doors and with vented as ~ell as unvented module side panels.
The
results of these tests were described in Test Report File 11-229/11-278,
Revision 1, dated March 15, 1996.
Based on the results of the above tests, the licensee concluded that the
currently used solid rack door and module side panels were acceptable.
The licensee's conclusions were based on their finding that, with a
normal room ambient temperature of 76 °F, the maximum rack ambient
temperature would be 91 °F and, hence, well below the design (rack
ambient) temperature specified by Westinghouse.*
For the equipment room, Section 3.11.1.3 of the FSAR specified a normal
operating ambient temperature of 70 °F +/- 15 °F.
The inspector also
determined that the design basis for the control area air-conditioning
system was based on its ability to maintain 76 °F with an outside summer
temperature of 95 °F.
Following a station blackout and loss of all
ventilation, a licensee calculation, No. S-C-AUX-MDC-0737, Revision 0,
showed that the control equipment room could reach a maximum temperature
pf 117 °F.
.
In discussing the result of the temperature test with the licensee, the
inspector determined that Westinghouse had conducted similar tests in
February 1971, although a report of these tests was apparently never
28
received by the licensee. The inspector's review of the report obtained
during the inspection (No. HC-25205, "Temperature Test on Racks and
- Modules of the Nuclear Protection System Public Service Electric & Gas
Company") revealed that, with considerable fewer modules and, hence,
with less heat load, the temperature in the rack rear airspace was
slightly higher than the one measured by PSE&G.
The inspector also
noted that the temperature in the rack front airspace and in the
airspace under the module shelves, not measured by PSE&G, was several
degrees higher than the one in the rack rear airspace.
The tables provided with the above Westinghouse report clearly indicated
that, when the rack fill _is considered, the modules located in the upper
shelves of the rack could be exposed to an ambient temperature higher
than 120 °F when the room temperature is within the "normal" range (less
_than 85 °F).
For instance, Table 19 of the report shows that, with a
room temperature of approximately 92 °F, the temperature in the airspace
under the shelf was as high as 122 °F.
The racks tested by Westinghouse
were sparsely loaded, when compared to the Salem racks.
The results of the Westinghouse tests raised several questions, such as
temperature effects on module accuracy and calibration, surveillance
requirements to ensure detectability of degraded module conditions, and
common cause failures due to heat during an abnormal event (station
blackout). Westinghouse addressed some of these issues in their
analysis of the results. For instance, on page 14 of the report,
Westinghouse pointed out the "definite temperature dependence" of the
module accuracy and, on page 16, recommended that the "rack ambient"
temperature not exceed 100 °F for normal operation. Because, however,
the rack heat load in the Westinghouse tests did not appear to envelop
that of the Salem racks, further analysis is required to determi~e the
applicability of all Westinghouse conclusions to Salem and to establish
the actions that are required to resolve the temperature issue.
The inspector discussed his concerns about the operating temperature of
the modules with the licensee who pointed out that the NUS modules
consumed less power and, therefore, would result in a lower temperature
in the racks.
The total impact, however, of the current changes was not
known without an evaluation by the licensee.
The operating temperature of the modules remains unresolved pending the
NRC evaluation of PSE&G's reconciliation of PSE&G and Westinghouse
temperature test results; determination of the "normal" ambient
temperature range for safety-related Hagan and NUS modules (considering
worst-case rack and module arrangements); determination of the impact of
this temperature on the maintenance, surveillance, and calibration
requirements of the modules; confirmation of acceptable module
performance under normal operating temperatures; and assurance of the
operability and performance of the modules under worst abnormal and
accident conditions affecting the equipment room ambient temperature .
' \\
29
(Updated) Unresolved Item 50-272 and 50-311/96-01-02: Electromagnetic *
and Radio Frequency Interference
During the January 1996 review of the Hagan module refurbishment and
replacement project, the inspector asked the licensee whether the
current program would address electromagnetic and radio frequency
interference (EMI/RFI).
Because of the NRC questions, the licensee
contracted the services of a consultant to evaluate whether the
replacement of the Hagan modules with NUS modules would represent an
increased EMI risk. A secondary objective of the review was to evaluate
whether the refurbishment of Hagan modules would similarly represent an
increased EMI risk.
The consultant's evaluation, described in Report No. CSR082, dated
April 11, 1996, concluded that the replacement of Hagan modules with NUS
modules would increase the immunity of the Hagan 7100 plant protection
and control system. Their conclusion was based on EMI immunity tests
performed on the modules, as well as on their review of the design
improvement included in the modules.
Regarding the refurbished module,
the consultant concluded that the replacement of aged electronic
components with new ones would improve module performance and,
therefore, reduce the potential for electromagnetic emission due to
deficient components.
The inspector reviewed the evaluation report and identified no areas of
concern regarding the replacement of the Hagan modules with NUS type
modules.
The inspector, however, observed that the report had not
specificallf addressed the increased speed of new solid state components
in the upgraded modules and the impact of the switching, integrated
power supplies used in the NUS modules on these components.
This item
remains open pending PSE&G's evaluation of these two issues and the NRC
review of the licensee's conclusions.
Configuration Control
The Salem Hagan 7100 series process control modules were supplied by
Although several years jgo Westinghouse sold this product
line to Rosemount, they continued to provide needed technical support to
their customers.
Westinghouse also continued to be the sole source for
replacement modules and spare parts and for the repair of failed units.
Apparently, from the time they were originally supplied to PSE&G, all
modules underwent several upgrades.
As stated in Salem Quality
Assurance audit report No. SNA95-092, dated October 21, 1995; when a new
order was placed, the modules were manufactured to their latest revision
level, unless PSE&G specified otherwise. Repairs, however, were made to
the extent necessary to render the modules functional again.
Drawings
of new or repaired modules were not furnished unless specifically
requested in the purchase order.
Due to this process, the Salem
instrument design used several revisions of the same module .
30
_PSE&G became aware of the existence of these design difference during
audits of Rosemount and Westinghouse in June and October 1995,
respectively, and when they reviewed the latest module drawings that
they had obtained to support their current module refurbishment effort.
The identified discrepancies resulted in the stoppage, on
October 10, 1995, of the refurbishment process, until engineering could
evaluate the impact of such discrepancies and propose a solution.
As
stated previously, a decision was made to upgrade all modules to their
latest revision level.
The inspector reviewed PSE&G's current configuration control process.
He determined that once the modules were refurbished and upgraded to the
latest Westinghouse drawing revision level, they were visually
inspected, catalogued and placed into stock in a controlled storage
facility. The inspector also determined that, when the licensee
received the latest Westinghouse drawings and part lists, they developed
a computerized data base that included both Westinghouse and PSE&G part
numbers.
This data base was then used to develop the bills of material
for the refurbishment of each module.
Components, properly labeled,
were kept in individual wrappers in a controlled storage facility. The
current material control program is described in procedure NC.NA-AP.ZZ-
. 0018(Q), Revision 2, dated April 10, 1996.
The licensee developed configuration control procedures for each type of
module.
The purpose of these procedures was to ensure that design
configuration is maintained when removing from or installing a module
into its assigned location within an instrument rack.
The configuration
process, e.g., procedure SC.IC-TI.ZZ-0102(Q) for the comparator.modules,
involved the dedication of a particular module to a specific instrument
loop, affixing the labeling necessary for its full identification, and
verifying that the design attributes for the specific application of the
module had been implemented.
If a module is removed from service for
any reason, the procedure requires that it be divested of its specific
identification, repaired if necessary, and returned to stock. A new
module is dedicated to the same service.
The inspector also addressed two specific issues regarding past
configuration concerns: first, control of safety and nonsafety-related*
inventory at the work area, and second, availability of adequate
documentation to the instrument and controls (I&C) department to ensure
that modules. were properly configured prior to installation in safety-
related systems.
Regarding the first issue, the inspector's review of the current
procedure, NC.NA-AP.ZZ-0018(Q), determined that it did not prevent
availability of both safety and nonsafety-related components at the work
area.
The procedure, however, included sufficiently specific
instructions about responsibilities and component control to prevent
inadvertent misuse of such components.
The procedure also provided
sufficient guidance regarding return of unused components to stock.
The
inspector considered the current requirements acceptable because of
c.
31
prestaging needs, technical personnel utilization, and existence of
routine as well as emergency work at the same time.
Currently, Hagan modules that are used .in both safety and nonsafety-
related applications are treated as safety-related. Therefore, the
inadvertent use of nonsafety-related parts in this application is
avoided.
The inspector's direct observation of the work area indicated
that materials were clearly labeled and properly controlled."
'Regarding the second issue, the inspector determined that currently,_ the
l&C group assigned to the refurbishment of the Hagan modules had up-to-
date documents available for their use. Until late 1995, the quality of
the documents available to the same group was less than adequate, as
evidenced by the results of the PSE&G's inspection of Rosemount and
Westinghouse, by the discrepancies identified during their review of the *
newly-obtained design drawings, and by the stop-work order on
October 10, 1995.
PSE&G's review of this issue in early 1995 had
incorrectly determined that the ~onfiguration control of the.Hagan
modules.was not a concern.
The inspector has no current concerns in
this area.
Review of UFSAR Commitments
A recent discovery of a licensee operating their facility in a manner
contrary to the Updated Fi.nal Safety Analysis Report (UFSAR) description
highlighted the need for a special focused review that compares plant
practices, procedures and/or parameters to the UFSAR descriptions.
While perfor~ing the inspections discussed in this report, the inspector
reviewed the applicable portions of the UFSAR that related to the areas
inspected. The inspector verified that the UFSAR wordi~g was cotisiste~t
with the observed plant practices, procedures and/or parameters.
Conclusions and General Comments
Based on hi~ review of .the refurbishment, testing, and configuration
- control of the Hagan module, the inspector concluded* that the licensee
had made progress in developing a viable process to ensure a good
product and a good control of future modifications. The procedures
developed for this effort were detailed and easy to follow.
Based on
his interview of several engineering, technical and supervising_
personnel, the inspector also concluded that they were knowledgeable of
the process and of the module performance requirements.
Effective communication among supervision, engineering, and technical
personnel was evident in their resolution of identified discrepancies.
When discrepancies were identified, whether in the original design or
the refurbishment process, they were documented, discussed, and resolved
- among appropriate groups.
The design change package process for NUS modules was acceptable,
although the instrument loop error is unresolved pending revision of the
I;)
32
calculation to include minimum calibration temperature and maximum
operating.module temperature.
The acceptability of design change package process for upgraded Hagan
modules is unresolved pending PSE&G's review of the packages to
determine the significance of the changes made to the circuit boards,
whether the design modifications constitute an unreviewed safety
question, and whether the design equivalence.process is applicable to
these modules.
The operating temperature of the NUS and Hagan modules was questioned by
the NRC previously, but a full and effective evaluation of the issue by
the licensee was not performed for its closure. Closure of this issue
is pending the licensee's review and assurance that the modules will be
capable of performing as intended under all anticipated operating
environments.
The inspector identified no EMI/RFI concerns regarding the NUS modules.
The susceptibility of the Hagan modules remains unresolved pending
PSE&G's verification that their performance is not affected by the speed
of new solid state components or by the switching, integrated power*
supplies used 1n the NUS modules on these components.
The review of past experience with the modules indicated the existence
of problems in the area of configuration control, some of which could be*
related to ineffective procurement guidelines in the past.
The licensee
efforts to regain contfol of the Hagan module corifiguration were good
and their development of procedures should enhance their ability to
maintain such control in the future.
E.8.4 Advanced Digital Feedwater Control System CADFCS) Modification*
a.
Scope of Review (IP 52002)
The ADFCS modification is involved in NRC Salem techni~al restart issues
114.
The review of the modification, Desig~ Package 2EC-3178, involved
interviews with members of the design and system engineering-staff,
review of design and software documentation, and a walkdown of the
equipment at the plant and maintenance facility. The inspectors
reviewed documents associated with the modification as follows:
- * * * * * * * * *
safety analysis and evaluation report;
design basis;
setpoints and control constants;
electromagnetic interference (EMI) susceptibility;
human-machine interface (HMI)
maintenance
software verification and validation (V&V);
cross-connected data points;
software configuration control;
software access control .
33
b.
Background
The ADFCS modification replaced an analog control system for steam
generator water level control with an advanced digital feedwater control
system (ADFCS) by Westinghouse.
The main purpose of the modification
was to minimize plant unavailability due to reactor trips caused by
analog system component failures and low power instability.
--1
The ADFCS equipment was installed in Salem Unit 1, and only wire pulling
was accomplished for Salem Unit 2.
The ADFCS equipment for Salem Unit 2
was set up in a maintenance facility on the Salem site, awaiting the
site acceptance test (SAT).
The ADFCS provides automatic control of steam generator water level
with out operator intervention at a 11 power levels above 2%.
Inventory
in the steam generators is maintained by automatically positioning the
generator.
The ADFCS also controls the main feedwater pump.
In addition, the ADFCS also automatically controls the modulation of the
atmospheric relief valves (ARV).
There are automatic/manual control
stations for all the feedwater valves and the ARVs.
The ADFCS is implemented using the Westinghouse Eagle Distributed
Processing Family (WDPF) and is powered from the redundant vital
instrument buses, which are backed by battery inverter units.* The
system has two distributed processor units (DPU), each internally
connected as a fail-over processor pair. The processors use an Intel
386 microprocessor. Only one part of the internal fail-over pair
processors controls the. system; the backup processor receives current
data over the data highway, monitors the status of the control
processor, and performs diagnostics. Automatic switchover to the backup
processor occurs on power interruption, failur~ of the control processor
itself, or its shared memory, or math co-processor~ or data highway
interface.
When automatic switchover occurs, the backup processor
_
receives active data from the input/output (I/O) bus, and the algorithms
incorporate measures to insure bumpless transfer ..
The two DPUs contain the control algorithms and signal processing, but
do not control identical outputs and are not consjdered to be .redundant.
The processing for the f~edwater flow control valves and ARVs are
divided between the two DPUs. -One DPU controls the main feed pumps.
A
limited set of data points are shared over the redundant data highway
between the DPUs for some algorithms.
The ADFCS control algorithms were developed by Westinghouse.
designs similar to Salem are installed and operating at Prairie Island,
-Catawba, Ginna, and Diablo Canyon nuclear ~ower plants.
The protective system low feedwater flow trip, which involved steam
flow/feed flow mismatch, was ,eliminated. Three narrow range steam
g.enerator 1 evel inputs per loop were added to the ADFCS to compensate
34
for the elimination of the low feedwater flow trip. The narrow range
steam generator level inputs are continuously validated in the ADFCS by
a median signal selection (MSS) software compartson technique.
The
purpose of the MSS is to prevent a failed instrument channel from
causing a disturbance.in the ADFCS, which could then initiate a plant
- transient that could require protective action.
The elimination of the
RPS low feedwater trip and the use of the MSS software technique was
previously reviewed and approved by NRR.
Three types of input signal validation using software techniques are
employed.
The first technique is the MSS, where the median of three
inputs is used for the control algorithms. This prevents. high or low
failures of a single input from affecting the control system.
The
narrow range steam generator level, steam flow, steam generator
pressure, feedwater flow, feedwater temperature, and feedwater header
pressure are validated using the MSS technique.
The second technique of input signal validation is arbitration, where
two inputs are compared; if they agree to a preset criterion, their
average is used for the control algorithms.
If the two channels differ
from the criterion, they are compared to an estimate of the variable
that ii calculated using other process measurements.
The input that is
closer to the estimate is then used for the control algorithms .. Turbine
first stage pressure is validated using the arbitration technique.
.
.
The third input signal validation technique is data quality check or
signal quality check for single input variables.
Diagnostics are incorporated that are automatically executed during the
normal operation of the system and do not disrupt the real-time
performance of the processor. Should a malfunction occur, the active
processor will fail over to the backup processor, *and a trouble alarm
will be generated.
An engineer/operator workstation connects to the system over the data
highway and provides the software engineering tools required to
configure and maintain the ADFCS.
The. workstation is used primarily for
on-line monitoring of control loops, and process inputs, outputs, alarm
status, hardware status, and high/lbw limits. The operator mode allows
graphics to be presented that allow defined actions, such as changing
selected alarm and process variables along with viewing process values.
The engineer mode allows construction of graphics for the operator mode,
. configuration of software, and downloading of application software for
use in the DPUs.
The workstation access is controlled by keylock and is
intended to.be used by system engineers and l&C technicians.
In normal
operation, after the programs and constants have been downloaded to the
DPUs, the workstations do not perform any control system functions.*
35
I.
c. Observations and Findings
Safety Anafysis/Evaluation
/.
The inspectors reviewed the licensee safety evaluation and noted the
following:
(1)
Any effects of the consolidation of the main feed pump speed
control were not considered. There is the possibility that the
main feed pump control may fail in conjunction with the control of
the feedwater regulator valves and the control of the ARVs.
This
is because the control systems are controlled by the same digital
processor. This possibility co~ld be classified as a different
type of an anticipated operational occurrence initiating event.
(2)
The effects of the consolidation of the ARVs may cause a different
type of an anticipated operational occurrence initiating event.
The licensee stated that the failure of one or more ARVs in
conjunction with one or more feedwater control valves was
analyzed.
The licensee changed the analysis of Chapter 15.4.8.2,
"Mass and Energy Releases Following a Steamline Rupture," to bound
this different type of initiating event.
(3)
The new control mode of the feedwater bypass valves may cause the
possibility of a malfunction of a different kind.
The licensee
stated that this malfunction was considered, and the analysis of
USFAR Chapter 15.2.10, "Excessive Heat Removal Due to Feedwater
System Malfunction," was changed to bound the possible
malfunction.
The above issues apparently conflict wtth 10 CFR 50.59(2)(ii) and the
NRC Inspection Manual Part 9900, "IO CFR Guidance."
The issues need
clarification from a licensing standpoint, since it appears that the
possibility for a different type of initiating event or a malfunction of
a different type than any evaluated previously in the USFAR may be
created, even though the licensee stated that the modified analyses show
the effects are bounded in some instances.
The above issues are considered unresolved pending NRC configuration of
the licensee's analyses and further NR.C regional and headquarters review
(URI 50-272/96-06-05).
Design Basis
The Westinghouse proprietary functional requirements and detailed design
documents covered generic criteria for the process sampling rates and
processing delays based on engineering judgement, but there were no
references to any specific analyses for sampling rates for Salem.
The
licensee calculated a worst~case processing delay and stated that it was
less than processing delays for other ADFCS plants.
' I
36
The inspectors determined through interviews that engineering judgement,
based on operational experience at other ADFCS plants and computer
simulations, most likely formed the bases for the Salem values of the
sampling rates and processing delays.
Setpoints and Control Constants
The inspectors determined through interviews that Westinghouse developed
the design bases for the setpoints and software constants using data
from the analog system and their experience with other PWR plants. * The
setpoints and control constants incorporated the Salem unique
characteristics of the steam piping, feed piping, valve Cv
linearization, and pump performance curves.
The inspectors concluded that the setpoint~ and control constants were
based on empirical data and analyses from other ADFCS plants, with the
necessary particularization for Salem.
EMI Testing
The ADFCS system was tested by Westinghouse for EMI-radiated
susceptibility with the cabinet doors open and closed.
The
configuration tested used the Intel 8086 microprocessor.
The
susceptibility field strength from 20 MHz to I GHz was three
volts/meter; from 20 MHz to 500 MHz the field strength was 20
volts/meter.
The I/O cards have surge-withstand capabilities, according
to Westinghouse documents.
The ADFCS was not tested specifically for conducted EMI susceptibility.
The licensee recognized this and plans to assess the ability of the
ADFCS to handle conducted susceptibility on the 1/0 and power lines.
The inspectors concluded that the licensee considered EMI
susceptibility.
Human-Machine Interface CHMil
.The licensee stated that the design objective for the ADFCS HMI was to
.minimize physical changes to the operator interface so as to minimize
impact on plant operations personnel.
The same console manual/automatic
(M/A) stations, indicators, and status lamps that were used for the
analog design were used for the ADFCS, with the appropriate
hardware/software interfaces. The console arrangement was slightly
changed and enhanced for the ADFCS, and the manual control pushbuttons
on the M/A stations were changed from linear control to exponential
control.
The "ADFCS Trouble" annunciator window was added as an indication for
certain classes of failures, such asi input signal failure, power supply
failure, loss of data highway, or processor hardware or software
diagno~tic failure.
The "ADFCS Switch to Manual" window was added as ari
indication for the operator to take manual control of the M/A stations.
. d.
37
The inspectors reviewed the operator training handouts, lesson plans,
and alarm response procedures.
The differences between the analog
operator controls and the ADFCS operator controls were* identified and
covered in appropriate detail.
The inspectors concluded that the lic~nsee met their HMI design
objective of minimizing impact on plant operations personnel.
l&C Maintenance
The licensee purchased a set *Of ADFCS equipment to use for maintenance
training. The training equipment was used in *a six ~eek_ vendor course
for ~ngineers and technicians that covered hardware and software, which
was held at the PSE&G training center. The maintenance procedures were
debugged on the trainer. The l.icensee plans to use the trainer to test
spare boards before they are installed in the operating system.
The inspectors walked down the trainer. Wrist grounding straps on
coiled leads were installed at the front and rear of the cabinets as a
precaution against electrostatic discharge pulses being introduced into
the electronic components by personnel.
The inspectors concluded that the use of the ADFCS trainer was an
effecti~e method for enhancing the technical quality of the l&C
maintenance activities.
Walkdown
The inspectors walked down the ADFCS installation at the site. The
processing and 1/0 cabinets and the workstation are in the relay room .
. The inspectors observed a demonstration of the monitoring of the ADFCS
. system using the workstation.
The ADFCS cabinets had louvered-doors.
The inspectors observed that
front and rear access to the cabinets were adequate and that the
interior temperature rise of the c*binets was not excessive.
e. Conclusions
The inspectors concluded that there are three 10 CFR 50.59 issues that
need clarification from a licensing standpoint. It appears that the
possibility for a different type of initiating event or a malfunction of
a different type than any evaluated previously in the USFAR may be
created, even though the licensee stated that the modified analyses show
the effects are bounded in- some instances (URI 96-06-05).
The inspectors determined through interviews that engineering judgement
based on operational experience at other ADFCS plants and computer
simulations most likely formed the bases for the Salem values of the
sampling rates and processing delays.
38
The inspectors concl~ded that the setpoints and control constants were
based on empirical data and analyses from other ADFCS plants, with the
necessary particularization for Salem.
The inspectors concluded that the licensee considered EMI
susceptibility.
IV. Plant Support
P4.
Staff Knowledge.and Performance in EP
P4.l Emergency Classification Performance during Training. *NRC Restart Item
III. 7
a.
Inspection Scope (71707)
During operator requalification training, the inspectors observed and
assessed licensed operator performance, senior reactor operator (SRO)
classification of adverse plant conditions, and training staff performance.
b.
Observations and Findings
Notable improvements were observed in operator performance.
For example,
operators showed significant improvement in repeating back instructions and
information relating to equipment status. Senior reactor operators
demonstrated better command and control in that the NSS led operator
response* by reading procedures and directing the NCO response while the SNSS
monitored overall plant and operator performance.
The NCOs displayed much
improved division of responsibilities, knowledge of plant system~, and
familiarity with procedures. Also, the operators demonstrated noticeable
improvement in team work, such as the NCOs insuring the NSS understood the
impact of a particular equipment problem on overall plant operation. The - .
instructors also demonstrated improved performance.
For exa~ple, during one
sim~lator sc~nario involving a seismic event, the instructors paused the
simulator after initial operator response to lead a discussion on the design
basis for seismic events.
During an operator requalification exam, however,
the SNSS incorrectly classified a loss of all offsite and onsite power, the
training staff did not notice the error,. and the simulator lesson plan did
not provide the correct classification. In addition, the inspector
. id~ntified that the training ~taff did not consider the incorrect
classification cause for failing the SNSS,'nor.did the lead in~tructor
consider it necessary to ini'tiate a Condition Report (CR) to document the
error in the lesson plan.
The Operations Manager, however, directed the
training staff to consider the SNSS exam a failure, and the other members of
the training staff initiated a CR.
c.
Conclusions
Overall, during requalification training operators demonstrated significant
improvement in command and control, technical competence, and teamwork.
In
one case, however, an SNSS_incorrectly classified a postulated event.
The
lesson plan for the event contained the incorrect classification, the
39
instructor did not detect the error and did not take appropriate corrective
action until the inspectors questioned the corrective acti~n and the
operations manager became involved~
P5
Staff Training and Qualification in EP
PS.I Staff Training and Qualification in EP. NRC Restart Item III.I3
a.
Inspection Scope (7I707)
During an unannounced call out EP exercise, inspectors observed emergency
response organization and EP staff performance.
b. Observations and Findings
The SNSS corr~ctly classified the event in accordance ~ith the event
classification guide.
The technical support center (TSC) and emergency
operating* facility (EOF) staffs met their goal of staffing the facilities
within 90 minutes of declaration of an Alert, despite confusion stemming
from the call out message.
Although a significant number of the ERO
responders expressed confusion over whether the call out message directed
them to respond, they appropriately decided to go to their designated
The TSC director demonstrated effective leadership.
For example, he called his staff together, briefed them on the nature of the
- event, and provided goals for event response.
The EOF director, on the
other hand, did not demonstrate effective command and control.
He discussed
the call out message with various EOF staff, and did not provide leadership
in responding to the event scenario. * The EP staff frequently confused their
roles as scenario controllers and referees with the roles of instructor and
player.
In some instahces, they made it difficult to determine the ERO
effectiveness.
For ~xample, one controller prompted the SNSS.to let the
controller perform the role of SNSS for the purpose of the drill.
In the
TSC, a controller showed an engineer coordinator how to perform his duties.
In the EOF, a controller performed the duties of director while the
designated director pursued clarification of the call out message.
c. Conclusions
During an unannounced call out EP exercise, the ERO met the goal of manning
the TSC and the EOF within 90 minutes. Overall, the ERO adequately
discharged. its duties required to protect the health and safety of the
public, however, the EP staff reduced the effectiveness of the traini.ng by
not allowing the ERO staff, in some cases, to make errors.
Sl
Conduct of Security and Safeguards Activities
SI.I Suspicious Substance Found in Protected Area
a.
Scope *cn 750)
The inspectors observed plant management's appropriate response to a
technician's discovery of a packet of white powder in a locker.
-
40
b.
Observations and Findings
On April 11, a temporary radiation protect ion (RP) technician discovered a
packet of white powder in a locker that had been his but that he had vacated
some time ago.
Temporary workers use the locker room located inside the*
Protected Area but outside Vital Areas.
The technician notified his .
supervisor, who notified security management.
Security management sent the
packet to local authorities for analysis. At about 3:30 p.m., authorities
- notified Salem management that the substance was cocaine and weighed
approximately 1/2 gram.
Subseque~tly, Salem m~nagement initiated a drug
sweep using a drug dog.
The sweep included Salem locker rooms, rest rooms,
the contractor change house, and similar buildings at the Hope Creek
facility.
The search did not uncover any additional examples of illegal
- substances. Salem management also tested the.thirty-six people that use the
locker room area for drug use.
No individµals tested positive.
At about 5:00 p.m. on April 12, *a temporary RP technician 'admitted that he
had planted the packet as a joke and that the packet contained coffee
creamer.
Because management had already for-cause tested the technician,
they elected not to retest him.
Salem management interviewed the individual
and concluded that his story was credible. They also sent the substance to
the State police crime lab for further testing to resolve what the substance
was.
On April 30, the Lower Alloways County Chief of Police reported that
the contents of the packet tested negative for illicit substances~
confirmi.ng the technician's story .. Salem management took appropriate
disciplinary action.
c. Conclusions
Salem management responded aggressively and quickly to the report of
discovery of an apparently illegal substance in the protected area. Their
actions were comprehensive and notifications were timely.
- By including a.
search of Hope Creek facilities, management showed sensitivity to the
generic implications of discovering an apparently illegal substance in the
protected area. The inspectors concluded Salem management responded
appropriately.
Laboratory tes.ts later concluded the. substance was a
powdered coffee creamer.
V. Management Meetings
Xl
Exit Meeting Summary
The inspectors presented the overall inspection results to members of licensee
management at the conclusion of the inspection on June 6, 1996.
The licensee
acknowledged the findings presented. Additionally, specialist inspectors
presented their findings on ~arch 22, April 19 and 23, 1996.
The inspectors asked the licensee whether any materials examined.during the
inspection should be considered proprietary .. No proprietary information was
identified.
I
X3
41
Management Meeting Sununary
On May 6 and 7, Mr. James Taylor, Executive Director for Operations, Mr.
Frank Miraglia, Deputy Director, NRR, and Mr. Thomas Martin, Administrator,
NRC Region IJ visited Salem to followup a meeting with the PSE&G Board of
Directors held during 1995 .
INSPECTION PROCEDURES USED
IP 37551:
Onsite Engineering
IP 60705:
Preparation for Refueling.
IP 61726:
Surveillance Observations
IP 62703~ * Maintenance Observations
IP 71707:
Plant Operations
ITEMS OPENED, CLOSED, AND DISCUSSED
Opened
50-272&311/96-06-01
Procedure non-compliance
50-272&311/96-06-02
UFSAR and licensing basis
nonconformances
50-272&311/96-06-03
Instrument loop inaccuracy
50-272&311/96-06-04
Hagon design change
50-272&311/96-06~05
Mods to feed control
Closed
50-272&311/94-18-01
Inadequate 125VDC acceptance
criteria
50-272&311/93-82-06
UPS output harmonica distortion
50-272&311/93-82-08
Safety related storage tanks volume
calculation
50-272&311/96-01-02
Radio frequency interference
50-272&311/96-01-01
Temperature of modules
Discussed
50-272&311/95021-02
IFI
Service water reliability issues
50-272&311/95029-00
LER
GE SBM control switch degradation
50-272&311/93-82-07
DEV
Onsite fuel oil requirements
50-272&311/94-18-02
EOG load fluctutttions
'
'
I
CA & QA
CAG
ccw
CCWHX
CR
ECACs
.EOG
FHAV
FHB
FPI
LER
MMIS
NAP
NCO
NBU
NRC
OHA
OSR
PSE&G
RC
SAC
SF P-
SNSS
sws
LIST OF ACRONYMS USED
Corrective Action and Quality Services
Corrective Action Group
Corrective Action Plan
Corrective Action Review Board
Component Cooling Water
Component Cooling Water Heat Exchanger
Condition Report
Emergency Control Air Compressors
c:;>
Engineered Safety Feature Actuation System
Emergency Response Organization
Fuel Handling Area Ventilation
Fuel Handling Building
Final Safety Analysis Report
Failure Prevention, Inc.
Jacket Water
Licensee Event Report
Maintenance Management Information System
Motor Driven Auxiliary Feedwater Pump
- Management Review Committee
Nuclear Administrative Manual
Nuclear Controls Operator
Nuclear Business Unit
Nuclear Regulatory Commission
Nuclear Safety Review
Nuclear Shift Supervisor
Overhead Annunciators
Operational Safety Review
Public Document Room
Public Service Electric and Gas
Quality Assurance
Restart Action Plan
Root Cause
Root Cause Manual
Radiation Protection
Station Air Compressor
Spent Fuel Pit
Safety Evaluation Report
Safety Injection
Senior Nuclear Shift supervisor
Service Water System
Updated Final Safety Analysis Report
I
DOCKET/REPORT NOS.:
LICENSEE:
FACILITY:
LOCATION:
DATES:
INSPECTOR:
APPROVED BY:
U. S. NUCLEAR REGULATORY COMMISSION
REGION I
50-272/96-06 AND 50-311/96-06
Public Service Electric and Gas Company
Salem Nuclear Generating Station
Units 1 and 2
Hancocks Bridge, NJ
March 18-22, 1996
A .. JJJY~l~
---"--/
A. Dell a Greca, Sr. Reactor Engineer
Electrical Engineering Branch
Division of Reactor Safety
[,/~tJ.;UJ
William H. Ruland, Chief*
Electrical Engineering Branch
Division of Reactor Safety
vf-c/l~t
Date
AREAS INSPECTED:
Announced.safety inspection to review ~nd evaluate PSE&G's
actions to address several unresolved issues resulting .from the electrical
distribution system functional inspection of the Salem plant and other
electrical system inspections.
-1
ENGINEERING REPORT DETAILS
50-272/96-06 AND 50-311/96-06
1.0
INSPECTION PURPOSE AND SCOPE
The purpose of the inspection was to review and evaluate PSE&G's actions to
resolve several issues identified previously by the NRC during the
electrical distribution system functional inspection (EDSFI).
Closure of
EDSFI-related issues is one of the items identified by the NRC as requiring
completion prior to the restart of Salem, Units 1 and 2.
The inspector's review, performed in accordance with the guidance of
Inspection Procedure 92903 and Temporary Instruction 2515/111, addressed the
licensee evaluation of each issue, the resulting corrective actions, and the
adequacy of these actions. Five previously identified issues were reviewed,
as detailed in Section 2.0 of this report.
2.0
REVIEW OF PREVIOUSLY IDENTIFIED ISSUES
2.1
(Updated) Deviation 50-272 and 50-31l/93-82-07: Onsite Fuel Oil
Requirement for Seven-Day Emergency Diesel Generator Operation
a.
Inspection Scope
The EDSFI team identified a discrepancy between the statement in Section
9!5.4 of the UFSAR and the available fuel in the diesel fuel oil storage
tank (DFOST).
The UFSAR stated that each 30,000 gallon DFOST could supply
one diesel with enough oil for seven-day operation.
In reality, the
licensee's calculation addressing this issue took credit for fuel available
in a nonsafety-related storage tank, capable of containing 20,000 barrels of
oil, for this commitment.
The licensee's resolution of this issue was reviewed previously
(IR 50-272/94-33 and 50-311/94-33), but it was left open pending the
licensee's resolution of two discrepancies identified by the NRC in the
supporting calculation and the acceptance by the NRC of a license amendment
submitted to the NRC on June 29, 1994.
The two discrepancies pertained to the protrusion of the suction pipe in the
DFOST and the licensee's basis for their use of the EDG-2C consumption rate
in the calculation of record.
The inspectors concern with the first issue
was that, because the tank design had specified only the minimum protrusion*
(1/2 inch), the licensee's assumption (2 inches) might not be sufficiently
conservative. Regarding the second. issue, the inspector understood that
fuel
o~l consumption measurements had been recently taken and their use was
more appropriate.
The purpose of this inspection was to revi~w the licensee's resolution of
the two calculation discrepancies and the status of the license amendment .
2
b *
Observations and Findings
. During the current inspection, a review of Calculation No. S-C-DF-MDC-1316,
Revision 1, determined that neither observation had been addressed.
Discuss ions with licensee engineering, however, determined that: .
1)
They had evaluated the measured consumption rates and concluded that
their bases for using the consumption rate of EDG-2C were appropriate
because:
EDG-28 and EDG-2C were equipped with turbochargers of a newer
design and had higher fuel consumption characteristics than the other
four engines; EDG-2C was more heavily loaded than EDG-28; and EDG-2C
always displayed a higher fuel consumption than the other engines.
2)
A license amendment that increased the minimum required fuel oil in
DFOST from 20,000 to 23,000 gallons per tank had been reviewed and
accepted by the NRC in their letter, dated June 20, 1995.
3)
At the time of the technical specification (TS) amendment request, the
licensee also submitted a FSAR change request. This change that served
as the basis for the TS amendment, specified that the TS minimum fuel
oil volume would be sufficient to operate two diesels for 4-1/2 days.
Operation beyond this time would require replenishment of the DFOSTs
from the onsite 20,000 barrel storage tank or from offsite sources.
4)
The suction pipe protrusion into the tank was not verifiable without
either emptying the tanks or using x-rays.
Because of the TS and FSAR
changes, however, and of the conservative assumptions used in
calculating diesel operating time, a more conservative value for pipe
protrusion would hav~ changed only minimally the estimated time when
more fuel oil was needed.
The inspector was satisfied with the licensee's justification.
Therefore, he evaluated their ability to secure oil within the specified
time and to transfer it to the DFOSTs.
The inspector determined that
the licensee had blanket orders with two fuel oil suppliers and the
ability to obtain oil by land or water within a very short time, if the
onsite 20,000 barrel tank was not available.
In addition, the licensee
had onsite fuel test capabilities.
This review of applicable documents and a walkdown of the fuel oil storage
and transfer systems also identified several discrepancies as follows:
The fuel oil emergency connections* had two different connection
fittings, neither of which was directly usable with hose connectors
typically found on tanker trucks.
A portion of the emergency connection pipe between the ground and the
only external support had not been seismically analyzed .
- -
3
The use of ~he oil from the 20,000 barrel storage tank relied on the
availability of hoses and pumps.
No hoses or pumps had been dedicated
for this service and no instructions were available on how to obtain
such equipment.
The emergency connections had never been tested to ensure their
availability following an accident.
Following the inspection, on May 20, 1996, the inspector questioned again
the pipe protrusion in the DFOST and licensee's ability to demonstrate the
availability of 23,000 gallons of fuel oil in each of the tanks.
He
determined that:
The licensee had contacted the vendor and had been informed that the
assumed two-inches protrusion was conservative. The specified 1/2-inch
minimum protrusion note was standard practice for fillet weld
inspection.
The li~ensee, nonetheless, issued an action request to
verify the pipe protrusion.
Following the approval of the TS amendment, the licensee had reduced the
administrative limit for minimum tank fill from 97 inches, corresponding
- to approximately 26,000 gallons (when a 5-inch gauge error is
considered) to 85 inches, corresponding to 23,005 gallons when the same
error is considered.
The change was apparently made to prevent
overfilling of the tank.
The surveillance procedure change did not undergo technical review
because it incorporated an approved TS and FSAR revision.
In addition,
a safety evaluation according to 10 CFR 50.59 was not performed for the
same reason.
- . The 5-inch gauge error stated in the storage tank volume calculation was
calculated (calculation No. SC-DGOl0-01) assuming a maximum fuel oil
specific gravity of 0.86. This number was based on the licensee review
of the fuel oil analysis history file.
An older version of the TS surveillance requirements (SR 3.8.3.4.b)
specified verification that the fuel oil absolute specific gravity was
greater than or equal to 0.83 and less than or equal to 0.89, in
accordance with ASTM standard 0975~ If 0.89 had been used, the meter
error could be approximately 8.7 inches, rendering the fuel oil lower
limit (85 inches) specified in the surveillance procedure too low.
The
current TS requirements (4.8.1.1.3.b) do not mention specific gravity,
butcontinue to refer to the ASTM standard for verification of viscosity
and water sediment.
A purchase specification was not immediately available to verify
specific gravity requirements .
-*~
- t
I
c.
4
Conclusions
The inspectrir concluded that sufficierit justifications were available to
support the license amendment regarding minimum volume requirement in the
DFOST and the proposed UFSAR changes regarding onsite fuel oil availability
for continuous EOG operation ( 4. 5 days) and other fue 1 oil sources. The
licensee's ability, however, to verify the quantity of fuel oil in the
storage tanks, is unknown pending measurement of the pipe protrusion and
licensee applicability review of the higher specific gravity value.
The
inspector also concluded that the ability to replenish the tanks for
emergency operation of the EDGs beyond the calculated 4.5 days had not bee_n
fully_ evaluated. This item remains open pending the NRC review of the
licensee actions to address the identified discrepancies.
2.2
(Updated) Unresolved Item 50-272/94-18-02:
Load Fluctuations Root Cause
a.
b.
- Inspection Scope
On August 16, 1994, during the monthly surveillance of EOG IA, the operator
noticed load fluctuations of approximately 100 kW (typical) from the target
value of 2600 kW.
In the follow-up inspection, Reports 50-272, 50-311, and
50-354/94:...18; the NRC evaluated the subsequent troubleshooting performed by
PSE&G and considered it adequate.
The issue was unresolved pending
completion of the licensee's root_ cause analysis and review by the NRC .
The purpose of this inspection was to determine the status of the corrective
actions initiated by the licensee during the inspection and to evaluate the
results of their root cause analysis.
Observations and Findings
The licensee's troubleshooting to address the event identified a number of
malfunctions, all of which, they believed, were responsible for the load
- control problem of the diesel. Therefore, the -licensee took a number of
actioris, including: replacement Qf the electrical governor controller -
(EGA); lubrication of the fuel rack (found to be stickirig)~ and replacement
of the mechanical governor controller (EGB).
Following the event, the licensee sent the EGB to the governor manufacturer.
In their analysis report, the manufacturer stated that they had found two
problems:
1) the "electrical P.V. plunger had a slight ring (worn area) on
-the center of control land"; and 2) "it was missing a gasket on the bushing
retainer for the drive shaft."
The manufacturer also stated that, although Problem 1 could cause the
oscillation problem, they could not duplicate the response during their test
of the unit.
Problem 2 could only cause a leak from around the drive shaft.
Neither problem would cause the shifting of null voltage noted by the
licensee when the unit was on the diesel. They faxed their conclusions to
PSE&G with a message that stated, "the oscillation of terminal shaft could
cause load fluctuations ... "
,.,
.f
c.
5
The subsequent root cause analysis by the licensee, that included change,
barrier, and causal-factor analyses, echoed the vendor conclusions and
attributed the oscillation to the worn area found on the P.V. plunger.
The inspector reviewed the root cause analysis and observed that the
licensee had not evaluated the impact of the EGA replacement, the fuel rack
lubrication, and other troubleshooting activities on the analysis results.
Also, they had not discussed the inability by the vendor to reproduce in the
laboratory the symptoms observed during the oscillations.
He discussed
these observations with the licensee who agreed that the analysis could have
addressed the other issues, but reaffirmed their convic;tion regarding the
EGB being the source of the oscillations. The licensee also believed that
it was not necessary to inspect or replace the EGBs of the other five
diesels because they had observed no abnormal symptoms in the performance of
the other diesels.
During the original inspection, the licensee had initiated actions to
enhance the EOG maintenance procedure.
The current inspection evaluated the
status of these actions and found that seven procedures addressing
lubrication, corrective and preventive maintenance, and inspection of the
diesel engine had been revised to address lubrication and maintenance of the
fuel rack and fuel pump adjustments.
While discussing the EOG IA event with licensee engineering personnel, the
inspector determined that on March I3, I996, EOG 2A had experienced load
oscillations. The event occurred while the licensee was returning EOG 2A
from an I8-month scheduled outage test. The licensee stated that the
oscillations were related to a governor oil change performed without proper
venting.
The licensee had corrected the problem and initiated a level I
root cause analysis of this new issue. lhe licensee also stated that the
new root cause analysis would reevaluate the EOG-IA load oscillations.
Conclusions
The inspector concluded that the licensee had taken acceptable actions to
correct the EOG IA malfunctions and to improve lubrication and maintenance
of applicable engine components.
The root cause analysis, however, should
have also evaluated the impact of the troubleshooting activities on the
analysis results.
The inspector deferred his conclusions regarding the cause of the EOG 1A
load oscillations until the oscillations recently observed in the EOG 2A
load control are evaluated and the new root cause analysis is completed.
This issue remains open pending the licensee's completion of the analysis
and the NRC's review of its results and actions.
2.3
(Closed) Violation 50-272 and 50-3II/94-I8-0I: Inadequate I25Vdc
Battery Test Acceptance Criteria
In August I994, during the subject inspection, the NRC found that the
acceptance criteria for the I25 Vdc station batteries service test
procedure, No. SC.MD-ST.I25-0004(Q), did not reflect the result of the
6
125Vdc system study, Calculation No. ES-4.003(Q). Although PSE&G had
already developed a procedure change request to address the discrepancy
before the next 18-month test, the NRC determined that the licensee's
failure to correct the discrepancy previously was a violation of 10 CFR,
Appendix B, Criterion XI requirements.
The same issue had been previously
identified, in early 1992, at Hope Creek.
The licensee did not dispute the violation, as stated for the Salem Station,
and attributed it to inadequate communication within the organization and
their failure to implement a formal requirement to identify procedures
impacted by calculations.
In their response to the NRC, Letter NLR-N94179
dated October 13, 1994, PSE&G proposed several actions to prevent
recurrence.
During the current inspection, the NRC inspector evaluated the adequacy and
status of the PSE&G's proposed actions.
He determined that the licensee had
completed all the actions as described in their letter to NRC to prevent
recurrence.
Because the violation was partly the result of inadequate communications
between the two plants, the inspector questioned the status of two action
items selected at random from a list developed for Hope Creek during a
meeting between the two engineering organizations. One of these issues
pertained to the EOG test procedure that permitted the EOG output voltage to
be below the degraded voltage setting; the other pertained to the need for
verifying the Uninterruptible Power Supply (UPS) output voltage total
harmonic content. *The inspector found that no actions had been taken by
Hope Creek engineering to close either issue and no tracking mechanism was
available to ensure their timely resolution and closure.
By the end of the
inspection, the licensee provided new action items and a schedule for
completing the actions.
The i-nspector concluded that actions taken by Salem engineering to address
the above violation were appropriate and sufficient and the issue is closed.
- The inspector also concluded that communications between plants continued to
be less-than-effective in that a meeting to evaluate potential issues
affecting the plants had taken place, but Hope Creek had failed to follow up
on the results of this meeting~
Communications between the two plants is part of the NRC restart action plan
for Salem. Actions taken by the licensee to improve communications and the
results of these actions will be evaluated during the inspection of this
area prior to the Salem plant restart.
2.4
(Closed} Unresolved Item 50-272 and 50-311/93-82-06: Uninterruptible
Power Supply Output Harmonic Distortion
a.
Inspection Scope
The purchase specification for the 115 Vac vital instrument bus
uninterruptible power supplies (UPS) required that the total output voltage
harmonic content did not exceed 5% of the fundamental voltage.
The licensee
b.
7
had never verified this attribute of the UPS, but agreed that they should
verify it. The purpose of this inspection was to determine the method and
results of the licensee's verification tests and the acceptability of these
results.
Observation and Findings
The inspector's review of the actions taken by the licensee to verify the
.UPS output voltage total harmonic distortion {THD} determined that:
The licensee had revised the inverter preventive maintenance {PM}
procedure {SC.MD-PM.115-000l{Q}, Rev. 4} to include steps to verify the
output voltage THD each time the inverter underwent PM.
Steps 5.3.56.
through .63 and .68 through .71 of this procedure require the use of a
harmonic distortion analyzer for the voltage THD verification. The
procedure also required adjustment of the inverter and ac line regulator
to less than 5% THD and notification of responsible engineering and
operation personnel if the adjustment could not be made.
Measurements taken in 1994 showed that the output voltage THD ranged
from 4.58% to 4.78%.
An evaluation had been made of the impact of their planned change of 75
Hagan modules to equivalent modules furnished by NUS.
This evaluation
had determined that the proposed changes would result in a THD increase
of only 0.15%.
Plans also had been made to verify the results of their
analysis during the Hagan modules change process.
The licensee was evaluating the UPS loads to determine the maximum
acceptable THD percentage.
c.
Conclusions
Based on his review of the measurements taken during the 1994 PM of the
inverters and the licensee's calculation of the THD change due to the
planned Hagan module changes, the inspector concluded that the licensee had
acceptably verified that the UPS output voltage did not exceed the specified
5% THD.
This item is closed.
2.5
(Closed) Unresolved Item 50-272 and 50-311/93-82-08: Safety-Related
Storage Tanks Volume Calculation
a.
Inspection Scope
While reviewing Calculation No. S-C-VAR-CDC-0095, the EDSFI team determined
that the calculation failed to account for unusable volume {e.g., vortex}
and.level instrument error in the determination of available fluids.
The
calculation applied to 28 tanks for both units.
The purpose of this inspection was to review the action taken by the
licensee to revise the calculation and evaluate the calculation results.
b.
8
Observation and Findings
The inspector's review of the licensee's corrective actions to address the
above finding determined that:
The licensee had revised Calculation No. S-C-VAR-CDC-0095 to delete all
calculations pertaining to safety-related tanks and had issued a new
calculation, No. s~C-VAR-MDC-1429, to address the same tanks.
The new calculation had addressed all the EOSFI team observations,
including vortex and loop accuracy.
The calculation was very detailed in the assumptions and methods used
for the determination of available volumes.
All volumes calculated were
evaluated for impact on technical specification or other requirements.
The revised calculation had only minimal impact on previously calculated
volumes.
In no case was a technical specification requirement affected
by the revised volumes.
c.
Conclusions
The licensee had properly ~ddressed the EOSFI team observations. The
clarity and quality of methods used in the calculation were good.
The
impact of the calculation revised results were pro~erly evaluated. This
item is closed.
3.0
MANAGEMENT OVERSIGHT AND GENERAL CONCLUSIONS
The inspector's review of the five previously identified issues addressed by
this report indicated acceptable resolution of most issues.
The
calculations were us~ally detailed and thorough and the evaluations, in
general, indicated good understanding of the issues. The evaluation of two
issues, however, was.less than thorough.
As a result, insufficient bases
were available for their closure. For instance, acceptable background
documentation had been developed to justify the reduction of onsite fuel
requirement from seven to four and a half days.
The evaluation, however,
had failed to ensure the availability of the emergency connection to the
DFOSTs.
Similarly, a good effort was evident in the troubleshooting of the
EOG load fluctuation.
The root cause analysis, however, had failed to
evaluate the impact of the troubleshooting activities on the analysis
results. Therefore, the acceptability of these results was unknown.
Management involvement in the resolution of the issues was evident in their
knowledge of the issues themselves and in their direct follow-up of more
critical issues. Their involvement, however, was less than effective in the
determination of the root cause analysis for the EOG load fluctuations and
in the emergency connection to the OFOST issues .
.....-------
_,
.
-
9
4.0
REVIEW OF UFSAR COMMITMENTS
A recent discovery of a licensee operating their facility in a manner
contrary to the Updated Final Safety Analysis Report (UFSAR) description
highlighted the need for a special focused review that compares plant
practices, procedures and/or parameters to the UFSAR descriptions. While
performing the inspections discussed in this report, the inspector reviewed
the applicable portions of the UFSAR that related to the areas inspected.
The inspector verified that the UFSAR wording was consistent with the
observed plant practices, procedures and/or parameters.
5.0
MANAGEMENT MEETINGS
The inspector presented the *inspection results to members of licensee
management at the conclusion of the inspection on March 22, 1996.
The
licensee acknowledged the findings presented~
_The inspector also asked the licensee whether any materials examined during
the inspection -should be considered proprietary.
No proprietary information
was identified.
6.0
PARTIAL LIST OF PERSONS CONTACTED
Public Service Electric and Gas Company
C. Manges
M. S. Bursztein
G. J. Overbeck
E. H. Villar
C. Warren
C. Bakker
M. McGough
L. Storz
D. Garver
D. Tauber
D. Garchow
Licensing Engineer
Nuclear Electrical Engineering Manager
Director, System Engineering
Licensing Engineer
General Manager, Salem Station
Operations, Salem Station
Design Engineering and Projects
Senior Vice President, Nuclear Operations
Salem System Engineering
Manager, Quality Assurance, Salem
Director, Engineering S/G Project
U. S. Nuclear Regulatory Commission
W. Ruland
Chief, Electrical Engineering Branch, DRS