IR 05000354/1998002
| ML20217H813 | |
| Person / Time | |
|---|---|
| Site: | Hope Creek  |
| Issue date: | 04/23/1998 |
| From: | NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML20217H804 | List: |
| References | |
| 50-354-98-02, 50-354-98-2, NUDOCS 9804300095 | |
| Download: ML20217H813 (35) | |
Text
I l
.
.
'
U. S. NUCLEAR REGULATORY COMMISSION i
REGION 1 Docket No:
50-354 License Nos:
NPF-57 Report No.
50-354/98-02 Licensee:
Public Service Electric and Gas Company J
Facility:
Hope Creek Nuclear Generating Station Location:
P.O. Box 236 Hancocks Bridge, New Jersey 08038 Dates:
February 22,1998 - April 4,1998
,
i Inspectors:
S. A. Morris, Senior Resident inspector S. M. Pindale, Senior Resident inspector J. D. Orr, Resident inspector R. L. Fuhrmeister, Reactor Engineer
K. Young, Reactor Engineer S. Barber, Project Engineer
,
Approved by:
James C. Linville, Chief, Projects Branch 3 Division of Reactor Projects
l l
!
'
i 9004300095 980423
'
PDR ADOCK 05000354 G
.
,
EXECUTIVE SUMMARY Hope Creek Generating Station NRC Inspection Report 50-354/98-02 This integrated inspection included aspects of licensee' operations, engineering, maintenance, and plant support. The report covers a six-week period of resident inspection in addition, it includes the results of announced inspections by four regional inspectors, two who evaluated the effectiveness of the fire protection program; and two who reviewed the maintenance program implementation.
Ooerations After a single control rod drive was mispositioned one notch, Hope Creek reactor operators assessed core thermal performance. The operators recognized the significance of reactivity management and they promptly reported the problem to the Hope Creek operations manager. The inspectors determined that the Hope Creek operators were conservative and safe during reactivity manipulations. (Section 01.1)
The inspectors toured the service water intake structure (SWIS) on a frequent basis and noticed three different examples of temporary equipment not properly controlled.
Specifically, (1) a SWIS floor drain system was modified before the required 10 CFR 50.59 safety evaluation was completed, (2) scaffold was erected without the administrative controls and inspections required by Hope Creek's scaffold program, and (3) floor drain plugs wera installed in both service water bays without following the requirements of Hope Creek's temporary modification procedure. The inspectors concluded that, although each problem considered alone was minor and in no instance challenged the operability of the safety-related equipment in the SWIS, the number of discrepancies suggests that PSE&G needs an increased awareness of SWIS material condition and equipment control. (Section 02.1)
Control room operators acted deliberately and cautiously during the conduct of a reactor power reduction from 100% to 60%, as well as during a subsequent control rod pattern adjustment. When abnormalindications and response were indicated with one of the control rods during the pattern adjustment, operators promptly and properly implemented off normal procedure guidance and consulted with cognizant reactor engineering personnel.
Good crew briefings were held to discuss individual observations and establish subsequent plans. The operations superintendent implemented appropriate actions upon declaring the associated control rod inoperable. (Section 04.1)
Operations, maintenance and engineering personnel effectively investigated an abnormal response of a torus-to-drywell vacuum breaker during testing, and the appropriate actions were taken to correct the situation. However, some minor weaknesses were apparent related to test conduct and follow-up by control room operators. (Section 04.2)
A Nuclear Review Board meeting was characterized by probing discussions and reflected a strong safety focus. The overall quality of the meeting was very good. (Section 07.1)
ii
.
.
Maintenance PSE&G's execution of the work week planning and scheduling process generally improved during the period, however several examples of poor implementation of work process requirements were identified. Work week critiques identified and documented deficiencies as lessons learned for future improvement. (Section M1.1)
Hope Creek maintenance personnel thoroughly prepared a troubleshooting procedure to locally withdraw a stuck traversing incore probe. The development of this plan included sufficient precautions from recent industry lessons learned to prevent any possibility of radiation overexposure. The Instrumentation and Controls maintenance technicians completed the troubleshooting procedure in a timely and error free manner. (Section M1.2)
Proventive maintenance activities conducted on the "B" and "D" service water trains were properly controlled in accordance with PSE&G's preventative maintenance procedures.
Corrective actions taken in the interim between the "B" and "D" service water outages ensured that some minor problems did not reoccur. (Section M1.3)
The on-line maintenance program was effective at balancing the benefits gained through preventive maintenance program against the costs of equipment unavailability. The licensee quantified the risk of component and system unavailability in the 12 week rolling schedule and limited preventive maintenance activities to manage these risks. A minor inconsistency was noted with evaluation of risk when maintenance was either deferred or added to a given work week. (Section M1.4)
The Work it Now team was effective at screening new work entering the system. The team has reduced existing corrective maintenance backlogs while maintaining a reasonable work-off rate for new work. (Section M1.4)
A wording ambiguity in a primary containment instrument gas compressor maintenance procedure could have led to an expansion of work scope not specifically identified by the maintenance procedure. This deficiency was corrected prior to beginning work. However, an extent of condition review noted other instances of poorly worded preventive maintenance procedures. By not writing action requests in these cases, the accuracy of information used for trending purposes could be susoect. (Section M1.4)
Root cause analyses were acceptable with one excepton where certain aspects of one analysis was not sufficiently probing and did not identify all of the root causes for the event. (Section M1,4)
PSE&G maintenance technicians methodically identified and corrected the source of unrelated electrical grounds that developed on the "A" and "B" diesel generators'
associated 125Vdc busses. The technician promptness was commensurate with the safety importance of the diesel generators. PSE&G engineers initiated a followup analysis l
to ensure that the "A" EDG speed switch failure was well understood. (Section M2.1)
l PSE&G management performed a thorough review into the circumstances surrounding a failed post-maintenance surveillance test of a filtration, recirculation, and ventilation unit.
iii i
l
l
.
.
i This investigation determined that inattention-to-detail by maintenance technicians and poor decision making by a supervisor led to the test failure. (Section M4.1)
i Enaineerino PSE&G engineers thoroughly evaluated the "B" residual heat removal pump minimum flow check valve failure. The engineers contacted the vendor, determined the failure mode, and performed radiographs of an expanded check valve population to support a conclusion that i
the failure was an isolated instance. The engineers completed a comprehensive l
investigation of the problem to determine the extent of condition. The operability i
determination and the engineers' followup assessment was thoroughly documented.
(Section E2.1)
PSE&G engineers promptly evaluated a 10 CFR Part 21 notification made by a relay vendor. Equipment repairs were completed in a timely fashion on those relays that potentially impacted plant operations. (Section E2.2)
l Hope Creek engineers evaluated inconsistent inservice Test data from several "back to back" surveillance tests performed on the "A" service water pump. The engineers'
detailed analysis of the flow measuring device, test methodology, and current plant conditions discovered test induced errors and prevented unnecessary intrusive inspection
'
of the "A" service water pump. (Section E2.3)
l Plant Succort The various departments cautiously considered the potential for radiation exposure while planning a recovery attempt for a stuck traversing incore probe. (Section M1.2)
PSE&G had good administrative controls for proper storage of combustibles and control of hot-work activities. (Section F1.1)
l Fire protection equipment conditions and housekeeping were good. Continuous fire watches were knowledgeable of station procedures for reporting fires, fire watch duties, l
~and responding to fires. Eight hour emergency light operation and illumination patterns j
j were good. (Section F2.1)
Penetration seals were in good condition and the "as-built" condition met the test criteria outlined in the vendor's test report for operational performance. (Section F2.2)
The Hope Creek and Salem fire pumps were well-maintained and ready for service. The t
l fire main loop was in good repair, and capable of providing the necessary water supply for j
fire fighting needs at the facility. (Sections F2.3 and F2.4)
Fire protection procedures met the requirements for fire protection program implementation, contained sufficient detail, and were technically sound. (Section F3.1)
Performance by the fire brigade team during a fire drill was very good. All expectations of the fire drill were met. (Section F4.1)
iv i
l I
~
.
<.
Selected fire brigade members were current on all required training and annual physicals.
Training provided to the fire brigade members was comprehensive, well organized, and complete. (Section F5.1)
Quality assurance audits focused appropriately on and verified selected fire program attributes for compliance with fire protection program requirements.
Audit findings were appropriately assessed and timely corrective actions were taken for identified deficiencies.
(Section F7.1)
l l
!
,
t l
V
.
,.
TABLE OF CONTENTS E X E C U TIV E S UM M A RY.............................................. ii TA B L E O F C O NTE NTS.............................................. vi 1. O pe r a ti o n s..................................................... 1
Conduct of Operations.................................... 1 01.1 Mispositioned Control Rod Drive During Weekly Exercising...... 1
Operational Status of Facilities and Equipment...................2 02.1 Service Water intake Structure Temporary Equipment Control.... 2
Operator Knowledge and Performance......................... 4 04.1 Observation of Control Rod Pattern Adjustment Evolution....... 4 04.2 Unexpected Valve Response During Surveillance............. 5
Quality Assurance in Operations............................. 6 07.1 Nuclear Review Board Meeting......................... 6 ll. M ainte nan ce................................................... 7 M1 Conduct o f M aintenance................................... 7 M 1.1 General Observations of Work Week Planning and Scheduling... 7 M1.2
"E" Traversing incore Probe (TIP) Stuck During Core Insertion.... 8 M1.3 Service Water System Preventive Maintenance.............. 9 M1.4 Hope Creek Maintenance Focus inspection................ 10 M2 Maintenance and Material Condition of Facilities and Equipment...... 14 M 2.1
"A" and "B" Emergency Diesel Generator 125Vdc Ground Isolation
..............................................14 M4 Maintenance Staff Knowledge and Performance................. 16 M4.1
"C" Filtration, Recirculation, and Ventilation System Maintenance and Te sti n g......................................... 1 6 M8 Miscellaneous Maintenance issues........................... 17 M 8.1 (Closed) LER 5 0-3 5 4/9 8-01........................... 17 l
I l l. Eng ine e ring................................................... 17 i
E2 Engineering Support of Facilities and Equipment................. 17 l
E2.1
"B" Residual Heat Removal (RHR) Minimum Flow Check Valve Failure
!
..............................................17 E2.2 Agastat and ETR Part 21 Notification Response............ 18 E2.3
"A" Service Water Pump Inservice Testing................ 19
!
IV. Pl a n t S u p p o rt................................................. 2 0 F1 Control of Fire Protection Activities.......................... 20 l
F1.1 Fire Risk Evolutions................................ 20 l
F2 Status of Fire Protection Facilities and Equipment................ 21 l
F2.1 Facility Tour
.....................................21 F2.2 Fire Barrier Penetration Seals.......................... 22 F2.3 Fire Main Loop Flow Testing.......................... 23 F2.4 Fire Pump Testing
.................................23 F3 Fire Protisction Procedures and Documentation.................. 24 vi
.
..
F3.1 Fire Protection Procedure Review....................... 24 F4 Fire Protection Staff Knowledge and Performance................ 25 F4.1 Fire Brig ade Drills.................................. 2 5 F5 Fire Protection Staff Training and Qualification.................. 26 F5.1 Fire Brig ade Training................................ 2 6 F7-Quality Assurance in Fire Protection Activities
..................27 F7.1 Audits and Surveillances............................. 27 V. M anage ment Meeting s........................................... 2 7 X1 Exit Meeting Sum mary................................... 27 X3 Site Visit Summary
.....................................28
.
i
,
vii i
.
.
Reoort Details Summarv of Plant Status Hope Creek was operated at or near full power for the duration of this inspection period.
l. Operations
Conduct of Operations M Mispositioned Control Rod Drive Durina Weekiv Exercisina -
a.
Insoection Scone (71707)
The inspectors reviewed the operator sctions taken following a control rod drive
~
(CRD) mispositioning. The inspectors also reviewed PSE&G's long term corrective actions for this human error.
b.
Observations and Findinas On March 14,1998, during weekly (CRD) exercising, CRD 14-35 was mispositioned a single notch in the wrong direction. The reactor operator intended to withdraw l
the CRD, but instead inserted the CRD. Prior to operating the CRD at the rod select matrix, the reactor operator received a rod block monitor (RBM) alarm. The reactor l-operator and his peer checker performed a rod position verification at the process I
computer in response to the RBM alarm. The process computer is located at the
'
back of the control room, several feet away from the rod select matrix. The reactor
operator inserted CRD 14-35 one notch when he returned to the rod select matrix, i
The reactor operator did not allow the peer checker to verify his actions and CRD 14-35 was inserted rather than withdrawn one notch. This failure to position CRD 14 35 to its correct position is a violation of minor significance and is being treated as a Non-Cited Violation, consistent with Section IV of the NRC Enforcement Poliev.
(NCV 50-354/98-02-01)
)
i The control room operators received guidance from the reactor engineer and withdrew CRD 14-35 to its intended position. The operators verified that the inadvertent mispositioning had no effects on core thermal performance. Although this problem was minor, the operators on shift promptly recognized the significance of reactivity management and they reported the event to the Hope Creek operations manager. The inspectors determined Hope Creek's followup actions for this problem to be very conservative and safe.
c.
Conclusions After a single control rod drive was mispositioned one notch, Hope Creek reactor operators assessed core thermal performance. The operators recognized the significance of reactivity management and promptly reported the problem to the Hope Creek operations manager. The inspectors determined that the Hope Creek operators were conservative and safe during reactivity manipulation.
.
02 Oprational Status of Facilities and Equipment 02.1 Service Water intake Structure Temocrarv Eauioment Control l
a.
insoection Scoce (71707)
The inspectors toured the service water intake structure (SWIS) on a frequent basis during periods of maintenance on the "B" and "D" service water pumps and also during periods of low activity.
b.
Observations and Findinas Temoorarv Hose Not Evaluated On February 14,1998, the operations department installed a temporary discharge hose to route past a clogged combined discharge pipe for the "A" and "C" service water bay sump pumps. The temporary discharge hose was installed for a few days using an operations troubleshooting plan. PSE&G left the temporary discharge hose installed at the conclusion of the troubleshooting and turned the control of this modification over to the temporary modification process. The engineers developing the temporary modification package determined that a 10 CFR 50.59 safety evaluation was required. Installation of the temporary discharge hose removed a i
pipe section from the SWIS building and equipment drain system. The piping diagram for the SWIS building and equipment drain system was completely described in the Updated Final Safety Analysis Report (UFSAR). PSE&G engineers completed the safety evaluation and determined that no unreviewed safety questions existed. Although engineering subsequently performed a safety evaluation for the temporary modification, the inspectors identified that the original troubleshooting plan should not have been approved because operations troubleshooting procedure, HC.0F GP.ZZ-0008 (Q), " Operations Troubleshooting,"
states that troubleshooting plans requiring a 10 CFR 50.59 safety evaluation should be rejected. This is a violation of procedure HC.OP-GP.ZZ-0008. (VIO 50-354/98-02-02)
'
,
Administrative Controls For Scaffoldina On February 24,1998, the inspectors noticed that a scaffold was installed around the "B" service water strainer to support maintenance activities on the strainer. The scaffold did not have an identifying scaffold control tag. The inspectors questioned the maintenance manager about why the scaffold was not controlled in accordance with Hope Creek's scaffold control program. The maintenance manager agreed with the NRC inspectors that administrative controls should be established to ensure that the scaffold is installed and removed in the proper time so that pump operability would not be challenged. The maintenance manager did not believe that the strainer work platform needed to be included within the scope of the Hope Creek scaffold control program since it was a prefabricated structure and it did not require the skills of qualified scaffold builder.
.
On March 31,1998, PSE&G quality assurance (QA) inspectors noticed that the same platform was installed on the "D" service water strainer for its scheduled maintenance. The QA inspectors independently developed similar questions that the NRC inspectors had asked. The QA inspectors discovered again that no administrative controls were established to ensure the timely and proper scaffold installation and removal. The NRC in pectors determined that, although the strainer scaffold was a prefabricated structure, it was still within the scope of Hope Creek's scaffold control program and all scaffold program requirements should have been adhered to. This is a violation of administrative procedure NC.NA-AP.ZZ-0023(O),
" Scaffold Program." (VIO 50-354/98-02-03)
Administrative Controls For Floor Drain Pluas During the frequent tours of the service water intake structure, the inspectors noticed several plugs installed in the floor drains. The plugs appeared to have been installed for several months or longer based on their physical appearance. " Control of Temporary Modifications, (NC.NA AP.ZZ-0013(O))" provides detailed instructions for the installation of drain plugs in the Hope Creek service water intake structure, as well as other safety-related equipment areas. The drain plugs were not installed in accordance with " Control of Temporary Modifications." The service water system manager could not determine when the plugs were installed and why they were installed. The floor drain plugs were removed and the system manager wrote an action request to document the problem. The improper installation of the floor
,
drain plugs is a violation of administrative procedure NC.NA-AP.ZZ-0013(O). (VIO 50-354/98-02-04)
Summarv The inspectors determined that this problem, along with the other two problems described above, were three examples of inadequate procedure implementation for temporary equipment that could affect other safety-related equipment. These are examples of violations of 10 CFR 50 Appendix B, Criterion V (Instructions, Procedures, and Drawings).
c.
Conclusions
'
The inspectors toured the service water intake structure (SWIS) on a frequent basis and noticed three different examples of temporary equipment not properly controlled. Specifically, (1) a SWIS floor drain system was modified before the required 10 CFR 50.59 safety evaluation was completed, (2) scaffold was erected without the administrative controls and inspections required by Hope Creek's scaffold program, and (3) floor drain plugs were installed in both service water bays I
l without following the requirements of Hope Creek's temporary modification l
procedure. The inspectors concluded that, although each problem considered alone was minor and in no instance challenged the operability of the safety-related equipment in the SWIS, the number of discrepancies suggests that PSE&G needs an increased awareness of SWIS material condition and equipment contro r.-
L
.
f
.
04 Operator Knowledge and Performance 04.1 Observation of Control Rod Pattern Adiustment Evolution
,
a.
- Insoection Scone (71707)
l The inspectors observed plant operators conduct a planned deep / shallow control rod exchange evolution.
b.
Observations and Findinos
,
t l
On March 28,1998, operators reduced reactor power to approximately 60% in
!
order to perform a planned control rod pattern adjustment. The inspectors were in the control room and observed portions of both the power reduction and control rod manipulations to support the pattern adjustment. The inspectors noted that, during-the reactivity manipulations, operators maintained the control room free from l
distractions and dedicated one reactor operator and one senior reactor operator to
!
the activity. Additionally, a reactor engineer and a nuclear fuels engineer were l
present to direct and monitor the overall evolution.
!
About mid-way through the rod pattern adjustment, operators received a " control rod data fault" alarm while continuously pulling rod 30-31 from notch 12 to 48.
Position indication for this rod was lost. This condition was promptly recognized and reported to all cognizant control room personnel. Operators appropriately entered the governing alarm response and abnormal operating procedures (HC.OP-AR.ZZ-0011 and HC.OP-AB.ZZ-106, respectively). A crew brief was held to discuss observed conditions and indications just prior to, during, and after the unexpected occurrence. Operators believed the rod stopped at notch 46.
Equipment operators were promptly dispatched to the associated hydraulic control unit (HCU), the rod position indication system cabinet, and the reactor manual control system cabinet to record as-found conditions.
After consultation with the reactor engineers, operators attempted to insert the rod i
using single notch demands. After two attempts, rod position indication returned
- .
and the rod data fault cleared, with indication that rod 30-31 was at notch 42.
l Another insert demand was made and the rod indication changed to notch 40.
i.
Based on these indications, which included normal directional control valve i
l-operation at the HCU, operators and engineers believed there was a position indication problem with the drive mechanism position reed switches between j
notches 44 and 46. Operators then attempted single notch withdrawals from notch I
40, and again received the rod data fault alarm after the rod was moved from notch 42. A control rod block was indicated at that point. The operations superintendent i
then declared control rod 30-31 inoperable, and directed the rod be fully inserted into the core and electrically disarmed at the HCU in accordance with technical l
specification 3.1.3.1.b. During the control rod insertion to notch 00 (fully inserted),
operators believed the rod moved slower than normal from notch 42 to 36. An action request was generated which accurately captured the actions and indications associated with this event. Operators completed the remaining control rod 1.
l
-
!
l l
.
.
manipulations for the pattern adjustment without incident. Engineering initiated efforts to develop a troubleshooting action plan to evaluate the unexpected control rod response.
c.
Conclusions Control room operators acted deliberately and cautiously during the conduct of a reactor power reduction from 100% to 60%, as well as during a subsequent control rod pattern adjustment. When abnormalindications and response were indicated with one of the control rods during the pattern adjustment, operators promptly and properly implemented off-normal procedure guidance and consulted with cognizant reactor engineering personnel. Good crew briefings were held to discuss individual observations and establish subsequent plans. The operations superintendent implemented appropriate actions upon declaring the associated control rod inoperable.
04.2 Unexoected Valve Resnonse Durina Surveillance a.
Insoection Scope (61726. 71707)
t The inspectors reviewed the results of HC.OP-IS.GS-0101(O)," Containment Atmosphere Control Systs-
'dves - Inservice Test," and assessed the operators response to the test resultt a inspectors reviewed operator logs, the surveillance procedure, an o ciated Action Request (980331068),and interviewed operators, b.
Observations and Findinas On March 30,1998, during a suppression pool-to-drywell vacuum breaker monthly surveillance, one of the eight valves ("B") failed to indicate fully closed on both redundant position indicators after the valve was cycled open ard then closed. One of the two position indication switches indicated closed, while the other indicated mid-position. Operators had just successfu!!y cycled the "A" vacuum breaker, and after observing the conflicting indication, successfully cycled the next two valves ("C" and "D"). The operators then stopped the test to further evaluate the valve j
position.
l The operators monitored the suppression pool and drywell pressures to determine
whether the valve was actually open or whether the indication was a valve limit switch malfunction. The operators also consulted Technical Specification 3.6.4.1
(entered the Action requirement), and requested support from engineering. Using normal operating procedures, the operators started nitrogen makeup to the drywell in order to establish a differential pressure between the torus and the drywell. The operators established about a 0.27 psi differentialin about 40 minutes. As a
!
conservative measure, and in the absence of information indicating a limit switch malfunction, the operators entered a twelve-hour shutdown action as per Technical Specification 3.6.4.1.b two hours after the initial indication. Within the next several i
minutes, the operators again cycled the "B" vacuum breaker, and the valve
.
.
indicated fully closed on both closed position indicators. The shutdown Action requirement was then exited. The test was placed on hold for further evaluation of the observed indications.
After continued evaluation, procedure review, and contingency planning, the operators performed surveillance procedure HC.OP-IS.GS-0101(O) on April 1,1998, for all eight vacuum breakers. No discrepancies were noted during the test and the results were acceptable.
The inspecter determined that operations, maintenance and engineering personnel effectively investigated the valve position indication discrepancy, and the appropriate actions were taken to correct the situation. However, the inspector noted some minor weaknesses during the test conduct. Specifically, operators continued the surveillance (two additional valves were tested) on March 30,1998, after the unexpected response for the "B" vacuum breaker was noted. This action was not consistent with management's expectations, which specify that test performers should stop the test when an unexpected equipment response is encountered during the surveillance. Additionally, although the operators implemented the more conservative Technical Specification Action 3.6.4.1.b for an inoperable vacuum breaker, there was not a clear and common understanding how they would implement the Technical Specification Actions 3.6.4.1.c.1 and 3.6.4.1.c.2, which are related to inoperable vacuum breaker position indicators only. Operations management acknowledged these minor weaknesses and were implementing actions to correct them, c.
Conclusions Operations, maintenance and engineering personnel effectively investigated an abnormal response of a torus-to-drywell vacuum breaker during testing, and the appropriate actions were taken to correct the situation. However, some minor weaknesses were apparent related to test conduct and followup by control room operators.
Quality Assurance in Operations
.QZJ Nuclear Review Board Meetina The inspectors attended Nuclear Review Board (NRB) meeting No. 98-02 on April 2
- 3,1998. The NRB is comprised of senior PSE&G managers and external consultants, and its responsibilities are described in administrative procedure NC.NA-AP.ZZ.0076(Q), " Independent Review Program." The inspectors observed the Hope Creek Report, presented by the Acting Operations Manager and the Quality Assurance Manager's Report, presented by the Quality Assurance Manager.
Both presenters were well prepared and were responsive to the NRB questions. The associated dialogues were probing and reflected a strong safety focus. The inspector concluded that the quality of the NRB presentations and discussions was very goo.
.
11. Maintenance M1 Conduct of Maintenance i
M11 General Observations of Work Week Plannina and Scheduling a.
Inspection Scooe (62707)
Throughout the report period, the inspectors reviewed on-line maintenance schedules and observed the execution of work plans. Additionally, self-assessments and quality assurance (QA) department observations with respect to work planning and implementation were evaluated, j
b.
Observations and Findinns j
i The inspectors noted that PSE&G maintained detailed performance indicators to monitor the effectiveness of the implementation of the work week management process. A review of these indicators showed an overall improving trend with respect to schedule stability, work completion rate, and maintenance backlog reduction. The inspectors did not identify any safety-related maintenance intentionally conducted outside its scheduled channel week. All emergent " cross-channel" work was appropriately evaluated for plant risk insights. Post work week critique meetings generally captured planning and scheduling deviations which developed. A QA surveillance which performed a detailed assessment of a scheduled on-line maintenance outage of the "C" filtration, recirculation, and ventilation train identified several work planning and execution deficiencies, many of which led to unnecessary increases train unavailability.
During the week of March 22,1998, the inspectors identified three different maintenance and surveillance activities that were planned and conducted on the reactor core isolation cooling (RCIC) system. All three of these activities were performed on different days, and all three required the system to be made inoperable for a period of a few hours each. Based on a review of operator logs, individual TS action statement entries were made for each scheduled activity, with the associated 14 day action statement re-initialized on each occasion. The inspectors discussed this issue with plant operators and the cognizant work week manager, and learned that the risk assessment conducted for the work week was based on the assumption that RCIC would be inoperable for the entire period, and as such, the three " mini-outages" did not increase overall plant risk. However, the inspectors determined that the overall RCIC system unavailability time, as well as the number of cycles put on the equipment, could have been reduced had the work activities been scheduled in parallel instead of in series. The work week manager generated an action request to document this issue.
On March 23, the mechanical maintenance department canceled an outage on the
"A" reactor water cleanup pump the day before its scheduled implementation due to the unavailability of critical repair parts. This outage was planned to correct long standing deficiencies associated with pump vibration, and required significant
-
coordination and planning with radiation protection personnel to ensure that j
exposures received during the maintenance would be as low as reasonable achievable. The inspectors noted that even though PSE&G's work management i
program requires that all necessary repair parts be available on site three weeks
)
prior to scheduled on-line maintenance, a decision was made to bypass this requirement in order to complete the outage as intended. When the parts did not arrive, the current work week was significantly impacted as was the future week to which the maintenance was deferred.
I Also on March 23,1998, station operators were unexpectedly required to tag-out the "B" emergency diesel generator (EDG) from service in order to support a pre-planned electrical maintenance activity identified one month earlier. Technicians j
originally believed that the activity could be performed without making the EDG l
inoperable, but when the work was initiated, they discovered that a control cabinet had to be de-energized to support the effort. Lack of specific pre-planning for this work activity resulted in increased unplanned EDG unavailability time and caused a perturbation of the pre-analyzed schedule of work week activities.
.
c.
Conclusions PSE&G's execution of the work week planning and scheduling process generally improved during the period, however several examples of poor implementation of work process requirements were identified. Work week critiques documented noted deficiencies as lessons learned for future improvement.
M1.2
"E" Traversina incore Probe (TIP) Stuck Durina Core Insertion
a.
insoection Scoos (62707)
The inspectors observed the development and implementation of a maintenance troubleshooting r!an to withdraw a TIP that could not be moved by the remote drive control unit.
b.
Observations and Findinas On February 18,1998, the "E" TIP was inserted into the reactor core for local power range monitoring (LPRM) calibrations. During the first attempt to insert the
"E" TIP, the TIP stopped at position 390 in channel 10. This located the TIP inside
{
the drywell, but not yet into the reactor core. All further attempts to withdraw the
'
"E" TIP remotely from the control room were unsuccessful. Control room operators promptly recognized that the "E" TIP primary containment isolation ball valve was inoperable in the current condition. The control room operators appropriately entered technical specification 3.6.3 action statement for primary containment isolation valves.
The operations manager organized representatives from the operations, engineering, i
maintenance, and radiation protection departments and the group developed a troubleshooting procedure to withdraw the stuck TIP locally from the drive
.
.
i
mechanism. The inspectors noticed that the operations manager and radiation protection personnel were especially cautious about the potential for radiation exposure should the TIP retract beyond its in-shield position. The troubleshooting
procedure included precautions to ensure the TIP would be stored in-shield. The maintenance technicians, in addition to participating in a thorough pre-job briaf, performed a dry run of the proposed activity using a spare TIP machine. The inspectors observed the implementation of the troubleshooting procedure from the control room. The "E" TIP was manually retracted to its in-shield position. Later troubleshooting could not discover any electrical problems with the "E" TIP drive control unit. The "E" TIP was remotely operated several times without any i
anomalies.
'
c.
Conclusions Hope Creek maintenance personnel thoroughly prepared a troubleshooting procedure to locally withdraw a stuck TIP. The development of this plan included sufficient precautions from recent industry lessons learned to prevent any possibility of radiation overexposure. The Instrumentation and Controls maintenance technicians completed the troubleshooting procedure in a timely and error free manner.
M1.3 Service Water System Preventive Maintenance a.
Insoection Scoos (62707)
The inspectors observed preventative maintenance activities on the "B" and "D" service water pump (SWP) discharge check valves, strainers, and traveling water screens.
b.
Observations and Findinos The inspectors observed the removal and installation of the "B" SWP discharge check valve. The inspectors noticed that the installed check valve had two disc springs compared to the removed check valve that had four disc springs. The inspectors questioned the acceptability of the installed check valve with the mechanical maintenance supervisor. The supervisor could not provide any explanation, nor did he notice that the check valve diagram in the procedure depicted a check valve with four springs. The mechanical maintenance supervisor stopped the check valve installation until he could resolve the problem with the service water system manager. The system manager verified that the check valve to be installed was correct and had been modified by a design change. The system manager agreed with the NRC inspectors that the procedure, although it did not preclude installation of a different style check valve, was misleading.
The inspectors walked down the material condition of the "B" service water train after the maintenance had been completed. The inspectors noticed that some equipment staged for the maintenance had been left at the job site even after the work was complete and after the pump was declared operable. The equipment was
_ _ - _ - _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ _ - _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _
l
-
i
.
not secured in accordance with Hope Creek's transient loads control. The inspectors discussed this with the operations manager who promptly had this minor problem corrected.
The inspectors observed similar maintenance activities five weeks later on the "D" service water train. The inspectors noticed that the procedure, " Folding Disc Check Valve Overhaul and Inspection, (HC.MD-GP.ZZ-0046)," had been revised to reflect the check valve spring modification. The inspectors also noticed that the job sites were promptly cleaned at the conclusion of the maintenance and that no transient loads were left improperly secured.
-
c.
Conclusions The inspectors observed the preventative maintenance activities conducted on the
"B" and "D" service water trains to be properly controlled in accordance with PSE&G's preventative maintenance procedures. The inspectors noticed that corrective actions taken in the interim between the "B" and "D" service water outages ensured that some minor problems did not recur.
M1.4 Hooe Creek Maintenance Focus insoection a.
insoection Scope (62700. 62707)
NRC inspectors reviewed various aspects of the Hope Creek Maintenance Program and its implementation. This initiative inspection was performed in response to recommendations from the Fall 1997 Plant Performance Review. Three NRC inspectors evaluated performance in this area from March 16 through March 20.
Preliminary findings and conclusions were discussed with PSE&G management on March 20,1998. The inspectors observed in-field maintenance activities, conducted interviews of 15 to 18 key personnel regarding their involvement with maintenance, conducted discussions, and reviewed pertinent documentation.
b.
Observations and Findinos On-line Maintenance PSE&G implements an on-line maintenance program which balances the benefits gained by a proactive preventative maintenance (PM) program against the potential cost or effect of equipment unavailability in any given work week. On-line maintenance is scheduled in recurring 12 week intervals with each of four channels scheduled for three separate weeks within each 12 week period. System and component outages within each period have been reviewed to ensure that unavailability of equipment does not pose any undue risk to plant safety. A Probabilistic Risk Analysis (PRA) with a resultant core damage frequency (CDF) has been generated for each of the 12 weeks. This PRA calculates a new CDF assuming all equipment scheduled for PMs within the week is unavailable. Any equipment or system unavailabilities that result in a CDF that exceeds 1.0 E-04 result in a " red" condition and require either a special evaluation and/or specific
-
.
.
.
.
- controls. In the current 12 week rolling schedule, two weeks list optional loop outages for either the Service Water or the Station Auxi:iary Cooling systems which result in a red condition and need special evaluations. The maintenance rule (10 CFR 50.65 (a) (3)) also suggests that evaluations be done. The inspector discovered that this was an area of inconsistent licensee performance. Specifically, maintenance may be rolled over between work weeks without documenting the extent of these evaluations. For example, Work Week Managers may consult a one-by-one color coded risk matrix which gives them a risk conclusion if only two major components are out of service at any one time. If three or more components are out of service, no overall risk conclusion can be derived from this matrix. In these cases, Work Week Managers should request a Plant Safety Analysis group formal evaluation to ensure acceptable results. This activity is not consistently implemented. PSE&G plans to review the current practices and to upgrade them to ensure appropriate evaluations are conducted.
Regarding normal work processing, potential work items normally enter the on-line maintenance program by the initiation of an Action Request (AR) which can result in up to three additional actionable documents: Condition Reports (CR), Corrective Maintenance (CM), or Business Practice (BP). CRs normally require the evaluation of some condition or event that affect or impact safety related equipment or procedures erJ are used primarily by engineering. bps relate to non-safety related equipment or procedures and are used island wide. cms are used primarily by the Maintenance Department to investigate and/or correct deficiencies in the performance of safety related and non-safety related structures, systems, or components. Typically,120 AR cms are generated weekly and are screened by the Work it Now (WIN) team. Of these, approximately 40% are worked and corrected by the WIN team,20% are corrected by Site Services, and 20% more are addressed through dispositioning/ validation and closed without work. This process eliminates duplicate activities, closes work already performed under another AR, and eliminates cms automatically generated when an AR is converted to a CR. The remaining 20% of cms are forwarded to planning for the preparation of work orders. On January 5,1998, there were 1524 open AR cms. Over the next two months,1237 AR cms were opened for a total of 2761 items. As of March 5, the WIN _ team was able to reduce this backlog to 402 items by screening und actual field work. This reflected a very successful effort by the WIN team.
To be worked by the WIN team, an AR CM must be coded as minor maintenance.
This requires a review against 10 specific criteria described in SD-11, Hope Creek Work Management Process. If an SSC can be worked as a minor maintenance item, it significantly reduces the administrative burden placed on the maintenance organization since it can be worked by the WIN. This streamlined process has been one key to PSE&G's backlog reduction efforts.
The planning department converts all forwarded AR cms to work orders. These WOs are scheduled based on their priority: as emergent work, during the next 12 week rolling schedule, during the next scheduled system outage, or for the next refueling outage. Major system outages are scheduled based on a three year schedule and consider impacts in addition to those dictated by the PRA. For i
.
.
example, the Service Water and Station Auxiliaries Cooling system preventive maintenance activities are not scheduled for summer periods because of elevated heat sink temperatures nor would heat trace PMs be scheduled winter periods.
.
Maintenance Observations The inspectors observed maintenance work in progress. Specifically, a number of work activities were observed on the "B" Primary Containment Instrument Gas (PCIG) compressor. These activities included initial setup such as draining the oil, and also evaluation and correction of deficient conditions discovered during inspections. During one of these inspections, an inspector noted a lack of clarity in a preventive maintenance (PM) procedure that listed the inspection criteria.
Specifically, the PM required a check for excessive oil on the piston guide tube. If detected, it stated that the guide piston seal assembly should be checked by the PCIG compressor overhaul procedure. Although the procedure had some limited inspection criteria, the bulk of the procedure for these sub-components addressed seal assembly removal and installation. This was interpreted to require replacement of the leaky seal assembly. A similar interpretation was applied when evaluating a small crack in the "B" PCIG's high pressure piston. Although these sub-components needed to be replaced, the inspector questioned both of these interpretations from a werk process standpoint because these actions expanded the scope of the original PM. PSE&G management agreed the wording was ambiguous and corrected it for the in-process work. The inspector questioned whether this wording was common in other PMs. The licensee researched 161 PM procedures and noted that the word " check" appeared in 79 of them. Each of these was reviewed and the licensee concluded that four of them could be interpreted to allow correction of a nonconforming condition without writing an AR or revising the PM.
The licensee stated that these four procedures will be corrected. By not writing ARs when nonconforming conditions are identified, the accuracy of information used for trending purposes (Level 3 ARs) would be suspect.
Root Cause Evaluations for Human Performance RelatedIssues The inspectors reviewed two Level 1 root cause analysis reports that PSE&G completed in order to assess the adequacy of PSE&G's root cause efforts. The l
inspectors reviewed the information contained in the reports and used the NRC's Human Performance Investigation Process (HPIP) to independently assess root cause. The inspectors found that the root cause analysis associated with Condition Report 970528252,HPCI Minimum Flow Valve Flow Transmitter Left Isolated After Surveillance, was thorough and it focused on several periphery factors that contributed to the event. However, the other report (PIR 971124338, Primary Condensate Pump Packing Failure), was not sufficiently probing and did not identify all of the root causes for the event. For example, several of the recommended corrective actions began with the statement " resolve the condition that caused."
The inspectors concluded that the failure to pursue all potential relevant root causes was a weakness, in addition, the inspectors reviewed several of the corrective actions to determine if the subsequent actions resulted in additional root cause analyses, and found that only specific actions were recommended and implemented
.
.
i rather than conducting further root cause analysis. Although no safety significant items were identified in this instance, the inspectors discussed this weakness with the maintenance standards superintendent and the corrective actions and self I
assessment supervisor.
i l
l The inspectors also reviewed an additional Level 1 root cause analysis, PIR V
980108107, Poor Maintenance Practices During RF07. This analy. sis was
!
conducted because a high number of issues related to poor maintenance work (
practices occurred during the most recent RF07 refueling outage. The analysis identified common causes for the poor performance to be inattention to detail and misjudgement, and the dominant factor was the absence of an adequately sized supervisory staff to compensate for the increased outage work force. The inspectors concluded that the associated recommended corrective actions were appropriate, and that the effort to collectively review several related performance issues was a good initiative.
Maintenance Trainino PSE&G was evaluating and implementing substantial changes in the maintenance training program. These changes were in response to previously identified deficiencies, such as on-the-job-training and '-evaluation (OJT/OJE) weaknesses, qualification tracking, and continuing training.
PSE&G was re-organizing the maintenance training department so that the trainers l
would report to the Hope Creek and Salem common maintenance standards department. This move is intended to provide improved interface between the
.
trainers and the field workers and to more directly meet the training needs of the maintenance organization. PSE&G also plans to utilize a mentoring program to further enhance communications training and field worker communications.
l
PSE&G recently implemented a new program consisting of periodic in Service Days,
which are designed to meet the specific training needs of each maintenance crew.
These In Service Days are replacing the second phase of the previously committed Hope Creek Maintenance Intervention. PSE&G documented this commitment change in a letter to the NRC dated March 31,1998 (LR-N980122). PSE&G stated that the first phase of the Hope Creek Intervention confirmed the same weaknesses identified in the Maintenance Interventions conducted for Salem maintenance and site services personnel. Although some skill-based deficiencies were identified, the
.
majority of the problems were related to " cultural traits." PSE&G plans to address l
improvement and reinforcement in several cultural areas, as well as reinforcement of
,
l certain skills, abilities and knowledge in the quarterly in Service Days. Maintenance l
craft and supervision attend the In Service Days. The inspectors observed an in
'
Service Day on March 18,1998, and determined that it provided relevant information to the work crews, particularly in the area of lessons learned from recent station events. Department goals and expectations, performance indicators, and improvement initiatives were also discussed. In addition, a mentor program,
l consisting of procedure writers, corrective action analysts and trainers, are assigned to support and provide focus to the various work crews. The mentor role is to l
.
e l
assist in areas such as repeat work, crew human performance, OJT/OJE, qualifications, and scheduling.
The inspectors interviewed several maintenance workers and supervisors and j
- discussed training issues. The interviewees generally discussed the known training i
weaknesses related to the quality of OJT and OJE, and some expressed concerns l
with the difficulty in verifying individual qualifications using the current system. No j
safety significant deficiencies were identified. Discussions with maintenance l
workers performing work in the field and observation of their work in progress
!
indicated acceptable skill and knowledge expertise.
!
c.
Conclusions l-The on-line maintenance program was effective at balancing the benefits gained
'
'through preventative maintenance program against the costs of equipment
l unavailability. The licensee quantified the risk of component and system unavailability in the 12 week rolling schedule and limited preventative maintenance
activities to manage these risks. A minor inconsistency was noted with evaluation of risk when maintenance was either deferred or added to a given work week.
'.
The Work It Now team was effective at screening new work entering the system.
The team has reduced existing corrective maintenance backlogs while maintaining a reasonable work-off rate for new work.
A wording ambiguity in a primary containment instrument gas compressor l
maintenance procedure could have led to an expansion of work scope not l
specifically identified by the maintenance procedure. This deficiency was corrected -
prior to beginning work. However, an extent of condition review noted other instances of poorly worded preventive maintenance procedures. By not writing I
action requests in these cases, the accuracy of information used for trending purposes could be suspect.
!
!
Root cause analyses were acceptable with one minor exception where certain l
aspects of one analysis was not sufficiently probing and did not identify all of the root causes for the event.
M2 Maintenance and Material Condition of Facilities and Equipment M2d
"A" and "B" Emeraency Diesel Generator 125Vdc Ground Isolation (
a.
Inspection Scope (62707)
The inspectors observed ground isolation troubleshooting activities on the "A" and
"B" emergency diesel generators (EDG).
l
,
!
.
.
b.
Observations and Findinas On February 23,1998, the "B" EDG was taken out of service for a planned maintenance outage. Maintenance activities included electrical panel work. On February 24,1998, operators performed a post maintenance surveillance test on the "B" EDG. During the."B" EDG operation, a ground developed on its associated 125Vdc bus. The "B" EDG passed all the acceptance criteria established by the surveillance test, however, the control room operators did not consider the "B" EDG operable yet. The operators did not know the source of the ground and considered the recent electrical maintenance activities a potential source.
The electrical maintenance department developed a troubleshooting procedure to identify the electrical ground source. The troubleshooting was complicated since
,
l the ground only appeared while the "B" EDG was operating. One portion of the
!
troubleshooting procedure included a visual inspection of the "B" EDG skid. PSE&G maintenance technicians discovered a damaged flex conduit to the 1 KJPSL - 6612B pressure switch. The technicians verified that slight physical movement of the flex conduit affected the ground magnitude. The maintenance technicians repaired the flex conduit. The "B" EDG subsequently operated without any grounds and the control room operators declared the "B" EDG operable. The NRC inspectors also verified that the."B" EDG operated without any grounds during its scheduled monthly surveillance test on March 24,1998. The NRC inspectors determined the maintenance technicians ground troubleshooting efforts to be thorough. The -
inspectors also considered a visual walkdown of all components that could generate electrical grounds to be a prudent activity.
On March 9,1998, during a scheduled monthly surveillance test of the "A" EDG, an electrical ground developed on its associated 125Vdc bus. No recent maintenance had been performed on the "A" EDG. Control room operators developed an operability determination for the "A" EDG and it considered the "A" EDG and its associated 125Vdc bus to remain operable, but degraded. Even though the "A" l
EDG was considered operable, the electrical mairitenance technicians expedited troubleshooting efforts to find the electrical ground. The troubleshooting discovered a faulty speed switch. The speed switch was replaced and the "A" EDG was operated with no electrical ground present.
i.
I~
The speed switch was enclosed in a " black box" that contained external lugs for electrical connection. This vendor supplied speed switch contained numerous electrical devices and its design was proprietary information. The maintenance technicians opened the cover to the faulty speed switch after it had been removed and identified a charred circuit board. PSE&G engineers sent the speed switch to
'
the vendor for a failure analysis. The inspectors considered the engineers' followup to evaluate the speed switch failure to be appropriate.
c.
Conclusions PSE&G maintenance technicians methodically identified the source of unrelated electrical grounds and corrected the electrical grounds that developed on the "A"
.
and "B" diesel generators associated 125Vdc busses. The promptness was commensurate with the safety importance of the diesel generators. PSE&G engineers initiated a followup analysis to ensure that the "A" EDG speed switch failure was well understood.
M4 Maintenance Staff Knowledge and Performance l
Ms.l
"C" Filtration. Recirculation, and Ventilation System Maintenance and Testina l
a.
Insoection Scone (61726. 62707)
The inspectors reviewed the circumstances surrounding a failed post-maintenance retest and monthly surveillance test of the "C" filtration, recirculation, and ventilation (FRVS) recirculation train.
b.
Observations and Findinas On March 20,1998, an operability retest of the "C" FRVS recirculation train failed following the installation of upgraded electrical contactors in the system's humidity control circuit. During the test, operators determined that a resistance heater in the ventilation train did not energize. Maintenance technicians subsequently discovered that a power lead had been inadvertently left unattached to one of the contactors during the upgrade installation. After a reverification of all the other leads in the associated electrical cabinet, the affected lead was properly installed, the surveillance testing was completed satisfactorily, and the unit was declared operable. Maintenance department personnel appropriately initiated an action request to document and evaluate the condition adverse to quality.
During PSE&G management's subsequent investigation into this issue, two specific concerns were identified in addition to the fact that the electrical lead was left unconnected. Specifically, PSE&G procedure NC.NA-AP-0005(O) revision 8,
" Station Operating Practices," Attachment 6 section 4.0 requires that, "regardless of the controlling administrative or implementing procedure, independent verification shall always be used for lifted leads..." and completion of this activity "shall be documented." During the contactor replacement, a second technician did document completion of an independent verification of the lifted leads, but failed to identify
- the lead which was left unconnected. Additionally, the work order for the contactor replacement required a voltage check at various outputs of the humidity control circuit after the new contactors were installed. When the technicians did not observe the expected results during one of these checks, they appropriately raised the issue to their immediate supervisor. However, the supervisor provided an incorrect basis for why the test results were satisfactorily, and closed out the work activity without any further discussion or investigation. Had a more thorough questioning and review been initiated, the unconnected lead would likely have been discovered before conducting the retest.
The inspectors determined that the failure to perform a proper independent verification of the contactor leads in the FRVS humidity control circuit was a
.
.
violation of technical specification 6.8.1.a in that a maintenance procedure governhg work on safety-related equipment was not adequately implemented.
However, this issue was self-identified during FRVS surveillance activities before the unit was declared operable. As such, this failure constitutes a violation of minor significance and is being treated as a Non-Cited Violation, consistent with Section IV of the NRC Enforcement Policy. (NCV 50-354/98-02-05)
c.
Conclusions PSE&G management performed a thorough review into the circumstances surrounding a failed post-maintenance surveillance test of a filtration, recirculation, and ventilation unit. However, this investigation determined that inattention-to-detail by maintenance technicians and poor decision making by a supervisor led to the test failure.
M8 Miscellaneous Maintenance issues M 8.1 (Closed) LER 50-354/98-01: Technical specification prohibited condition - missed high pressure coolant injection (HPCI) surveillance resulting in unavailability of HPCI.
The inspectors performed an onsite review of LER 98-01. In addition, this event was completely reviewed and described in inspection Report 50-354/98-01 Section 01.1, and was characterized as a Non-Cited Violation. No new issues were revealed by the LER. Based upon onsite follow-up and inoffice review of LER 98-01, this LER is closed.
Ill. Enaineerina E2 Engineering Support of Facilities and Equipment E2d
"B" Residual Heat Removal (RHR) Minimum Flow Check Valve Failure a.
Insoection Scoce (37551)
The inspectors reviewed PSE&G's identification and followup assessment of a failed four-inch Anchor-Darling swing check valve on the "B" RHR pump minimum flow line, b.
Observations and Findinos On February 23,1998, PSE&G engineers determined that the "B" RHR pump minimum flow check valve was stuck open under no flow conditions. The problem was found during radiography required by Hope Creek's check valve program. The check valve program was established at Hope Creek in response to INPO's Significant Operating Experience Report (SOER) 86-03.
Hope Creek had just started radiography on a limited population as part of its SOER 86-03 check valve program. Other testing methods had previously been used in the
.
.
SOER 86-03 check valve program. This wn.3 tre first radiograph that had been performed on the "B" RHR pump minimum flow check valve. The American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code Section XI inservice testing, using methods other than radiography, had never detected a similar failure of the "B" RHR pump minimum flow check valve. Even after the failure was detected by radiography, the check valve did not fail its ASME Section XI inservice stroke test when performed during troubleshooting.
The PSE&G engineers contacted the check valve manufacturer to more clearly understand the failure and to determine if a common mode failure existed in other i
safety-related applications. PSE&G also expedited radiography on an additional 13 similar check valves and did not detect any failures. Three similar check valves had recently been radiographed with successful results. Based on the failure analysis of the check valve performed during disassembly, information supplied by the vendor, and successful radiography of an expanded population, the engineers determined that the "B" RHR pump minimum flow check valve failure was an isolated instance and that no common mode failure would exist in similar check valves. The check valve failure was attributed to excessive " play" in the disc hinge arm which allowed the disc to catch and hang up on the inside of the seat ring.
The inspectors reviewed PSE&G's operability determination (OD) and followup assessment (FA) that was developed to assess the potential impact of similar check valves in other safety system applications. The inspectors judged the OD and FA to provide sufficient supporting documentation that no common mode failure existed.
However, the inspectors also noticed that some credit was taken for the ASME Section XI IST program. The IST program had never identified this particular failure,
,
and did not identify any abnormalities when the check valve was stroke tested i
.
during troubleshooting after the failure identification.
c.
Conclusions PSE&G engineers thoroughly evaluated the "B" RHR pump minimum flow check
. valve failure. - The engineers contacted the vendor, determined the failure mode, and performed radiographs of an expanded check valve population to support a
,
conclusion that the failure was an isolated instance. The inspectors concluded that i
the engineers had completed a comprehensive investigation of the problem to determine the extent of condition. The operability determination and the engineers'
followup assessment were thoroughly documented.
E2 2 Anastat and ETR Part 21 Notification Resoonse a.
Insoection Scone (37551)
i The inspectors reviewed PSE&G's response to a recent 10 CFR Part 21 noncompliance report made by Thomas & Betts, a subvendor to General Electric Nuclear Energy, regarding possible failures of Agastat and ETR relay l
-
!..
!
l b.
' Observations and Findinas General Electric Nuclear Energy notified PSE&G that Hope Creek had previously j
received Agastat and ETR relays that were the subject of a 10 CFR Part 21 notification. Hope Creek received the notification on February 10,1998.
'
l'
Insufficient soldering led to the relay failures that prompted the Part 21 notification.
l Hope Creek was able to account for all suspect relays that were ever purchased.
!
PSE&G engineers determined the application of each installed suspect relay and L
quarantined all the suspect relays that had ever been entered into the warehouse.
One relay had a control function in the "A" primary containment instrument gas (PClG) compressor, one relay was used for the "B" nuclear steam supply shut-off l
system (NSSSS) manual pushbutton, and four relays provided indication functions only. Hope Creek maintenance technicians replaced the suspect relay in the "A" PCIG compressor on March 25,1997, and the "B" NSSSS manual pushbutton relay l
on March 27,1998. An operability determination was made for the remaining
!
installed relays and all were evaluated as operable but degraded.
l c.
Conclusions i
l PSE&G engineers promptly evaluated a 10 CFR Part 21 notification made by a relay l
vendor. Equipment repairs were completed in a timely fashion on those relays that potentially impacted plant operations.
l
-
fl.3
"A" Service Water Pumo inservice Testina
a.
inspection Scone (37551)
l The inspectors reviewed a PSE&G engineer's evaluation of a failed "A" service l
water pump (SWP) inservice test.
l b.
Observations and Findinas l
On March 10,1998, the "A" SWP was operated for an ASME Boiler and Pressure l
Vessel Code,Section XI inservice test (IST). The control ling inservice test i
procedure, ("A" Service Water Pump - AP502 - Inservice Test, (HC.OP-IS.EA-
'
0001(Q)), was recently revised and included new reference test conditions. PSE&G engineers established new test conditions at a higher flow,17,000 gpm compared to 13,500 gpm for the previous tests.
When the equipment operators performed the "A" SWP IST on March 10,1998, the pump did not pass acceptance' criteria for pump differential pressure (flow was acceptable). The "A" SWP IST was performed later that day with PSE&G engineers present. The "A" SWP delivered about 400 gpm less flow and was below the acceptance criteria. PSE&G engineers believed that the "A" SWP did not operate on its characteristic curve and that its performance had degraded. However, PSE&G engineers could not explain the ince.nsistent results within one da.
!-
l l
1 l
l Subsequent troubleshooting verified that the installed instruments used to measure
!
acceptance criteria were accurate. PSE&G engineers e.lso verified that all available pump flow was measured and did not "short circuit" the installed flow measuring i
device. However, PSE&G engineers learned that a time constant of about 4-6 j
minutes was associated with the flow measuring device. The engineers also l
noticed that the current high detritus levels in the Delaware River, combined with l
the new higher test flow rate, require that the strainer backwash line be unisolated to clear high strainer differential pressure alarms on n Irequent basis. The strainer i
backwash line was isolated during the conduct of the IST per procedure so that all pump flow was accounted for in the flow measuring device. PSE&G engineers concluded that if system flow was not maintained stable, and it was not when the backwash line was unisolated, and if sufficient time was not allowed for the flow measuring device to settle, about 4-6 minutes, the data that was gathered for the
,
IST was inaccurate.
l After these lessons were learned, the IST was subsequently performed with repeatable successful results. The inspectors determined that PSE&G engineers appropriately analyzed all factors that could possibly effect the "A" SWP IST results. The results of the engineers' evaluation conclusively determined that the testing methodology needed additional precautions. The inspectors concluded that the engineers' thorough followup prevented unnecessary intrusive inspection of the
"A" SWP.
c.
Conclusions j
Hope Creek engineers evaluated inconsistent IST data from several "back to back" surveillance tests performed on the "A" service water pump. The engineers'
detailed analysis of the flow measuring device, test methodology, and current ph.nt conditions discovered test induced errors and prevented unnecessary intrusive inspection of the "A" SWP.
l IV. Plant Suonort F1 Control of Fire Protection Activities f1J. Fire Risk Evolutions a.
insoection Scoce (64704)
The inspectors reviewed the administrative processes for controlling and evaluating fire hazards, including limiting the interaction of combustible and flammable materials with ignition sources. This review was conducted to verify that adequate guidance and proper authorization requirements existed for identifying and limiting fire risk.
_. - - _
.
.
b.
Observations and Findinas The inspectors found that the administrative process for controlling ignition sources included the use of a permit system for authorization to perform hot-work activities.
The authorization form was approved by the loss prevention superintendent. Prior to authorizing a hot-work activity, fire protection personnel inspected the hot-work area to identify potential fire protection problems and to ensure that appropriate additional fire watches were provided. The inspectors determined that the administrative process provided a comprehensive review of fire areas to identify any potential interaction of combustible and flammable materials with ignition sources.
c.
Conclusions The inspectors concluded that good administrative controls had been established for proper storage of combustibles at the site and for control of hot-work activities.
F2-Status of Fire Protection Facilities and Equipment E2d Facility Tour a.
Inspection Scoos (64704)-
The inspectors toured the Hope Creek / Salem site and inspected fire suppression and detection systems / components.
b.
Observations and Findinas The inspectors found that the fire protection equipment material condition was good and that combustible fire-loading was properly maintained in those inspected. Fire i
brigade members' protective clothing and gear was found in good condition and it was adequately organized in the site fire house and brigade lockers. The inspectors determined that housekeeping in areas containing safety-related equipment, as well as housekeeping throughout the nonsafety-related areas was also good. Proper combustible material control was observed.
The inspectors verified that the fire suppression system pressure was adequate.
'
Fire hoses did not exhibit any cracks of fraying and all observed nozzles were properly rated. The inspectors noted that the diesel-driven pump fuel storage tanks
- were full and that the diesel engines were properly heated for quick start operation.
The inspectors also noted that terminal connections for the' diesel-driveri fire pump starting battery banks were clean, tight, and corrosion free.
The inspectors found that gauges on fire equipment including fire extinguishers, halon tanks, and carbon dioxide tanks registered in their appropriate ranges. The inspectors verified that all observed fire extinguishers were current with monthly surveillances. The inspectors noted appropriate smoke detection', fire detection, and alarm panels were installed throughout observed areas at the site. The inspectors observed that fire doors latched properly and were clearly posted with door
.
.
i'
requirements regarding closure. When fire doors did not latch properly, the inspectors noted that the licensee had appropriately tagged the doors and placed
,
them in their corrective action program for repair. There were no instances observed where access to fire suppression devices was restricted by any materials
or equipment.
{
The inspectors interviewed two continuous fire watches. The fire watches were knowledgeable mgarding station policy on reporting fires, fire watch duties, and
!
responding to fires. The inspectors reviewed the logbooks of each fire watch. 'The
'
inspectors determined that the fire watches were knowledgeable of their duties and responsibilities regarding the fire protection program.
[
The inspectors found that recent actions by the licensee had been implemented at.
the site to improve the effectiveness and operation of eight hour emergency lights for access / egress routes to safety-related equipment areas. The licensee demonstrated emergency light operability and illumination patterns on selected emergency lights during the plant tour. The inspectors noted that an emergency l
light in the emergency diesel generator access way for Salem Unit 2 failed to illuminate when the test button was depressed. A tag on the light indicated that the test button had failed and that the licensee had already entered it into their corrective action program. The inspectors verified this by reviewing the appropriate action request.
c.
Conclusions-l The inspectors concluded that fire protection equipment conditions and housekeeping were good. The inspectors also concluded that the licensee maintained good control of combustibles. Continuous fire watch personnel were knowledgeable of station procedures for reporting fires, fire watch duties, and j
- responding to fires. Eight hour emergency light operation and illumination patterns l
on observed emergency lights were good. The inspectors concluded that fire
'
systems were capable of providing protection against fire and were consistent with the defense-in-depth principle.
F12 Fire Barrier Penetration Seals I
a.
Inspection Scoos (64704)
I The inspectors reviewed selected penetration seals to determine the adequacy of installation and testing.
j b.
Observations and Findinas I
The inspectors selected penetration seals N-5304-008 and 009 located in Hope Creek's "D" emergency diesel generator room for review. The inspectors found that
- the licensee performed visual inspections of penetration seals every 18 months to ensure that required barriers were not degraded and remained operable. The inspectors found that the selected penetration seats were in good condition and that they met the appropriate installation criteri.
.
The inspectors reviewed Brand Industrial Supply Company (BISCO) fire test report No.1064-10, which documented tests performed on the above seal type, and acceptance criteria for seal detail, as validated by the destructive examinations.
The destructive examination demonstrated that the fire barrier penetrations had withstood the fire endurance test without the passage of flame for a period of time equivalent to the barrier fire resistance rating. The inspector found that the licensee installed the above seals in the same configuration as outlined in the BISCO test report.
c.
Conclusions The inspectors concluded that fire penetration seals were in good condition and that the "as-built" condition met the test criteria outlined in the vendor's test report for operational performance.
f2,ja Fire Main Looo Flow Testina a.
Insoection Sconed%704)
The inspectors reviewed test results of the last three yard hydrant loop flow tests for both Hope Creek and Salem. The inspectors also performed a walkdown of the yard hydrant loop.
b.
Observations and Findinas i
The inspectors found no evidence of deterioration or blockage of the fire mains i
based on loop flow test results. The inspectors noted loop flow tests for both Hope Creek and Salem resulted in flow coefficients well above the acceptable minima. In addition, the Salem test procedure recorded flow coefficients from previous tests to establish trend / data. The inspectors found that the yard hydrant loop components I
material condition was good.
c.
Conclusions Based on the tests results reviewed, test data trends, and observation of the condition of the hydrants and post indicator valves, the inspectors concluded that the fire main loop is in good repair and capable of providing the necessary water supply for fire fighting needs at the facility.
EL4 Fire Pumo Testina a.
Insoection Scooe (64704)
The inspectors reviewed the fire pumps tests data for the past five years. The inspectors also inspected the material condition of the installed fire pumps and fresh water storage tank i
.
.
l
b.
Observations and Findinas The inspectors determined that fire pump operating data recorded during pump tests l
were within the acceptance criteria. The inspectors also found that test data showed consistent performance for pumps and drivers.
,
The inspectors found that fire pump tests had formerly been performed on an I
automatic timer, but that this practice was discontinued after a diesel engine driver l
failed due to the cooling water supply having been inadvertently isolated (in May
.
1993). Tests are currently initiated using the test switches on the control panels,
!
which bleed the pressure off the automatic start pressure switches. An operator is l
on station during the entire pump test. The inspectors verified that the licensee had appropriate procedures in place to perform this test.
The inspectors found that the installed equipment for Hope Creek / Salem showed the fire pumps to be in a good state of preservation, with the water supply valves open, and the pumps ready for operation. One exception was the Salem i fire main jockey pump. Trouble tag 48965 was attached to the pump to document that excessive packing gland leakage had caused corrosion of the pump casing. The inspectors noted that the jockey pump maintained system pressure while running for only a l
few minutes each hour. The inspectors verified that the licensee had entered the pump into their corrective action program for replacement. The inspectors noted leakage from the test manifold for the Hope Creek fire pump. The inspectors verified that the licensee had entered an action request into their corrective action program to replace the isolation valve for the fire pump test manifold. The inspectors judged that the operation of the Hope Creek / Salem fire pumps were not affected by these conditions.
c.
Conclusions Based on the observed conditions of the equipment and the review of the test data, the inspectors determined that the Salem and Hope Creek fire pumps are well-maintained and ready for service. Conditions which could adversely affect pump performance were identified and corrected.
F3 Fire Protection Procedures and Documentation Ell Fire Protection Procedure Review a.
Insoection Scoos (64704)
,
. The inspectors reviewed fire protection procedures to determine if they provided sufficient detail and were technically sound. The inspectors also reviewed changes l
that occurred in these procedures in the past three years.
j
.
.
.
I l
b.
Observations and Findinas l-The inspectors found that the fire protection procedures govern all facets of the fire
~
protection program. This included the operational fire protection program, fire rnanning and qualifications, fire protection surveillance and periodic test program, fire protection training program, and actions for inoperable fire protection. The
.
inspectors found that the procedures provided sufficient detail and were technically sound for implementing the fire protection program at the site. The inspectors also found that changes to the fire protection procedures were minor in nature and did not impact the program as it is stated in the Updated Final Safety Analysis Report.
c.
Conclusions The inspectors concluded that fire protection procedures were acceptable for fire protection program implementation, contained sufficient detail, and were technically sound. The inspectors also concluded that changes to fire protection procedures did not impact the licensing basis of the fire protection program.
F4 Fire Protection Staff Knowledge and Performance F4.1 Fire Briaade Drills a.
Insoection Scone (64704)
The inspectors observed a fire drill to evaluate the effectiveness of the fire brigade and their understanding of fire attack strategies. The drill was conducted to demonstrate the following:
i o
effectiveness of fire alarm response, timeliness of department notification; e
response of fire brigade, time required to initiate fire attack or mitigation; e
ability to assess the fire properly; e
an understanding of the fire attack strategy; i
e an awareness of vital equipment in the area; e
ability of each member to physically perform required tasks; e
effective communication between fire brigade members; and e
an awareness of additional hazards in the fire area.
b.
Observations and Findinas The inspectors observed the fire drill on March 26,1998. A fire was simulated at the Hope Creek service water intake structure.
The inspectors determined, based on drill observations and post-drill discussions with responding brigade members, that the performance and knowledge of the drill participants was very good. This determination of acceptability was based on the following:
.
.
e use of appropriate fire type suppressant; e
command and control demonstrated by the fire brigade leader; e
team work displayed by fire brigade members; and e
communications among brigade members.
The inspectors found the quality of the critique following the drill to be effective; in that, it provided constructive feedback to the fire brigade regarding performance.
The inspectors noted that each member of the fire brigade team was allowed to provide constructive comments to the team leader and supervision regarding fire brigade performance. Additionally, the inspectors noted that this fire drill was also used as a training vehicle to increase the effectiveness of the fire brigade.
c.
Conclusions The inspectors determined that the fire brigade performance during a drill was very good. All expectations of the fire drill were met.
F5 Fire Protection Staff Training and Qualification Eid Fire Briaade Trainino a.
Insoection Scone (64704)
The inspectors reviewed the training program requirements and the training provided for fire brigade members to verify that members had completed all required training for qualification and duty.
b.
Observations and Findinas The inspectors verified that twelve fire brigade members randomly selected for review had successfully completed the required training courses, drills, respirator training and passed their annual medical physicals. No deficiencies were identified.
The inspectors found that the initial and continuing training programs appropriately emphasized potential fire hazards and precautionary measures, supported brigade member readiness, and complied with NRC requirements and the Hope Creek / Salem i
licensing basis.
]
i c.
Conclusions l
The inspectors determined that fire brigade members were current on all required training and annual physical examinations. The inspectors also concluded that the PSE&G fire brigade training program was comprehensive, well organized, and complete. The training program complied with NRC requirements for preparing fire brigade members to combat fire.
.
F7 Quality Assurance in Fire Protection Activities EL1 Audits and Surveillances a.
Insoection Scoce (64704)
The inspectors reviewed the quality assurance audit reports completed to satisfy the Updated Final Safety Analysis Report (UFSAR) requirements that evaluated the effectiveness of fire protection measures, equipment, program impismentation, and problem identification and resolution.
b.
Observations and Findinos The inspectors reviewed the three most recent quality assurance audit reports, several quality assurance surveillance reports, and several quality assessment feedback forms. The reports reviewed were:
e Audit Report No. 97-032-Nuclear Business Unit Fire Protection Audit (1997)
e Audit Report No. 96-032-Nuclear Business Unit Fire Protection Audit (1996)
e Audit Report No. 95-032-Nuclear Fire Protection (1995)
The inspectors determined that these documents demonstrated good problem identification, had been appropriately completed, and clearly communicated findings in the reports. The inspectors noted that the audit scopes, findings, and observations were good and met the requirements of the program. The inspectors verified that proper revisions and actions were taken to effectively resolve any identified deficiencies. Corrective actions were found to be implemented for resolving these deficiencies in a timely manner.
c.
Conclusions The inspectors concluded that fire protection quality assurance audits appropriately focused on fire program attributes and compliance with program requirements. The inspectors also concluded that the fire protection audit findings were appropriately assessed and timely corrective actions had been taken for any identified deficiencies.
V. Manaaement Meetinas X1 Exit Meeting Summary The inspectors presented the inspection results to members of licensee management at an exit meeting on April 14,1998. The licensee acknowledged the findings presente f l
.
.
<
The inspectors asked the licensee whether any material reviewed during the inspection should be considered as proprietary information. No proprietary information was identified.
X3 Site Visit Summary On March 30 and 31,1998, Richard V. Crlenjak, Deputy Director, Division of Reactor Projects for NRC Region I and James C. Linville, Chief of the Reactor Projects Branch No.
3, NRC Region I, toured Hope Creek and met with PSE&G Senior Vice President - Nuclear Operations and the managers and staff of Hope Creek.
l l
t I
l I
l l
I
!
l l
i l
!
,
.
.
(
INSPECTION PROCEDURES USED
,
L IP 37551:
Onsite Engineering IP 61726:
Surveillance Observations l
lP 62707:
Maintenance Observations l
IP 64704:
,
IP 71707:
Plant Operations l
IP 92700:
Onsite Followup of Written Reports of Nonroutine Events at Power Reactor l
Facilities ITEMS OPENED, CLOSED, AND DISCUSSED Ooened
,
l l
50-354/98-02-02 VIO Inadequate procedure controls over temporary equipment. (O2.1)
{
50-354/98-02-03 VIO Inadequate procedure controls over scaffold program.
(O2.1)
50-354/98-02-04 VIO Improper installation of the floor drain plugs.'(02.1)
Ooened/ Closed
,
i 50-354/98-02-01 NCV CRD 14-35 mispositioned. (01.1)
50-354/98-02-05 NCV
"C" FRVS maintenance and testing. (M4.1)
,
!
Closed i
-
-
l 50-354/98-001 LER TS Prohibited Condition - missed HPCI surveillance
resulting in the unavailability of HPCI. (M8.1)
i i
i i
I
,
e
'
.
LIST CF ACRONYMS USED AR Action Request ASME American Society of Mcchanical Engineers BISCO Brand Industrial Supply Company BP Business Practice CDF Core Damage Frequency CM Corrective Maintenance CR Condition Reports CRD Control Rod Drive EDG Emergency Diesel Generator FA Followup Assessment FRVS Filtration, Recirculation, and Ventilation HCU Hydraulic Control Unit HPCI High Pressure Coolant injection HPIP Human Performance Investigation Process IST Inservice Test LPRM Local Power Range Monitoring NRB Nuclear Review Board NRC Nuclear Regulatory Commission NSSSS Nuclear Steam Supply Shut-off System OD Operability Determination OJT/OJE On-The-Job-Training and -Evaluation PCIG Primary Containment Instrument Gas PDR Public Document Room PM Preventative Maintenance PRA Probabilistic Risk Analysis PSE&G Public Service Electric and Gas OA Quality Assurance i
RBM Rod Block Monitor RCIC Reactor Core Isolation Cooling
]
SOER Significant Operating Experience Report
.
SWIS Service Water intake Structure SWP Service Water Pump
TIP Traversing incore Probe
'
UFSAR Updated Final Safety Analysis Report WIN Work it Now