ML17306A384

Responds to Concerns Raised in Insp Repts 50-528/91-30, 50-529/91-30 & 50-530/91-30.Corrective Actions:Work Order Process Changed to Provide for in-line Review of Emergency Lighting PM Work Orders

ML17306A384
Person / Time
Site:	Palo Verde
Issue date:	12/27/1991
From:	Conway W ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR
To:	Martin J NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V)
References
102-02091-WFC-T, 102-2091-WFC-T, NUDOCS 9201070257
Download: ML17306A384 (23)

Text

WILLIAMF. CONWAY EXECUTIVEVICEPRESIDENT NUCLEAR RECEiVED HRC REGiOH Y Arizona Public Service Company I9I'tIJ OE'{; 3 f pg I2: i 3 P.O. BOX 53999

~

PHOENIX. ARIZONA85072-3999 102-02091-WFC/TRB/DLK December 27, 1991 Mr. John B. Martin, Regional Administrator U. S. Nuclear Regulatory Commission Region V 1450 Maria Lane, Suite 210 Walnut Creek, CA 94596-5368

Reference:

1) NRC Inspection Report 50-528, 529, 530/91-30

2) Conversation between P. Narbut, NRC and T. R. Bradish, APS on November 22, 1991 - Response Extension Request to December 27, 1991

Subject:

Palo Verde Nuclear Generating Station (PVNGS)

Units 1, 2 and 3 RESPONSE TO NRC CONCERNS

~2il:

1-7

- 2 Reference 1 requests'that Arizona Public Service Company (APS) provide information r garding the management of the emergency lighting program. APS's response is e

ment provi e in

'd d 'ttachment 1 which describes completed engineering and manageme lied lant actions, and actions being taken to enhance trending and improve insta e p an equipment.

System Performance Monitoring and Trending is part of an improvement.

~

initiative that is being implemented by APS.

Due to the length of time needed to fully implement the program, interim measures are being taken to enhance monitoring of the performance of the emergency lighting system by System Engineering.

These interim measures are described in Attachment 2.

In addition, APS is providing a description of the plan fortransferring the functions of the Emergency Lighting Team to the Fire Protection Engineering Section. This description is provided in Attachment 3.

e ij l'

page 2 102-02091-MFC/TRB/DLK December, 27, 1991 Should you have any further questions or require additional information, please contact Mr. E. C. Simpson of my staff.

Very truly yours, WFC/DLK/css Attachment 1 - Response to NRC Concerns Attachment 2 - Interim Emergency Lighting Equipment Performance Monitoring Attachment 3 - Emergency Lighting Team Transition

0 f

0 t

ATTACHMENT1 RESPONSE TO NRC CONCERNS

RESPONSE TO NRC CONCERNS N

E NIT Rl E

E I

PERFORMANCE You do not appear to have effectively implemented a method to monitor whether recent system modifications are actually resulting in improved emergency lighting system performance.

APS RESPONSE At PVNGS, monitoring of system performance, which includes the effectiveness of system modifications, is accomplished by two means:

~

Failure Data Trending (FDT) monitors equipment. deficiencies over time to detect recurring deficiencies and negative trends that are not immediately obvious; and,

~

System Engineering monitors and evaluates the day-to-day activities on specified systems to assess system performance and to anticipate problems.

In addition, the Emergency Lighting Team provided an oversite function which resulted in feedback to management and engineering on the effectiveness of the emergency lighting system.

A. Enhancements to the FDT program that have been completed since August, 1990, or are being implemented include:

~

Providing component-specific failure criteria to more clearly focus FDT efforts on those components that can prevent a system from performing its intended function.

~

Performing increased analysis of component histories prior to issuance of the FDT Quarterly Report to facilitate the identification of recurring failures and negative trends.

Reviewing all failure reports submitted to ensure consistency prior to being made available for on-line review.

These enhancements have been completed for emergency lighting, but are still in progress for other systems.

The FDT program effectiveness is demonstrated in that the three specific examples of "failed equipment" noted in the NRC inspection Report were reported through the FDT program activities to engineering for further evaluation and action.

(See Section ll, below.)

0 I

t

B. The following actions are being taken to enhance the monitoring of system performance by System Engineering.

-'eview monitoring programs at other plants to help determine the best approach to effective performance monitoring by April, 1992;

~

Establish revised monitoring and trending program guidelines by June, 1992;

~

Communicate the program guidelines to the appropriate engineers to establish a clear and common understanding of the expectations by June, 1992; and,

~.

Implement the new guidelines on emergency lighting as one of the pilot systems by August, 1992.

This is a long-term action plan.

The interim methodology for monitoring the'erformance of emergency lighting equipment is detailed in Attachment 2.

The tong-term plan discussed above and the measures being taken in Attachment 2 will form the basis of our long-term trending program for emergency lighting.

1 During the NRC emergency lighting follow-up inspection conducted in August, 1991 (Reference 1), the NRC expressed an interest in reviewing updated information on Exide and Holophane inverter availability similar to that presented to the NRC on August 28, 1990. Development of the information provided in August, 1990, required an intensive effort by APS to research and develop information to a level of detail that was'well in excess of normal Failure Data Trending or system performance monitoring.

In response to the NRC's

interest, however, APS will provide the updated information on Exide and Holophane inverter availability by January 31, 1992. This is being done in order to determine the impact of increased engineering and maintenance on the availability of the emergency lighting system. APS does not intend to maintain trending information to this level of detail on an on-going basis.

J I

(

I I

I 1

NRC CONCERN - REC BRIN DEFICIENCIES Several types of.emergency lighting system deficiencies, similar to those noted during previous NRC inspections, appear to be recurring.

iLPP~F NMB ln

1990, there were some recurring emergency lighting deficiencies.

Improvements made to preventative maintenance, the use of more thorough testing, 'and modifications. to the plant design have resolved most of these conditions.

One condition not yet fully resolved relates to recovering the specific gravity of the Exide batteries following a discharge.

Some improvements have been made and

'engineering and management are in the process of determining additional improvements to the design and testing method to resolve this issue.

ln Reference 1, the NRC identified Exide battery low specific gravity and low voltage, relay (LYR) problems as examples of recurring failures. These examples were based on a review of a FDT report.

APS concurs that there was a problem with the Exide LVR's in June, 1990.

LVR deficiencies were. initiallyidentified to APS engineering and management as a result of the emergency lighting work order review of July, 1990.

The deficiency was determined to be'an inappropriate setpoint that did not fullyaccount forthe length of'able between the DC supply batteries and the Exide inverter unit.

Once the deficiency was identified, the LVR setpoints w'ere revised and adjusted by Site Modification 1,2,3-SM-QD-008 in August, 1990.

Since the initial setpoint change, the LVR setpoints have been changed again to accommodate a removal of loads from the Exide UPS's by Design Change Package (DCP) 1,2,3-XE-QD-026.

These changes to the LVR's have resolved the problems associated with the LVR's. This has been confirmed through subsequent system testing and no further problems have been observed in this area.

With regard to low specific gravity, a review of the emergency lighting work orders performed in July, 1990, identified instances in which the recharge time for the Exide batteries exceeded 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> before meeting the specific gravity acceptance criteria. The Exide UPS's were considered unavailable until the specific gravity criteria was met. APS considered this a prudent course of action, even though the 72-hour time limit was administratively established.

Specific gravity recovery time requirements are not specified in the manufacturer's instructions or IEEE 450-1 980.

0

An Engineering Evaluation Request (EER) was issued to determine the factors that prevent rapid recovery of specific gravity.

While the limited output of the battery chargers is the major factor, excessive cable lengths was identified by the EER as a contributing factor.

As a result of DCP 1,2,3-XE-QD-027, the cable length was reduced, thus improving battery charger efficiency. The EER also recommended reducing the load on the Exide inverter to improve the capacity of the UPS.

As previously mentioned, DCP 1,2,3-XE-QD-026 was issued,to accomplish this.

Retests associated with DCP's 1,2,3-XE-QD-026 and QD-027 in October, 1991 revealed that, while the design changes did provide an improved discharge margin, they did not fully resolve the slow recharging issue.

Engineering is considering a number of additional actions to resolve the issue of slow recharging of the Exide batteries, including changes to the battery chargers, alternative battery designs, and revised acceptance criteria. Until a permanent resolution is implemented, a maintenance work instruction will be developed to improve specific gravity recovery times of Exide batteries following a freshening charge.

This instruction is expected to be developed during the second quarter of 1992.

III.

NRC CONCERN - ENGINEERING AND MANAGEMENTINVOLVEMENT Appropriate licensee engineering and management personnel do not appear to be properly involved in the timely evaluation and correction of several of those recurring deficiencies.

APS RESPONSE The engineering and management involvement and commitment to proactiveness is clearly demonstrated by the following:

~

PVNGS senior management established the Emergency Lighting Team and closely monitors its actions and reports.

~

Engineering is proactive in identifying design weaknesses and senior management has committed resources to expeditiously implement required design changes.

As a result, since March, 1989, engineering has issued eight design changes (some of which address multiple issues) on design weaknesses.

In addition, management was proactive in committing funds to expeditiously implement all eight design changes.

~

Senior management'has elevated the priority level applied to corrective and preventative maintenance work performed on emergency lighting.

MNCR's are used to report and resolve emergency lighting deficiencies.

~

The Dual-Lite Pilot Program was established to determine the proper PM.

frequency for Dual-Lites located in high temperature environments.

0

~

Increased QC inspection time is now being devoted to emergency lighting.

~

Engineering performed a failure mode analysis on Dual-Lite fixtures to more clearly focus Failure Data Trending efforts in that area.

The recurring deficiencies cited by the NRC (in Reference

1) demonstrate that engineering and management are deeply involved in the emergency lighting program, since each deficiency was being addressed appropriately, as described in Section II, above. APS management willcontinue to closely monitor the progress of

'he emergency lighting program to ensure that pending issues are resolved and improved performance is maintained. describes enhancements to the program for monitoring performance of the emergency lighting system that are being implemented on an interim basis.

ATTACHMENT2 INTERIMEMERGENCY LIGHTING EQUIPMENT PERFORMANCE MONITORING

INTERIMEMERGENCY LIGHTING EQUIPMENT PERFORMANCE MONITORING To monitor equipment performance, the System Engineer must be able to trend critical parameters, such as inter-cell voltages, charging voltages and specific gravity. The System Engineer also needs to be cognizant of system problems. To expedite obtaining the necessary parameters and increase System Engineering's real time awareness of emergency lighting performance, the following willoccur:

~

The work order review process will be changed to provide for in-line review of emergency lighting PM work orders by the System Engineer following the Work Group Supervisor review. This willallow the System Engineer to sort and trend critical parameters.

~

Copies of all emergency lighting Material Non-Conformance Reports (MNCR's) and Condition Reporting/Disposition Requests {CRDR's) will be provided to the System Engineer to sort and categorize like failures.

~

Failure Data Trending information will be provided to the System Engineer on a weekly basis to identify similar failures or problems.

ln addition, the following willalso be provided:

N

~

Trending software and training to provide the System Engineer with more sophisticated tools to sort and categorize data.

~

Any additional assistance that may be needed to initiallyset-up and support the enhanced trending program.

These measures willultimately be phased into the System Performance Monitoring and Trending Program as discussed in Attachment 1,Section I.B.

0

ATTACHMENT3 EMERGENCY LIGHTINGTEAMTRANSIT(ON

I Vr 0

EMERGENCY LIGHTINGTEAMTRANSITION As previously stated at the NRC Exit meeting, the Emergency Lighting Team has a published Demobilization Plan. Per this Plan, emergency lighting oversight responsibility has been transferred to the Fire Protection group. Ultimately, emergency lighting issues willbe fullyintegrate'd into permanent, PVNGS programs a'nd procedures. The Emergency Lighting Team willnot demobilize until:

~

Senior management has assurance that the emergency/essential lighting system wiilfunction as intended and that current activities, including those required to meet PVNGS commitments identified in the UFSAR, are adequately controlled to ensure the activity is completed as scheduled.

Adequate performance trending is in place and the results demonstrate emergency lighting is reliable.

ln addition, the PVNGS Engineering Vice President will meet with the responsible department managers to reinforce the APS commitment to timely complete remaining open issues.