ML20199G356

Responds to NRC Re Violations Noted in Insp Repts 50-528/98-14,50-529/98-14 & 50-530/98-14,respectively. Corrective Actions:Assured That All Potentially Affected Check Valve Discs Were Correctly Aligned

ML20199G356
Person / Time
Site:	Palo Verde
Issue date:	01/13/1999
From:	James M. Levine ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR
To:	Lieberman J NRC OFFICE OF ENFORCEMENT (OE)
References
102-04233-JML-A, 102-4233-JML-A, 50-528-98-14, 50-529-98-14, 50-530-98-14, EA-98-382, NUDOCS 9901220270
Download: ML20199G356 (15)

Text

_..*.

& PbQ:

I Jarnes M. Levine TEL (602)393-5300 timi Station 7602 Palo Verde Nuclear Senior Vice President FAX (602)393-6077 P O Box 52034 Generating Station Nuclear Phoenix. AZ 85072-2034 102-04233-JML/AKK/ RAS January 13,1999 Mr. James Lieberman Director, Office of Enforcement U. S. Nuclear Regulatory Commission ,

One White Flint North,11555 Rockville Pike Rockville, MD 20852-2738 i

Reference:

NRC letter dated December 21,1998, NOTICE OF VIOLATION AND PROPOSED IMPOSITION OF CIVIL PENALTY - $55,000 (NRC  !

Inspection Report No. 50-528/98-14; 50-529/98-14; 50-530/98-14)

Subject:

Falo Verde Nuclear Generating Station (PVNGS)

Units 1,2 and 3 Docket Nos. STN 50-528/529/530 4 Reply to a Notice of Violation and Payment of Civil Penalty (EA l 98-382) l In a letter dated December 21,1998 (Reference), the NRC concluded that violations representing a Severity Level 111 problem occurred at PVNGS, and proposed the l imposition of a civil penalty in the amount of $55,000. In brief, the violations involve three l main issues: (1) failures to adhere to technical specification limiting conditions for operation when ECCS subsystems were inoperable; (2) a failure to identify and correct l significant conditions adverse to quality despite numerous indicators; and (3) l inadequate procedures which caused the problems and prevented timely identification l of the significant conditions adverse to quality.

I Pursuant to the provisions of 10 CFR 2.201, APS hereby submits this response to the Notice of Violation (NOV). Enclosure 1 to this letter is a restatement of the NOV and APS' response to the NOV is provided in Enclosure 2. APS admits that each of the three violations occurred and in accordance with 10 CFR 2.205, APS has elected to pay the proposed civil penalty in the amount of $55,000. Payment is being made in the form of a check made payable to the Treasurer of the United States (Enclosure 3).

o90 M I t

9901220270 990113 g= ^== oS9ua . 7

l ,

Director, Office of Enforcement U. S. Nuclear Regulatory Commission Reply to a Notice of Violation and Payment of Civil Penalty (EA 98-382)

Page 2 APS acknowledges that the analysis methods used to demonstrate the effects of the back  !

flow condition during a loss-of-coolant accident were not NRC approved. However, the l realistic evaluation models used by APS provided a conservative assessment of the l- event that demonstrated the accident consequences would meet the same acceptance l

criteria as that specified in the approved methodologies. Additionally, simulator enactments have demonstrated the reverse flow condition would have been adequately identified and addressed by operator actions and emergency operating procedures.

l l

l APS is confident that the immediate and long term corrective actions that have been taken to correct these violations are sufficient to restore compliance with the l regulations and prevent recurrence of the condition. Accordingly, APS makes no new regulatory commitments in this correspondence.

Should you have any questions, please contact Ms. Angela Krainik at (602) 393-5421.

Sincerely, i I l

4

/4 JML/AKK/ RAS /mah l Enclosure 1 Restatement of Notice of Violation Enclosure 2 Reply to Notices of Violation Enclosure 3 APS check made payable to Treasurer of the United States, in the amount of $55,000.

cc: E. W. Merschoff w/o Enc. 3 I M. B. Fields w/o Enc. 3 l J. H. Moorman w/o Enc. 3 1

f J

J

l 1

1 l

ENCLOSURE 1 f

RESTATEMENT OF NOTICE OF VIOLATION NRC INSPECTION REPORT NO.

50-528198-14; 50-529/98-14; 50-530/98-14 I

I 2

h

RESTATEMENT OF NOTICE OF VIOLATION NRC Inspection Report No. 50-528/98-14; 50-529/98-14; 50-530/98-14 During an NRC inspection conducted May 26 through July 21,1998, violations of NRC requirements were identified in accordance with the " General Statement of Policy and Procedure for NRC Enforcement Actions," NUREG-1600, the NRC proposes to impose a civil penalty pursuant to 1 Section 234 of the Atomic Energy Act of 1954, as amended (Act),42 U.S.C. 2282, and 10 CFR j 2.205. The particular violations and associated civil penalty are set forth below: '

A. Technical Specification 3.5.2 requires, in part, that in Modes 1,2, and 3, two independent emergency core cooling system (ECCS) subsystems be operable with each subsystem  !

comprised of one operable high pressure safety injection (HPSI) pump, one operable low l pressure safety injection pump, and an independent operable flow path. The action i statement requires that with one ECCS subsystem inoperable, restore the inoperable subsystem to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or be in at least hot standby within the next 6 l hours and in hot shutdown within the following 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />. ,

Technical Specification 3.0.3 requires, in part, that when a Limiting Condition for Operation is I not met, except as provided in the associated action requirements, within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, action shall I be initiated to place the unit in a Mode in which the specification does not apply by placing it, as applicable, in at least hot standby within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />, and at least cold shutdown within the following 30 hours3.472222e-4 days <br />0.00833 hours <br />4.960317e-5 weeks <br />1.1415e-5 months <br />.

l

! l l Contrary to the above: )

1. Between May 1,1992, and May 15,1998, while Unit 1 was in Mode 1, the Train "B" ECCS subsystem was inoperable because it did not have an independent operable ,

flow path, due to the inoperability of Valve 1 PSIA-V404, and action was not taken to j l either restore the su. .ystem to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or to place the unit in l hot standby within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in hot shutdown within the following 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.

2. Between April 14,1993, and May 16,1998, while Unit 2 was in Mode 1, the Train "A" l ECCS subsystem was inoperable because it did not nave an independent operable l flow path, due to the inoperability of Valve 2PSIB-V405, and action was not taken to l either restore the subsystem to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or to place the unit in hot standby within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in hot shutdown within the following 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.

3. Between October,1992, and April 1994, while Unit 3 was in Mode 1, the Train "B" ECCS subsystem was inoper;ble because it did not have an independent operable flow path, due to the inoperability of Valve 3 PSIA-V404, and action was not taken to either restore the subsystem to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or to place the unit in hot standby within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in hot shutdcwn within the following 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.

l

4. On September 4,1996, while Unit 1 was in Mode 1, both independent ECCS subsystems were inoperable in that the Train "A" HPSI system was out-of-service for i maintenance, Valve 1 PSIA-V404 was inoperable, and Train "B" was not isolated from Train "A". With both flow paths inoperable, the licensee failed to take action within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, to place Unit 1 in a Mode in which Technical Specification 3.5.2 does not apply.

5. On October 21,1994, while Unit 2 was in Mode 1, both independent ECCS subsystems were inoperable in that the Train "B" HPSI system was out-of-service for maintenance, Valve 2PSIB-V405 was inoperable, and Train "A" was not isciated from Train "B". With both flow paths inoperable, the licensee failed to take action within 1 ENCLOSURE 1 PAGE1 i

RESTATEMENT OF NOTICE OF VIOLATION NRC Inspection Report No. 50-528/98-14; 50-529/96-14; 50 530/98-14 hour, to place Unit 1 in a Mode in which Technical Specification 3.5.2 does not apply.  ;

(01013) i B. 10 CFR Part 50, Appendix B, Criterion XVI, requires, in part, that measures shall be j established to assure that conditions adverse to quality such as failures, malfunctions, '

deficiencies, deviations, defective material and equipment, and nonconformances are promptly identified and corrected. In the case of significant conditions adverse to quality, the measures shall assure that the cause of the condition is determined and corrective action taken to preclude repetition.

Contrary to the above:

1. A cocked open HPSI pump discharge check valve caused unexpected draindowns of l a Unit 2 safety injection tank on October 10 and 28,1997. The cause of this significant condition adverse to quality was not promptly identified and corrected, nor were corrective actions taken to preclude repetition until May 16,1998. )

2. The misalignment of the intemals of Valve 1 PSIA-V404 was not promptly corrected in that it was improperly reassembled on April 9,1998, which resulted in the potential for excessive reverse flow through this HPSI pump discharge check valve. This significant condition adverse to quality was not identified and corrected until May 15, 1998.(01023)

C. 10 CFR Part 50, Appendix B, Criterion V, requires, in part, that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances, and that the instructions, procedures, or drawings include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily accomplished.

Contrary to the above:

1. During the period of July 26,1992, through April 11,1998, Procedures 73ST-9XI29, "Section XI Check Valve Operability Verification - Mode 6 - Full Stroke Testing of Safety injection Check Valves," and 73ST-9XI33, "HPSI Pump and Check Valve Full Flow Test," failed to include appropriate quantitative or qualitative acceptance criteria to determine that the check valves would close satisfactorily during reverse flow condition. Because of the lack of appropriate acceptance criteria, the licensee failed to identify that Valves 1 PSIA-V404,2PSlB-V405, and 3 PSIA-V404, would not close I properly during re /erse flow conditions for approximately 6 years, 5 years, and 1%

years respectively.

2. During the period of November 30,1992, through November 17,1994, Procedure 31MT-9ZZ17, " Disassembly and Reassembly of Borg-Wamer Check Valves," was not appropriate to the circumstances in that the procedure did not ensure correct vertical alignment and resulted in the disk assemblies of Valves 1 PSIA-V404,2PSIB-V405, and 3 PSIA-V404 being incorrectly positioned in the body of the valve. The incorrect disk positioning caused improper seating of the valves during reverse flow conditions which resulted in the potential for excessive reverse flow through the valves. (01033)

ENCLOSURE 1 PAGE 2 A P

1 ENCLOSURE 2 REPLY TO NOTICE OF VIOLATION ,

NRC INSPECTION REPORT NO.

50 4 28/98-14; 50429/98-14; 50 430/98-14

NRC 5:;:I+5 Report No. 60-824/98-14; 50429/98-14; 60430/98-14 EA 94 342 REPLY TO NOTICE OF VIOLATION "A" Admission or Denial I APS admits the alleged violation occurred.

l Reason For the Violatiori Violation A cites five examples where action was not taken to either restore the High l Pressure Safety injection (HPSI) subsystem to an operable status or place the unit in  !

the applicable mode in accordance with Technical Specification 3.5.2 or 3.0.3. The 1

reason that the violation occurred was that Operations department personnel did not i l

recognize that the subsystem flowpath was inoperable because they relied on I 1

l inadequate surveillance testing procedures to demonstrate operability.

l The fact that some of the HPSI pump discharge check valves (hereafter " check valves")

l had been assembled incorrectly was concealed by surveillance testing which i demonstrated (albeit falsely) that the HPSI subsystem would perform its intended I l

function. Acting on what was believed to be acceptable surveillance test results, 1 Operations department personnel unknowingly allowed the units to operate outside of piant design and Technical Specification conditions. Once the condition had been identified by Palo Verde staff, appropriate action was taken to addiess operability, transportability and reportability. I i

Corrective Steps That Have Been Taken and the Results Achieved i immediate corrective actions were taken to assure all potentially affected check valve i discs were correctly aligned. By May 17,1998, each HPSI pump discharge check valve had been tested and demonstrated acceptable performance in accordance with the revised surveillance test and acceptance criteria.

ENCLOSURE 2 PAGE1OF8 4

NRC inspection Report No. 60424/94-14; 60429/98-14; 80430/94-14 EA 08-342 l

l Additionally, a surveillance test procedure has been developed and implemented to ensure the check valves perform properly. The procedure includes a component-level closure test and quantitative acceptance criteria to demonstrate acceptable

! performance.

Corrective Steps That Will Been Taken to Avoid Further Violations No additional corrective steps are necessary.

Date when Full Comoliance Will Be Achieved Full compliance was achieved on May 17,1998, when the affected check valves were verified to be operable.

i i

ENCLOSURE 2 PAGE 2 OF 6

, u m a NRC inspection Report No,80 824/9814; 80429/94-14; 8043W98-14 EA 98-342 3EPLY TOJ.OTICE OF VIOLATION "B" Admission or Denial APS admits that the alleged violation occurred. ,

1 l

Reason For the Violation Violation B cites two examples where prompt actions were not taken to identify, correct and preclude recurrence of significant conditions adverse to quality. Specifically, a cocked open check valve caused an unexpected loss of inventory in a Unit 2 Safety j injection Tank (SIT) on October 10 and 28,1997, and final corrective actions were not j completed until May 16,1998. Additionally, check valve 1PSIAV404 was improperly assembled on April 9,1998 and was not corrected until May 15,1998.

The reason that the final corrective actions for the Unit 2 SIT inventory losses were not ,

1 promptly completed was that engineering personnel did not understand the significance j of the reverse flow through the HPSI pump discharge check valve. Specifically, engineering personnel understood that reverse flow through check valve 2PSIBV405 caused the SIT to drain down, but they incorrectly assessed the operability implications.

As part of the event evaluation, engineering personnel reviewed the results from the previous reverse flow test of check valve 2PSIBV405 and noted that the check valve had a reverse flow rate of approximately 30 gallons per minute (gpm). The 30 gpm reverse flow was determined to be acceptable because adequate HPSI subsystem flow had been obtained and there was no quantifiable reverse flow acceptance criteria in the surveillance test procedure to challenge check valve operability.

As a result of the event svaluation, engineering personnel initiated a corrective action document to evaluate whether a cautionary note should be added to the boration procedure to alert operators of the potential change in SIT level when performing the cold-leg borations. Engineering personnel initiated a work request to rework check valve 2PSIBV405 during the next refueling outage, and an acceptance criteria of 20 ENCLOSURE 2 PAGE3OF8

l NRC inspection Report No. 80 824/98-14; 80429/98-14; 80430/98-14 EA $8-382 gpm reverse flow was added to the surveillance test procedure to validate check valve closure. These corrective actions were taken even though surveillance testing appeared to demonstrate system operability.

The reason that valve 1PSIAV404 was improperly assembled on April 9,1998, and not corrected until May 15,1998 was personnel error. Specifically, a measurement error resulted in maintenance technicians incorrectly determining the vertical offset of the check valve disc.

Corrective Steos That Have Been Taken and Results Achieved immediate corrective actions were taken to assure all potentially affected check valve discs were correctly aligned. By May 17,1998, each HPSI pump discharge check valve had been tested and demonstrated acceptable performance in accordance with the revised surveillance test and acceptance criteria. ,

The maintenance instruction for check valve assembly had been revised on November 7,1994 and provided adequate instructions for the disc alignment activities performed on April 9,1998. However, this maintenance instruction has been further enhanced to prevent this type of personnel error from recurring by providing more detailed instructions and drawings.

Engineering personnel performed a transportability review to determine if other Borg-Warner bonnet hung pressure seal check valves were susceptible to the vertical disc misalignment condition. The transportability review confirmed that the vertical misalignment condition was limited to the HPSI pump discharge check valves.

Engineering personnel performed a transportability review of other Inservice Testing (IST) check valves with safety closure functions that may be susceptible to disc cocking. Based on the transportability review, Engineering personnel concluded that ENCLOSURE 2 PAGE4OF8

NRC inspection Report No. 60428/98-14; 60-62919814; 60430f98-14 EA 98-382 existing surveillance testing was adequate to verify no additional Borg-Warner check valves were cocked.

Engineering personnel reviewed other IST check valve surveillance test procedures to determine if closure acceptance criteria was appropriate. Based on the review, Engineering concluded that with one exception, the other IST check valve surveillance test procedures were adequate. The surveillance test procedure that contained the inappropriate acceptance criteria was revised and the affected valves satisfactorily passed the revised test.

Corrective Steps That Will Be Taken To Avoid Further Violations No additional corrective steps are necessary.

Date when Full Compliance Will Be Achieved Full compliance was achieved on May 17,1998, when the affected check valves were verified to be operable.

l l

l ENCLOSURE 2 PAGE 5 OF 8

l .

NRC Inspectkm Report No. 804W88-14; 804W88-14; 804W84-14 EA 88-382 l-REPLY TO NOTICE OF VIOLATION "C" Admission or Denial APS admits the alleged violation occurred.

Reason For ths Violation l l

Violation C cites instances where maintenance procedure 31MT-9ZZ17, " Disassembly )

and Reassembly of Borg-Warner Check Valves," was not appropriate to the circumstances in that the procedure did not ensure correct vertical alignment of check valve disc assemblies and surveillance test procedure 73ST-9XI33, "HPSI Pump and Check Valve Full Flow Test," failed to include appropriate quantitative or qualitative acceptance criteria.

The reason the maintenance reassembly procedure did not ensure correct disc vertical l alignment was that it was developed from a vendor maintenance manual that did not contain disc alignment techniques. The vendor technical manual, which was revised in l

October,1990, did not contain adequate instructions to ensure correct valve assembly. j The correct factory assembly techniques required the retainer ring to be backed out l

l from the bottomed out position until the disc was centered on the valve seat.

l l The reason the surveillance testing procedure did not include appropriate acceptance l

cnteria was that a specific quantifiable acceptance criteria for check valve reverse-I direction flow had not been incorporated into the surveillance test procedure until just l prior to Unit 1 refueling outage in March 1998. Prior to the incorporation of the reverse flow acceptance criteria, HPSI subsystem performance had been evaluated by its ability to deliver the required forward flow. However, this assumption was flawed in that it was based upon flow delivered under ambient pressure conditions, and did not correctly factor the effect of accident pressures into the testing results.

ENCLOSURE 2 PAGE 6 OF 8

~'

NRC Inspection R: port No. 60-628198-14; 60-629/98-14; 60-630/98-14 EA 98-382 Corrective Steps That Have Been Taken and Results Achieved immediate corrective actions were taken to assure all potentially affected check valve discs were correctly aligned. By May 17,1998, each HPSI pump discharge check valve had been tested and demonstrated acceptable performance in accordance with the revised surveillance test and acceptance criteria.

The maintenance instruction for HPSI pump discharge check valve assembly had been revised on November 7,1994 and provided adequate instructions for the disc alignment activities performed on April 9,1998. However, this maintenance instruction has been further enhanced to prevent this type of personnel error from recurring by providing more detailed instructions and drawings.

Engineering personnel performed a transportability review to determine if other Borg-Warner bonnet hung pressure seal check valves were susceptible to the vertical disc misalignment condition. The transportability review confirmed that the vertical misalignment condition was limited to the HPSI pump discharge check valves.

Engineering personnel performed a transportability review of other Inservice Testing (IST) check valves with safety closure functions that may be susceptible to disc cocking. Based on the transportability review, Engineering personnel concluded that existing surveillance testing was adequate to verify no additional check valves were cocked.

Engineering personnel reviewed other IST check valve surveillance test procedures to determine if closure acceptance criteria is appropriate. Based on the review, Engineering concluded that with one exception the other IST check valve surveillance test procedures were adequate. The surveillance test procedure that contained the inappropriate acceptance criteria was revised and the affected valves were retested satisfactorily.

ENCLOSURE 2 PAGE 7 OF 8

NRC Inspecslon Report No. 50428198-14; 5042919814; 60430/98-14 EA 98-382 l Corrective Steps That Will Be Taken To Avoid Further Violations No additional corrective steps are necessary.

Date when Full Compliance Will Be Achieved Full compliance was achieved on May 17,1998, when the affected check valves were verified to be operable.

l I

l t

l l 1 i

i l

l

- ENCLOSURE 2 PAGE 8 OF 8

... *. 1 ENCLOSURE 3 Reply to a Notice of Violation and Payment of Civil Penalty i (EA 98-382)

,