ML17303A606

LER 87-024-00:on 870902,two Dma Interface Cards Providing Input to Core Operating Limit Supervisory Sys Found Incorrectly Installed.Caused by Cognitive Personnel Error. Computer Technician counseled.W/871001 Ltr

ML17303A606
Person / Time
Site:	Palo Verde
Issue date:	10/01/1987
From:	Bradish T, Haynes J ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR
To:	NRC OFFICE OF ADMINISTRATION & RESOURCES MANAGEMENT (ARM)
References
192-00284-JGH-T, 192-284-JGH-T, LER-87-024, LER-87-24, NUDOCS 8710080068
Download: ML17303A606 (12)

Text

REGUL INFORMATION DISTRIBUTI YSTEM (RIDS)

ACCESSION NBR: 8710080068 DOC. DATE: 87/10/01 NOTARIZED: NO 'OCKET 5 FACIL: STN-50-528 Palo Verde Nuclear Stations Unit 1 Arizona Publi 05000528

~

AUTH. NAME AUTHOR AFFILIATION BRADISHi T. R. Arizona Nuclear Power Prospect (formerly Arizona Public Serv HAYNES'. G. Arizona Nuclear Poeer Prospect (formerlg Arizona Public Serv RECIP. NAME RECIPIENT AFFILIATION

SUBJECT:

LER 87-024-00: on 870902'wo DMA interface cards providing input to core operating limit supervisory sos found incorrectly installed. Caused bg cognitive personnel error Comp uter technic ian counsel ed. W/871001 l tr.

DISTRIBUTION CODE: IE22D COPIES RECEIVED: LTR ENCL SIZE:

TITLE: 50. 73 Licensee Event Report (LER) i In'cident Rpt> etc.

NOTES: Standardi zed plant. 05000528 REC IP IENT COPIES RECIPIENT COPIES ID CODE/NAME LTTR ENCL ID CODE/NAME LTTR ENCL PD5 LA 1 1 PD5 PD 1 1 LICITRAiE 1 1 DAVIS> M 1 INTERNAL: ACRS MICHELSON 1 1 ACRS MOELLER 2 2

• EOD/DOA 1 1 AEOD/DSP/NAS 1 1 AEOD/DSP/ROAB 2 2 AEOD/DSP/TPAB 1 1 DEDRO 1 1 NRR/DEST/ADS 0 NRR/DEST/CEB 1 1 NRR/DEST/ELB 1 1 NRR/DEST/I CSB 1 1 NRR/DEST/MEB 1 1 NRR/DEST/MTB 1 1 NRR/DEST/PSB 1 NRR/DEST/RSB 1 1 NRR/DEST/SGB 1 NRR/DLPQ/HFB 1 1 NRR/DLPQ/QAB 1 NRR/DOEA/EAB 1 1 NRR/DREP/RAB 1 1 NRR/DREP/RPB 2 2 NR D S/SIB 1 NRR/PMAS/ ILRB 1 1 REG FILE 02 1 1 RES DEPY GI 1 1 RES TELFORD. J 1 1 RES/DE/EIB 1 RGN5 FILE 01 1 EXTERNAL: EGGG GROH> M 5 5 H ST LOBBY WARD 1 LPDR 1 1 NRC PDR. 1 1 NSI C HARR ISe J 1 1 NSIC MAYST G 1 NOTES:

TOTAL NUMBER OF COPIES REQUIRED: LTTR 46 ENCL 45

I I

1 t

I l

NRC Form 355 US, NUCLEAR REOULATORY COMMISSION (933 I APPROVED OMB NO. 3)500104 LICENSEE EVENT REPORT {LER) EXPIRES; 5/31/SB FACILITY NAME (I) DOCKET NUMBER (2) PA 3 nit 1 0 5 0 0 0 1 OF 4 Core Operating Limit Supervisory System Rendered Inoperable Due to Reversed Circuit Cards EVENT DATE ISI LER NUMBER (SI REPORT DATE (7) OTHER FACILI1IES INVOLVED (SI MONTH OAY YEAR YEAR SERVE NTIAL IIEvsstors DAY FACILITYNAMES DOCKET NUMBER(S)

cscyh NUMBER NUMSESI MONTH YEAR N/A 0 5 0 0 0 0 902 8787 0 2 4 0 1001 8 7 N A 0 5 0 0 0 THIS REPORT IS SUBMITTED PURSUAN'1 T 0 THE REOUIREMENTS OF 10 CFR (): (Check one or mori of thi follow/no/ (11 OPERATINO MODE ( ~ ) 20.402(el 20.405(c) 50.734((2( l iv) 73.71 (II)

POWER 20.405 (~ ) (I I (il 50.35(el(1( 50.734) (2) 4) 73.71(c)

LEYEL 2 0 20.405(el(1)(ii) 50.35(c) (2) 50.73(e)(2l(rQI OTHER ISpec/fy In Ahstrect below era/ /n Teat, HIIC Form 20.405( ~ I (11(oil 50.73(e) (2)(i) 50.734) (2)(r(EI(AI 3$ SAI 20.405( ~ II1)(ir) 50.734) (2)(E) 50.7 3(e I (2)(riiiI I 5 I 20.405( ~ I(1)(v) 50.7 34)(2) I IE I 50.73(e)(21(al LICENSEE CONTACT FOR THIS LER (12)

NAME TELEPHONE NUMBER AREA CODE Thomas R. Bradish, Compliance Su ervisor 602 393- 353 1 COMPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRI ~ EO IN THIS REPORT (13)

CAUSE SYSTEM COMPONENT MANUFAQ MANUFAC.

TURER CAUSE SYSTEM COMPONENT TURER

4 Bl)NA))".c(p-saic.

IGC PU 2 6 N X IGC PU 2 6 N SUPPLEMENTAL REPORT EXPECTED (14) MONTH DAY YEAR EXPECTED SUBMISSION DATE (15)

YES Ilf yeL COmpleie EXPECTED St/Sht/SSIOSI DATE/

X NO AssTRAGT ILimlt to te00 spaces, I 4, epproaimetely fifteen ssnpre speci typewri Hen IinNI (15)

On September 2, 1987 at 0815 MST, with Unit 1 in Mode 1 (POWER OPERATION) operating at approximately 59 percent power, it was identified that two DMA Interface Cards providing input to the Core Operating Limit Supervisory System (COLSS) had been incorrectly installed. This rendered the COLSS inoperable.

Technical Specifications 4.2.1.2 and 4.2.4.2 which require monitoring of certain reactor core operating characteristics every 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> when COLSS is inoperable above 20 percent rated thermal power were not met beginning at 0106 through 0815 on September 2, 1987.

The root cause of the event has been determined to be a cognitive personnel error by a computer technician who inadvertently reversed the two cards during troubleshooting under approved work order documents. The technician did not obtain independent verification of the serial numbers and addresses for the cards which were being changed during the troubleshooting efforts. This is contrary to an approved procedure (work order).

As immediate corrective action, the cards were returned to their correct locations and COLSS was restored to an operable status. In order to prevent recurrence, the computer technician has been counselled on the importance of the accuracy of his work as well as when it is necessary to have independent verifications conducted.

There have been no previous similar events reported.

8710080068 871001 PDR ADOCK 05000528 NRC Perm 345 S PDR

NRC Form 455A 19451 US. NUCLEAR REOULATORY COMM)55)ON LICENSEE EVENT REPORT (LER) TEXT CONTINUATION APPROYEO OMS NO 5)EO~ICe EKPIRES; 4)S) lSS FACILITY NAME III OOCKET NUEISER LT)

LER NUMSEII I ~ ) ~ AOE LT)

YEAR 5E QV 5 NTI*L r) 5 v l5 Io rr NVM ER RVM ER Palo Verde Unit TEKT llfmare eaece )I reaaeerE eee ~ 1 H)IC Farm JSSA'5) ) IT) 0 5 0 0 0 5 2 8 8 7 024 0 0 2 oF 0 On September 2, 1987 at 0815 MST, with Unit 1 in Mode 1 (POWER OPERATION) operating at approximately 59 percent power, Interface Cards (CPU) had been incorrectly installed in Remote Input Subsystems it was identified that two DMA (RIS) "B" and "D". This resulted in the Core Operating Limit Supervisory System (COLSS)(IG) receiving reversed incore detector (DET) signals for detectors processed through RIS's "B" and "D", thereby rendering COLSS inoperable. With COLSS inoperable, Technical Specification Surveillance Requirements (TSSR) 4.2.1.2 and 4.2.4.2 require monitoring of the reactor core (AC) operating characteristics Linear Heat Rate (LHR) and Departure from Nucleate Boiling Ratio (DNBR) margin every 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> when the unit is operated at greater than 20 percent rated thermal power. Unit 1 increased power above 20 percent at approximately 0106 on September 2, 1987. Therefore, for approximately 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> COLSS was inoperable and the TSSRs were not met. In addition, the ACTION Statement for Technical Specification 3.3.3.2 was exceeded since approximately 50 percent of the incore detectors were rendered inoperable due to the reversed cards.

While conducting routine monitoring of plant status during startup of Unit 1 on September 1, 1987, Operations identified that portions of the COLSS data being obtained were not as expected, but were still within the Technical Following discussions with -Reactor Engineering, Unit 1 continued power Specification'imits.

ascension beyond 20 percent. On the morning of September 2, 1987, Reactor Engineering was comparing the results of the previous surveillance tests for incore detectors (72ST-9RX08) with data obtained at approximately 8 percent power and identified a potential discrepancy. Incore detectors which were known to be out of service were now operating and detectors which had been operating were now out of service. The Operations Computer Department investigated the problem and discovered that two DMA Interface Cards were reversed. This resulted in the computer (CPU) system considering RIS "B" incore detectors to be in the "D" quadrant of the reactor core and RIS "D" detectors in the "B" quadrant. The discovery of the improperly installed cards resulted in the COLSS being declared inoperable at 0815.

The root cause of the event has been determined to be a cognitive personnel

'error by a computer technician (utility non-licensed) who inadvertently reversed the cards while troubleshooting under approved work order documents. The troubleshooting consisted of changing cards between different RISs to determine which, if any, card(s) were not functioning properly. The technician did not obtain independent verification of the serial numbers and addresses for the cards which were being changed during the troubleshooting efforts. This is contrary to an approved procedure (work order). The cards which were identified as not functioning properly (Honeywell, Inc., Model Nos. 51302564 and 51302567) will be evaluated/reworked under the existing work control program.

"rRC rORM 555k 19 45r

jl I

NAC Porrrr 3CCA 19S) 31 U.d, NUCLEAR AEOULATOAT COMM)SCION LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPAOVEO OMd NO 3IEO~ICS EXP)RES) 3/3) IIX)

FACILITY NAME III OOCKET NUMCEA 13)

LER NUMEEA Id) PACE )3)

YEAR CCOVCN'IIAL :Ir)< OCVICION NVM CA NVM CA Palo Verde Unit TEXT IIImort soon is ttlrttsL t>> ostsooo) 1 PIAC forrII 3CC)A'I) I 17) o s o o o 528 87 024 0 0. 03oFO 4 As immediate corrective action, the required surveillance test (72ST-9RX03) was initiated at 0820 to meet the applicable Technical Specification ACTION Statements for the inoperable COLSS. Concurrently, the computer technician reinstalled the two cards in their correct locations and obtained independent verification of this action. Independent verification was also performed on those cards which had been removed/reinstalled during the troubleshooting work orders to ensure they were in their correct locations. The incore detector surveillance test (72ST-9RX08) was successfully completed and COLSS was declared operable at 1256 on September 2, 1987.

As corrective action to prevent recurrence, the technician has been counselled on the importance of the accuracy of his work as well as when it is necessary to have independent verifications conducted. As a prudent measure, the work order forms utilized for removal/reinstallation of components will be evaluated for adequacy. Upon completion of this evaluation, computer technicians who perform similar types of work will be briefed on the event, the requirements for independent verification of their work, and planned changes to the removal/reinstallation forms, if any.

The COLSS provides the plant operators a means of directly monitoring the reactor core status (LHR and DNBR) in order to ensure that the Technical Specification Limiting Conditions for Operation are maintained. However, it is the Core Protection Calculators (CPC)(JC) which utilize the incore detectors (DET) and initiate the automatic protective function (reactor trip) when the predetermined values for Local Power Density (same as Linear Heat Rate) and DNBR are exceeded. Although the values for LHR and DNBR margin could not be correctly monitored from 0106 through 0815 using COLSS, the CPCs were unaffected by the mispositioned cards and available during the event to provide the automatic protective function.

CPC Channel "D" (which represents approximately one quadrant of the reactor core) was monitored during the event by the Control Room Operators every 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> for Local Power Density (LPD) and DNBR margin as required by Operating Department Guideline No. 46. These values were within the Technical Specification values. The shiftly surveillance test (41ST-1ZZ33) which was performed at 2325 on September 1, 1987 prior to the event and at 1235 on September 2, 1987 after the event verified that the maximum deviation between the four channels of LPD and DNBR margin were within the limits specified in the surveillance test. A review of the maximum deviation values and CPC Channel "D" monitoring results indicate that the values for LPD and DNBR for the other three CPC channels would have been within the limits specified in the Technical Specifications. Also, the values which were obtained for LHR and DNBR margin during the performance of surveillance test 72ST-9RX03 following the discovery of the reversed cards were found to be within the acceptance criteria for all four CPC channels. Therefore, this event posed no threat to the health and safety of the public or to the safe operation of the plant, NIIC I OIIM 555k 19 CTI

r'l II t

NAC form 344A U.S. NUCLEAR AEQULATORY COMMISSION (9 83)

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION APPAOVED OMS NO. 3150MIOS EXPIRES: 8/81/88 FACILITY NAME III DOCKET NUMSER )QI LER NUMSER )4) PACE C))

YEAR SEQUENT/AL )yet NEVIS/ON NUM tr)  :~O NI/M Palo Verde Unit TEXT /// moro sof Io /I tot/irod. rrw ~ 1 H/IC form 8//SAT/ l)1) 0 5 0 0 0 5 2 8 8 7 024 components, or systems that were inoperable at the 000 4 oF 0 There were no structures, start of the event, other than those previously described, that contributed to the event. There were no unusual characteristic's of the work location which contributed to the event. There were no automatic or manually initiated safety system responses. Should other concerns or information pertinent to this event be discovered, a supplement to this report will be issued.

There have been no previous similar events reported.

4oc roAM ssso 19 8) I

Arizona Nuclear Power Project P.O. BOX 52034 ~ PHOENIX, ARIZONA 85072-2034 192-00284-JGH/TRB/TJB October 1, 1987 Document Control Desk U.S. Nuclear Regulatory Commission D.C. 20555 'ashington,

Subject:

Palo Verde Nuclear Generating Station (PVNGS)

Unit 1 Docket No. 50-528 Licensee Event Report 1-87-024-00 File: 87-020-404 Attached please find Licensee Event Report (LER) No. 1-87-024-00 prepared and submitted pursuant to 10CFR 50.73. In accordance with 10CFR 50.73(d), we are herewith forwarding a copy of the LER to the Regional Administrator of the Region V Office.

If you have any questions, please contact T. R. Bradish, Compliance Supervisor at (602) 393-3531.

Very tr ly yours, UA uy~~

J. G. Haynes Vice President Nuclear Production JGH/TJB/cld Attachment cc: 0. M. DeMichele (all w/a)

E. E. Van Brunt, Jr.

J. B. Martin R. C. Sorenson E. A. Licitra A. C. Gehr INPO Records Center