ML18096B245

LER 92-017-00:on 921213,unrecognized Loss of Overhead Annunciator Sys Alarm Indication Occurred.Caused by Personnel Error & Design,Mfg,Const/Installation Deficiency. Personnel Disciplined & Design assessed.W/930129 Ltr

ML18096B245
Person / Time
Site:	Salem
Issue date:	01/29/1993
From:	Pollack M, Vondra C Public Service Enterprise Group
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
LER-92-017-01, LER-92-17-1, NUDOCS 9302110009
Download: ML18096B245 (7)

Text

e PS~G Public Service Electric and Gas Company P.O. Box E Hancocks Bridge, New Jersey 08038 Salem Generating Station January 29, 19931 U. s. Nuclear Regulatory Commission Document Control Desk Washington, DC 20555

Dear Sir:

SALEM GENERATING STATION LICENSE NO. DPR-75 DOCKET NO. 50-311 UNIT NO. 2 LICENSEE EVENT REPORT 92-017-00 This Licensee Event Report is a voluntary report made pursuant to the Code of Federal Regulations 10CFR 50.73.

Sincerely yours,

c. A. Vondra General Manager -

Salem Operations MJP:pc Distribution 050038 9302110009 930129 PDR ADOCK 05000311 s PDR The Energy People

NRC FORM 366 U.S. NUCLEAR REGULATORY COMMISSION 16-891 r~:

,_:;

APPROVED OMB NO. 3150-0104 EXPIRES: 4/30/92 ESTIMATED BURDEN PER RESPONSE TO COMPLY WTH THIS INFORMATION COLLECTION REQUEST: 50.0 HRS. FORWARD LICENSEE EVENT REPORT (LERI COMMENTS REGARDING BURDEN ESTIMATE TO THE RECORDS AND REPORTS MANAGEMENT BRANCH IP*530), U.S. NUCLEAR REGULATORY COMMISSION, WASHINGTON, DC 20555, AND TO THE PAPERWORK REDUCTION PROJECT (3150-01041, OFFICE OF MANAGEMENT AND BUDGET, WASHINGTON, DC 20503.

FACILITY NAME 111 l°OCKET NUMBER 121 PAGE 131 Salem Generatinq Station TITLE 141

- Unit 2 o 1s Io Io Io I 3111 ~ 1 loF o 16 Unrecognized loss of Overhead Annunciator System alarm indication EVENT DATE 151 LER NUMBER 161 REPORT DATE 171 OTHER FACILITIES INVOLVED (Bl SEQUENTIAL MONTH DAY YEAR YEAR }{:' NUMBER }] REVISION NUMBER MONTH DAY YEAR FACILITY NAMES DOCKET NUMBERISl 01s1010101 I I 1 12 il3 9 2 912

- ol ii?

- olo 0!1 2 19 913 01s1010101 I I OPERATING THIS REPORT IS SUBMITTED PURSUANT TO THE R"-OUIREMENTS OF 10 CFR §: (Ch*ck on* or mor* of th* following/ 1111 POWER MODE 191 1

Z0.402(bl 20.4061*111 llil -- 20.405lcl 50.311lcll1l 50,73111121livl 50.73111121M ------x 73.71lbl 73.71 lcl LEVEL 1101 0 9 9 20.4051*11111lil 20.4081*111 llilll 50.311lcll21 50.731*11211il . ..

- 50.731111211viil 50.7311112llviiillAI

- OTHER (Specify in Abstr1ct bolow *nd in Tt1xt. NRC Form J66AI

,... 20.40611111 llivl 20.40811111 JM

- 50.7311112lliil 50.731*11211iiil LICENSEE CONTACT FOR THIS LER 1121

- 50.7311112llviiillBI 50.731*112llxl Voluntary NAME TELEPHONE NUMBER AREA CODE M. J. Pollack - LER Coordinator 61019 31319 I- 12 I 01212 COMPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRIBED IN THIS REPORT 1131 MANUFAC- REPORTABLE :-:*:*:-:*:-:.:*:*:*:*:*:*:*:*:*:*:.:*:*:-:*:-:*:*:*.

-:-:-:-:-:*:.:*:*:*:*:*:*:*:.:*:*:*:-:*:*:*:*:*:*:* MANUFAC* REPORTABLE CAUSE SYSTEM COMPONENT TO NPRDS CAUSE SYSTEM COMPONENT TUR ER TUR ER TO NPRDS ::*:.:,,,:.::::*'Ji'.:':.:,::::::::::*:::::::::::!:

'

:::::::::::;:;:::::::::::;:;:

i!!::::'j:*,*::*:=!*JJ.:::::::.:*:::::.*:.::::::::.*: .*.*.:,:,:,:,::*'**::.):**::.:,j,,,.:,:::'::::::':::

I I I I I I i ....,...,.............................

I I I I I I I

.;o;.;o;o;.;.;o

.;*:*:*:*:*:*:*:.:.:*:<*:<*:*:-:* *:*:*:*:*:*:*:*:*:*:<*:*:.:

• • • • • • • • • • • • • • -:*:.:*:*:-:*:*:*:*:*:* ........,...........,...... :::::?::::::::::::

I I I I I I I  :;:::::::::::::::::::::::::::::::::::::::::::::::::: I I I I I I I ."."*:*:*:*:*:*:-:*:.:.:*:*: ::::::::;:::::::;::::

SUPPLEMENTAL REPORT EXPECTED 1141 MONTH D y YEAR EXPECTED n ~

SUBMISSION DATE 1151 YES (If yos. comp/ato EXPECTED SUBMISSION DATE/ NO I I I ABSTRACT (Limit to 1400 S/JllCtJS, i.e.. 1Jpproxim*tt1ly fiftst1n 1inglt1*spact1 typewritten lines) (16)

On 12/13/92, at 2122 hours0.0246 days <br />0.589 hours <br />0.00351 weeks <br />8.07421e-4 months <br />, control Room personnel observed that Overhead Annunciators (OHAs) were not alarming upon receipt of alarm signals. The OHAs were returned to service at 2123 hours0.0246 days <br />0.59 hours <br />0.00351 weeks <br />8.078015e-4 months <br /> that day. They had stopped annunciating at 1946 hours0.0225 days <br />0.541 hours <br />0.00322 weeks <br />7.40453e-4 months <br />. The Auxiliary Alarm System and other Control Room alarms and indicators continued to function. The root cause of this event is "Design, Manufacturing, Construction/Installation". The OHA system did not provide indication to the control room operator that the system had been reconfigured to a non operational mode preventing OHA alarm actuation. This manifested when Operations personnel did not follow procedures in assessing the cause of the spare OHA A~45 annunciations. The main controller will stop sending events to any connected display devices when a specific combination of commands are entered into the computer. Operations personnel involved in this event have been disciplined. This event will be reviewed with applicable personnel. This event will be reviewed by the Nuclear Training Center.

A third party assessment of the Beta OHA System design modification is being performed. A design modification is being prepared to install an independent alarm circuitry system to monitor OHA operation. Procedure S2.0P-SO.ANN-0001 was revised. OHA System preventive maintenance and corrective maintenance procedures will be developed. Abnormal Operating Procedures, for OHA system partial or total loss, have been issued.

NRC Form 366 16-891

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION Salem Generating Station DOCKET NUMBER LER NUMBER PAGE Unit 2 5000311 92-017-00 2 of 6 PLANT AND SYSTEM IDENTIFICATION:

Westinghouse - Pressurized Water Reactor Energy Industry Identification System (EIIS) codes are identified in the text as {xx}

IDENTIFICATION OF OCCURRENCE:

Unrecognized loss of Overhead Annunciator System alarm indication Event Date: 12/13/92 Report Date: 1/29/93 This report was initiated by Incident Report No.92-822.

CONDITIONS PRIOR TO OCCURRENCE:

Mode 1 Reactor Power 99% - Unit Load 1170 MWe DESCRIPTION OF OCCURRENCE:

On December 13, 1992, at 2122 hours0.0246 days <br />0.589 hours <br />0.00351 weeks <br />8.07421e-4 months <br />, Control Room personnel observed that Overhead Annunciators (OHAs) {IB} were not alarming upon receipt of alarm signals. As identified in the Sequence of Events section, the OHAs were returned to service at 2123 hours0.0246 days <br />0.59 hours <br />0.00351 weeks <br />8.078015e-4 months <br /> that day.

Investigation identified that the OHAs had stopped annunciating at 1946 hours0.0225 days <br />0.541 hours <br />0.00322 weeks <br />7.40453e-4 months <br /> (that day). The Auxiliary Alarm System (AAS) and other Control Room alarms and indicators continued to function. The Nuclear Regulatory Commission (NRC) was notified of the OHA system loss per Code of Federal Regulations 10CFR 50.72(b) (1) (v).

The OHA system electronics, for both Salem Units, were modified during the recently completed refueling outages (i.e., lRlO design modification completed on June 12, 1992 and 2R6 design modification completed on March 26, 1992). This new system is microprocessor based. It is manufactured by Beta Products Division of Hathaway Industries.

SEQUENCE OF EVENTS:

Date Time Event (Hours) 12/12/92 -1500 OHA A-45, a spare alarm, annunciates - the alarm is reset 12/13/92 1200 OHA A-45 annunciates - the alarm is not cleared in support of further investigation

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION Salem Generating Station DOCKET NUMBER LER NUMBER PAGE Unit 2 5000311 92-017-00 3 of 6 SEQUENCE OF EVENTS:

Time Event (Hours)

-1soo Operations personnel access the Beta Remote Control Workstation to obtain information on OHA A-45 1946 Beta System CRT display clock stops updating 1955 AAS prints "Chilled Water Expansion Tank Level Low"; the associated OHA for "AUX ALM SYS PRINTER" does not alarm Operations responds to the AAS printout by directing an Equipment Operator to fill the tank; the absence of the OHA actuation is not recognized by Operations 2008 Radiation Monitor channels 2R13A and 2R13B causes the "Radiation Alarm Process" alarm to actuate on the 2RP1 Control R90m panel; OHA A-6, "RMS TRBL",

does not alarm Operations personnel respond to the 2RP1 alarm 2122 The "Chilled Water Expansion Tank Level Low" alarm returns to normal. This prints out on the AAS typewriter.

NCO's recognize that the OHA A-41 does not annunciate and that the clock on the OHA CRT is indicating 1946 hours0.0225 days <br />0.541 hours <br />0.00322 weeks <br />7.40453e-4 months <br /> and not updating 2123 Operations personnel reset Sequential Event Recorders (SERs) "A" and "B"; four (4) OHA alarms annunciate

1. Annunciator Logic;

2. RMS Trouble;

3. 104 Panel Trouble; and

4. AAS Printer.

APPARENT CAUSE OF OCCURRENCE:

The root cause of this event is "Design, Manufacturing, Construction/

Installation" (per NUREG 1022, "Licensee Event Report Program"). The OHA System did not contain adequate protection to prevent inadvertent access to software control functions which would place the system in an indefinite "lockup" condition. Investigation identified that the

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION Salem Generating Station DOCKET NUMBER LER NUMBER PAGE Unit 2 5000311 92-017-00 4 of 6 APPARENT CAUSE OF OCCURRENCE: (cont'd) main controller will stop sending events to any display devices that are connected when a specific combination of commands are entered into the computer. This occurs when the RCW is in the "PROCOM PLUS" program and the "Black Box" switch is in RCW-A (rather than SER-A) and the "Cntrl L" command is entered twice.

A causal factor of personnel error was also a contributor to the event. The software design inadequacy was manifested when Operations personnel did not follow procedure S2.0P-SO.ANN-0001(Q), "Overhead Annunciators Operation", in assessing the cause of the spare OHA A-45 annunciations. Contrary to the procedure, Operators did not ensure the "Black Box" switch was in the "SER A" position. The operator then loaded the PC installed "PROCOM" software program and inadvertently pressed the "Cntrl L" keys twice, instead of the "Alt L" keys, resulting in the controller being in a:* "wait for information from the keyboard" mode.

Causal factors associated with the root cause of this event include:

1. OHA System lockup was not readily detectable. The OHA System did not provide a direct means to inform Operations personnel that the SER had been reconfigured such that it was no longer processing inputs through to the alarm windows.

2. The design specification for the Beta OHA System did not adequately specify software security.

3. Procedure S2.0P-SO.ANN-OOOl(Q) was inadequate. It implied that the Beta OHA System could not be affected without use of a password.

ANALYSIS OF OCCURRENCE:

The control room OHA System consists of a Betalog 4100 (a high performance sequential events recording system) a Betalog 1500 (a microprocessor based serial input distributed annunciator system, and a RCW Computer with printer. The OHA consists of ten (10) overhead boxes with forty-eight (48) windows per box and a redundant Control Room CRT.

The OHA system electronics, for both Salem Units, were modified during the recently completed refueling outages (i.e., lRlO design modification completed on June 12, 1992 and 2R6 design modification completed on March 26, 1992). This new system is microprocessor based. It is manufactured by Beta Products Division of Hathaway Industries. In addition to the principal design modification, the alarm window displays were rearranged, relabeled and system reflash capability modified. Also a CRT with keypad controls and new

A LICENSEE EVENT REPORT (LER) TEXT CONTINUATION Salem Generating Station DOCKET NUMBER LER NUMBER PAGE Unit 2 5000311 92-017-00 5 of 6 ANALYSIS OF OCCURRENCE: (cont'd) pushbutton/switches was installed on the control console.

The Updated Final Safety Analysis Report (UFSAR) states that the OHA System is not safety related. System alarms are not part of the plant protection scheme and failures cannot affect protective system operation.

Those component failures which would result in OHA System annunciation, during OHA system inoperability, were addressed by .

Control Room personnel as appropriate. Therefore, failure of the OHA System did not affect the health or safety of the public.

Review of operator response to this event identified that a procedure for partial or total loss of the OHA System did not exist at the time of this event. Abnormal Operating Procedures, Sl/S2.0P-AB.ANN-0001, which address this, have been issued.

CORRECTIVE ACTION:

Operations personnel involved in this event have been disciplined as appropriate.

This event will be reviewed with applicable Operations, Engineering and Technical Department personnel.

This event will be reviewed by the Nuclear Training Center for inclusion in applicable training programs.

The engineering department is performing a third party assessment of the Beta OHA System design modification. Appropriate corrective actions will be taken based on the assessment findings. The assessment includes the role of the Nuclear Computer Group's responsibilities for the review of design modifications and design specifications that involve digital systems. Other proposed digital system design change packages are being reassessed for adequacy of software design.

A design modification is being prepared to install an independent alarm circuitry system to monitor OHA operation. The design will provide OHA failure alarms in the Control Room.

Procedure S2.0P-SO.ANN-0001 (and the comparable Unit 1 procedure) was revised. It was revised to include resetting and testing the OHA System with an operability determination description.

OHA System preventive maintenance and corrective maintenance will be developed.

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION Salem Generating Station DOCKET NUMBER LER NUMBER PAGE Unit 2 5000311 92-017-00 6 of 6 CORRECTIVE ACTION: (cont'd)

Abnormal Operating Procedures, Sl/S2.0P-AB.ANN-0001, have been issued which address operator response to OHA System partial or total loss.

~~General Manager -

Salem Operations MJP:pc SORC Mtg.93-009