IR 05000498/1996012

Insp Repts 50-498/96-12 & 50-499/96-12 on 960108-22. Violations Noted.Major Areas Inspected:Events Associated W/Reactor Trip on 951218 & Malfunction of Motor-Operated Valve SI-16B During Surveillance Test on 951223

ML20149L459
Person / Time
Site:	South Texas
Issue date:	02/20/1996
From:	Tapia J NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:
Shared Package
ML20149L454	List: IR 05000498/1996012 ML20149L452 ML20149L456
References
50-498-96-12, 50-499-96-12, NUDOCS 9602260350
Download: ML20149L459 (23)

Text

<

.

e

, ENCLOSURE 2 i

U.S. NUCLEAP, REGULATORY COMMISSION

REGION IV

I

! Inspection Report: 50-498/96-12 50-499/96-12 Licenses: NPF-76 NPF-80 Licensee: Houston Lighting & Power Company P.O. Box 1700 Houston. Texas Facility Name: South Texas Project Electric Generating Station. Units 1 and 2 Inspection At: Matagorda County, Texas Inspection Conducted: January 8-22, 1996  ;

i Inspectors: S McCrory, Lead Inspector. Operations Branch  !

Division of Reactor Safety {

A. Gody Jr., Senior Resident Inspector at Comanche Peak Division of Reactor Projects M. Biamonte, Inspector, Human Factors Branch Office Nuclear Reactor Regulation Approved: ~ J f. Operations Branch 2_-2 o 'E Josep

~

I. Tapial Acting C Date Div' ion of Readtor Safet Insoection Summary Areas Insoected (Units 1 and 2): Special, announced inspection of the events associated with the reactor trip on December 18, 1995 and the malfunction of Motor-Operated Valve SI-16B during surveillance testing on December 23, 199 Results (Units 1 and 2):

Ooerations

. Following a reactor. trip on December 18, 1995, three control rods remained six steps withdrawn. Contrary to emergency operating procedure requirements, operators did not emergency borate. This was an apparent violation of Technical Specifications (Section 1.1).

i 9602260350 960220 PDR ADOCK 05000498

_ % . .- ..

PDR

-- -. -- ..- . - . - - - - - . - - - - . . - . - . - - - . - - .

...

.

-2-

External communication by the crew following the reactor. trip was

'

inconsistent with management expectations. The initial notification of licensee management was untimel The initial report to the NRC did not contain the fact that three control rods remained six steps withdrawn following the reactor trip (Section 1.3).

• During the reactor trip event. operators exhibited several deficient

behaviors that had been observed in previous inspections (Section 1.4).

* Excessive auxiliary feedwater flow to the steam generators created a negative-impact in that it required operators to manually decrease flow to prevent excessive cooldown seven minutes following the reactor tri This has been a long standing situation that disrupts the intended flow l of the emergency operating procedures (Section 1.5). j Maintenance l I

e A violation of Technical Specifications occurred when operators failed I to declare a surveillance test unsatisfactory after an acceptance {

criterion was not met as a result of a malfunction of a safety injection system valve (Section 2.5).

Enaineerina e The licensee's schedule for implementation of the molded case circuit breaker setpoint modifications was not a risk informed schedule

<

(Section 2.4).

Safety Assessment /0uality Verification

. The licensee's initial event review process did not adequately address the operator response to the reactor trip event (Sections 1.3 and 3).

• Deficiencies in human resource management during the peak vacation demand period created by the holiday season contributed to poor crew performance and poor initial licensee event review (Sections 1.3 and 3).

,

l Summary of insoection Findinas: l e An apparent violati.7 of Technical Specification 6.8.1 related to 3 failure to follow pNcedures was identified (Section 1.1).

. Violation 499/9612-01 was opened (Section 2.5).  ;

e Unresolved item 498:499/9529-01 was closed as a result of finding that 1 one violation and one apparent violation had occurred (Sections and 2.5).

• An Inspection Followup Item 498:499/9612-01 was opened (Section 1.5).

. .-. . .

...

.

.

-3 Attachments:

.

Attachment 1 - Persons Contacted and Exit Meeting

. Attachment 2 - Sequence of Events

.

Attachment 3 - Documents Reviewed

1 l

l l

l l

l l

l l

-- - - - -.- - _ . - - - . - . - . . . - . . .. - ._-

.

F l -4-l l

,

DETAILS l 1 FOLLOWUP (IP 92701) REACTOR TRIP ON DECEMBER 18, 1995 l This inspection was a-followup to a concern first reported in NRC Inspection Report 50-498/95-29: 50-499/95-29 and tracked as Unresolved

'

Item 498:499/95029-01. A detailed description of the event was included in Section 2.1 of that report. An updated sequence of events is provided in this

report as Attachment 2.

t On December 18, 1995, following a reactor trip with a partial loss of off-site l power., o]erators observed indications on the digital rod position indication i system tlat three control rods each remained six steps withdrawn. The crew

!

concluded that the rod position indications were inaccurate and that the three

,

rods were at the bottom limit of travel. That conclusion permitted the crew l to declare that all rods had fully inserted and proceed with emergency

! operating procedure implementation without initiating emergency boration. A l subsecuent evaluation on December 19. 1995, determined that the three control L rods cid in fact stop at six steps withdrawn following the reactor trip.

!

l The objective of the special inspection was to better understand the reasons l for the crew's determination that emergency boration was not required with

'

indication of three control rods six steps withdrawn following a reactor tri The inspectors were to further determine if there was evidence of a generic issue of inappropriate procedure interpretation or lack of adherenc .1 Ooeratina Crew Performance l The inspectors found that the operators on shift in Unit 1 during the reactor

! trip inappropriately determined that the indications of the digital rod

'

position indication system were invalid and then concluded that all control

.

rods were FULLY INSERTED by the indications available in the control room.

l There was no history of unreliable or inaccurate digital rod position indication system performance with all control rods inserted to the bottom limit of travel. Further, the consistent interpretation of the procedural recuirement had been to look for rod bottom 1ights lit. This interpretation hac been consistently reenforced through recurring trainin Finally, there was no safety significant basis to warrant consideration to refrain from initiatingemergencyboration. Therefore, the inspectors determined that the operators failure to emergency borate when three control rods each indicated six steps withdrawn was an apparent violation of Technical S)ecification 6. which requires, in part, that written procedures be establisled, implemented, and maintained covering the emergency operating procedures recuired to implement the requirements of NUREG-0737 and Supplement 1 to buREG-0737 as stated in Generic Letter 82-33.

'

The crew engaged in a number of deficient behaviors ranging from rejecting rod

! position indication as inaccurate without supporting information or 4 indications to manually starting emergency diesel generators on the basis of

,

invalid indications. The inspectors determined that operators at the South l

'

,

. - _ _ , _ _ _ _ , _ ~ - _ . . , .

. -

- . _ _ --

.

.

-5-

Texas Project had exhibited similar behavior on multiple occasions over the ;

past year (refer to Section 1.4 below). The inspectors concluded that a l negative trend in operator performance was confirmed by the actions of the l operators in response to the reactor trip on December 18. 199 .2 Safety Imoact l The inspectors agreed with the licensee's evaluation that there was negligible safety consequence resulting from the specifics of the reactor trip and subsequent control of the plant. The remaining reactivity worth of the control rods that did not fully insert was minimal. Further, as a result of automatic isolation of the letdown system the outlet valve of the volume control tank eventually shut on low level, causing an automatic shift of charging pump suction to the refueling water storage tank. That resulted in injection of water at a boron concentration greater than that of the reactor coolant system. However, the potential safety impact of the operators'

behavior, in the context of the indicated plant condition, was significan The success of symptom based emergency operating procedures has depended heavily on operators acting in response to the most reliable plant indications to determine an appropriate course of action. In this event, operators concluded that the only available indication of actual rod position was inaccurate without sufficient evidence to support that conclusion and did not initiate emergency boration as a consequence. This behavior is contrary to the NRC's expectation that operators at the South Texas Project would act conservatively when faced with unexpected plant condition .3 Human Factors Assessment A human factors assessment was conducted as part of the inspection of the event which occurred on December 18. 199 The assessment was made by conducting interviews with those operators involved with the event as well as operators in similar positions on other shift crews. by interviewing the plant manager and the operations manager, and by observing several meetings. The interviews focused on understanding the human performance aspects of the reactor trip, the licensee follow-up actions and event investigation, and included questions to determine the level of understanding of management expectations and to assess the degree to which those expectations were being me .3.1 Management Expectations I

l Interviews with individuals as well as a review of documents outlining

management expectations indicated that site management had specifically and consistently outlined expectations in the areas of problem identification, work focus, procedure use and improvement, and self-checkin Discussions in meetings, especially the plan-of-the-day and shift turnover, demonstrated appropriate reinforcement of management expectations. All interviewed operators stated that verbatim procedure compliance was expected with the exception of those items outlined in the emergency operating procedure user guide or as allowed under 10 CFR 50.54. All those interviewed indicated a

.

.

-6-

!

high degree of compliance to procedures despite the circumstances associated with the December 18, 1995, event. Based on the interview results and the meeting observations, the inspectors determined that there was a common understanding of management expectations in the areas of procedural adherence

and self-checking.

l 1.3.2 Crew Composition and Support Staffing i

Crew composition was investigated when interviewing the operating crew and l

'

operations management. Prior to the event, management had an informal policy regarding the number of substitute operators allowed on a shift. The policy called for no more than one senior reactor operator and one reactor operator

, to be replaced for any single shift. However, the crew in the control room l' for this event consisted of two substitute senior reactor operators, one l substitute reactor operator and a substitute shift technical advisor.

'

Licensee management cited a desire to allow holiday time off as a reason for not following their informal policy. The event occurred on the third night of

'

l the crew's rotation. By that time, the crew felt that everyone was accustomed i to working together during steady state plant conditions. Crew members did .

not Jerceive that the number of substitute operators was a contributing factor !

,

in t1e event. The shift supervisor indicated that dealing with the additional

! plant problems subsequent to the trip proved to be a challenging workload for l the crew but that the availability of plant operators from Unit 2 helped to

,

accomplish all tasks.

l l While reviewing the licensee event documentation, the inspectors observed that l all the principal participants to the decision that all control rods had fully l inserted were substitute crew members (the shift supervisor, unit supervisor, and 3rimary operator). Although the crew had time to become accustomed to one anotler during nearly three shifts of steady state operation, they were not exercised as a team to respond to abnormal or emergency conditions. When the unit supervisor read the step from the emergency operating procedure to verify all control rods fully inserted, the primary operator questioned the system indications with regard to the recuirements of the emergency operating procedure sten That precipitatec a brief discussion with the shift and unit supervisors w1erein they determined that the indications were within the allowable tolerances of the digital rod position indication system and thereby concluded that all rods were actually at the lower limit of insertio Therefore emergency boration was not initiated (previously discussed in Section 1.1). Through interviews with training personnel, the inspectors concluded that the behavior of the crew was uncharacteristic of the behavior reenforced through training regarding emergency operating procedure us Further. training had conditioned operators to look for all rod bottom lights lit to satisfy the requirements of Step 3 in Procedure 0 POP 05-E0-ES01.

<

" Reactor Trip Response." Virtually all operators interviewed confirmed a

'

consistent understanding (prior to the event) that Step 3 of Procedure ES01

-

required that all rod bottom lights be lit. Therefore, the inspectors

concluded that excessive substitution on the crew may have disrupted the expected team dynamic for abnormal and emergency events that led to behavior l inconsistent with that reenforced through trainin '

_ _ _ __

____ ._ _ __ . _ _ _ _ - . -

_ . . _ _ _ _ _ _ . . . _ _ _ _ _ _ _ . _ _ _ - . . _ _ . _ _. ..

, .

'

.

-7-

!

The initial event review appeared to also have been influented by off-normal support staffing. At the time of the event, the operations manager for Unit I was also assuming the responsibilities of the plant manager. The operations manager stated that having to cope with the responsibilities of two positions while trying to resolve the issues associated with this event delayed him from exploring operator performance issues. Additionally, some of the managers who were interviewed indicated that a significant portion of the initial event review team was comprised of individuals less experienced in event review than would normally have been assigne A senior manager indicated during an interview that the initial event review team appeared to focus on technical issues because the event review team members were more comfortable with technical issues than human performance issues. As a consequence, operators were not interviewed regarding the event during the first 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> as expected by management. The inspectors concluded that off-normal staffing of the initial event review team contributed to a lack of human performance evaluation. The lack of human performance evaluation may have contributed to management's initial assessment that the operators' performance in this event as adequat ,

1.3.3 Communication Communication within the crew, crew communication with their management, and crew communication with the NRC were all reviewed as part of the event investigatio Ineffective resource utilization made communication less that optima In the first hour after the reactor trip, the Unit 1 shift supervisor refused assistance when the Unit 2 shift supervisor offered to make the necessary notifications to the NRC or licensee management. The crew became so involved with establishing natural circulation within the reactor, as well as with handling secondary plant problems, that licensee management was not notified until about 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> and 10 minutes after the reactor trip. Ultimately, the on-coming shift supervisor assisted in compiling the information so that the NRC notifications could be made within the required 4-hour windo An off-normal management staffing arrangement, due to holiday time off may also have contributed to some communication problems. In the past, when notified of plant trips, the plant manager routinely offered to handle.some of the notifications for the shift supervisor, including notifications to the NRC resident inspector. However. since the plant manager was in transit to his vacation when he was notified, he did not offer to hel) with the noti fications. Although the shift supervisor stated tlat he clearly understood that he was responsible for making the necessary notifications, this off-normal response by the >lant manager represented a changed work-load for the shift supervisor from otler similar events. Based on this information, the inspectors determined that resource utilization appeared to impact the response to this even :

,

-

e , - - - ,- --

.

!

.

-8-

The inspectors were concerned with the completeness and accuracy of the initial report of the reactor trip made to the NRC Operations Center. During that communication, the shift supervisor reported that all rods had fully inserted following the trip. When the operations center asked questions to

,

!

confirm the accuracy of what they had recorded, the shift supervisor continued '

to report that all rods had fully inserted. When asked if all systems performed as expected, the shift supervisor did not report that the digital rod position indication system showed three rods at six steps out. During an interview, the shift supervisor reiterated that he believed that the control rod indications in the control room following the reactor tri) satisfied the procedure requirement for all rods to be fully inserted and tlat his report to the agency was consistent with his interpretation of plant indications. The inspectors concluded that there was no intent to 3rovide incomplete or inaccurate information to the NRC. but that the saift supervisor reported what he believed to be true. The inspectors further concluded that the shift supervisor was not aware of the level of detail desired by the NRC as part of an initial event notification and, therefore, did not report the unexpected rod position indicatio The inspectors concluded that external communications of the crew were untimely, and failed to identify information important to the NRC. Licensee management was not notified of the event for over an hour following the reactor trip. When the event was reported to the NRC Operations Center, the report did not include the fact that the digital rod position indication system displayed three control rods at six steps withdrawn.

4 1.3.4 Procedure Quality When interviewed, the plant and operations managers indicated that they thought the wording of Step 3 of the reactor trip response Procedure ES01, was ambiguous. The step states. " Verify all control rods FULLY INSERTED."

However, when the operators were asked how they normally verified this step they all responded that the rod bottom lights would be li The operators were also asked to compare and contrast Step 1 of Procedure E000, which requires the operators to verify that the reactor has shut down by verifying that. " Rod bottom lights are LIT." with Procedure ES01, Step 3. Procedure ES01. Step 3 allows operators to use judgement in making the determination of whether or not the reactor has shut down while Procedure E000. Step 1 calls for verbatim complianc All the operators were able to differentiate between the steps. They clearly understood the adequate shutdown margin basis for Step 3 in Procedure ES0 However, this knowledge, combined with their knowledge that the indicated position of the control rods was within the tolerance limits of the digital rod position indication system, a) pears to have influenced them to assume that they met the intent of Step 3 ratier than following the procedure verbati Based on the operators' common understanding of the steps, the inspectors determined that the wording in Procedure ES01. Step 3 was not ambiguou l l

l

.

i

)

.

-9-1.3.5 Training Operators indicated that training on the use of both Procedures E000 and ES01 had been extensive. However, the scenarios presented extreme cases of control rods failing to insert properly or moving out of step with others in a grou !

Under those conditions the tolerances of the digital rod position indication system were less of an issue and the necessary response was clea When !

presented with the control rod failures to insert the final 6 steps, the issue 1 of the digital rod position indication system tolerance was introduced into the problem and presented a situation for which the operators had not been specifically trained. Although the operators had not been specifically trained in this situation. the inspectors determined that the overall training on the emergency operating procedures and abnormal events was adequat .4 Prior 00erator Performance Findinas or Concerns The inspectors reviewed inspection reports and other documentation generated in the past year to determine if operator performance deficiencies similar to those displayed during the event had been previously observe The inspectors characterized those performance deficiencies as:

• Disregard or disbelief of indication . Operations without supervisory direction or concurrence, and

. Failure to detect inaccurate indication 'In the year of operation preceding the event, several findings were made that pointed to operator behaviors similar to those exhibited during this even .4.1 Disregard or Disbelief of Indications

The inspectors concluded that the o]erators responding to the Unit 1 reactor trip on December 18. 1995, did not ]elieve that the digital rod position indication system accurately displayed actual control rod position. A similar situation wherein operators did not believe indications was documented in NRC Inspection Report 50-498:499/95-23. The report stated that a fire occurred in the- packing area of Motor-Operated Valve 1-MS-MOV-0084 on the main reheat steam line to the moisture separator / reheaters. However, upon initial receipt of the report of the fire, control room operators delayed activating the fire brigade to require the reactor plant 03erator who reported the fire to confirm that there was an actual fire rather tlan a small steam leak in the valv .4.2 Operations without Supervisory Direction or Concurrence The inspectors concluded that the reactor operator who started the Emergency Diesel Generators 12 and 13 within 30 seconds of the reactor-trip on December 18, 1995. acted without proper supervisory concurrence. After referring to indication on a deengergized control panel, the reactor operator

_ __ __ _ _

- - _ . _ . _ _ _ _ .. _ ._ _ _ _ _..._.- _. _ _ _ . _ _ _ . _ _ _ _ _

l .

.

-10-

,

concluded that power had been lost to all engineered safety features busses l and started Emergency Diesel Generators 12 and 13 without obtaining the concurrence of either the unit or shift supervisors. This was inconsistent l with management expectations regarding independent operator performance.

.

'

Similar behavior had been reported in NRC Inspection Reports 50-498/95-23:

50 499/95-23 and 50-98/95-27: 50-499/95-2 . On August 18, 1995, a thermal power excursion above licensed limits occurred when a reactor operator arbitrarily increased the inservice time of the boron thermal regeneration system to control reactor coolant system temperature without the knowledge or concurrence of shift supervisio . On October 18. 1995. a licensed o3erator manipulated a safety-related pump control handswitch 3rior to )eing directed to during the test. when a different pump was to 1 ave been use . On October 27, 1995. the actuation of cold overpressure protection was attributed, in part. to a lack of supervisory oversigh .4.3 Failure to Detect Inaccurate Indications l The inspectors concluded that the reactor operator determined that power had been lost to all engineered safety features busses following the reactor trip on December 18, 1995, because he did not recognize that the indications were inaccurate as a result of loss of power to the control panel. Alternate instruments were readily observable in the control room which indicated that off-site 3ower had not been lost to the engineered safety features busses l supplied )y Emergency Diesel Generators 12 and 13. Similar performance was reported in NRC Inspection Report 50-498/95-23: 50-499/95-2 . On July 28. 1995, operators failed to recognize that Reactor Coolant Flow Instrument FT-0447 was reading out of tolerance. The condition I warranted placing a channel of the reactor protection system in the l l tripped condition. The condition was not recognized for more than 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />, which exceeded Technical Specification requirement . On September 5, 1995, a reactor o)erator failed to recognize that i

Radiation Monitor Instrument C2RA1T8034 had failed lo The condition

! required placing the control room ventilation system in recirculation.

, However, it went unrecognized for an additional 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> before it was

! identified as having failed low and declared inoperabl .5 Auxiliary Feedwater System Performance Imoact l Following the reactor trip, the reactor coolant system cold-leg temperature went from 564 degrees F to 536 degrees F in a period of about 10 minutes, i That represented a cooldown rate of 168 degrees F per hour. Since the main steam isolation valves were shut within a minute of the reactor trip, the

'

i cooldown was primarily the result of excessive auxiliary feedwater flow

__ _ _ _ -_ - . _ - - _ --.

.- _ _ - _ . _ _ _ _ _ . - - _ _ _ _ _ _ _ _ . _ . - . _ . _

!. .

i

.

!*

i -11-

.

i

(approximately 2500 gallons per minute total) to the steam generators. The cooldown rate and rapid refill of the steam generators forced the operators to i take manual control of auxiliary feedwater flow approximately 7 minutes after j the reactor tri .

Step 1 of Emergency Operating Procedure ES01 required the operators to verify reactor coolant system cold-leg temperature steady or trending toward

567 degrees F. If temperature was less than 550 degrees F and decreasing, the

, " Response Not Obtained" column for Step 1 required that emergency boration be initiated. Less than a minute after operators assessed reactor coolant system

.

cold-leg temperature greater than 550 degrees F and steady (approximately three minutes after the reactor trip). reactor coolant system cold-leg

temperature went below 550 degrees F. If reactor coolant system cold-leg i temperature had been less than 550 degrees F when the operators performed i Step 1 of Emergency Operating Procedure ES01, the operators would have had to i respond to an artificially created excessive cooldown event that resulted from

<

excessive auxiliary feedwater flow.

k The inspectors expressed concern over the impact that the auxiliary feedwater system performance created which compelled operators to take action outside the intended sequence of the emergency operating procedures. The licensee

, responded that operators were conditioned through training to monitor and

, control auxiliary feedwater flow following a reactor trip to minimize the  :

cooldown created by full auxiliary feedwater flow. That practice stemmed from

'

l l a study of contributors to excessive cooldown following a reactor trip, which

~

included auxiliary feedwater system 3erformance, conducted between March 1992 and March 1994. A memorandum dated iarch 31, 1992, and titled, " Rapid i Cooldown After Reactor Trips Requiring Closure of MSIVs." was provided to the inspectors. The memorandum identified contributors to excessive cooldown j following a reactor trip as.a percentage of full thermal power. The four  :

i largest contributors were:  !

. Steam drains - 5.4 percent.

4 . Excess auxiliary feedwater flow - 3.3 percen . Main steam reheat vents - 2.3 percent, and

. Maximum auxiliary steam flow - 2.1 percent (actual expected -

O percent).

, The memorandum provided a range of possible technical solutions that included i installing flow orifices in steam drain lines, automatically shutting main steam reheater supply or vent valves, automatically closing the auxiliary steam supply valves, and automatically tripping the feed pump turbines on

indications of excessive cooldown or reactor trip. The memorandum also discussed a variety of operator actions to achieve similar results to the

'

technical solutions. No specific technical recommendations regarding

,

auxiliary feedwater flow were mad i l

._- - . --. - - . . - . . - - - - - . - - - -

.

O-12-The study employed a cost / benefit analysis that used selected event probabilities and revenue generation amounts to derive a revenue impact for the identified options and combinations of those options. A baseline comparison value was derived from the "take no action" option. The final resolution was to install steam drain orifices and to rely on operator action to mitigate the remaining contributors to excessive cooldow The inspectors asked the licensee about any risk analyses that considered the various options originally identified. The licensee responded that unless a modification was being considered for implementation no analysis was performed. Therefore, since modifications to the auxiliary feedwater system were not considered, no comparative analysis was performed to gain insight to the risk significance of modifying the auxiliary feedwater system versus relying on o]erator action to control cooldown. Further the licensee provided notling that assessed the lack of operator availability to respond to complications due to devoting time and attention to monitoring cooldown and controlling auxiliary feedwater flo The inspectors concluded that the auxiliary feedwater system created an impact to emergency response. Operators were required to monitor and control flow in the early stages of emergency operating procedure implementation to prevent overcooling. The impact was not unique to the specific event but affects response to virtually all emergency situations as a result of performance characteristics of the auxiliary feedwater system. The licensee's decision to rely on operator actions to control the cooldown due to the auxiliary feedwater system is considered an inspection followup item (498:499/9612-01).

2 FOLLOWUP (IP 92701) MOV MALFUNCTION DURING SURVEILLANCE TEST 2.1 Backaround On December 23. 1995, while performing Step 5.6.26 of Procedure OPOP03-SP-00098. "SSPS Actuation Train B Slave Relay Test."

Revision 5. dated December 20. 1995, the breaker for the Containment Sump 2B to Safety Injection Train A pumps suction Isolation Motor- j Operated Valve SI-16B tripped. Since the successful stroke open of Valve I-16B was a required acceptance criteria of the surveillance test and Valve SI-16B did receive an open signal from the protection system, the Unit 2 shift supervisor authorized a surveillance test exception in accordance with Procedure OPGP03-ZE-0004. " Plant Surveillance Program." Revision 1 Step 4.4.6.1. and operators closed the tripped open Valve SI-168 breake )

Once the breaker for Valve SI-168 was closed. Valve SI-16B immediately stroked '

ope The shift supervisor directed that the solid state protection system slave relay test be restarted at Step 5.6.1. Valve SI-168 stroked properly during the subsequent performance of Step 5.6.26 and the solid state protection system slave relay test was completed. Operators noted the Valve SI-16B discrepancy in the remarks section of the solid state protection .

system slave relay test package and initiated Condition Report 95-14538. to ;

!

. - -

.

.

-13-i further evaluate the root cause of Valve SI 168 breaker trip. Later that

. evening, the shift supervisor and other licensee management determined during a conference call that the Valve SI-16B breaker failure had no operability

'

-

impact on the facilit On December 26, 1995, a Priority 3 work package was developed and approved.

The work package authorized maintenance personnel to take as-found measurements of the Valve SI-16B actuator (82SIMOV0016B) current readings and

. to inspect the Valve SI-16B molded case circuit breaker (E2B2 Cubicle M1) for damage in accordance with licensee Test Procedure OPMP05-NA-0004, " Molded Case

. Circuit Breaker Test."

On December 27, 1995. the Valve SI-168 molded case circuit breaker was .

replaced with a new one because maintenance personnel sus 3ected that the old i breaker was " weak." Test Procedure OPMP05-NA-0004 was suasequently performed and completed satisfactorily on the new breake .2 Imolementation of the Plant Surveillance Proaram l

,

'

The inspectors reviewed the shift supervisor's utilization of I Procedure OPGP03-ZE-0004. Step 4.4.6.1 to grant an exception to the solid I state 3rotection system slave relay test failure because Valve SI-168 did not meet t1e stroke-open acceptance criteria of Procedure OPSP03-SP-0009 ' Step 4.4.6 of Procedure OPGP03-ZE-0004 stated:

"E surveillance test results are unsatisfactory or do not meet the acceptance criteria, as specified by the surveillance procedure. THEN the surveillance is considered failed (unsatisfactory) and the Shift Supervisor is notified. Furthermore. E the test data obtained during l'

the performance of a surveillance test indicates that the acceptance

. criteria will not be satisfied. THEN the surveillance, once completed, is considered failed (unsatisfactory)."

The exception stated in Step 4.4.6.1 of Procedure OPGP03-ZE-0004 indicated that:

"An exception to Step 4.4.6 [ stated above) is allowed E during the-surveillance test a deficiency is noted that can and SHOULO be corrected / resolved by immediate test coordinator or operator actions without the generation of an additional / external documentation (i.e.,

CR. etc.) and not within the scope of existing deficiency programs (Ref. 5.31). In those cases the appropriate action (s) SHOULD be taken, the surveillance test completed and the corrected deficiency documented in the surveillance test data package. This step does not allow the Test Coordinator to bypass procedural requirements under existing maintenance rules and procedures."

. - - - _ - . - ..

.. - . . - . --.- . - - - - . - -~ .- -- .. . .-

.

.

-14-l The inspectors noted that Condition Report 95-14538 was written on December 23, 1995, to investigate the cause of the SI-16B breaker tri The inspectors questioned the licensee on their interpretation of Step 4.4. l particularly the portion which indicated that a test exception could be given I if the deficiency could be corrected without the generation of a condition ;

report. The licensee indicated that since the surveillance test was being I performed to verify that the solid-state protection system relays actuate the intent of the 3rocedure was met when the breaker was subsequently closed :

and the valve stroced. The licensee's position was that once the breaker was closed, the deficiency, at least as it related to the solid-state protection system slave relay test, was corrected. The inspectors were concerned because i

it appeared that the shift supervisor incorrectly used the process described ;

in Step 4.4.6.1 to grant an exception to the Valve SI-168 stroke open !

acceptance criteria in Step 5.6.26 of Procedure OPSP03-SP-0009 Further review by the inspectors revealed that molded case circuit breaker setpoints were already in a deficiency 3rogram and if the molded case circuit breaker setpoint was suspect, it would lave not been appropriate to utilize the exception described in Step 4.4. The inspectors concluded that the utilization of Procedure OPGP03-ZE-000 Step 4.4.6.1 to grant an exception to the solid state 3rotection system slave ,

relay test failure because Valve SI-16B did not meet t1e stroke-open acceptance criteria of Procedure OPSP03-SP-00098, was a failure to follow procedures and is considered a violation of Technical Specification 6. (499/9612-01). Troubleshootina and Root Cause Determination of SI-168 Failure The inspectors reviewed the completed condition report and developed several observations and concerns regarding the licensee's troubleshooting of the cause of the Valve SI-168 failur The first concern involved the limited scope of the troubleshooting of Valve SI-16B which according to the condition report. was limited to measurement of actuator peak and running current, and circuit breaker

setpoints. The inspectors believed that the troubleshooting plan did not

investigate sufficient information to accurately determine the root cause of l the breaker tri). Specifically, the inspectors were concerned that the l licensee's trou)leshooting did not consider the possibility of an intermittent locked or near-locked rotor condition on the Valve SI-16B actuator. The

licensee indicated that they had eliminated the locked rotor concern early in

!

their review of the breaker trip event and produced additional information to support their position. The additional information although not part of the condition report package, logically supported the licensee's decision to limit

!

troubleshooting to actuator current and breaker setpoints. The inspectors

concluded that, although the additional information provided by the licensee

'

i satisfied the. inspectors scope of troubleshooting concerns, the condition report package alone was not adequate to justify the limited troubleshooting performed, l

'

- , . -

.

.

-15-Another concern involved the utilization of Procedure OPMP05-NA-0004, to determine as-found data on the Valve SI-168 breaker. The inspectors noted

'

that Step 6.3.7 directs the electrician to cycle the breaker open and closed a minimum of five times to verify that there was no apparent binding. Although, the inspectors recognized that cycling breakers prior to testing them was a standard industry practice, the inspectors were concerned that utilizing this practice during the troubleshooting of a potential breaker failure could result in the loss of potentially valuable as-found information. The licensee indicated that by the time maintenance personnel even began troubleshooting, the Valve SI-168 breaker had already been cycled a number of times. The licensee indicated that since the breaker had already been cycled a number of times, it didn't matter if it was cycled a few more. The inspectors found that although this was most-likely true, the as-found information was never capture The third concern was noted when the inspectors read the condition report i disposition documentation and found that the breaker had tripped several times l during previous testing by motor-operated valve diagnostic testers who !

performed testing on Valve SI-16B following an actuator upgrade from a l SMB-type to an SB-type and a pressure locking modificatio '

Additionally, the inspectors noted that the as-left breaker trip setpoints for j the replacement breaker were actually slightly lower than the as-found breaker l setpoints on the old breake l l

The. instantaneous trip data for the SI-168 breakers taken in Step 6.6 of Procedure OPMP05-NA-0004, were as follow I Breaker As Left Dial Minimum Trip Maximum Trip As Left Trip l Phase Setting Current Current Current Old Breaker A phase / 2 78.75 amps 147 amps 100.1 amps Positive B-phase / 2 78.75 amps 147 amps 111.4 amps Negative C-phase 2 78.75 amps 147 amps 115.3 amps New I Breaker

. A-phase / 2 78.75 amps 147 amps 108.9 amps Positive B-phase / 2 78.75 amps 147 amps 105.7 amps Negative C-phase 2 78.75 amps 147 amps 96.8 amps ,

. q

.

-16-

,

'

The ins)ectors noted that the actual as-left instantaneous trip setpoint for the C-plase of the new breaker was 96.8 amps, which was only 1.26 times greater than its maximum voltage locked rotor current. NRC Information .

Notice 92-51, " Misapplication and Inadecuate Testing of Molded-Case Circuit 1 Breakers " dated July 9,1992, indicatec that licensees occasionally ;

underestimate the inrush transient (starting) current during the first few cycles after a motor is started. NRC Information Notice 92-51. Supplement 1, indicated that the magnitude of inrush transient current, depending on the motor characteristics and the-actual timing _of the motor start, could be as high as two times the motor's locked rotor curren The inspectors questioned the licensee on their molded case circuit breaker setpoint methodolog The licensee indicated that they committed to increase the molded case circuit breaker setpoints to two times full locked rotor -

current in a letter to the NRC dated October 16, 1995. The schedule for i implementation of the design changes documented in the licensee's letter dated October 16, 1995, was prior to startup from the fifth refueling outage for Unit 2 and the seventh refueling outage for Unit 1. .The inspectors also noted that the licensee's corrective actions were previously reviewed by inspectors and documented in NRC Inspection Report 50-498/9512: 50-499/951 .4 Risk Imoact of the Failure of SI-168 to Stroke Ooen Voon Demand The inspectors questioned the licensee on what risk contribution the containment sum) to safety injection Jump suction isolation motor operated valves had on t1e safe operation of t1e facility. The licensee indicated that these valves had a common mode risk achievement worth of 2.705E+02 and a :

Fussell-Vesely importance contribution of 70.77 percent. The inspectors noted that the licensee's schedule for implementation of the molded case circuit breaker setpoint modifications was not a risk informed schedul The inspectors concluded that the licensee's planned increase in molded case circuit breaker instantaneous trip setpoints was appropriate with one exception. The one exception involved the inspector's observation that the schedule for implementing the design modification was not risk informed. This was a particular concern because the inspectors found that the actual instantaneous trip setpoint for the Valve SI-16B breaker, which provides power to a relatively risk significant valve, was only 1.26 times maximum locked )

rotor current, which was significantly lower than the recommended two times locked rotor current industry recommendation endorsed in NRC Information Notice 92 5 l 3 SAFETY ASSESSMENT /0UALITY VERIFICATION

'

When interviewed, managers expressed an operations philosophy that encouraged operators, particularly senior licensed operators, to use judgement to strike 8 a balance between reliance on procedures and skill-of-the-craft to carry out the daily operations of the plant. However, managers were slow to acknowledge

.

.

.

-17-that operators may have exceeded the intended scope of that philosophy during the reviewed events. Management was informed of the facts in both events on which this inspection focused within a few hours of each event and initially supported the actions of the operator The licensee's initial event review for the reactor trip was deficient in that it did not assess human performance. Management reported having identified a large number of human performance errors site-wide in recent months. In response. management created a " Human Performance Review Day" training  ;

requirement that was repeated about every 10 weeks. The review day consisted of a dedicated 4-hour period in which all personnel on site and not on shift participated in training focused on human performance error avoidance relevant to each employee's functional area. However, no specific effort had been directed toward analyzing operator performance errors in the context of the ,

operations philosophy. Management expressed awareness of human performance l

'

errors related to procedure adherence and interpretation that had been identified through NRC inspections over the past year. However, management did not communicate a perception that their operations philosophy was being improperly applied. Further the recent events on which the inspection focused had not prompted management to consider a broader problem beyond the specifics of each even The inspectors discussed their concerns regarding the adequacy of shift and  !

o)erations de)artment staffing to accommodate the large demand for vacation at 1 t1e time of t1e reactor trip event with licensee management. As previously i

'

noted, management exceeded their informal guidelines for crew substitutions and the personnel who performed the initial event review did.not meet management expectations. The inspectors expressed a Jerception that management of personnel resources was a likely contriautor to both crew and event review team performanc The licensee expressed disagreement, particularly with regard to the crew that responded to the reactor tri Although the disagreement remained unresolved by the end of the inspection, the licensee expressed their intent to implement several corrective actions-focused on ensuring that event review teams could satisfy management expectations even during periods of reduced staffing. The inspectors continued to hold the perspective that excessive substitution in the control room was a contributor to the crew's performance. By substituting four members of the crew, the licensee had created a rcw crew for all practical purposes since the normal control room complement was six operators including the shift technical advisor. This crew assumed the responsibility of operating the )lant for several days without the benefit of preparation and conditioning tlat may normally have been given to a reconstituted crew. As previously noted the crew was not exercised in the simulator to build or reenforce team dynamics or to evaluate crew response to abnormal or emergency conditions. The inspectors found that increased management attention is ,

warranted in this are I

-

_- - - , . _ _ _ _ _ _ ,

.

.

ATTACHMENT 1 l

.1 PERSONS CONTACTED I Licensee Personnel )

l

• J. Calvert. Licensed Operator Requal-Training Supervisor

• J. Carlin. Nuclear Training Manager

• T. Cloninger. Vice-President-Nuclear Engineering  ;

• W. Cottle Group Vice-President. Nuclea *D. Daniels. Manager. Operating Experience

• Dowdy Continuing Improvement l

• R. Fellingham J Event Review Team Leader i

• J. Groth. Vice President. Nuclear Generation  !

• S. Head. Supervisor. Compliance  !

• C. Johnson, Manager. South Texas Project Activities. Central Power and Light

• T. Jordan Manager, Systems Engineering l

• M. Lashley, Supervisor.Section XI

• J. Lovell. Unit 1 Operations Manager 4

• M. McBurnett Licensing Manager .

!

• L. Martin, General Manager. Nuclear Assurance and Licensing i

• R. Masse. Unit 2 Plant Manager

• A. Mikus. Supervisor. Engineering

• L.-Myers. Unit 1 Plant Manager

• R. Rehkugler. Director. Quality

• P. Serra. Manager. Emergency Response

• J. Sheppard. Assistant to Group Vice President

• A. Spencer, Manager, Operation Support

• K. Taplett. Supervisor. Licensing

• S. Thomas. Manager. Design Engineering

• T. Underwood. Administrator. Participant Services

• D. Wohleber Project Manager 1.2 NRC Personnel

• P. Gwynn. Director. Division of Reactor Safety

• J. Keeton Resident Inspector

• D. Loveless. Senior Resident Inspector

• J. Pellet. Chief. Reactor Projects Branch A

• Sifre. Resident Inspector In addition to the personnel listed above. the inspectors contacted other personnel during this inspection perio * Denotes personnel that attended the exit meetin EXIT MEETING An exit meeting was conducted on January 22. 199 During this meeting, the inspectors reviewed the scope and findings of the inspection. The licensee acknowledged the inspection findings as they were presented. The licensee did not identify as proprietary any information provided to. or reviewed by, the inspector ]

_, - . ,mr -- --- .,,--,,--,-, ,- -- - --- - --

.

.

.

ATTACHMENT 2 SEQUENCE OF EVENTS 12/18/95 0336 Turbine trip / reactor trip on main transformer lockou Emergency Diesel Generator 11 started and tied to respective engineered safeguards bu Auxiliary feedwater pumps started and began feeding all steam generators (levels 2-4% narrow range).

Emergency Diesel Generators 12 and 13 and their respective emergency cooling water pumps were started manuall Operators carried out immediate actions for E0Ps from memor Main steam isolation valves were shut manually (Valve B did not indicate fully closed).

Auxiliary feedwater flow at maximum ~2500 gp Operators com)leted verification of immediate actions and entered ES01, Reactor Trip Respons RCS cold-leg temperature 557-553 degrees Operators observed indications on digital rod position indication system -l that three rods remained six steps withdrawn. Operators interpreted rod indications as satisfying the requirement to be FULLY INSERTED and l proceeded without initiating emergency boratio l 0341 RCS cold-leg temperature went below 550 degrees F Operators were beyond Step 1 of ES01 and did not initiate emergency boratio An operator took manual control of auxiliary feedwater and began to throttle flow to the steam generator Charging pump suction shifted to the reactor water storage tank as a result of low volume control tank leve Crew observed instrument air pressure at 95 psig and increasin An operator manually tripped the start up feedwater pum MSIV B indicated fully close Shift technical advisor observed RCS cold-leg temperature of 536 degrees F and recommended to one of the SR0s that emergency boration be initiated. The STA was told that the operators had already gone beyond the E0P step that required emergency boration and that the requirement was not continuous: therefore, the su)ervisor did not act on j the STAS recommendation. The STA then observed tlat the charging Jumps had shifted suction to the RWST and did not pursue the matter furt1e Operators stopped AFW flow to the steam generators (levels 28-33% narrow range).

0350 A pressurizer power operated relief valve cycled open three times over about a 30-second perio Operators placed auxiliary spray in servic l

. . _ .

'.

.

-2-t 0351 Operators increased AFW flow to ~360 gpm total.

1 0352 Source range NIs energized - counts decreasin Reactor coolant system cold-leg temperature increased above 550 degrees F.

l 0355 The shift technical advisor from Unit 2 arrived and was assigned to perform Procedure OPOP04-AE-000 Normal letdown was restored (pressurizer level - 70.3 percent).

l 0405 Plant operator reported all AFW pumps running normall The emergency diesel generator engineered safety features sequencer was rese The operators began using the steam generator power operated relief

.

valves to control cooldow Opened main condenser vacuum breakers, i

(RCS pressure - 2146 psig, pressurizer level 68.2 percent),

0414 Verified that only undervoltage relays actuated on 13.8 kV auxiliary and standby busse Reenergized 13.8 kV Auxiliary Bus 1 Containment HI/LO alarm actuated - containment pressure at 0.3 psig, entered 1-hour Technical Specification action statemen Smoke (no fire) reported coming from LC 12 J1 feeder breaker. Unit 2 personnel investigat Reenergized Auxiliary Bus I Load Center 12J1 cross tied .from Unit 2 via 12J2. Load Center 12J1 supply breaker racked ou Plant status - RCS pressure 2125 psig, pressurizer level 67.5 percen Main turbine on turning gea Auxiliary Bus 1G reenergize Unit 1 operations manager notified (within the next hour, the group vice 3 resident for nuclear operations arrived in the control room).

)lant status - reactor coolant system pressure 2176 psig. pressurizer j level 57.6 percen Auxiliary Bus 1H reengergized.

'

0454 Placed Emergency Diesel Generators 12 and 13 in cooldown.

,

i

, , - - -

0 3-0459 Stopped Emergency Diesel Generator 12 and unloaded Emergency Diesel Generator 1 Started reactor containment building supplemental purg Started open loop auxiliary Cooling Water Pump 1 Started open loop auxiliary Cooling Water Pump 1 Containment pressure HI/LO alarm cleared - exited Technical Specification action statemen Started Reactor Coolant Pump 1 Plant status - RCS pressure 2208 psig, pressurizer level 37.1 percent, Average RCS temperature 567 degrees Started Reactor Coolant Pump 1 Opened volume control tank outlet valv Shut charging pump suction valve to the reactor water storage tan Started Train B reactor containment fan cooler I 0548 RCS boron sample - 963 pp Secured turbine driven auxiliary Feedwater Pump 1 Established normal control room envelope HVAC Trains A and Secured Essential Chiller 12 Stopped Emergency Diesel Generator 1 Restored normal offsite power to Engineered Safety Features Bus E1A l Placed Emergency Diesel Generator 11 in cooldow ,

0606 Secured containment purg Stopped Emergency Diesel Generator 1 ]ened charging pump suction valve to the reactor water storage tank and i slut volume control tank outle !

0610 Completed shut down margin verification using short form. Required xenon free baron at 567 degrees F - 957 ppm. actual boron - 963 pp I

0618 Opened volume control tank outlet valv Shut volume control tank outlet valv Started open loop auxiliary Cooling Water Pump 1 _ _ _ _ _ _ __

o

.

C

.

0624 Opened volume control tank outlet valve and shut reactor water storage tank to charging pump suction valv Exited E0P ES01 and entered Procedure OPOP04-Z0-0003. Secondary Plant Stabilizatio (NRC resident staff became aware of the trip sometime between 0630 and 0700 upon arrival onsite.)

0644 Oncoming shift supervisor prepares NRC report and validates DRPI-B indicates ALL rods on botto NRC notified via ENS of RPS and ESF actuation (The licensee reported that all rods were fully inserted, but did not report that the DRPI system indicated three rod six steps withdrawn.)

12/18/95 An event review team was established and began to investigate the technical cause for the trip. Operators provided written statements but were not interviewed by the event review tea /19/95 In the 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> subsequent to the trip, the licensee determined that the rod position indication was accurate and that three rods initially stuck at six steps withdraw /20/95 Subsequent to discussions with the NRC, the licensee issued a memorandum to all operators clarifying management expectation regarding E0P use and interpretation as well as accuracy and completeness of information reported to the NR The trip initiator was identified and corrected control rod testing was performed and it was determined that rod drop times satisfied Technical Specifications. and the reactor was restarte /28/95 A human performance review team was established to investigate operator per formance immediately following the reactor tri o e

o ATTACHMENT 3 DOCUMENTS REVIEWED ,

l OPOP05-E0-E000, " Reactor Trip or Safety Injection," Revision 5 OPOP05-E0-ES01 " Reactor Trip Response " Revisions 6 and 7 l OPOP05-EO-FRS1, " Response to Nuclear Power Generation - ATWS," Revision 4 l

OPOP03-ZG-0006. " Plant Shutdown from 100% to Hot Standby," Revision 4 OPGP03-ZA-0010. " Performing and Verifying Station Activities," Revision 20 OPOP01-ZA-0018. " Emergency Operating Procedure User's Guide," Revision 7 OPOP04-CV-0003, " Emergency Boration " Revision 3 l OPGP03-ZE-0004, " Plant Surveillance Program," Revision 15 l OPGP03-ZX-0002, " Condition Reporting Process," Revision 11 OPSP03-SP-0009B, "SSPS Actuation Train B Slave Relay Test " Revision 5 Memorandum, " Rapid Cooldown After Reactor Trips Requiring Closure of MSIVs "

dated March 31, 1992

Memorandum, " Nergency Operating P^'edure Compliance," dated December 20, l 1995 Operations Policies and Practices Manual. Revision 2 Human Performance Post-Trip Review Team Report, CR-14621 dated January 5, 1996 Condition Report 95-14355, SCAO-S, Event Review Team Report

.

Executive Training Review Board Meeting Minutes dated March 30, 1995 Licensed Operator Training Technical Advisory Council Meeting Minutes dated January 9, 1995, february 7, 1995, March 1, 1995 March 23, 1995, and August 8, 1995 Licensed Operator Training / Licensed Operator Requalification Combined Operations Program Curriculum Review Committee Minutes dated May 18, 1995, July 27, 1995, and October 31, 1995 Nuclear Training Department Lesson Plan, LOT 501.06, " Instrumentation and Control Failure Analysis," Revision 4 Nuclear Training Department Lesson H an, LOT 803.08, " Respond to Off-Normal Conditions," Revision 3

, Nuclear Tra.ning Department Le m n Plan, LOT 504.04, " Introduction to Emergency Operating Procedures," Rev" sn 5 l

Nuclear Training Department Lesson Plan, LOT 504.06, " Reactor Trip Response OPOP05-E0-ES01," Revision 5 l

i l

l 4