ML20198L274

Insp Rept for Clinton Power Station on 970923-1003 & 1020-31.Violations Noted.Major Areas Inspected:Operations, Engineering,Maintenance & Plant Support

ML20198L274
Person / Time
Site:	Clinton
Issue date:	12/22/1997
From:	Collins S, Perkins K NRC
To:
Shared Package
ML20198L247	List: ML20198L242 ML20198L274
References
NUDOCS 9801150213
Download: ML20198L274 (80)

Text

. _. . .

- OFFICE OF NUCLEAR REACTOR REGULATION Licensee: lilinois Power Company Facificy: Clinton Power Station Location: Route 54 West Clinton, IL Docket No: - 50-461 Evaluation Period: September 23-October 3,1997 and October 20-31,1997 Team Manager: Kenneth E. Parkins Administrative Assistant: Judy A. Kilcrease Team Members: Henry A. Bailey Robert J. Bores Frederick H. Burrows Michael E. Ernstes Forrest R. Huey Paul J. Kellogg Richard J. Laufer Michael D. Tschiltz John W. York Contractor: Jonathan M. Wert Illinois Department of Nuclear Safety Observer: Alwyn C. Se : 7.s Fire Protection Evaluation: Philip M. Qualls Submitted By: t2 m i ~

2 A -

O'7 K'enneth E. Perkins, Team Manager Date Spe  ! Evaluation Team for li n Power St Approved By: 't A / M2 S'amuel J. Collins, Director Dat6 Office of Nuclear Reactor Regulation 9901150213 900102 PDR ADOCK 05000461 0 PM

q TABLE OF CONTENTS EAnt EXECUTIVE SUMM ARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ill

' A B B R EVI ATi r N S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi 1.0 I NT R O DU CTI O N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Ba c k g rou nd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Scope and Objectives ....................................2 1.3 M e t h od olog y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 1.4 Facilit/ Description ......................................3 1.5 O r g a niz a t io n . . . . . . . . . . . . . . . . . . . . . . _ . . . . . . . . . . . . . . . . . . . . . 3 2.0 - THF. LICENSEE'S INTEGRATED SAFETY ASSESSMENT ,................ 5 2.1 ISA Scope and Mission ...................................5 2.2 I S A Evalu a tio n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2.1 Effactive Diagnostic Assessment .......................5 2.2.2 Changas to the Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.3 Changes in Team Composition . . . . . . . . . . ............ 6 2.2.4 Communications with IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Ce nc l u s io n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.0 , FUNCTIONAL AREA ASSESSMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1 O p e ra t i o n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1.1 Management and Leaderchip ..........................9 J.1.2 Programs, Processes, and Procedures . . . . . . . . . . . . . . . . . . . 10 3.1.3 Human Performance ...............................12 3.2 Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.2.1 - Management and Leadership .........................14-3.2.2 Programs, Processes, and Procedures . . . . . . . . . . . . . . . . . . 17 3.2.3 Human Performance ...............................23 i

o 3.3 M aint e n anc e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 5  ;

I 3.3.1 Management and Leadership .........................25 3.3.2 Programs, Processes, and Procedures . . . . . . . . . . . . . . . . . . . 26 3.3.3 Human Performance ...............................29 3.4 Plant Support .........................................30 l 3.4.1 Management and Leadership .........................30 3.4.2 Programs, Processes, and Procedures . . . . . . . . . . . . . . . . . . . 32 3.4.3 Human Performance ...............................34 3.5 Managament and Organization . . . . . . . . . . . . . . . ............. 35 3.0.1 Management and Leadership .........................35 3.5.2 Programs, Processes, and Procedures . . . . . . . . . . . . . . . . . . . 38 3.5.3 Human Parformance .................... ......... 40 4.0 R O OT C A U S E S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.1 Ineffec,tive Performance Standards Established by Management . . . . . 41 4.2 Ineffective Defense in Depth Provided by Programs, Processes, and Pr oc e d ur e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4 4.3 inconsistant Problem Identification and ineff active Evaluation and Corrective Actions . . . . . . . . . . . . . . . . . . . 46 4.4 ineffective infrastructure To Support Major Changes ............. 47 5.0 EXIT MEETING ATTENDEES LIST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 APPENDIX A: EVALUATION OF FIRE PROTECTION . . . . . . . . . . . . . . . . . . . . . . . . . 51 A.1 Fire Protection Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 A.2 Facilities and Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 r A.3 Staff Knowledge and Performance ..........................54 A.4 Fire Brigade Perf ormance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 APPENDIX A VERTICAL SLICE FIRE PROTECTION EVALUAnoN APPENDIX B EXIT PRESENTATION E

f li .

A EXECUTIVE

SUMMARY

Backaround From August 25 through December 11,1997, a Special Evaluation Team (SET) from the Nuclear Regulatory Commission (NRC) evaluated the performance of the Illinois Power Company (IP)in ensuring the safe operation of the Clinton Power Station (CPS). The evaluation included an assessment of the efficacy of the lic,ensee's Integrated Safety Assessment (ISA).

Con 'asions The CPS ISA was performed by a well quellfled team of industry experts. The ISA team conducted an effective diagnostic assessment of the station's performance, and reached substantive conclusions, which the licensee recognized as generally valid. The root causes identified by the ISA were confirmed by the SET's independent evaluation and were consistent with the SET's de'ermination of root causes. ,.

The ISA used a barrier analysis technique to arrive at root causes and focused its root cause effort on a recent and relatively short time frame. in contrast, the SET's observations, validation, and causal analysli. approach was more historically based and focused on a longer performance period than considtred by the ISA. Although the timeframes considered were different in length, the SET confirmed the ISA's process and found their root causes to be consistent with those of the SET. Further, the SET found some additional examples of problems observed by the ISA in the Operations, Engineering, and Plant Support functional areas. Potential safety and compliance issues that were identified during the ISA and SET assessments were provided to Region 111 for appropriate regulatory follow up.

Functional Area Assessments Operations The SET confirmed the ISA's assessment that Operations management and leadership were weak in oversight, establishing expectations, conservative decisionmaking, and supervision. Programs, processes, and procedures were weak in procedure quality, equipment labeling, corrective action (CA) program, safety tagging program, and equipment status control p ocess. Human performance weaknesses were found in the conduct of operations, procedural adherence, staffing, human factors, and work prioritization.

Ena!neerino The SET confirmed the ISA's assessment that Engineering management and leadership  !

_ were weak in the areas of organization oversignt, establishing expectations, training, performance monitoring, major program implementation, and corrective actions. Programs, procestes, and procedures were weak in design control, design basis documentation, ill 5-ep- ., -- .n -

, y ,, - - - . - - , , e-

safety evaluations, opersbility evaluations, and Engineering support wr surveillance and maintenance procedures. The fire protection program d.d not provide sufficient bases to ensure safe plant shutdown following a fire. Human performance was weak in engineering training and skills, workload prioritization, living with problems, and lack of ownership.

Maintenance The SET confirmed the ISA's assessment that Maintenance management was weak in communications, establishing expectations, and providing training. Programs, procest,es, and procedures were weak in preventive maintenance, industry experience, corrective actions, work control, and material control. Human performance was weak in attention to detail, procedural adhert.rce, and training.

Plant Sunoort The SET confirmed the ISA's assessment that Plant Support management and leadership were weak in ownership of programs, staffing and training, deelslonmaking, establishing expectations, communications, and coordination. Programs, processes, and procedures were weak in corrective actions, the "as low as is reasonably achievable" (ALARA' program, the quality assurance of analytical measurements, work planning and control, and procedures. Human performance was weak in procedural adherence and worker practices.

Manecement and Oraaniri . x t The SET confirmed the ISA's assessment that CPS's management and leadership were generally ineffective in establishing expectations, communications, independent oversight, performance measurement and monitoring, decisionmaking, and human resource management. Programs, processes, and procedures were generally ineffective in self-assessments, corrective actions, root cause analyses, planning prioritization, and scheduling. Human performance was weak in procedural adherence, resource allocation, t.nd time management and prioritization.

Moot Causes The SET determined, based on assessment of CPS performance from June 1995 through October 1997, that the following were the root causes of the licensee's problems concerning management and leadership; programs, processes, and procedures; and human performance.

1. Management generally did not establish and implement effective performance standards. The SET concluded that the failure of IP and CPS management to establish and implement effective performance standards was a root cause of the significant decline in safety performance. Management failed to establish and communicate appropriate, clearly defined expectations ar,d priorities, and failed to monitor their implementation for the desired performance. Management decisions iv

-# . -, - - r

. . - . . _ _ ~ _ . - - . - . . _ _ - - - - . - -. -- -- - - - .

6 that were inconsistent with stated expectations contributed to declining performance, in addition, management did not give the ctaff sufficient feedback and failed to establish accountability.

2. CPS programs, processes, and procedures did not consistently provide defense in depth to assure plant activities were conducted in a safe manner. The SET concluded that programs, processes, and procedures failed to integrate activities ,

across departments, incorporate industry information, and clearly delineate ownership and accountability. Program implementation was not effective in attaining the intended objectives. Processes and procedures were overly cumbersome and by failing to provide appropriate guidance unnecessarily challenged workers performing an activity. Programs and processes did not provide effective ,

monitoring and feedback.

3. Problem identification was inconsistent and evalestion and corrective actions were generally ineffective. The SET concluded that the inability to idendfy, evailuate, and correct -7blems was a major impediment to improvement. Inconsistencies in proble. . identification resulted in failure to ensure that problems were effectiveiy captured. Ineffective evaluation of identified problems contributed to failure to develop effective corrective actions. Failure to monitor and ensure implementation of CA p'ans contributed to recurring problems and an attitude of living with problems.

4. Management did not ensure that the infrastructure was suitable to support major changes. The SET concluded that management did not recognize that the infrastructure at CPS was insufficient to support major changes. As a result, management made organizational, programmatic, and resource decisions in the context of reengineering without appropriately considering the longer term and integrated effects of the decisions. Management did not ensure that there were '

appropriately qualified staff, integrated programs and processes, and appropriate resources to support implementation of the reengineering and downsizing effort.

V

.. -- ._ _ - - . . - = _---_._--_.-. .--- .-.- .-.--.-

ABBREVIATIONS ac alternating current ADS automatic depressuriration system ALARA as low as is reasonably achievable AR/PR area radiation / process radiation (monitor) ,

C/B circuit breaker C&l controls and instrumentation CA corrective action ,

CAL confirmatory action letter CARB Corrective Action Review Board CCF comment control form CFR Code of Federal Regulations CPS' Clinton Power Station ,

CR condition report d/P differential pressure -

de direct current ECCS emergency core cooling system ECN engineering change notice EDG emergency diesel generator EOF emergency operations facility EOP emergency operating procedure EPRI Electric Power Research Institute FME fore!gn material exclusion FPFI fire protection tunctional inspection FRG Facility Review Group GE General Electric Company  :

GL generic letter HPCS high pressure core spray HVAC heating, ventilation, and air conditioning IDNS lilinois Department of Nuclear Safety IN information notice IP lilinois Power Company IPE individual clant examination IPEEE Individual plant external event examination lSA integrated saisty assessment JUMA Joint Utility Mariagement Audit vi

- - - - - . . - - . . =. - - - - - - - _ . - . - - . . . - --

t i

j LASS line sasistant shift supervisor  ;

limiting condition for operation  ;

LCO LOCA loss of coolant accident l LOOP loss of offsite power  !

LPCI low pressure coolant injection l LPCS low pressure core spray  ;

Long Term improvement Plan LTIP MCC motor control conter MCR main control room ,

MCV motor operated valve MSIV main ateam isolation valve MWR mainterience work request ,

NFC Ns.lonel Fire Code NPRDS Nuclear Performance Reliability Data System NRAG Nuclear Review ano Audit Group i

NRC Nuclear Regulatory Commission NRR Nuclear Reactor Regulation, Office of NSED Nuclear Station Engineering Department OD operability determination ODCM Offsite Dose Calculation Manual OSHA Occupational Safety and Health Administration OSTI operational safety team inspection i

PM ' preventive maintenance Pt.iT post maintenance testing

? REM personal radiation exposure management QA quality assurance OC quality control' RCA root cause analysis RCIC reactor coro isolation cooling ,

REMP radiological environmental monitoring program -

RHR residual heat removal RO reactor operator RP&C. Radiation Protection and Chemistry RP Radiation Protection i RPWI radiation protection work instruction RPV reactor pressure vessel RR reactor recirculation RWP radiation work permit SALP- Systematic Assessment of Licensee Performance vil

.. .. ._-. . - . . . . . - _ - . - . . - - _ - - _ = - .. _= _,.= .

l SBGT standby gas treatment SBO station blackout SCBA self contained breathing apparatus SET Special Evaluation Team '

SMM senior management meeting SRAP Startup Readinest. Action Plan i shift resource manager SRM SRO senior reactor operator SRP Strategic Recovery Plan SRV safety relief valve SS shif t supervisor STP surveillance test procedure SX safety related service water

.TS Technical Specifications USAR Updated Safety Analysis Report VG standby ges treatment system VOC volatile organic chemical VOTES valve operation test and evaluation system WX solid radweste transfer system 4

viil

0 1.0 INYRODUCTION

1.1 Background

The decline in safety performance.of the Illinois Power Company (IP), licensee for the Clinton Power Station (CPS), became apparent to the Nuclear Regulatory Commission (NRC)in SepMmber 1996, hoblems were apparent in CPS's response to the known leakage of the recirculation pump seats that ultimately failed and resulted in the declaration of an Unusual Event. NRC responded to the event with a special team inspection. On the basis of NRC's concern for the deelslons made and actions taken by the licensee during the event and the slow response of IP senior management to thoroughly assess the actions of the operating crew, NRC sent a confirmatory action letter (CAL) to IP to establish conditions for restart. As a followup, NRC performed an operational safety team inspection (OSTI), which revealed widespread problems involving nonconr,ervative decisionmaking, ineffective procedures, ineffective management oversight, ineffective planning and evaluation of infrequently performed evolutions, longstanding equipment problems, and untimely recognition of the significance of the September 1996 event. NRC issued a $450,000 civil penalty for violations identified by the OSTl.

In response to the special team inspection, the OSTl, and the CAL, IP developed and submitted to NRC (December 1996) a Startup Readiness Action Plan (SRAP) which included corrective actions in the areas of concern. In January 1997, NRC sent a second CAL to IP to ensure that certain problems would be resolved before restart. The problems to be resolved were weaknesses in the safety culture, in human performance and Jnderstanding of expectations, in procedural adherence and adequacy, in management oversight effectiveness, and in plant material condition, in addition, the CAL required CPS to establish criteria for assessing the effectiveness of the SRAP.

At the January 1997 NRC senior management meeting (SMM), NRC senior managers discussed CPS performance an.d concluded that the licensee's recent regulatory and "

operational performance was not an isolated incident but represented a broadly based decline. NRC sent a declining trend letter to IP expressing concern for the licensee's nonconservative safety focus in decisionmaking, poor procedural adherence and adequacy, weak Engineering and Maintenance support to Operations, and a lack of discipline and attentiveness during the conduct of operations. Subsequently, IP determined that a more deliberate and more comprehensive approach was needed. The Strategic Recovery Plan (SRP), established in March 1997, included the SRAP and cal!ed for the development of a Long Term improvement Plan (LTIP).

In June 1997, NRC issued the CPS Systematic Assessment of Licensee Performance (SALP) report in which the functional areas of Operations, Engineering, and Plant Support received a Category 3 rating for adequate performance and the functional area of Maintenance received a Category 2 rating for good performance. NRC also sent a third-CAL to IP in June to address concarns with the impact of coatings in the containment and drywell on the operability of the emergency core cooling system (ECCS) pump suction

. strainers in the suppression pool.

1 l

t

At the June 1997 SMM, NRC senior managers again discussed the continuing decline in safety poformance of CPS and the ineffectiveness of IP management initiatives to correct '

the problems. The senior managers decided that they needed additional information to make an informed decision on the overall performance of CPS. In a letter dated June 30, 1997,IP committed to perform an Integrated safety assessment (ISA)in order to prepare a diagnostic assessment of CPS performance. Subsequently, NRC sent a fourth CAL to IP on August 0,1997, to require CPS to investigate the failure of a 4160 V circuit breaker (C/B) and to meet with NRC before resuming operations. This C/B failure was an example of a poor root cause analysis leading tJ an ineffective Corrective action.

1.2 Scope and Objectives The Executive Director for Operations instructed the NRC staff to form a Special Evaluation -

Team (SET) to assess both the efficacy of the licensee's ISA through direct observation and independent assessment and the overall performance of CPS. The goals of the SET were to (1) conduct an independent evaluation of data and information available in NRC and CPS documents, (2) Independently obuerve selected nctivities of the ISA team and review plans, records, and reports associated with the ISA, (3) evaluate the adequacy and independence of the ISA evaluation and recommend whether to proceed with the NRC evaluation, and (4) obtain additional information to determine the root cause(s) for CPS safety performance problems to allow NRC senior management to make an informed assessment of plant safety.

1.3 Methodology The SET consisted of a team manager,10 NRC team members, a management and organization contractor, and an administrat;ve assistant, in addition, the Illinois Department of Nuclear Safety (IDNS) provided an observer to the SET, who facilitated communication between the team and the State. The team was organized into the areas of Operations, Engineering, Maintenance, Plant Support, and Management and Organization. The team spent several weeks on preparations that included extensiva review of NRC and CPS documents from June 1995 through the time the team was on site, concentrating on performance after September 1996. The team also was briefed by NRC representatives from Region lli (including the resident inspectors), the Office of Nuclear Reactor Regulation (NRR), the Office for Analysis and Evaluation of Operational Data, and the Office of Nuclear Regulatory Research.

Between September 23 and October 3,1997, the SET assessed the performance of the licensee's ISA. This assessment was accomplished by reviewing documents; interviewing the ISA team; observing the ISA team's field work, meetings, and root cause analyses; and attending the ISA exit meeting. During the development of the ISA's conclusions on the fire protection program, a decision was made by NRC to perform a vertical slice evaluation of the fire protection program and to augment the SET membership with a fire protection engineer. An NRR fire protection engineer joined the team during the week of October 13, 1997. -

2 l

l

5 J N T %I v On OcMber 20,1997, the SET returned to the site for 2 weeks to independently assess a sample of the ISA's results and to pursue areas potentially containing performance problems that the ISA did not address or did not address in depth. The SET's onsite independent assessment included team observations of onguing activities, IP management and staff interviews, and additional document revie'ws, such as licensee responses to the i team's requests for info,mation, it also included discussion with the IDNS Clinton resident l Inspector for CPS to obtain his insights on performance problems at the station. NRC - '

resident inspectors frequently observed the SET meetings at the site. Representatives from the SET met daily with their licensee counterparts to discuss team activities and findings, i 1.4 Facility Description CPS is located on Lake Clinton, 6 mtios east of the city of Clir. ton, Illinois. The plant consists of one General Electric (GE) Type 6 boiling water reactor with a Mark 3 ,

containment. The plant's rated electrical output is 930 megawatts. The f acility was  ;

designed by Sargent and Lundy and was constructed by Baldwin Associates.

1.5 Orgenlaation I

CPS is owned and operated by the Illinois Power Company. Several key management changes were made before and during the evaluation.

The chart that follows illustrates the IP organizational structure at CPS at the close of the onsite evaluation. .

3 I

.o CLINTON POWER STATION NUCLEAR PROGRAM ORGANIZATION ,

=.

Semor Vee Presedest J.C.Coom l

Assestent Vee W W. o. Monte, Manager - CPS Meneger Quetty Myeger Nucteer Staten Enynoonng Manager poucteur Manager semesser Ses oty and W. D. Montorg. Actmg Amourence L S. WW Trenung av ?mpeert Pwoopmance 1. _ _ . .

G. L Baker J. F. Polchst fl F. Phasee

- Asesstent Plant ""-Asesstent Plant Meneger Owoeter - Enyneanne De octor - Plant Dweeter- Laconseg Drector - 0 . ;;

Meneger Mowieenonce Operatene Propects Enyneenne J. V. Sapest Anas vons

, R. A. Joyce M. W. Lyone J. 8L L% R. L feoed D. W. Weddet i

Dweetor - Plent Dweeter- Pione Support Dweeter - Resource Dweeter - Dweetor - Strategic -- - . -- -, ,

fleesten & Chenostry .._ Serences L. - s., Carrective Actone __ & Propoet ?"_-

.- - -; Long.Teren J. J. Ptece W. P. Souseguet K. A. Beker J. M. Gruber 1 ... ~- ; Ptose Vocost t

o-octee - Pionnas sad Sc'eduhng J. A. Heee t

4 l I

, , . ,- -_._.-n.-.-..._.--.-----_~.--

i l

1 i

2.0 THE CLINTON INTEGRATED SAFETY ASSESSMENT - k 2.1 ISA Scope and Mission I

- The licensee performed e detailed integrated safety assessment (ISA) to evaluate the performance of Clinton Power Station (CPS) in the areas of Operations, Engineering, j Maintenance, Plant Support, and Management and Organization. The licensee based the ISA on the guidance for condu sting NRC diagnostic evaluations and incorporated j

. experience gained from othc. dustry initiatives.

The ISA team consisted of a . tm leader, an assistant team leader,30 technical i evaluators, and an administrativ . assistant. The team members were organized into I functional area teams for each of the areas described above, A separate section of the !SA ,

team performed a vertical slice inspection of the residual hast tornva! (RHR) system, low.

pressure coolant injection (LPCI) mode, and supporting systems, The team spent 4 weeks on site performing the assessment (September 2 to October 3,1997). On October 20, ,

1997, the team leader issusd Volume I (the ISA functional assessment) ano Volume 11 (the  :

vertical slice report). Volume 111 (the ISA's detailed observ='lons) was issued on .!

October 31,1997.

. 2.2 ISA Evaluation During the final 10 days of the ISA, the Special Evaluation Tearv assessed the performance of the ISA both by Interviewing the ISA counterparts and observing the ISA field inspections, team meetings and root cause analyses, ISA technical debriefings with the

. licensee, and the formal ISA exit meeting.  ;

2.2.1 Effective Diagnostic Assessment i

The licensee assembled a large, multi disciplined, and experienced independent safety I assessment team at the station following the NRC notification of an impending diagnostic '

assessment of performance at CPS. This team possessed substantial experience in the design, operation, maintenance, plant support, and management and organization of  ;

boiling water reactor plants. The technical strength of the team enabled the members to i overcome in progress changes in the assessment methodology rnd composition of the i team.

The ISA found strengths in the t reas of a willing work force, security program leadership,

- respontes to ISA identified issues, and recent initiatives to upgrade the radiation protection ,

1 program. Significant weaknesses were noted in conduct of operations, oprations.

orger.iration and teamwork, operations management and supervision, operations command anu control, operations procedures, equipment _ status, tegouts, performance of system

~ engineers, long term equipment problems, maintenance of the station design bases, t,uality of technical reviews, essential maintenance processes, the radiation protection program, '

chemistry, fire protection, and industrial safety. Additionally, weaknesses were identified

. In the barriers to pravent safety problems. Weak barriers are not typical of a good nuclear -

U S

==..==_=:_x-. ~ . .. - - - . - ;w~ _. --.; _- >

5 safety culture. The weak barrlers included senior management, plant management, administrative controls, procedures and instructions, operations control, supervision and oversight, worker performance, and verification and testing.

The ISA identified the following four root causes for the declining performance at CPS:

e ineffective leadership throughout the organization in setting standards of excellence and management's expectations, intrusive monitoring of performance for all plant functions, feedback based on the standards set, and management help to adjust performance; e complacency throughout the organizations, i.e., the convict!cn that CPS's performance was good, unchallenged by critical self assessments and benchmarking against industry standards of excellence; e barrier weaknesses in many areas that were important to achieve the nuclear safety culture present at well performing nuclear plants; and

c. weaknesssa in teamwork at alllevels at CPS so that synergy was lost and departments functioned independently.

2.2.2 Changes 'o the Process Although the objectives of the ISA emained the same, the causal root cause methodology was abandoned during thu third week. The methodology adopted was a nuclear quality / safety culture barrier analysis. Although this caused some team members difficulty, ,

the competencies of the team members were well used to arrive at the root causes.

2.2.3 Changes in the Team Composition During the 4 week onsite assessment period, the composition of the ISA team fluctut.ted:

some joined the team after the assessment had begun, some took personal leave, and some left before the assessment was completed. Such changes complicated the SET interface somewhat, but did not appear to compromise the process. In addition, the ISA team was augmented with technical experts to strengthen assessment in the fire protection, environmental, and effluents areas.

2.2.4 Communications with IP The communication of ISA findir.gs to and coordination with IP personnel were effective.

The counterpart debriefings observed by the SET were comprehensive, and accurately conveyed the findings, their importance, and their implication to the designated counterpart.

The exit meeting was presented to a broad cross section of the plant staff.

6

1 . . . . .

On October 8,1997, the ISA's preliminary findings were presented to the IP Board of Directors by the ISA team leader at a routinely scheduled Board meeting. The team leader answered questions regarding the findings of the ISA.

2.3 Additional SET Findings and Conclusions The Clinton ISA was a diagnostic assessment of the r,tation's performance problems; it reached substantive conclusions, which the licensee has recognized as valid. The root causes identified by the ISA were consistent with the SET's independent root cause determinations. Correction of the ISA identified findings and root causes should lead to improved plant performance.

The ISA used a barner analysis technique to arrive at root causes and focused its root cause effort on a recent and relatively short time frame. In contrast, the SET's observations, validation, and causol analysis approach was more historically based and focused on a longer performant:e period than considered by the ISA (June 1995 through October 1997). Although the timeframes considered were different in length, the SET confirmed the ISA's process, identified some additional examples of ISA observed problems, and found their root causes to be consistent with those of the SET.

Additional examples of problems identified by the SET that the licensee was not aware in the Operations, Engineering, and Plant Support functional areas included: (1) The plant labeling program was not effectively coordinated with Operations or Engineering procedures. (2) The service water system was not always chlorinated in accordance with the Updated Safety Analysis Report (USAR). (3) The high pressure core spray (HPCS) emergency diesel generator (EDG) did not have emergency override of the test mode in the event of a loss of offsite power / loss of coolant accident (LOOP /LOCA) during testing. (4)

Not all required HPCS and automatic depressurization system (ADS) logic system and design basis functions were verified by existing surveillance test procedures (STPs). In addition, required HPCS and ADS surveillances were inappropriately performed through preventive maintenance (PM) activities and v'ere not tracked as (TS) requirements. Several HPCS steps were not correctly referenced by the EDG integrated STP (5) The Division til EDG was not able to achieve its TS-required frequency band within 12 seconds of starting.

(0) No safety evaluation for the non functional emergency core cooling system (ECCS) high pressure alarms was performed. (7) Acoustic matorial was installed on the main control room panels without sufficient evaluation or post maintenance testing (PMT). (8)

Compliance with 10 CFR Part 50 Appendix R for achieving and maintaining cold shutdown was not clearly demonstrated. Fire brigade access to the Division ll and lli service water pump rooms was impeded. (9) Fire hoses were clogged with mud dauber nests.

(10) Scaffolding was constructed in the low-pressure core spray (LPCS) pump room without an engineering evaluation of the potential effect on seismic qualifications. (11)

The program ior assuring quality of chemical analyses was not thorough. Information related to radioactive material use, storage, and spills required for decommissioning was not readily retrievable.

7

Potential safety and compilance issues that were identified during the ISA and hET assessments were provided to NRC Region ill for appropriate regulatory follow up, s

6 N

8 i

_._____m. . . _

3.0 FUNCTIONAL. AREA ASSESSMENTS The Special Evaluation Team (SET) assessed the functlonal areas of Operations, Maintenance, Engineering, Plant Support, and Management and Oversight. The assessments for each of the functional areas are categorized into three toples: (1) management and leadership, (2) programs, processes and procedures, and (3) human performance.

3.1 Ooerations 3.1.1 Management and Leadership The SET verified the weaknesses obsu cod by the ISA in Operations management oversight and expectations, decisionmaking, and supervision.

Overslaht and Excectations Managers did not consistently oversee plant staff activities and clearly communicate their expectations to alllevels of the plant staff; as a result, activities were not performed in accordance with expectations. SET interviews with operato.s indicated on inconsistent and infrequent presence c' managers in the main control room (MCRI and in the plant to monitor activities Operations managers stated that a large portion of their time was spent in meetings, precluding them from providing oversight in the field. Interviews conducted by regional NRC inspectors in June 1997 indicated that many Clinton Power Station (CPS) personnel had minimal contact with plant management and that the flow of informaon up and down the chain of command was limited. In December 1996, NRC noted in an inspection report that the Operations Department's expectations for MCR Journal entries were not clearly defined and resulted in entries that lacked detail. A recent Operations staff self assessment documented that this expectation was not yet effectively implemented. Also, there were many documented problems with procedure adherence until management's expectations were proceduralized in April 1997.

Additionally, managers did not take advantage of opportunities to communicate their expectations to the staff. For example, during initiallicensing training and requalification training, instructors did not always reinforce management's expectations regarding annunciator response and three part communications.

Decisionmakina Safety was not consistently the priority in management's decisions, and management's focus on production and schedule contributed to nonconservative decisions. For example, the decision to maintain the reactor in hot standby and the repeated use of safety relief valves (SRVs) for pressure control following the reactor scram on April 9,1996, challenged operators with constantly changing conditions. This method of pressurs control also contributed to increased SRV leakage. During the September 5,1996, reactor recirculation 9

- . . - . -- -,. - -. ~ .-_ - . - --

l (RR) pump seal event, the Operations Department again demonstrated the lack of an appropriate safety focus by failing to follow procedures and attempting to keep the unit at operating temperature and pressure.

The lack of a ss,fety focus in decisionmaking was not limited to major events but was also evident during certain routine evolutions. For example, in October 1997, CPS tagged out the Division ll emergency diesel generator (EDG) to perform the breaker rework activities i while the Division i EDG was degraded due to sitting of the service water (SX) system  !

intake bay and the Divisions I, ll, and IV battery chargers were Inoperable. Following this event, CPS management stated that significant improvement was needed in conservative decisionmaking and maintaining sufficient defense in depth.

ISA identified several examples in which a Technical Specifications (TS) limiting condition for operation (LCO) was not entered for equipment that was "available" but not in full compliance with the TS operability requirements. CPS identified another example of nonconservative decisionmaking in a condition report (CR) when Division ll Westinghouse circuit breaker (C/B) refurbishment was interrupted on the weekend of October 12-13, 1997, without sufficient evaluation of the TS immediate action statements.

Supervision Shift supervisors (SSs) and line assistant shift supervisors (LASSs) did not consistently provide crew supervision, and missed opportunities to reinforce management's ,

expectations in communications, procedural adheience, and conduct of cperations. On September 11,1997, the LASS was preot cupied with reviewing paperwork and f ailed to appropriately direct the activities of the reactor operators (ROs). This contributed to a draindown of reactor pressure vessel (RPV) level to approximately 16 inches lower than the preestablished administrative limit.

During SET interviews, LASSs stated that their administrative responsibilities took time away from supervising their crews. A UPS self assessment performed in August 1997, noted that SS visits to the MCR were infrequent, and that during thne visits the status and progress of work were not discussed. During May 1997, an industry audit observed that the SS was in the MCR only during shift turnover and pre Jeb briefs. Shift supervisors recognized ths need for more time to observe and monitor shift acWylties, but said that administrative duties interfered with this interaction. During the same visit, several LASSs were observed coordinatmg manpower for tagging and post maintenance testing when they should have been focused on the major equipment restorations taking place. The LASSs indicated that a majority of their time was used coordinating, integrating, and performing maintenance tasks.

3.1.2 Programs, Processes, and Procedures The SET verified the weaknesses observed by the ISA in operations procedure quality, the equipment labeling program, the CA program, the safety tagging program, and the equipment status control process.

10

Procedure Quality  ;

Procedures did not always provide operators with sufficient guidance to complete tasks.

Procedure weaknesses caused operators to develop a lack of confidence in their procedures; as a result, they did not adhere to procedures. Walkdowns of procedures by the SET indicated poor verification and validation of procedures. Additionally, tnere was no process to coordinate label changes, engineering change notices (ECNs), and operator aids to match the operations procedures. For example, the procedure for de load shedding during a station blackout (SBO)is one of the most significant operator actions identified in the CPS Individual Plant Examination (IPE) in reducing the core damage frequency. During a walkdown it was noted that the nomenclature in the procedure did not match the operator alds pested at the breaker cubicle, it was also noted, during a walkdown of a table in an off normal procedure for reactor cavity leakage during refueling, that a valve required for isolation was missing from the table. These were obstacles that would make the operator's job more difficult in a strersful situation.

A large procedure change backlog at CPS made it difficult to change procedures in a timely fashion. Since CPS issued a formal policy on strict procedure adherence in April 1997, operators made efforts to take ownership of their procedures and report procedure problems when encountered. This was a positive move for improving procedures but added to the backlog. The Operations Department contributed to the backlog by giving a low priority t- te review and approval process following revisions to the procedures by the Plant Support procedures group. The procedures group recently received a comment control form from an operator to make the same procedure change that had already been incorporated into a procedure and was awaiting review and approval by Operations for 2 months.

Eautoment Labelina Plant labeling did not appropriately identify equipment in several instances and placed an unnecessary reliance on IndividualInterpretation. Some valves were not labeled or were labeled incorrectly. NRC inspection reports documented labeling errors that led to tagging problems. CPS Identified that a small turbine vent valve had been left open while running a vacuum pump because the valve was not labeled.

During a procedure walkdown with an operator, the SET noted that were no labels identifying the motor control center (MCC) cubicle numbers. The operator had to count the cubicles to determine the right cubicle. The operator also said hat when cubicles were numbered, the convention was not consistent in that some cubicles were numbered from left to right, and some from right to left. The ISA identified that black markers were used

% designate pumps, valves, and throttle positions in lieu of approved labels. During interviews, Operations staff said that the plant labeling program had been in place intermittently for several years and was only 20-30 percent completed.

11

Corrective Action Proaram Operations was generally ineffective in preventing known problems from recurring.

Employees were not confident that the corrective action (CA) program could resolve their issues. Recent condition reports (CRs) documented problems that had recurred several times. NRC lasued a notice of violation in July 1997 for corrective actions that failed to preclude the third loss of service building security lightmg in a year. NRC also issued a notice of violation in October 1997 for failure to implement corrective actions to improve personnel performance with respect to the tagout program. Deficiencies with implementation of the tagout program continued to be pervasive throughout the Ope 7ations and Maintenance Departments.

Egletv Taaaina Prooram Chronic errors occurred in hanging and removing tags and verifying appropriate system bounderles; this created a challenge to protecting personnel and equipment. NRC inspection reports and the ISA described a number of tagging errors at CPS. During a tagout walkdown, mechanical maintenance staff found that a pump was not isolated from the deaerator tank recirculation line. The tagout had a ma;ntenance work request (MWR) added to rebuild the pump; the original tagout, however was only to lubricate the coupling and change the oil.

The plant manager issued a stop work order on September 11,1907, in part due to five recent tagging errors, which included working on the wrong component and insufficient system isolation boundaries.

Eauloment Status Control Operators on shift did not consistently remain aware of the current status of equlpment and systems resulting, on occasion, in an inability to predict system response in safely operating the plant. A recent CR identified that operators lined up reactor water cleanup to the "A" train, unaware that component cooling water was not aligned to the "A" train.

Another CR written during a review of MCR documentation indicated that some full system lineups had not been conducted for more than 6 years, during which time the valve lineup check list had several revisions. The ISA identified that because proce?ses to track out of-service equipment were not used throughout the site, scheduling errors occurred.

3.1.3 Human Performance The SET verified the weaknesses observed by the ISA in corduct of operations, procedural adherence, staffing, human factors, and work prioritization.

CSDduct of Ooerations A lack of thoroughness and a lack of formality in conduct of operations contributed to instances of human performance errors. Three-part communications were not consistently 12

used by the crew or enforced by rhift supervision. Thera was limited self checking or peer checking. This was evident recently when an operator induced a trantlent on the SX system by manipulating the wrong switch.

Most errors were due to operator inattentiveness rather than to insufficient operator knowledge of the task or equipment, as demonstrated by the September 11,1997,RPV low water level event. Poor attention to detail was also ir +. rated in the continuing personnel errors and tagging errors.

Procedural Adherence Procedures were not consistently performed as written. Procedural adherence was a chronic problem in the Operations area. During the September 5,1996, RR pump seal f ailure, operators' f ailure to adhere to procedures, while isolating the "B" RR loop, contributed to the seal failure. Before April 1997. guidance condoned flexible adherence to procedures. As long as the intent of the procedure was being met, the operator could choose which steps to perform and in what order. CPS lssued a formal policy of strict procedural adherence in April 1997. This policy had a positive effect on Operations.

However, during a recent exercise, the NRC resident inspector observed that a step in an emergency operating procedure (EOP 06) to initiate standby gas treatment (SBGT) was not performed. Operators indicated that they no longer attempted that step because it had little effect and was resource intensive, initial acceptance of this situation by management failed to support an attitude of procedural adherence and demonstrated a basic failure to recognize the iraportance of EOPs.

Staffina CPS failed to maintain a viable source of quellfied operators. An init al licenso class was 8

last held in 1993. The reduction of Operations staff through attrition resulted in the extensive use of overtime. For example, the SET found that a group of 51 reactor operators, nuclear equipment operators, and auxiliary operators worked an average of 46.5 percent overtime for the 12 week period. Reduced staff also limited the opportunities for operators to rotate to other areas and support Operations' ownership throughout the organization. Operators indicated that the lack of operational experience in other departments contributed to poor cooperation with those groups.

Human Factora Poar human factors hindered operators in both routine and off normal plant operation. The ISA noted operator aids in use that were not controlled by the operator aid program. RCt indicated that the noise levelin the MCR, due to the ventilation system, made communications difficult and created the need to shout when acknowledging certam annunciators. Some effort has been made to redt.ce the noise level by installing acoustic material on MCR cabinets. As discussed earlier, plant labeling was insufficient, and resulted in operating or tagging the wrong components. During a walkdown of a portion of the SBO procedure with the SET, a licensed operator noted that the steam tunnel had no 13

battery powered lighting. The lack of lighting would create a significant challenge as well as a safety hazard for the operator conducting the task since, after a reactor scram, all of the piping in the tunnel would be at operating temperatures. On the positive side, interviews with ROs indicated that the current program for reducing MC.R deficiencies was effective.

Prioritization F'.ough operators were burdened by many cumbersome processes, operators contributed

'eir own inefficiency by poor work prioritization. Shift supervisors indicated that they

,sent a large portion of their time on activities that took them away from their supervit,ory sle. For example, one SS stated that he spent 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> of a shift reviewing one CR.

A 'fitionally, Operations related work packages were reviewed by the shift resource manager (SRM) and the shift related work packages were reviewed again by the LASS.

Complex work packages, poor quality work packages, and rehandling work packages contributed to the administrative workload of senior reactor operators (SROs). Control board operators also spent a portion of their time reviewing work packages.

Failure of operators on shift to prioritize activities have led to plant events. For example, contributing to the September 11,1997, RPV low water level event, was the RO's decision to give a training checkout during the draining evolution. Additionally, the LASS was reviewing work schedules rather than monitoring the evolution. Quality Assurance (QA) revealed in an ISA interview that the SRM, SS, and LASS were not routinely going into the plant with the operators. QA concluded that there appeared to be enough slow time in Operations to accommodLte this effort, but it was not accomplished. The SET interviews confirmed that SROs were not consistently placing sufficient priority on providing onshift oversight of MCR and plant activities.

3.2 Enaineerina 3.2.1 Management and Leadership The SET verified the weaknesses observed by the ISA in Engineering in the areas of organization, establishing expectations, training, performance monitoring, major program implementation, and corrective actions.

Qmanization Oversiaht Management did not effectively implement its reengineering program. Many of the engineering program, process, and human performance problems appeared to be a consequence of ineffective management attention and follow through in this area. Several significant t .amples were found by the January 1997 independent assessment, the ISA, and SET. Elimination of supervisory engineering positions resulted in ineffective supervision. Emphasis on unmonitored self direction by individual engineers resulted in ineffective work prioritization and a tendency toward consensus driven decisionmaking.

Also, a lack of effective supervisory oversight and direction resulted in a work environment 14

that was overly focused on emergent work activities, with decreasing attention to proactive engineering programs for system monitoring and performance trending.

Excectations Management did not consistently define, communicate, or reinforce expectations and accepted performance different from expectations, creating a credibility gap and inhlLiting accountability. The ISA and SET observed management expectation problems that were identical to findings of a January 1997 independent assessment of Engineering. For example, although system engineer responsibilities were clearly and specifically addressed in Nuclear Station Engineering Department (NSED) procedures, management did not take action to empower engineers to fulfill those responsibilities. Both ISA and SET interviews confirmed that expectations for detailed system walkdowns by engineers with operators, proactive engineering review of preventive maintenance (PM) activities and MWRs, trending of system performance, and long range system engineering planning were not being met. Management did not establish clear expectations or minimum standards for Engineering involvement in root cause analyses (RCA). NRC inspections and CPS self-assessments identified that a lack of refresher training contributed to inconsistent implementation of formal RCAs. As a result, many RCAs did not identify potential contributors to an event, or did not appropriately consider similar events.

Management did not reinforce its expectations when it accepted different performance standards. For example, management failed to demonstrate a clear commitment to stated expectations for plant material condition. CPS self assessments documented repeated examples of management toleration of longstanding material condition problems.

Noteworthy examples included recirculation pump sealleaks, diesel generator oilleaks, and heat exchanger corrosion and fouling. The January 1997 independent ascessment identified several examples of plant material condition degradation due to management's acceptance of overuse of engineering analyses to justify existing conditions.

Trainina Management did not ensure that engineers were appropriately trained or qualified. SET interviews determined that NSED did not establish a well coordinated method for process and technical qualification and certification of design and system engineers. Some engineers stated that they were assigned tasks or system responsibilities for which they had not been acceptably trained. Although these engineers expressed discomfort with their level of technical knowledge, they were reluctant to request training in light of the large workload in Engineering. The January 1997 independent assessment team made the same observation. Recent CPS self assessments identified that some important engineering process tasks such as operability evaluations, safety evaluations, and calculations were conducted by unqualified personnel. Another example of management's failure to ensure appropriate training was provided was that engineering supervisors stated they had not received supervisory training in more tha.) 6 years.

15

Performance Monitorina Management did not establish effective measures for monitoring performance of engineering programs and processes. Management did not effectively monitor the performance of design engineering activities. Problems identified during peer and interdisciplinary review of design modifications were seldom documented, and were not trended. Also, problems identified during design modification implementation were not consistently documenteci and trended. None of the engineers interviewed by the SET recalled any structured feedback during the last 2 years of their involvement with design modifications. Recent performance indicators for Engineering's work backlog were not consistently focused on the priority or magnitude of ths work effort, but only on the number of work items. These performance indicators were of limited value for assessing the technical or resource significance of the backlogs. Further, established work priorities were not followed. For example, the current NSED backlog contained approximately 30 high priority work tasks that were more than 3 years old.

The SET viewed as a strength CPS's reestablishment of an engineering assessment group, which performed several well focused and probing evaluations.

Maior Proaram imolementation Management generally did not insist on thorough and timely implementation of important engineering programs. Delays in completion of activities associated with Generic Letter (GL) 8913, " Service Water System Problems Affecting Safety Related Equipment," GL 96 01, " Testing of Safety Related Logic Circuits," and Information Notice (IN) 9218,

" Potential For Loss of Remote Shutdown Capability During a Control Room Fire,"

contributed to significant service water system, logic system functional testing, and fire protection problems that were identified during the ISA and SET evaluations.

The ISA identified numerous examples of deferred tests and inspections of safety related heat exchangers. The heat transfer methodology utilized at CPS did not address design basis conditions of operation as specifiecI by GL 8913. Quarterly chemical treatments of the service water system were GL 8913 commitments, but were suspended in mid 1994 and were subsequently altered. Heat exchanger tests were not validated with periodic inspection, and many test results appeared questionable because acceptance criteria were not re baselined following chemical cleaning. As a result, on the basis of tests and inspections conducted, it was difficult to determine the effect of heat exchanger fouling on equipment operability.

Tt.e SET Identified several problems with the performance of logic system functional testing required by plant Technical Specifications, as described in Section 3.2.2 of this report. Also, more thorough and timely action in response to IN 9218 may have revealed many of the Appendix R issues described in Section 3.2.2 of this report.

16

Corrective Actions Management did not effectively enforce consistent standards for Enginesing involvement in corrective action activities. The ISA observed that root cause evaluations were not consistently required for potentially significant generic engineering problems. The SET observed that the Corrective Actions Review Board (CARB) did not require a root cause analysis for a failure to properly account for test measurm !naccuracles during Technical Specification surveillance tests or for fallu. to properly account for line resistance changes due to corrosion in the service water system. It was also observed that eng' eering reviews of some CRs were not thorough, similar to findings of CPS self.

assessments.

Appropriately thorough Engineering evaluation of safety related service water (SX) silting was not evident until after SET involvement in the issue. CPS responses to ISA concerns associated with insufficient GL 8913 implementation initially dismissed potential equipment operability problems without appropriate Engineering involvement or thorough evaluation.

3.2.2 Programs, Processes, and Procedures The SET verified the problems observed by the ISA in Engineering in the areas of design control, design basis documentation, safety evaluations, operability evaluations, Engineering support not surveillance and maintenance procedures, and fire protection. The SET developed some additional concerns related to fire protection.

Deslan Contrgj Weak design control procedures and Engineering reviews contributed to design errors and configuration control problems. A March 1997 self assessment noted that design change procedures did not provide sufficient guidance to ensure a consistent product, resulting in a lack of appropriate documentation of the bases for Engineering's judgment. Design changes did not consistently document detailed analyses and reviews for each discipline.

For example, a January 1997 independent assessment identified that a design change implemented to modify valves to prevent containment overpressurization did not properly document design inputs, impacts, calculations, or associated design analyses for the change. The assessment concluded that the design change package did not clearly document appropriate bases for determining that the modification corrected the problem.

Interdisciplinary review and use of the General Design Review Standard were not well specified or implemented by design change program procodures. Lacking specific requirements, appropriate reviews were not always performed. The January 1997 independent assessment identified severalinstances of inappropriate interdisciplinary reviews, including a failure to perform appropriate review: of fire hazards analysis impacts and of control room heating, ventilation, and air conditioning (HVAC) loading. CPS also identified that design changes had not been properly reviewed for impact on the containment hydrogen burn analysis.

17

SET interviews noted that independent technical reviews of deJign changes were not comprehensive and failed to identify deficient design changer. Engineers stated that abolishment of a configuratior management group during reengineering had eliminated previously effective checks and balances in the design control process. Also, some engin ers stated that reliance on a multi discipline team structure within Engineering made it difficult for engineers to obtain assistance from other engineers within the same discipline. During the same period, system engineers with limited design change experience were assigned to perform design changes. These factors, cornbined with weak process controls, appeared to result in inconsistently performed interdisciplinary reviews and insufficient independent technical reviews. As a result, several design errors were not identified. For example: a deficient 1995 modification made drywell floor drain flow monitors inoperable; a January 1996 drywellleakage detection system modification created a loop sealin instrument sensing lines; and a 1996 installation of new regulating transformers failed to properly incorporate industry guidance for use of embedded microprocessors, ineffective design processes resulted in loss of plant configuration control. For example, in May 1996, following improperly controlled installation of an uninterruptable power supply, installation of a regulating transformer resulted in spurious tripping due to load instability and breaker miscoordination.

1:npkinentation of ECNs without updating related design documents resulted in additional nstances of loss of plant configuration control. A July 1995 CPS audit identified that n.imerous ECNs failed to require revision of applicable design drawings resulting in ireccurate drawings.

CPS failed to ensure that the Division lli EDG design described in the strtion's in:tial licensing documents matched equipment configuration. The SET observed that the Division lli EDG was operated in the droop mode when tested in parallel with the offsite grid. Equipment design specified automatic transfer capability to the isochronous mode in response to a loss of offsite power / loss of coolant accident (LOOP /LOCA) signal during testing Installed equipment configuration required a manual action to restore the EDG to isochronous mode.

Deslan Basis Documentation Processes for control and use of design basis documents were generally cumbersome and contributed to design errors. Design basis information was contained in numerous documents, which were not cross referenced, making it difficult to identify documents associated with or affected by design changes. SET interviews indicated that design basis research became a daily task. Management's decision that reference drawings, vendor manuals, and calculations need not be maintained up to date created a cumbersome design process. This has resulted in reliance upon the user of the reference to review the backlog of outetanding revisions to determine applicability and impact on current work. TheISA concluded that it was difficult for CPS to respond in a timely manner to basic questions on the LPCI mode of operation of the RHR system although the design basis documentation 18 i

. . _.. . ._._ m . m .- ._ ___ ____.

I

\

may ultimately be retrievable and identifiable. The GE instrument setpoints to support LPCI mode of RHR were not available for review. The ISA noted several documents each with approximately 30 changes that had not been incorporated. A review of approximately 4,500 pre 1993 ECNs was required to respond to an ISA request for up to date design <

documents associated with the residual heat removal (RHR) system, indicating the i difficulty and level of effort required to determine system design basis.

Coloulation revleions generally were difficult to link to other coloulations and procedures }

effooted by changes, resulting in calculations that were not always updated. Although e i ca!culation index with more than 70,000 calculations was establishwd, it did not include {

i General Electric Company (GE) calculations nor riid it include the capability to cross-  :

reference calculations. The ISA identified two instrument setpoint calculations that  :

superseded GE setpoint calculations. However, no tracking system existed that identified  !

that the GE calculations were superseded.

The capability to conduct a search M the index. based upon the calculation title provided

limited aid. Therefore, considerable reliance was placed upon the experience and 1 knowledge of the individual revising a calculation to know or identify the affected l

- calculations and procedures. CPS recognized the lack of cross referenced calculations as a (

- weakness in 19g3, but corrective actions were assigned low priority and limited resources,  !

and were not completed. Reliance upon an individual to cross reference calculations has ,

resulted in errors. For example, revisions to breaker coordination curves were not updated in a coordination calculatioa because it was not recognized that the coordination curves ,

had an effect on the calcu.ation. l The ISA noted that field completed modifications involving design calculation revisions 1 were not evaluated for impact on other design calculations in a timely manner. Personnel  !

in Document Control were relied upon to identify affected calculations end notify the  :

owner to incorporate the change into the base calculation. This resulted in outdated 3 i

c calculations whose significance was increased by a lack of time limits for subsequent revisions. ,

?

Procedures for control of setpoints did not clearly define a standard setpoint methodology.  !

Engineers were relied upon to decide which methodology to utilize. As a result, setpoints for safety related instrumentation were not properly controlled, having que,stionable or i c - missing bases. The ISA'noted a program weakness in that no upper tier document i

o described the instrument setpoint program nor the roles and responsibilities of involved organizations. For. exermle, an instrument reset setpoint was not srocified in related -

design documents. As a result, engineers assumed a value that was greater than allowed by calculations in the associated surveillance procedure.

The ISA identified that calibration accuracy was not always considered in setpoint control, i thus creating the potential for "as left" bands greater than those allowed by the GE design  ;

specification. This could be nonconservative. The SET noted a strength in that the NSED l

manager questioned and revised an initial Engineering response to issues involving setpoint >

-  ! control that would have failed to correct the problems noted by the ISA. +

19  !

1. -- "*d1s 7"eYi'r Nt"-W*"eet'W*+=+tt"**h' P 9'irm 'N W-frr-t*=u-T'r@'NNAY-a9-W TeTFr*D" V rri g gy,"-y sy qg g y 3,q9qgr y aipeeirp --gF sp-11r Oihtt9%-r'ke=9- -m+J-'B

• MP=p mmW fIIf='t*'P*'"M4-F ' -'t-== W *d'9'"'re +T+-Tv-rm Pgr /

I Processes for ensuring that the Updated Safety Analysis Report (USAR)is updated were not always effective. This observation was made in the ISA vertical slice. SET interviews indicated that engineers had neglected updating of the USAR following design changes.

Also, SET document reviews revealed numerous inaccuracles in the USAR. Some had existed since initial plant operation; others were a result of failure to perform required updates following design changes. In addition, CPS was slow to correct inaccuracies in the USAR, some of which were known for more than 3 year- Examples of USAR Inaccuracles noted in CPS assessments and NRC inspection .eports included operation of control room chillers differently than described in the USAR, which impacted EDG loading analysis; USAR main steam isolation valve (MSIV) closure times that did not correspond to <

the GE design specificatlW or the Technical Specifications; and USAR motor operated velve (MOV) differential pressure (D/P) values that conflicted with calculated values to support the GL 8910 program. ,

Gafety Evaluatirms Insufficient safety screenings resulted in failure to perform required safety evaluations.

More than 50 CRs were written for deficient safety screenings and safety evaluations  ;

within the last year. These problems continued despite findings in a January 1997 independent self assessment that the 10 CFR 50.59 safety evalu% ion program had systemic implementation problems. The independent assessment noted that the Facility Review Group (FRG) reviewed only changes that required full safety evaluations. As a result, FRG reviews were ineffective in identifying problems since safety screenings -

frequently failed to require appropriate safety evaluations, in September 1997, an FRG review, which included safety screening submittals, rejected more than 50 percent of the submittals due to insufficient evaluation or information. Several examples of inappropriate safety screenings were identified in a January 1997 independent assessment, including modification of containment isolation valves to prevent overpressurization, installation of a relief valve to prevent overpressurization of a containmer.t penetrat!on, and modifications to control rod drives. The SET observed that a safety evaluation associated with a modification to add soundproofing to control room cabinets was deficient in that it failed to document a technical basis that cabinet internal temperature limits were not exceeded.

Furthermore, that the same issue was raised in a 1996 self assessment; however, no corrective action was taken.

Other instances wve noted involving failure to perform required safety screenings. The component cooling system expansion tank automatic level control system had been 1 operated for several years differently than described in the USAR without proper evaluation. The SET noted that the service water chemical treatment system had been removed from service for extended periods of time, contrary to the USAR, without performing any evaluation or implementing other compensatory actions.

Ooerability Evaluations Operability evaluations failed in severalinstances to properly address degraded eqely $nt conditions. Before December 1396, CPS had no formal procedure for evaluating 20

operability. After the procedure was established, numerous operability evaluation predoms [

were dccumented in CRs. Problems included Engineering evaluations that were not documented, were untimely, did not address all aspects of an issue, or were not performed in accordance with procedural requirements. The SET found that engineers did not recognize a need to perform an operability evaluation of the buildup of slit in the screenhouse pump bays. After attempting to justify operability, CPS ultimately determined  !

that there was not sufficient technical justification to ensure that the SX system would perform its function during a design basis ) vent. Additionally, after the discovery of heat exchangers that were significantly fouled, Enginecting did not perform operability evaluations to assess and justify operability of other SX heat exchangers that were subjected to similar conditions.

The SET noted an operability determination, supported by an Engineering evaluation, which concluded that Division 111 EDC respons6 was acceptable even though the time to achieve rated frequency exceeded the 12 second acceptance criterlon allowed by the Technical Specifications. Review of test data showed that the EDG frequency had dropped below the minimum value after 12 seconds and, therefore, did not meet the Technical Specifications testing requirement. Another determination noted that corrosive solder flux s was utilized on source range and intermedlute range monitor cabling. Although the consequence of the use s:f corrosive flux was time related, no Engineering evaluation, additional inspections, or compensatory measures were conducted to assess continued ,

instrument performance. Also, an operability determination (OD) for three high pressure alarms on emergency core cooling system (ECCS) piping with setp;lnts above associated relief valves was too narrowly focused. The alarms were Identifieo as safety system ,

components incapable of performing their intended l unctions as described in the USAR.

The OD concluded that the ECCS was operable, however, it did not reach an operability conclusion for the alarms.

Enaineerina Suonort for Surveillance and Maintenance Procedures Engineering generally did not provide appropriate support for surveillance and maintenance procedures, resulting in technically insufficient procedures and maintenance problems. SET interviews of Engineering managers and engineers revealed that vendor manuals were not kept current. A backlog of more than 200 vendor technical drawings that required updating. A 1997 Engineering self assessment identified that vendors were not contacted on an annual basis as CPS committed to in response to GL 83 28, " Required Actions Based On Generic implication Of Salem ATWS Events"; engineers did not screen changes to vendor manuals upon receipt to determine the significance of the change (s); additionally, ECNs did not always identify vendor drawing revisions, or the revisions were not incorporated in a timely manner. The SET noted that NSED Procedure P.4, " Vendor Manual Program," lacked sufficient detail to ensure a consistent change process for vendor ,

information.

Engineering was not effectively involved in the development, revision, and review of maintenance and surveillance procedures. Without proper system engineer review and concurrence, CRs continued to document vendor recommended PM activities that were not 21

~ _ _ _ _ _ _-_ _. _ _ .- _ _ , _

1 A  :.

l included in the PM program. Examples c,f problema that resulted from ineffective. .

Engineering review of procedures included inappropriate C/B mal _ntenance requirements, torque limits.above vendor recommended values, and inappropriate lubricant requirements for MOVs. o Procedure comment control forms (CCFs) remalned open after several years. The backlog  ;

grew to nearly 2,700 procedure commer:ts requiring resolution. One of the contributing

~

.i causes to the backlog was untimely Engineering technical review of proposed revisions, as docuraented in several 1997 CRs. The large backlog for the Pro:edures group resulted in untimely resolution of Engineering procedural concerns.

CPS identified numerous problems with surveillance procedures related to insufficient j en#neering involvement. For examp's, a 1997 Engineering self assessment identified that the Technical Specifications basis, USAR. and GE design specifications all indicated that 1 the automatic depressurization system (ADS) air supply was designed for two actuations. = i However, the value specifitd in the surveillance procedure only ensured one actuation after

-instrument tolerance was taken into account. A similar situation existed with the EDG nterting air requirements. Also, a 1997 CR identified that an MSIV surveillance procedure

. did not acceptably time the MSIVs as required by Technical Specifications. The measured

- time was based upon position indication supplied by. limit switches that actuated before full valve movement. The SET found several problems in the performance of logic system functional testing required by plant Technical Specifications. These problems included

. insufficient testing of the HPCS initiation sealin circuit, the high drywell pressure sealin for the ADS, the ADS Logic A/B 105 second timer initiated annunciator, and the reset of

the ADS initiation 105 second timer.

I The SET also found several examples of surveillance testing of safety-related components that were inappropriately performed through preventive maintenance ano that were not tracked as satisfying TS requirements. Examples included time delay and ac driver devices far low-pressure core spray (LPCS), HPCS,'and ADS logic system fr .n circuits.

Although the PM activities for the involved devices appeared to havr ;.sen done, PM procedures did not provide the same assurance or controls appropriate for surveillance tests.

. The SET also identified instances of inappropriate control of logic system functional testing requirements in surveillance test procedures. Examples included testing of several ECCS h component functions credited by performance of the integrated EDG surveillance test procedure. However, th'. EDG surveillance test procedure did _not clearly define which procedure steps were credited for testing ECCS components, and created the potential for missing required testing. .

Many'PM activities were deferred witLout appropriate technical justification. NSED Procedure M.02,-" Review of Preventive Maintenance Documents," provided instructions for engineers to follow for processing PM deferral requests. The procedure required the ,

NSED engineer.to ensure that the deferral did not adversely affect equipment operability or existing qualification programs.' However, the SET observed that the procedura did not 22

..s

)i ow-, . . - . - ~ - n w~. ~ , , , -. - - , n ~ , . , , - . c.,-- .,

- appropriately require the engir,aer to document the technical basis for PM deferral.

Furthermore, the procedure did not specifically require the engineer to address the individual or the cumulative effect of PM deferral on equipment operability.

[he Protection The SET conducted a vertical slice evaluation of fire protection. The resuks of th3s review are summarized below. Details of the evaluation are in Appendix A.

The fire protection program did not provide sufficient bases to demonstrate safe plant shutdown capability following a fire. An August 1997 independent audit identified insufficient evaluation of the ability to suppress a fire and associated consequences. CPS identified that SRVs required to support cold shutdown would be available for approximately 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> rather than for the 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> required. The SET observed that CPS hr.d not resolved a 1995 issue involving the potential for a single fire to cause a simultaneous LOOP and loss of all EDGs as a consequence of spurious carbon dioxide system actuation. The SET also noted that CPS had not resolved a concern that a single fire could cause all 16 SRVs to simultaneously open.

The fire protection program implementation had numerous problems. The SET found l

obstructions to sprinkler flow patterns in the Division 111 switchgear room. CPS was unable to provide data to demonstrate that three randomly selected fire seals were installed in a configuration validated by a fire test. CPS had extended surveillance for some fire protection systems beyond that specified in National Fire Codes, resulting in the failure to identify 11 fire hoses plugged with mud dauber nests. Counter to GL 91-18. "Information to Licensees Regarding NRC Inspection Manual Section on Resolution of Degraded and Nonconforming Conditions" guidance, CPS used hourly fire watches in lieu of fixing several problems identified in 1995, including the following: structural steel components with thermal shorts, inaccessible and inoperable fuel pool fire detectors, f ailure to install some fire barrier penstration seals in the control room, and the potential for a single fire to cause a LOOP and the loss of EGDs.

In contrast, the SE f observed that fire protection administrative controls, procedures, and equipment required for program implementatien were generally well maintained and available for immediate use. Good housekeeping and control of transient combustible materials was observed. Also, the fire protection staff was well qualified and had a good wo-king relationship with other station organizations. Fire brigade knowledge and performance were program strengths.

3.2.3 Human Performance The SET verified the weaknesses observed by the ISA in engineer training and skills, workload prioritization, living with problems, and lack of ownership.

23

= e, j V ., N 1

i Trainina and Skills 1 Engineering staff did not receive appropriate training and in several instances lacked ecoeptable skills and knowledge.- SFT walkdowns revealed that system engineers were not

~

fully aware of system design and operating parameters 'sech as operating margi_ns,

~

' functions of specific system componentsi system leakage trends, system pressure

. nperational controls, andl limits on system out-of service times. Also, in the absence of 1 system design basis documents, engineers were left to deterrline design bases from i

~

- several documents that were not easily retrievable or well maintained. This process was j

- extremely cumbersome and placed too much relience on individual skills. The combination. ,

of effects has resulted in human errors. For example, Section 3.2.2 of this report addressed a breaker coordination error resulting from calculation retrieval difficulties.  ;

The qualification and certification program for engineers was not wellintegrated or

< formalized.10ualifications for performing design changes were in many cases .

f t

- grandfathered and of little value. Some engineers certified to perform design changes had limited design experience and acknowledged that they did not have a clear understanding >

! of the design bases for their assigned system. System engineers were reluctant to request >

formal training in light of the large work load in Engineering. The NSED manager stated that CPS has not paid enough attention to training engineers in technical skills or processes

[ .

and procedures, and that engineers, managers, and supervisors needed more training to improve the safety culture, in interviewing system engineers, SET found that, following reengineering, supervisors were not of the same discipline as the engineers working for  ;

them and some were unable to effectively resolve technical issues. Engineering team leaders had little time to coach and~ mentor assigned engineers.

Workload Prioritization s

Some system engineers did not effectively trend equipment and system performance. SET  ;

' interviews of system engineers found that some were ineffective at trending equipment  ;

performance. Following reengineering, predictive maintenance and trending responsibilities wore transferred to system engineers and contributed to their overload. _ As a result, some system engineers neglected requirements for monitoring equipment performance and did-

- not set goals for improving poor equipment perfomance, f Livina With Problems Engineers failed to resolve numerous longstanding equipment problems. The ISA concluded that some engineers tolerated longstanding equipment problems. The January 1997_ independent ' assessment identified that engineers' overuse of analysis justifying degraded conditions resulted in several examples of decreased plant material condition.

The assessment also identified that engineers haci become " experts" in removing ~ available

> l margin from design analyses to avoid making changes to equipment or performing maintenance.. In an interview with SET, the NSED manager confirmed that one of the ,

biggest problems with degraded equipment conditions was using evaluations to reduce margin'to avoid. expending the effort to fix the problem properly. Also maintenance 24 p-4,. ,,m. . , - .5 % ,+ . h. . .[., m, ,.mm 5, m m.w, .vm.um_.. . # .yv ,[...w.., .w. -=m .-- W- w

3 , _ _ _ . _ _ , c- .,,_ ,. ._ . , _ _ _ _ _ _ . _ _ _ _ _ _ _ _ _ _ . _ ._

)

e 4

.-- i i.-

~

_j

~ .

planners indicated ', hat system engineers in general were not proactive in_ pursuing g corrective maintenance. Examples of longstanding equipment problems were long term unreliability_of the V notch ~ monitor of drywell leakage, Division ill EDG bearing failure due to_ deferred repair of an oilleak,'and operation of Division ll EDG with exhaust valve timing ,

out of specification since 1992. Interviews also revealed that engineers had not demonstrated "an appropriately questioning attitude.- An NRC inspection concluded that the _;

technical assessment of water in the reactor core isolation cooling (RCIC) turbine lube oil was nonconservative because the involved engineers did not demonstrate a questioning i attitude. -

Lack of Ownershin - -

Engineers generally did not demonstre+.e effective ownership of programs, systems, and

equipment. The January 1997 independent assessment concluded that the Nuclear l

- Performance Reliability Data System (NPRDS) program had not been actively st.pported for more than 2 years and the responsible engineer felt no ownership of the program. The ISA ,

observed that such lack of system engineer ownership was evident in the degraded

-material conditions in the switchyard and relay house. When sitting problems were

- identified in the service water pump bay, the ISA and SET observed that a lack of engineer ownership contributed to the .'ailure to properly evaluais the impact of silting in the safety-related service water pump bay.

3.3 - Maintenance 3.3.1 Management and Leadership t

The SET verified the weaknesses observed by the ISA in maintenance in the areas of communications, management's establishing expectations, and training, Communications Management generally did not ensure that effective communications existed in the planning and craft groups of the Maintenance Department or with other organizations, which led to compartmentalization. The ISA and SET noted inoffective cominunications within the department, through the managem'ent chain, and with other departments. As an 4 example, in the planning of work, Operations contributed to the plann;ng process in a -  !

limited way. The craft contributed nothing relative to the content _of the packages to the work planners generating the work package. System engineering made no contribution to work plarening unless requested. The only communications to the planners were craft inquiries'as to when the package would be ready.-

Manaaement's Exnectations Managers did not consistently oversee _ plant staff activities and clearly communicate their expectations to alllevels of the plant staff; this resulted in activities that were not

- performed in accordance with expectations. The ISA noted that management's-I 25

. - . - iL c. . , , , ._ ; ~ . ..- ,,. G _, _,__ - _ . _ . . ,_....-_.__~_._Jl....- __ , -

w

%- ,. i p

expectations had not been established'or communicated to the Maintenance staff. The SET noted 'during interviews that promulgated performance measures were not emphasized .

In the department. Although guidance for disciplinary actions for inappropriate- >

performance was disseminated, these measures were no+ enforced.; As a result, managers in the Maintenance Department lacked credibility and wrukt,rs lacked accountability. =

Additionally,Lthe PM prograni was fragmented among depetments, and no single' _

4 department was, responsible for the program. This contributed to the failure to determine _  ;

the impact on equipment of not performing PM activities. For example, radiation -

monitors / process monitors were not returned to service for extended periods and maintenance of the emergency operations facility (EOF) HVAC system was suspended.  ;

The ISA and SET, during interviews with craft and first line supervisors, determined that nonconservative decisions had been made in some areas. One of these decisions deferred safety related PM activities past their due dates without evaluating the effect of such deferral on the equipment. Another decision was to re-use a corrugated metal gasket in a -

valve repair. During restoration of that system, the gasket leaked profusely, contaminating an individual. Inspection reports identified that some design changes were installed in the plant using MWRs before the design change was reviewed and approved. Examples of this were raising the containment equipment sump float switch setpoints and removing springs from the air dryer check valves for the EDG air start system before the formal engineering '

approval.-

Trainina Management did not provide appropriate training to supervisors and craft. The ISA and SET noted through interviews with supervisors and group leaders, that supervisory training, had not been conducted for several years. This contributed to the ineffectiveness of 6

coaching and mentoring in reinforcing management's expectations. CPS identified this lack of superviso;y training and recently started a new training program. Additionally, CPS held a retreat for managers and supervisors in October 1997, during which an outline of the

" Plan for Excellence" was developed. Interviews with the craft indicated that initial training was good, but that retraining and reinforcement of initial training were deficient.

3.3.2 Programs, Processes and Procedures The SET verified the weaknesses observed by the ISA in maintenance in the areas of preventive maintenance, industry experience, entrective actions, work control, and material control.

Preventive Male tenance-Ownership of the PM program was not delineated: this led to program implementation problems. -The ISA and SET identified that the Maintenance Department assumed -

' ownership of the PM program. The ISA noted that Engineering typically owns the PM

. program content, frequency, and bases, and that Maintenance implements the program.

The Maintenance Department's presumed ownership caused several problems with the PM 26

• wi --

cd- 4'- <e <wg y-as- -+ y i,iyw- .w gwi,p4--w - e,- pm ,. g -+- --,+yr y q yw y- sw

l l

l 9[

P

_ program. One of these problems was that the Engineering bases for PM activit es'and their frequencies were not known or understood by Maintenance. The failure of Maintenance to '

understand Engineering's role and involve Engineering in the' PM program contributed to a- .

. jarge number of _PM activities being deferred without the effects being evaluated. ,

' Additionally, the ISA and SET noted that the responsibilities were not clearly defined as a '

result of organizational' changes and vaguely defined processes.

Industry Exnerience

• l The program for disseminating and incorporating industry experience was generally

- Ineffective and directly contributed to several equipment failures. The ISA and SET, through observations and interviews, noted that industry experience was not being 3

disseminated and incorporated in an effective or timely manner at CPS. An example of this

[ waF the lack of industry experience used in the repair of the failed C/Bs. CPS received-

- three NRC information notices but did not incorporated them into work procedures for the i C/Bs. CPS did not receive GE service advice letters, which also discussed PM activities for the C/Bs. Vendors were not consistently contacted for the latest information concerning ,

their products and changes were not incorporated into the manuals. The procedure for installing a freeze seal did not contain the Electric Power Research Institute guidelines on appropriate installation of freeze seals. The failure to incorporate these guidelines resulted-l in an improperly installed freeze seal, which damaged the reactor water cleanup piping in

_1996.

G.gn.edye Actions An ineffective corrective action program resulted in repetitive equipment and human r performance problems. The ISA and SET noted through interviews and direct observations

~

of the site CA program that the program did not effectively capture failure trends, was

, narrow in focus, had no repetitive failure analyses, and had no generic application for e common mode failures. Root causes were not well developed in some cases. The root cause of the failure on July 22,1997, of a safety related 4160-V Westinghouse C/B to open was shallow in that the true failure mechanism was not identified. This led to an additional breaker failure on August 5,1997, following a meeting.in which the Illinois Power Company-(IP) Informed NRC that reasonable assurance existed that the safety-related breakers would function when called upon.

e The ISA and SET, through interviews and observations, determined that Maintenance had

stopped trending human performance errors in 1996 und that the termination inhibited the-

!- identification of a declining performance trend. A root cause investigation, initiated in June i1997 to determine the factors behind an increasing number of performance errors in Maintenance activities, identified the root causes as ineffective parformance measures, a

lack of control measures, and a lack of individual accountability.

27

t ,:

Work Control

-Work scheduling was ineffective and work packages were generally very cumbersome; this contributed to large Maintenance backlogs and errors in the field. The ISA and SET, through document review, interviews, and field observations, concluded that work

" scheduling and package generation were ineffective and contributed to the growing backlogs of MWRs and PM activities. CPS utilized two scheduling processes, one when on line and one during outages.: The outage schedule was in use for14 months; consequently, the PM activities scheduled by the'on-line process were not scheduled, contributing to the PM backlog.- Since the completion of refueling, neither the outage planning not the on-line planning process was used successfully and no_ integrated site-wide planning or scheduling system appeared to be operable at the time of the ISA and .

SET site evaluations.- Individual departments established their own priorities and  !

' schedules, but did not coordinate with other groups, resulting in a lack of support and schedule slippages, if work was not scheduled on Operat;on's schedule and Radiation Protection's (RP's) schedule, the craft were turned back to the shop, in addition,

oopartment personnel often spent significant time waiting for other organizations to be ready to begin a scheduled activity that was not sufficiently planned. - Lack of timeliness in establishing schedules often left support organizations without appropriate resources to support activities thet appeared on short notice. _This was causing c significant problem in performing work on schedule, as indicated by the 50-percent daily schedule accuracy.

-Work packages were generally cumbersome, ineffective, and inefficient, sometimes leading to human error while attempting to identify the correct steps and specifications from tha attached matenals. For example, the work package to repair an 8 inch butterfly valve contained the technical manual for the valve, the foreign material exclusion (FME)  ?

_ procedure, the torque specification procedure, the valve operation test and evaluation '

system (VOTES) testing procedure, the valve operator's technical manual, and a valve

- packing procedure. The work package for replacing a heat exchanger on the Division 1

EDG contained more than 500 pages. The craft needed to set up a table in the diesel room to spread out the procedures and engaged one person to keep place in the various (

procedures. This process put too much reliance on individuals to ensure effective work performance.

Material Control Control of consumables~end volatile organic chemicals (VOCs) was weak. The ISA and l SET through observations and interviews, identified inconsistent control of consumables.

This included the use of unapproved cleaners and lubricants in safety related _C/Bs. Some lubricants stored for use by the craft were untabeled or had expired shelf lives.

Additionally, some lubricants required by PM procedures had been out of stock for years..

. CPS control of VOCs was limited to the control of containment coatings. Other.VOCs, such as dye penetrant developer, were not controlled and could impact the operability of-safety related filter trains.

28 i

o.

% e 3.3.3 Human Performance The SET verified the weaknesses noted by the ISA in Maintenance regarding attention to detail, procedural adherence, and training.

Attention to Detail Attention to detal'in Maintenance was inconsistent. NRC inspection reports and SET interviews, detern.Ined that there was a general perception at CPS that attention to detail was not as important as getting the job done. An example of this was identified in a 1997 inspection report which stateo 1.iat supervisors did not pay enough attention to the accuracy of sign-offs on quality records, in another instance, an inadvertent actuation of the standby gas treatment (VG) system occurred in April 1997, when a controls and instrumentation (C&l) technician misread a digital voltmeter and a local trip awitch was not placed in " normal" following tagout removal. Scaffolding was constructed in the LPCS pump room, affecting cooler seismic qualifications without an Engineering evaluation.

Adherence Maintenance's focus on production contributed to the use of procedures as guidance. The ISA and SET noted that procedural adherence was a weakness. The attitude that procedures did not have to be followed as written, but could be regarded as guidance was evident in the performance of Maintenance procedures, PM activities, and Maintenance shift turnovers. For example, Maintenance parsonnel changed torque values on flange bolts when installing a rupture disk. This was done by lining out portions of the job step.

Before April 1997, this attitude was proceduralized. These expectations were changed and Procedure CPS 15005.15 " Procedure Use and Adherence," was issued on April 1,1997, to specifically address procedural adherence. All personnel were trained on the new procedure. The number of temporary procedure deviations, procedure advance changes, and CCFs increased following the issuance of this procedure and the training, indicating an increased awareness on the part of oersonnel for procedural adherence.

Trainina Training on and common understanding of craft skills was waak. Tool box skills were not well defined. ' The craft had signoffs for some tasks requiring only tool box skills.

However, the definition of what constituted a tool box skill or just how much skill was involved was not defined. Supervisors did not know the contents of the tool box skills and planners had an incomplete listing of the skills. Examples of equipment degradation caused by the lack of understanding of tool box skills include (1) overgressing 480-V motors, when the drain plug was not removed to allow the old grease to escape and (2) the use of soldering flux which was conductive and corrosive on the neon light assemblies in the MCR. In both cases, tool box skills were not sufficiently understood.

29

t .

Ma3ntenance craft were not fully trained on management's expectations and were not held accountable. The craft believed that management was most interested in getting the job

- done and keeping the plant on line. -

Craft Attitudes Craft attitude was generally good. By interviews and field observations, the ISA and SET concluded that the craft were willing to work safely. The attitude of the craft generally remained good and their morale did not appear to be suffering from the long outage.

interviews and field observations indicated that the craft wen. Willing to work inside the procedurel boundaries and were willing to meet management's expectetions with respect to nuclear safety.

3.4 Plant Support The following programs were induded in the Plant Support area of the SET evaluation:

Radiation Protection (RP), Chemistry, Effluents, Radiological Environmental Monitoring Program (REMP), Emergency Preparedness, Industrial Safety, and Security.

3.4.1 Management and Leadership The SET verified the weaknesses observed by the ISA in Plant Support in the areas of ownership of programs, staffing and training, decisionmaking, management's expectations, communications, and coordination.

Ownershio of Proarams Several programs had no clear owners and no effective communications between organizations to ensure effective functioning. The REMP and Offsite Dose Calculation Manual (ODCM) programs were conducted by staff individuals in separate groups, each with a different supervisor. These programs were related and required frequent communications and coordination. The site organization did not foster such coordination and communications. For example, the ODCM was fragmented and did not effectively involve Operations, Chemistry, Engineering and Maintenance. In another example of fragmented ownership, a system engineer was recently assigned responsibility for approximately 120 area radiation / process radiation (AR/PR) monitors without sufficient authority to obtain the support necessary for keeping the monitors operable and calibrated. Many monitors were overdue for r:alibration or out of service, some for several years. Most of the monitors were routinely scheduled for calibration at 125 percent of the TS intervals.

Fragmented ownership and poor communications existed within RP. One group collected and analyzad REMP samples, another determined setpoints of monitors for the ODCM, and a third group monitored the AR/PR panel in the control room for Operations. Each RP group had a different supervisor. - Poor communication between these groups resulted in 30

i

.1 i y j r

L- _ calibration iriformation not being routinely provided to either _ Chemistry for use in - l t

-determining effluent releases or RP for determining any setpoint changes.1 -

iThe *as low as is reasonably achievable" (ALARAlprogram and the source term reduction _ '

program did not have site wide support. ALARA was viewed as solely an RP problem, and 4 the source term reduction committee had not met in more than 2 years.- The SET _

i determined that CPS failed to have a system for collecting and maintaining information for '

use in site characterization at the time of decommissioning. The failure to have such a j

^ system was due,'in part, to unassigned responsibility. 4 During the past year, several improvement programs were developed with little staff -

involvement. These initiatives were not supported by some workers and supervisors who believed that the improvement program would not address the real problems. Some of >

these programs were superseded before they were fully implemented.

Staffino and Trainina Gtaffing for several programs was ineffective since the reorganization in 1995, and training / retraining of staff in technical skills was not a priority at CPS. Until recently the '

ALARA program had a single individual assigned less than full time. The industrial safety L - program also had one individual assigned, even though as noted by the ISA, CPS was ,

among the worst in the industry relative to the Occupational Safety and Health Administration's (OSHA's) reportable incident rate. Supervisors had been given the

responsibility to oversee industrial safety in the field, but without commensurate training in ,

industrial safety. Many managers and supervisors spent significant portions of their work week in meetings and on other duties, which kept them from coaching and counseling their staff.

Refresher training in fundamentals for RP technicians was discontinued several years ago.

The ISA noted severalincidents pointing to a lack of knowledge in fundamentals by RP technicians. During the SET visit, maragers end supervisors.were being trained in supervisory skills, error _ reduction techniques and conservative decisionmaking. For many,-

' this was the first training they had received in sunervisory skills. The SET determined that technical training opportunities for supervisors were limited. Recently CPS personnel began again to visit other sites to baseline their programs against other plants.

pecialonmakina ,

A number of nonconservative decisions were made in the Plant Support area. Examples included the management decision to reduce the staffing of the ALARA program,' the -

~

~ failure to recognize the importance of ALARA to safety, and to classify the September 1996 reactor recirculation pump. seal failure event as an Unusual Event instead of an Alert,

+ :despite the loss of capability to assess the leak rate. . Additional examples included the ifailure to effectively control worker exposures during the reactor recirculation pump motor

,  : recoating in July 1997 and the decision in January 1997 to repressurire the leaking solid

= radweste transfer _ (WX) system without evaluating the radiological hazard, resulting in the 31 I ..

a

~

.m . .mE- - . .. 4-... . i ...,_ i m_.4 , . . .....U. . . - , . . ~ , . , . . , . . . - . - - , . -,-m.-

N .;

' apra ing of contaminated water.and resin from a burst rupture disc. Recent events -

provided no assurance that nonconservative decisionmaking has ended. .

Exnectations. Communications, and Coordination Management generally failed to establish and communicate effective standards for

_ procedure adherence and priorities fc site wide programs. Until April 1997, flexibility to-deviate from procedures was incorporated into the procedure system. Failure of management to emphasize personal accountability for procedural adherence and radiation worker practices gave the perception that there continued to be flexibility in procedure-

. adherence, as evidenced by the number of NRC cited violations and CRs on this subject.

Management had not established leadership at the site in terms of overall goals and priorities. As'a result, the various station departments were often working on different prioritiesi frequently conflicting in the area of interdepartmental resource needs. This lack of coordinated planning resulted in unnecessary personnel exposures as a consequence of reduing scaffolding, and shielding, or other work in the drywell during outages, and some' unnecessary clean up effort during the reactor recirculation pump motor recoating in July 1997.

3.4.2 - Programs, Processes, and Procedures The SET verified the weaknesses observed by the ISA in Plant Support in the areas of the corrective actions, ALARA, work planning and c.ontrol, and procedures. The SET developed an additional concern regarding the quality of analytical measurements.

Corrective Actions in the Plant Support area, CPS was generally good at identifying problems, but was generally ineffective in trending and resolving them. Problems were identified by a number of systems at CPS, including audits by Quality Assurance and outside groups, self-assessments, and CRs for self disclosing events. CRs were written for identified problems. Many recent CRs resulted from problems with worker practices. Self-assessments of some areas, such as REMP, the effluents program and CDCM implementation, and industrial safety were not effective because of limited staffing. In some areas, such as emergency preparedness, the self assessments were more of a

- regulatory compliance nature than of a quality / performance nature, in general, other Plant Support areas were critically self assessed.

. The ISA and SET noted that problems were not consistently trended. Radiation Protection-and Chemistry (RP&C) only recently began reviewing CRs for adverse trends. Multiple systerns/ files were maintained in RP&C to track aedit findings from various organizations,

NRC inspection findings, self-assessment items,' station commitments, CRs, and other

issues. The use'of multip'e systems for' tracking these items reduced focus on the overall significance for improvement in performance.

32

V ,.

. The ISA noted that few root cause determinations for adverse trends were performed in the Plant Support area. This led to ineffective and often narrowly focused cor ective

~ actions that did not prevent recurrence cf problems, especially in the area of worker L practices. Often there was no follow up to determine the effectiveness of corrective

- actions.

ALARA

. The ALARA program wat generally ineffective in accomplishing its function.' Insufficient ALARA program staff limited the amount of time available to monitor. jobs for-1 implementation of ALARA plarsning,- Some jobs received neither ALARA preplanning nor effective exposure monitoring and control during the activity and resulted in significant-unnecessary personnel exposures. An example was the recoating of the reactor recirculation pump motors in July 1997.

- The ISA noted that a large backlog of completed jobs, dating to 1996, had not received post-Job ALARA reviews. sThe delay in evaluating thers jobs precluded the use of this experience in subsequent work plans. Additionally, the SET determined that CPS had no system to capture " lessons learned" in the ALARA process. The current radiation work permit (RWP) syst.m and personal radiation exposure management (PREM) system did not always provide effective task specific data for monitoring exposure from jobs in progress for comparison with exposure goals.- As a result, exposure monitoring for job specific . ,

goalr. versus status of job completion was difficult.

As previously noted, the ALARA program wa:, considered an RP problem and received limited support from other departments. Annual exposure goals were not set for individual departments; consequently, they felt no obligation to control exposures. Sines radiation

. exposures were relatively low as a result of a clean fuel history, source term reduction had

- a low priority. The source reduction committee had not met in several years and clear ownership of this program had not been demonstrated. The ISA noted that ALARA Committee meetings did not have a high priority and had inconsistent membership.

Sometimes the department performing the activity under roview was not represented at the ALARA meeting.

Quality of Analvtical Measurements

- The process for assuring the quality of analytical measutoments was not thoroogh. The SET determined that CPS vendor acceptance audits of contractor labvratories perfonning-chemical analyses were not provided to the Chemistry group. Further, the Chemistry group had not reviewed vendor laboratories' quality assurance / quality control (OA/QC) programs to__ ensure the quality of measurements provided by the vendors.

With the exception of the REMP, the results of vendor QC measurements were not f routinely reviewed by the Chemistry group. Routine OC sample (other than background and reference level checks) were not used on a consistent basis to monitor the quality and consistency of chemical analyses at CPS.

33 h

. s _

.m . n , , . _

a' C , .. ,

i

- Work Plannina and Work Control-Tne work planning and work control processes were cumbersome and inefficient. Since

_lthe refueling portion of refueling outage RF6, the lack of timeliness in establishing ,

schedules often left support organizations, such as RP&C, without appropriate resources available to support activities that appeared on short notice. Likewise some training ,

-activities fo.' RP&C were imposed at short notice, further challengi ng available resources. .

- The lack of integrated site wide scheduling and priorities sometimes resulted in c inappropriate department schedules and priorities. Some processes resulted in the

. untimely return of systems to operation. For example, the pror.ess for calibrating radiation  ;

- monitors had no provisions to replace it if it failed calibration. Consequently, the radiation monitor could be out of service for days while a new work package for replacing the monitor was scheduled, prepared, and routed for approval.

Procedures Procedures sometimes did not provide effective guidanus and the change process was backlogged. Some procedures, particularly in RP, did not contain sufficient guidance to conduct some activities, in those cases supplemental guidance was provided in Radiation Protection Work Instructions (RPWis), which unlike the RP plant procedures were not

- subject to review by the FRG, but were reviewed and approved at the department director l-vel.. The SET noted that the content of some of the RPWis appeared suitable for plant

, procedures.. Other departments, notably Security, also made significant use of such

" standing orders" to conduct routine business.

Because of the sizable backlog in procer'ure change requests waiting to be processed, staff '

considered the change process cumbersome. In most instances, however, the longest delays in the process were with the originating department, awaiting department walk-down/ validation before station approval.

3.4.3 Human Performance

- The SET verified the weaknesses observed by the ISA in Plant Support in the areas of

, procedural adherence and worksr performance.

Procedure Adherence Workers did not consistently perform procedures as written. Procedural adherence was a persistent problem in the areas of RP&C as illustrated by numerous examples of procedural L - violations cited by NRC during the past year. Before April 1997, flexibility was ,

j. Incorporated into the procedure system. Consequently, plant staff did not strictly adhere to procedures as long as the intent was met, in addition, some departments used -

" standing orders" to provide additional guidance for performing tasks. This may also have ,

L contributed to the workers' attitude that procedures did not have to be perfor.ned as written.

34

(.

[

, _ , _ - _,-_ .m..,_ , , _ . . _ . , -

3 V-y_  :.

Worker Performa0GA~

Workers generally did not have a clear understandhg of performance expectations and -

received mixed and changin0 messages. Workers did not clearly understand the expectations for work performance. Differences between the guidance promulgated through procedures and training and the expectations communicated by supervisors and RP

-in the field cor:tributed to this lack of understanding. This lack of understanding was

. klustrated in the numerous CRs written for workers f ailing to comply with radiation wotk practices. Although some expected worker practices changed frequently, the changes were not effectively communicated to the staff. For example, dress out requirements and PREM usage were changed several times during the ISA and SET. Frequent changes contributed to inconsistent compliance with management's expectation for these practices.

The SET noted that until recently, workers were not held accountable for deviating from the stated practices. Management's failure to enforce its expectations, perpetuated problems with worker performance.

Exnertise and Leadershin The SET. observed strengths in the technical expertise in the emergency preparedness coordination program and in a number of the RP supervisors and staff that were 6terviewed. - Alsoi as noted by the ISA, the leadership of the Security program was a strengch.

3.5 Mananoment and Oreanization 3.5.1 Manegement and Leadership

. The SET verified the weaknesses observed by the ISA in management's expectations,

-- ccmmunications, independent oversight, performance measurement and monitoring, decisionmaking, and human resource management.

4 Manaaement's Exnectations Management generally did not establish appropriate and clearly defined expectatione and performance standards; this failed to foster an effective safety culture. Management's

_ expectations generally were not clearly defined and effectively communicated. in sddition, management generally did not provide effective context or rationale for decisions, creating an opportunity for staff speculation, particularly during downsizing and reengineering..

Expectations were not always effectively translated into action plans at lower levels in the organization, such as ALARA goals for individual departments. Additionally, clear performance standards were not established for the workers.

' CPS,ISA, and NRC documents described numerous attributes of an ineffective safety Lculture. Examples included an emphasis on low cost production, nonconservative

- decisions, failure to maintain a questioning attitude, a willingness to live with problems, lack of ownership and accountability,' poor teamwork, working around procedures, and an 35

ineffective CA program. CPS's business plans from 1994 to 1996 emphasized cost reduction as the primary concern. In the 1997 Business Psan management stated that

" implementing revised work processes or reengineering changes was the site's key plan for reducing costs. However, until a safe and conservative safety culture is again established for the long-term, focus will not be placed on this area." Management's failure to maintain appropriate safety expectations and standards did not foster an effective safety culture in most functional areas.

Commu,11 cations Weak communications contributed to instances of a lack of understanding of management expectations, departmental isolation, poor teamwork, and the lack of cooperation and coordination. CPS, ISA, and SET found numerous reasons for ineffective communication at CPS. Management had not successfully communicated its expectations to the staff.

Management had sent mixed messages on expectations in that their actions did not match their words in some cases. A January 1997 independent assessment of NSED found that "another area where management has been unable to clearly communicate expectations is the quality of the work product."

During interviews, the SET found that the downsizing and reengineering changes had not been effectively explained. Failure to provide the basis and context for these changes contributed to unnecessary organizational instability and insecurity, and a reluctance on the part of some individuals to seek assis+.ance or to challenge changes in direction.

The ISA observed that departments frequently operated indep?ndently of each other, resulting in poor cooperation and inefficiencies. Onshift organizational lines of communication, command and control were not clearly defined or effectively utilized. The problems with communications in most functional areas kept people from sharing information, discussing new ways to approach problems, and building a team.

Indeoendent Overslaht ineffective Independent oversight resulted in many missed opportunities to improve station performance. The ISA observed that independent oversight was not aggressive. The Board of Directors' lack of nuclear operations experience contributed to the ineffective performance monitoring and feedback at CPS. In addition, the Board did not ensure that the management team had varied nuclear plant operating experience, in some cases, the Board failed to ensure that senior CPS management had any recent nuclear operating experience. -

The Nuclear Review and Audit Group (NRAG) focused oa specific technicalissues as opposed to overa!! site performance. The ISA observed that NRAG was effective at identifying technical issues but was often not effective in ensuring corrective actions, in addition the ISA observed that NRAG did not effectively communicate its findings to senior

- management and had almost no communications with the Board of Directors. The SET 36

observed that senior management did not effectively consider NRAG findings and confirmed that there was httle communication between the Board and NRAG.

Performance Measurement and Monitorina ineffective performance measurement, monitoring, and feedback contributed to poor accountability. Performance monitoring at the systems, program, and organizational levels did not exist or was deficient in some areas. For example, the performance of design engineering was not effectively monitored, and Maintenance errors were not trended.

CPS,ISA, and SET found that insufficient quantifiable measures and processes limited managerial evaluation of performance. The performance appraisal system had been terminated for a considerable time. There were no centralized work status tracking system, insufficient coaching and counseling, and little feedback on performance to individuals, in addition, there was inconsistent recognition for above average or less than expected performance. Management was ineffective in establithing individual accountability because it had no performance measures or monitoring.

Decisionmakina Instances of nonconcervative decisionmaking by management communicaten an inappropriate message to the staff. The CPS, ISA, and SET identified a number of examples of nonconservative decisions. One of the most safety signif: cant examples was the decision to delay shutdown during the recirculation pump sealleakage event of September 5,1996. Anothet example was the deferral of PM activities beyond the due dates without evaluating the effect of such deferral on the equipment. As a result of these nonconservative decisions, the staff received mixed messages from the inconsistencies between mana;iment's stated policies and decisions.

Human Resource Manaaement Human resource management was weak and contributed to a failure to match the jobs with qualified people and to provide management team unity and contineity. The CPS, ISA, and SET identified several problems in human resource management. For example, management paid insufficient attention to human resource development issues as evidenced by training problems noted in all functional areas. An April 1996 NSED self assessment of human performance reported that, as a result of reengineering, the primary responsibility for showing up at technical training sessions was now placed on the individual, and there were few mandatory requirements for attendance. In lieu of training senior personnel were relied upon to coach and mentor less experienced personnel. This approach collapsed when organizational responsibilities were realigned and the increased workload eliminated the time for mentoring and coaching.

There was ineffective management succession planning, particularly at the end of 1995 and beginning of 1996, when numerous key management changes led to the loss of management and technical expertise. CPS identified that the "buyout" program, which 37 i

i

4-was part of the downsizing, contributed to a lack of unity, lack of continuity, and loss of corporate history. The leadership and management team, down through first-line supervision, was not prepared to handle the organizational transformation of downsizing and reengineering. At the time of this assessment, several key positions, including Plant Manager, Engineering Manager, and Maintenance Manager were vacant; some vacant por!tions were filled with acting or temporary managers for the long term.

3.5.2 Programs, Processes, and Procedures The SET verified the weaknesses observed by the ISA the areas of self assessment, corrective actions, root cause analysis, and planning prioritization and scheduling.

Self Assessments The self assessment program was not aggressively implemented to enhance station performance. Self assessments were not consistently perfor.ned and not required by procedure untillate 1996. Expectations for conducting self assessments were not effectively defined or understood, resulting in the performance of self assessments that were overly dependent on the individuals assigneo the task. Srme of the assessments were not comprehensive or sufficiently self critical, and some did not effectively identify weaknesses. Some of the staff stated that management had not placed enough emphasis on the importance of self assessments. When the self assessments were critical and thorough, management often failed to correct identified problemc. The ISA and SET observed this failure to correct problems as the most serious weakness associated with self assessments.

Corrective Actions The corrective action program generally did not prevent recurrence of problems. In 1997, an NRC inspection report noted that failure to recognize the significance of multiple failures of safety related components and to take prompt, effective corrective actions demonstrated a lack of owr'ership in the facility, a poor questioning attitude, and a willingness to accept substandard workmanship.

The ISA observed that the CA program was limited to CRs and MWRs for capturing problems. The ISA also observed that QA did not always review for resolution CRs it had initiated. In addition, the SET and ISA observed that the CARB was not consistently challenging the classification and evaluation of problems brought before it. The CARB did not consistently request or use CR trending information. In addition, the ISA observed that membership at meetings varied from day to day, organizational representation was often at too low a level, and some CARB members were not well prepared. The CARB did not routinely receive input from, or report to, station senior management.

The August 1997 Joint Utility Management Audit (JUMA) exit summary reported that follow-up on 1995 JUMA findings found recurring problems. For example, QA personnel 38

-- - . - ~ ~. . . - - - - . - -. ..- .- - - - - - - - -

r  :.;

)

- failed to identify costitions adverse to quality. SET interviews of management indicated i f that QA staff had little technical credibility.  !

QA audits found many problems with the CA program, but routinely concluded it was effective overall. -The OA audit report on the CA program, conducted during August 1997, reported: " implementation of the program requirements'and expectations is considered-  ;

ineffective in the following areas: identification and resolution of personnel errors, trending-of hardware and conditions, and the overall effectiveness of corrective actions.' .

... Corrective actions specified by Engineering, Maintenance, and Training appear weak, and-not supported by the investigation and root cause determination." NRC's augmented '

inspection team that assessed the July 22 and August 5,1997, electrical C/B failures

- concluded that IP corrective actions and PM activities were ineffective. Therefore, in a September 1997 Demand for Information, NRC required IP to provide further information to .

establish reasonable assurance that the CA program was effective.

In July 1997, the Director of CA position was moved from the Plant Manager's line organization and the scope of his responsibilities was narrowed to the CA program. The objective of this move was to provide increased management attention and support for the

. CA program.

899t Cause Analvsis The root ceuse analysis program was weak ~and contributed to instances of ineffective  ;

_ problem resolution. Root cause analyses in support of CAs were not consistently required or thorough. Interviews by the SET indicated that management was not fully committed to a strong root cause analysis program. Root cau:e analyses were sometimes not performed because the safety significance of CRs was not recognized. CPS did not have a formal

' human performance evaluation system. There had been numerous personnel errors, yet there had been few root cause analyses of these errors. When root cause analyses were '

performed, they were often shallow and made non specific root cause determinations such as " operator error." Other root cause analyses were so narrowly focused that the real root cause was missed, e.g., the failure of the electrical breakers in 1997. Workload and backlog contributed to accepting first causes rather than determining the root cause and to the failure to consider broader implications. Additionally, some individuals involved in the root cause investigation process were not effectively trained.

Plannina? Prioritization, and Schedulina

~

Planning,'prioritization, and scheduling were fragmented and provided ineffective guidance. --

Both the ISA and SET observed that there was no effective site-wide.and integrated planning, prioritization, and work management process. The CPS staff believed that there

- were too msny initiatives and progra.ns to permit effective definition of priorities. -

The.LTIP was a plan for a plan, it was not fully implemented to provide effective guidance t for improved performance at CPS. In the past, plans that had been poorly executed produced limited, if. any, measurable results.- There were neither goals nor measurable 39

_ _ . _ . _ _ _ ,. -_- - _ _ - ~ . - _ _ . ..---__ _ . . . -.

= . . w . . . - a _. ~_m w-, . - n .

n-- . - -.- ---- -

l m +

1 Lobjectises at the group and worker level. _There was no monitoring and use of foodback to - .

make revisions to the plans; The practica of a

• top down" approach to planning made iti  ;

. difficult to obtain buy in" at. lower levels in the organization. Individual organizations

~

Eworked independently,- because there were no integrated or consolidated schedules. -  ;

Planning, prioritization, and scheduling were identified as problems in most functional ,

areas.r  ;

l 3.5.3 Human Performance -

, The SET verified the weaknesses observed by the ISAin procedural adherence, resource ,

allocation, time management, and prioritization.  ;

Procedural Adherence Procedural adherence generally was not enforced. Management did not demand I accountability for failure to follow procedures and allowed procedures to be viewed as ,

- guidelines, in April 1997, CPS addressed this problem by revidng procedures to require adherence, in a June 1997 letter forwardir g the notice of violation regarding the reactor -

recirculation pump seal failure, NRC considered two of the causes of the event to be careless disregard for procedure requirements and a lack of conservative decisionmaking.  ;

The subsequent Systematic Assessment of L!censee Performance (SALP) report noted that management failed to provide appropriate control over procedures, and was unable to provide effective guidance or. procedure use until after significant NRC involvement. The ISA and SET noted that procxdural adherence was a site-wide problem.

Resource Allocation s

Human performance was impaired by management failure to provide the resources 5~ appropriate to programs and processes to ensure their effective implementation and ,

problem resolution, and to prevent a large backlog Management's decision to discontinue the training program for newly licensed operators significantly reduced the ability to move individuals with Operations experience into other organizational units to provide Operations insights into problem solving. According to a CPS maintenance rule self-assessment, e

ineffective management attention and resource allocation resulted in industry experience and NRC inspection reports not being reviewed and the maintenance rule not being

effectively implemented.

Management did not allocate adequate effective time to train engineers or adequate

- Engineering staff to effectively resolve recurring equipment problems, such as the Westinghouse C/Bs and recirculation pump seal failu.es.

Management did not provide sufficient attention or appropriate resources to pre rent the t

following: (1) large backlogs of ECNs dating back to 1993, contributing to the significant

_ backlogs of engineering work, (2) the backlog of 400 safety-related PM activities,  ;

- (3) backlog cf 2700 MWRs and (4) the backlog of about 2700 procedure CCFs. The allocation of inappropriate _ staff resources also contributed to the ' untimely review of 40 6

-wery e v. . -y w r,w oe. -, --e+--,-m-- m m-e w-m+ -

wm-e-=v+i*t-xg v y w en- 6.i--a- v+---* +d--+a "%-ie+e54 =e -- - +> + -=v - - +W-r-4 = m' - * - '- -N+*

- . . - -- - --. . - ~.

- e- .-

a generic, industry, and vendor information and the backlog of revisions to drawings and vendor manuals.

Management recently recognized that the resources it provided were inappropriate to resolve these problems. Management also recently hired additional staff and obtained contractor assistance to address the backlogs and workload.

Time Manaaement and Prioritization Poor time management and prioritization impaired managers' effectiveness. Many management and supervisory personnel commented that they did not have sufficient time to do their job due to an excessive number of meetings and other interruptions, in addition, these managers reported a lot of overtime. Some meetings were not held to an agenda or timetable, and unnecessarily consumed management's time. Managers could have delegated meeting attendance in some instances, if subordinates had been effectively developed to represent the managers' expectations.

Managers did not demonstrate a clear vision for improved performance needed and did not establish priorities on the basis of the significance of the need. CPS appeared to be in a reactive response mode to problems and focused on the day to-day issues. CPS confirmed that time management and prioritization were recognized problems.

4.0 ROOT CAUSES The Special Evaluation Team (SET) concluded that based on the assessment of Clinton Powei Station's (CPS's) performance from June 1995 through October 1997, there were four root causes for the declining performance at the Clinton Power Station. The first root cause identified by the SET was that management generally did not provide effective performance standards. The second was that programs, procedures, and processes were not effective in ensuring that station activities were consistently conducted in a safe manner. The third was that problem identification was inconsistent and evaluation and a corrective actions were generally ineffective. The fourth was that management did not ensure the infrastructure was sufficient to support changes. The SET's root causes are stated and developed in Sections 4.1, 4.2, 4.3, and 4.4.

- 4.1 Ineffective Parformance Standards Established by Manaaement

-The general failure by lilinois Power Company (IP) and CPS management to establish and implement effective performance standards was a root cause of the significant decline ia -

performance. Management generally failed to establish and communicate appropriate, clearly defined expectations and priorities and monitor their implementation for the desired performance. Management decisions that were inconsistent with stated expectations also 4 contributed to declining performance. In addition, management did not give sufficient feedback to the staff and failed to establish accountability.

41

. Management did not clearfy define appropriate expectations for performance and prie ties.

' Some of senior management's formally promulgated expectations were not appropriate.

Sonict managemern expectations during the 1994-1995 downsizing and reengineering-were 'ocused on lowering the cost of production. The 1996 CPS Business Plan clearly

=

placec the greatest emphasis on low-cost production. Proper balance and priority were lacking between reducing costs and' meeting schsdules versus assuring nuclear safety.

This wac evidenced by reduced emphasis on conservative decisionmaking. CPS's root ~

cause invotigation of the September 1996 reactor recirculation pump seal event noted that management's expectations for conservative decisionmaking were vague and unclear.

When expectations were established, management failed to provide performance indicators for station, department, and individual performance measures.

Following the September 1996 reactor recirculation pump seal event, senior management attempted to define safety as the first priority. This change in focus was stated in the

- January 1997 version of the CPS Business Plan. However, corporate and senior station management's newly stated (1997) expectations were not effectively translated into station goals and objectives and internalized throughout the organization. Therefore, the organization did not understand them and was unable to accept them. As a result, these attempts to define expectations with a safety focus did not result in significant changes in performance. The inability of management to fully develop and implement the Long-Term improvement Plan was indicative of this problem.-

Management did not effectively communicate recent safety focused expectations and obtain staff acceptance. Corporate management seldom visited the site and, as a result, had limited opportunity to define and reinforce expectations with senior plant management.

The lack of senior plant management presence in the plant also contributed to ineffective promulgation of management's expectations to the plant staff.

!nsufficient explanation by senior management of the bases and context for changes made during downsizing and reengineering during 1994-1995 contributed to a reluctance to communicate between organizational units that continued until the Integrated Safety Assessment (ISA) and SET were on site, in addition, plant management did not clearly c.mmunicate expectations for organization coordination priorities and the quality of the work product. This, in part, contributed .to departments operating independently of each

- other with different schedules and conflicting standards and priorities. This lack of coordinated plannmg resutted in an increase in PM activity, engineering work, and procedure change backlogs, and unnecessary personnel exposure.

Middle management and supervisors had a limited interaction with their staff. As e result, there was insufficient reinforcement of expectations through coaching and counset'g.

= Many of these managers were relatively new in their jobs and either were too busy or not fully familiar with their proper role, in addition, as a result of ineffective vertical-communications, there was a lack of acceptance of management's decisions, including goals and objectives.. This was substantiated by staff reports that there was a "we versus c- they" attitude between managers and workers. Plant' personnel stated that performance measures had not been clearly communicated throughout the departments.

s 42

%7

Management's actions were at times inconsistent with stated expectations. On : .

of several recent nonconservative decisions, it appeared that management did n; consistently consider safety les the first priority. Certain decisions appeared to I--

made with production and schedule as the driving force in contrast to safety. /

management's actions at times conflicted with stated expectations, sending a rn message to the organization. The disparity between management's formally stc -

expectations and the implied expectations created a credibility gap. For examp!' ,

management failed to demonstrete a clear commitment to stated expectations it material condition by not giving a high enough priority to the longstanding matc-problems on the " top ten" list.

Management set expectations for conducting technical and supervisory training, little importance on accomplishment of training. Engineering supervisors indicat they had not received any supervisory training in more than 6 years. As a resuh priority given to training and retraining staff in techr.ical skills, important enginer process tasks, such as operability evaluations, safety evaluations, and calculati:

being performed by unqualified personnel. Although management's expectatior system engineer responsibilities were clearly addressed in Nuclear Station Engin.

Department (NSED) procedures, these responsibilities were given low priority en:

met.

Management did not effectively monitor performance and enforce expectations.

little performance monitoring of activities such as problem tracking, performance field supervision, and completion of work in accordance with priorities. The ISA found that management performed limited review and evaluation of achievemcr upon defined goals and objectives. Meaningful performance indicators were nc*.

established to monitor organizational performance.

Management did not effect;vely monitor the performance of design activities, T.

-elimination of supervisory Engineering positions rer: ed in ineffective monitorin; Engineering activities. Management did not track or trend design-related probler though significant errors had occurred when making design modifications. Eng" interviewed could not recall any structured feedback during the last 2 years rett performance of design modifications.

. Management monitored and evaluated human performance inconsistently and i .

The termination of the performance appraisal system and the absence of an incc:

program linked to results caused performance, both above and below standard, unrecognized.'

Management did little to reinforce expectations by mentoring, coaching, giving f; and training. As a consequence, engineers failed to meet expectations 'or dete!..

walkdowns, proactive review of maintenance, and trending of system performa:

Operators indicated an inconsistent and infrequent presence of managers in th:

m room and in the plant to monitor activities. Shift supervisors (SSs) and line as aupervisors (LASSs) placed to'o low a priority on the supervision of the crew. T 43

L in missed opportunities to reinforce expectations for communications, procedu al adherence, and conduct of operations.-

4 Management did not consistently hold organizational units or personnel accot.ntable. ,

Without clear performance measures, organizational units could not be held accountable.

Management placed little emphasis upon personal. accountability for procedure adherence or established work practices.- This gave the staff the perception that adherence to established procedures and work practices did not have high priority. Management promulgated disciplinary measures for inappropriate actions; however, these measures

- were not enforced, contributing to the lack of accountability. For exernple, although management recognized recurring equipment tagging problems, no specific actions were taken to hold personnel accountable for mistakes CPS identified the lack of accountability i as one of the root causes for the decreasing trend in the performance of maintenance.

4.2 Ineffective Defense in Death Provided by Proarams. Processes, and Procedures CPS programs, processes, and procedures did not provide defense in depth to ensure that plant activities were consistently conducted in a safe manner. .

The programs, processes, and procedures generally failed to integrate activities across departments, incorporate industry information, and clearly delineate ownership and accountability. Program implementation was not effective in achieving the intended objectives. Processes and procedures generally were overly cumbersome and failed to provide appropriate guidance and unnecessarily challenged workers performing ca activity.

Programs and processes did not provide effective monitoring and feedback.

I Programs, processes, and procedures failed in several instances to incorpomte industry information and to integrate activities across departments. Programs, processes, and procedures did not benefit from the lessons of industry. The decline of programs, processes, and procedures with respect to industry standards was undetected by CPS

, because isolation from industry removed the benchmark. Setpoint, calculation, surveillance, and PM programs were not consistent with those of industry counterparts, i For example, the setpoint program did not define a standard setpoint methodology nor was the program described in an upper tier document. The program for maintaining CPS's calculation index was not comprehensive and did not provide the ability to cross reference i calculations. Safety tagging, surveillance, and PM programs were fragmented among

- organizations and did not stringently ensure safety and timely completion of maintenance

> and Technical Specifications (TS) surveillance testing. In addition, the reengineered organization became overly compartmentalized, significantly limiting opportunities for peer checking, information transfer, and efficient coordination. The ISA and SET identified that ineffective interdepartmental reviews contributed to several procedure problems. Also, the

. lack of an effective site wide, integrated planning, prioritization, and work management

, process contributed to an increasing backlog of work activities and delayed completion of

important tasks.

~

4 f

44 l

__----L----- - - . _ - . _ - , ~ - - - - - -..-..a... n . , .-, c , -

...,n, .- . , - ,,,. , , - .,, n, n - ,,, , -- , ,.

x s-f Program implementation was not effective in consistently achieving the intended objectives. Program implementation and follow through were not aggressive in programs such as fire protection, rquality assurance (QA), and the corrective action review board ,

(CARB). Some programs were implemented by going through the actions without meeting the intent. For example, the CARB process did not ensure that problems were appropriately evaluated and corrective actions were effective. The QA program identified problems but corrective actions taken did not ensure effective resolution. Lack of follow.

through in the generic letter program implementation contributed to problems involving the sai ity related service water (SX) system and the automatic depressurization system (ADS).

important program changes and TS amendments were implemented without disciplined follow through. The program for changing surveillance test procedures failed to ensure that design bases were effectively tested. For example, the high pressure core spray (HPCS) initiation sealin signal was not verified by a revised surveillance test procedure.

Significant programs, processes, and procedures did not clearly delineate ownership and accountability. The ISA and SET observed a widespread problem with delineation of ownership and accountability for program and process implementation. The CARB process did not effectively ensure appropriate problem ownership for CA plans. The Nuclear Performance Reliability Data System (NPRDS) program lacked ownership and was terminated for a period. Poorly defined program ownersh;p and accountability were reflected in the uncorrected siltmg problems and lapses in chemical treatment of the SX system. Fragmented ownership of the preventive maintenance (PM) program contributed to the inappropriate deferral of PM activities. Weak program ownership and accountability resulted in some programs, such as "as low as is reasonably achievable" (ALARA) and the performance improvement programs, being neg'ected in favor of emergent activities.

Processes and procedures generally were cumbersome, failed to provide appropriate guidance and relied too heavily on individuals for effective implementation. Cumbersome procedures contributed to poor attitudes on procedure compliance. Poorly coordinated processes and procedures allowed gaps in assurance that importar t requirements were i appropriately performed. Maintenance work packages were observed to be cumbersome and overly reliant on craft skill and experience for successful completion. Processes for obtaining updated design information were fragmented and susceptible to errors resulting

> from failure to ensure timely update of design-basis documentation. Operating procedures

. did not provide effective guidance to complete tasks and relied on operator knowledge.

The setpoint program lacked appropriate guidance on methodology. Engineering lacked an i effective method to cross 7 reference calculation outputs to other affected calculations. The 4 cumbersome and inffective processes and procedures failed to provide effective defense in depth and too uften challenged the individual.

. Progs ' ms and processes generally did not provide effective monitoring and feedback. The'-

reengineerino program, for example, failed to incorporate effective mechanisms for i

management monitoring and feedback. Similarly, the engineering design change process was not effective in monitoring performance or providing' management with needed feedback. Processes for evaluating important generic issues lacked effective requirements for timely review or monitoring. Suspension of the performance appraisal process -

45 J

e--.. - e .--w--m.-* .-,- ,a- w. -. _ - , - + e . = -r,r - * -+,w -..- - e,w -- % .--e.=-. +wt,- ,--, e-,s-r= -- - . + = -

A

+ .

oliminated an important mechanism for providing feedback to station personnel. Also, the Nuclear Review and Audit Group (NRAG) process did not ensure that the Board of Directors understood the group's concerns, such as the impact on CPS from thw loss of continuity of management expertise.

4.3 inconalstent Problem identification and ineffegtive Evaluation and Corrective Actions The persistent inability to identify, evaluate, and correct problems was a major impediment to improving performance inconsistencies in problem identification resulted in failure to ensure that problems were effectively captured. Ineffective evaluation of identified problems contributed to failure to develop effective corrective actions. Failure to monitor and ensure implementation of CA plans contributed to recurring problems and an attitude of living with problems, inconsistencies in problem identification resulted in a number of failures to ensure that problems were effectively captured and fully resolved. Problem identification by the CPS staff was not effective because equipment and system performance trending was inconsistent. CPS was not proactive in that industry experience contained in publications such as General Electric (GE) service advice letters and NRC information notices was not incorporated into repair and PM activities for circuit breakers (C/Bs). Also, Electric Power Research Institute (EPRI) guidelines were not used in the procedure to install freeze seats.

CPS problem identification through self assessments was not required by procedure until 1996. Therefore, expectations for conducting self assessments were not effectively definsJ or understood. The August 1997 Joint Utility Management Audit (JUMA) exit summary noted a failure of QA personnel to identify conditions adverse to quality.

CPS reduced opportunTis for problem identification when surveillance intervals were increased.- For example, the SET discovered that mud dauber nests blocking a numtwr of fire hoses went undetected because monthly hose house inspections were replaced oy annual inspections. The Unit 1 SX pump bay had not been inspected recently for sitting.

During the ISA,3 feet of accumulated silt were discovered.

An NRC inspection concluded that the engineers did not demonstrate a questioning attitude in the assessment of water in the reactor core isolation cooling (RCIC) turbine lube oil.

Some staff became reluctant to report problems bec6use identified problems were not effectively resolved, ineffective evaluation of identified problems contributed to the high incidence of ineffective corrective actions. Root cause aralyses were sometimes not performed because the significance of some condition reports (CRs) was not recognized. The Corrective Action Review Board (CARB) failed to challenge the classification or evaluation of CRs presented at their mcetings._ Trending informa. ion associated with CRs was not presented to the CARB. Causes of adverse trends were not always determined, in addition, root cause analyses did not consistently consider generic and common mode implications.

Management did not effectively enforce consistent standards for Engineering involvement 46 I

I

in corrective actions. _ in other cases, the root cause was so narrowly focused that the real root cause was missed, leading to corrective actions that did not prevent recurrence of problems. CPS failed to recognize the significance of multiple failures of safety related components. Frequently, the first or easy answer was accepted without a strong, self-critical questioning attitude. This resulted in shallow corrective actions that did not completely eliminate the problem, ineffective safety evaluation screenings have resulted in the failure to perform required safety evaluations. Examples of ineffective evaluations of root causes have contributed to the longstanding problems with reactor recirculation pump seals, C/Bs, and personnel errors.

Failure to monitor and ensure implementation of corrective action plar s resulted in a number of instances of recurring problems and an attitude of living with problems. The CPS staff were not confident that the CA program would resolve their issues. A number of CRs documented problems that occurred for a second or third time. When corrective actions were developed, management frequently failed to establish performance measures

. to track the effectiveness of the corrective actions. A lack of ownership for corrective actions hindered effective implementation. For example, actions to correct problems with radiation worker practices received little support from the plant staff. Few performance measures had been developed to assess the effectiveness of corrective actions, ineffective implementation of corrective actions was demonstrated when the failure to revise an outage procedure checklist resulted in a second inadvertent isolation of the reactor water cleanup system. Correctives actions to improve personnel performance with respect to the tagout program were not implemented. Consequently, problems with implementation of the tagout program continued.

4.4 Infrastructure ineffective to Suncort Maior Channes in 1994 IP set cost reduction as an objective in its business plan. IP believed that CPS was a good performer with significant margin in their performance. Consequently, reengineering was initiated with the focus on cost reduction and downsizing. IP management failed to assess the CPS infrastructure to determine whether it could support the changes planned and sustain defense in depth and a healthy safety culture. IP management did not recognize that CPS's programs, processes, and procedures that had functioned prior to reengineering, were cumbersome and not sufficient to properly function in a downsized organization. Also, management did not recognize that, although CPS employees were willing to be led, they required effective leadership to establish appropriate ownership and accountability. As a result, management made organizational, programmatic, and resource decisions in the context of reengineering without appropriately considering the longer term, integrated effects of their decisions. Management did not ensure that there were appropriately qualified staff, integrated programs and processes, and appropriate resources to support implementation of the reengineering and downsizing effort. The SET considered that this root cause made the greatest contribution to CPS performance problems.

47 1

Management's failure to develop managers, supervisors, and staff to implement the organizational changes contributed to weaknesses at alllevels of the organization. Two senior CPS management positions were filled for a period of time with individuals who lacked any recent nuclear operating experience. Succession planning was ineffective. -

Several key positions at the station were inappropriately filled, left vacant, or filled by temporary employees when several managers left in late 1995 as a result of a "buyout" program that was part of downsizing. For example, in Radiation Protection and Chemistry and in OA, directors were assigned areas to manage, although they had no technical expertise in these areas. This inhibited communications within the departments and to plant management, in addition, the elimination of supervisory engineering positions resulted in ineffective oversight of Engineering activities. This led to reduced attention for proactive Engineering programs for monitoring and trending system performance. The change to a multi-discipline team structure at a time when the rest of the site organization was becoming compartmentalized made it difficult for engineers to obtain assistance from other engineers in the same discipline. The team approach confused owner = hip of programs, systems, and issues, making accc mtability and responsibility unclear.

Technical and supervisory training was not given a high priority in any department.

Engineers were assigned tasks or system responsibilities for which they had not been appropriately trained, but were reluctant to request training because of the Engineering workload. Refresher training in fundamentals for Radiation Protection (RP) technicians was discontinued. Qualification requirements for Maintenance tool box skills were not well defined and not consit,tently implemented.

Management's failure to ensure that programs and processes were effectively integrated across departments generally resulted in departments working independently and hindered teamwork between departments. Management did not initially recognize that its foremost concern for cost reduction and downsizing was causing people and organizational units to become apprehensive, contributing to departmental isolation. As a result, programs and process were redefined within departments rather than integrating the changes across departments, in addition, by using a top-down approach to redefine programs and processes, management did not get buy-in from the staff, creating problems with program

> and process ownership. The work control process, for example, became fragmented and weak. Site wide integrated planning, prioritization, and work planning processes becsme ineffective. For example, if maintenance work was not scheduled on both the Operations and RP schedules, it was delayed or the craft was sent back to the shop, unable to complete the work. Several plant programs functioned ineffectively because they had no clear owners and lacked effective communications between organizations. Management did not consistently insist on thorough implementation of important programs, such as for generic letters, the maintenance rule, or ALARA. The elimination of the Configuration Management group of Engineering removed checks and balances in the design control process.

Management did not consistently ensure that appropriate resources were available to effectively suptiort major programs and to prevent CPS's isolation from industry.

Management's decision to discontinue regular initial operator licensing classes resulted in 48

insufficient staffing to support the Operations Department workload without the extensive use of overtime, it also limited the opportunities for operators to move into other areas of the organization contributing to departmental isolation and lack of Operations' ownership _

throughout the plant, in addition, staffing for several programs, including the ALARA program and the industrial safety program, were ineffective since the 1995 reorganization.

Many defined system engineering functions, such as incorporating industry experience, were not accomplished in the reengineered organization.

Management's decisinn to reduce or eliminate staff visits to other plants for benchmarking contributed to the isolation of CPS from industry. As a result, management did not recognize that performance at CPS was declining relative to industry standards. In addition, management's decision to discontinue some industry subscriptions, such as GE service advice letters, contributed to the failure to capture industry experience.

9 49

ATTENDEES LIST For Exit Meeting on December 11,1997 l IP Attendees G.L. Baker, Manager, Quality Assurance K.A. Baker, Director, Resource Management, NSED 1 W.P. Bousquet, Director, Plant Support J.G. Cook, Senior Vice President, Illinois Power J.M. Gruber, Director, Corrective Action, NSPl L.D. Haab, Chairman, President, CED J.A. Hale, Director, Planning and Schedullrg, R.A. Joyce, Assistant Plant Manager, Maintenance J.R. Langley, Director, Engineering Projects, NSED ,

R.F. Phares, Manager, Nuclear Safety and Performance Improvement  !

J.J. Place, Director, Chemistry and Radiation Protection [

W.D. Homberg, Assistant Vice President, Illinois Power t J.V. Sipek, Director, Licensing, NSPl J.R. Taylor, Directct, Administration, NT&S D.W. Waddel!. Director, independent Analysis, NSPI J.C. Womlinger, Director, Operations Training and Emergency Response, NT&S L.S. Wigley, Manager, N'Jclear Station Engineering Department R.M. Wyatt, Assistant Plant Manager, Clinton Power Station ,

and other CPS staff E. Fuller, Lead ISA Team NRC Attendees .

S. . Collins, NRR A.B. Beach, Rlli

~ J.A. Grobe, Rill

• T.J. Kozak, Rlli J.A. Hopkins, NRR K.K. Stoodter, Rill K.E. Perkins, RIV F.R. Huey, RIV T.W. Pruett, Rlll P.J. Kellogg, Ril M.E.~ Ernstes, Ril R.J.~ Bores, RI J.A. Kilcrease, RIV 50 ,

m y . , , . . . . , . , - -

a t

[

t APPENDlX A

SPECIAL EVALUATION TEAM *  !

i VERTICAL SLICE EVALUATION OF FIRE PROTECTION FIRE PROTECTION PROGRAM  !

- t Sackersund J

Following resolution of generic Thermo Lag 330 lesues, NRC had scheduled a pilot fire l protection functional inspection (FPFI) to be performed at the Clinton Power Station (CPS)  !

in August 1997. In preparation for the FPFI, the licensee performed an independent audit of fire protection. The audit identified insufficient documentation of the consequences of a  :

fire or the ability to suppress a fire. Of 16 cundition reports (CRs) issued,11 were f attributed to insufficient analyses.  !

IE028 l The Special Evaluation Team (SET) reviewed CPS's fire protection program to verify that the station had properly implemented and maintained the fire protection program required ,

i by the operating license. The SET reviewed fire protection procedures, administrative ,

controls, quality assurance findings, fire brigade qualifications, and fire bngade staffing in accordance wnh the approved fire protection program. The SET reviewed the licensee's  ;

Ouality Assurance Audit 038 97 05, " Fire Protecticn," issued on July 24,1997,CRs ,

resulting from the audit, and additional CRs related to fire protection. The SET also ,

conducted extensive walkdowns of the facility to verify licensee implementation of the approved fire protection program.  :

Problems Related to Safe Shutdown The SET reviewed CR 197 06 310 which captured the safe shutdown issues id%tified in the independent audit. The licensee concluded that the root cause was "ir. '4 terminate."

The SET noted that many of the issues identified in the audit had not yet been evaluated  ;

by Engineering. Some of the issues could require plant modifications, revision to the l remote shutdown procedure, and addtional emergency lighting. The licensee also i identified that air for the safety relief valves (SRVs) to support taking the plant to .

cold shutdown conditions would only be avaltable for approximately 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> instead of the 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> required by NRC regulation. Another issue potentially affecting ability to  !

achieve fire-relowd safe shutdown was a ceccern that the licenses had not resolvsd  ;

actions necessary to address the possibility of a single fire caucing all 16 SRVs to open simultaneously. The licenses had also not resolved an issue raised in 1995 of a single fire. j causing the loss of offsite power and spurious actuation of the emergency diesel generator (EDG) toom carbon dioxide system that could result in the loss of all diesel- -

generators.-

1 51 n

+ -

. - . ~ < . . . _ . - _~6-,_----, m - . . . , . . _ , ...,_.-.,--..-.--,m,-.,m.m... -,--.-~,,--a,--- _,.,,,,---.;

_ ._.e.____..___._

I l

fire Barrier Penetration $ pals

The licensee was unable to give the SET the requested test data to demonstrate that three J - randomly selected fire barrier penetration seals were installed in a configuration validated t by a fire test. The licensee did not cle.asify the fire barrier penetration seals as inoperable.  !

National Fire code comollana.e j The SET noted, in general, that fire detection systems appeared to comply with the .j National Fire Code (NFC). However, the service water pump rooms were observed to have  ;

on elevated ceiling and the fire detection system did not appear to comply with the current .[

NFC for such a ceiling. Also, the licensee's individual plant extemal event examination i

. (IPEEE) stated that credit was taken in three fire areas (two cable spreading rooms and the- l Division ill switchgear room) for automatic fire suppression allowing a reduction in ]

core damage frequency by a f actor of 266. Ahhough the licensee had evaluated and taxon l 4

corrective actions for the sprinkler obstructions in the cable spreading rooms, the SET  !

= noted significant obstructions to sprinkler flow patterns in the Division ill switchgear room.

Fire Protection Eaulament Surveillance Freauencies _

The SET noted that the licenses had extended surveillance for some fire protection systems

beyond that previously approved by NRC or specified in NFCs. Monthly hose house t

- inspections had been extended to an annual inspection. The SET noted mud douber nests j blocking the inside of fire hoses in each of two hose houses opened. Subsequently, the licensee identified a total of 11 fire hoses blocked by mud dauber nests. The last ,

inspection had been performed in June 1997. The licenses committed to revert to monthly l hose house inspections.

4 The SET noted that the licensee, during plant licensing, had been granted numerous ,

deviations from NRC fire protection guidance. Many of tl.a deviations were,in part, based

- on having fire detection and suppression in a fire area. The SET observed that the

- licensee's staff was closely monitoring the performance of associated fire protection i systems to ernure that extending maintenance and surveillance frequencies beyond the j NFC did not affect system performance. ,

f

. Fire Protection Staffina ,

t The SET noted that the current number of fire protection staff was consistent with the industry standard. However, in 1995, the fire protection staff was est.igned additional

duties. One engineer responsible for program implementation told the SET that only 20 l

- percent of his time was available for ;.ie protection activities. Ths SET also noted that during that period,'there was a significant increase in the number uf CRs related to fire U - protection and in the number of fire protection impairments. The SET concluded that the 1 increased number of CRs'and impairments occurred, in part, because the staff had insufficient time to devote to fire protection activities.

62 9

f w+v v-gt,,'ys,gog^-witg-'v-1r -

aT wa ee--s e b es a .4-e*-J-*-u--1'NC-i e J en 59 Mar" e e--se g--e s ase --hr-**% tr e- **we4---*st w s.=- u su'e evem i prv31,-eyy y-w ye y v-n vi w -y - r vy-wwyn reswwwpy ir iygyg w tggr grgyn gwe - g -

I

(

.. .. j 1 J

4 planolution of Lonn Term Fire Protection issues j

- The SET noted that several fire protection issues identified in 1995 had not been effectively addressed, including structural steel components with thermal shorts, i inaccessible and irsperable fuel pool fire detectors, some fire barrier penetration seals not  ;

installed in the control room, and the potential for a single fire to cause loss of offsite f power and loss of all diesel generators. The licensee had used an hourly fire watch in lieu

, of addressing identified problems. This action is counter to NRC guidance in GL 91 18,

'information to Licensees Regarding NRC Inspection Manual Section on Resolution of l

- Degraded and Nonconforming conditions,* which discourages the use of compensatery l measures instead of restoring equipment to full operability.  ;

i conclualon 1 in general, appropriate fire protection administrative controls and procedures were - i implemented, fire watch personnel were well qualified, plant housekeeping was effective for control of transient combustible materials, and, with one exception, station fire -

response equipment was well maintained. However, the SET concluded that the licensee could not demonstrate the design adequacy of installed fire barrier penetration seals.

Additionally, the SET concludec' that the licensee could not demonstrate the ability of the existing post fire safe shutdown analysis, equipmerst, and procedutos to offectively ensure ,

that the plant could achieve and maintain safe shutdown conditions following a fire.

FIRE PROTECTION FACILITIES AND EQUIPMENT- .

The SET performed a walkdown of all areas of the facility containing safe shutdown -

equipment. The SET performed in depth walkdowns of the screenhouse, the Division lli  ;

switchgear room, and the cable spreading rooms. These areas were chosen because of ,

their relatively high contribution to core damage frequency. The SET also randomly l

- selected components required for safe shutdown during a control room fire to verify that  ;

they were accessible, and welllabeled, and that they had effective emergency lighting to  ;

perform required tasks. 2 The SET observed that fire response equipment was well maintained, accessible,  :

I calibrated, and in good working order. All valves observed by the SET in the fire suppression system were in their proper position. Fire water purops and equipment were operable and well maintained. All fire brigade response equipment located in the storage locker in the t'nbine building was well maintained and ready for immediate use.

' A walkdown of the service water pump rooms showed that access to the Division 11 and .

Division til pumps was through a door from the Division I pump room. The primary doors

- to the Division ll and 111 rooms were chained shut from the inside. The chains were -

installed to meet a security commitment to NRC.- The other door opening from the Division I toom to the Division ll and 111 rooms.was restricted by a door stop installed to protect a

- motor control center.: The NRC approved fire protect 6n program required the fire brigade <

leader to have access to all plar,t areas in an emergency. The SET noted that fire brigade 1

- 53 i

t a

. . .. ..r

t

+- -.

members in turnout gear and self contained breathing apparatus (SCBA) nsay not fit  !

through the door opening from the Division I pump room, in addition, accussing a fire in

- the Division il pump room from the Divislen i pump room would expose both i soft shutdown trains to damage from a single fire.

i fanrdalen  :

Fire protection equipment required for program implementation was generally well  !

maint sined and available for immediate use. The SET considered fire detection and alarm capatility to be good. The Divisior, il and lll pump rooms were not readily accessible for i fire brtgade response-FIRE PROTECTION STAFF KNOWLEDGE AND PERFORMANCE Discussions with engineers indicated that they understood the NRC requirements for ths  !

t fire protection program and the NFC. They also demonstrated a detailed knowledge and understanding of station fire hazards, systems, testing, and analyses associated with the . i fire protection program. The SET observed that the fire protection personnel had a very l good working relationship with other onsite organizations.

Conclualon The fire protection staff was observed to be well qualified and had a good working relationship with other station organizations. I

. FIRE BRl0ADE PERFORMANCE The fire brigade consisted of five individuals per shift. The fire brigade leader was a -

licensed operator who had sufficient knowledge of safety related systems to understand t the effects of fire and fire suppresunts on plant safe shutdown capability. All personnel [

received initial and requalification training and were required to participate in at least two

- drills annually, in addition, the licensee performed quarterly fire drills. Fire brigade-personnel were also interviewed by the SET to determine their knowledge of the fire brigade program requirements and the use of water on an eier,trical cable fire. The fire brigade members had a good understanding of their duties and of fire fighting requirements. Fire brigade members stated that they were not assigned duties that would i interfere with their ability to respond to a fire. -L The SET observed an unannounced fire drill in the Division ll cable spreading room. The

. fire brigade promptly responded with proper equipment to suppress the firs. The SET noted that the licensee demonstrated proper use of the equipment and had good d communication capability. The fire brigade leader displayed good command and control during the drill. .

t 64 q

t

,.--..m. . - . -,E.,.,

.. ,-, , = - --.v...-.e, ,w,.v-,-.-,.yw,v,..- .'--. , - . , , - -' . .~w,,. , - . . . , mr.~-,.-

~.. . _ . . - _.. _ _.. ._.__ -- _ .. _ . .. .-.--_ _ . . . - . ___._. _ _ _ . - _

i t

.o .,

.6 . i e

it

,- t 4 t Conclusion i i

Fire brigade knowledge and performance were a program strength. -  :

L i

?

i i

t 6

1 2

J T

-U

+

h b

5 i

e s

4 O

E f s

P b

f a

a 1

55 r-W P

4

,,4,Lm - .gy'.. , , , --- -,.- .,.s f r... ~

-,_.,-....,,,--,,,y-, ,yy y ,.,p , , , ., ,v.y,---- ,y--w-.,.y--, r-

.>,I!t ) ,,1  !:' \

7 .

9 -

9 S 1 N IER KG

~

1

_ RA C T1 I

E P N R.

XR EE HM.

T A N.

S B E M N M U.

A E NE C ET K .

E D

, i I,  ;

4 i

l I

AGENDA  !

i e INTRODUCTION & BACKGROUND ,

i e ISA CONCLUSIONS l e SETASSESSMENT e SET ROOT CAUSES 1

i l

i

. . _ . . , _ . _ . . , . _ ,.,_..-_._.-_.,.__..-_.m_ -

l SET TEAM LEADERS I

e TEAM MANAGER - KEN PERKINS l e ADMINISTRA TIVE ASSISTANT - JUDY KILCREASE l e ENGINEERING - RANDY HUEY

e OPERATIONS . MIKE ERNSTES e MAINTENANCE - PAUL KELLOGG e PLANT SUPPORT- BOB BORES

e MANAGEMENT & ORGANIZA TION - HENRY BAILEY

}

l l

_-. . - . _ . _ . - _ - . _ , . . - - . . . _ _ . _ . _ _ _ . _ - . _ _ _ . _ _ . . _ - - _ . _ _ _ _ _ _ . _ _ _ _ _ . . . _ - .I

. _ _ _ _ _ _ _ _ _ . . _ - - - - - - _ _ = = ..

.i SET TEAM MEMBERS e OPERATIONS - RICH LAUFER l e ENGlNEERING - MIKE TSCHILTZ, FRED RURROWS, PHil QUALLS l e MAINTENANCE - JOHN YORK

e MANAGEMENT & ORGANIZA TION - JONATHAN WERT e OBSERVER -IDNS, CECIL SETTLES 2

e l

BACKGROUND i

I l e SEPT 1996 EVENT - INCREASED INSPECTION e BROAD BASED CONCERN FOR PERFORMANCE e SMM 1/97 - TRENDING LEII2R

e ADDITIONAL PERFORMANCE CONCERNS l e SMM 6/97 - DIAGNOSTIC ASSESSMENT e IP PROPOSES ISA e NRC SETMONITORS ISA e NRC SET VAllDATES ISA e ISA / SET ASSESSMENT 1

l i

i

. . . . - -. ... . - - . _ , . . . . - _ . . . . _ - - . - . _ ~ . . . . - . . . - - _ . . _ .

! r!! \! i !!l ll*

Y T

E FT R _

E _

AN SE RA D

EE DM L L L

ES TS UM F

E A .

AS DE E T RS _

G A EA T

S I

N I

9 i

I I

l

( t l SET PROCESS i

! i e DOCUMENT REVIEW

! e OBSERVED ISA IN PROGRESS e EVALUATE ADEQUACY AND INDEPENDENCE OFiSA i

e ROOT CAUSE DETERMINATIONS I

e SAFETY AND COMPLIANCE ISSUES PROVIDED TO REGION l

4 t

i i

FUNCTIONAL AREAS  !

i I

e OPERATIONS 1  :

i i

! e OVERSIGHT / EXPECTATIONS i e

~

l SUPERVISION .

I e PROCEDURES i l e LABELING i e STAFFING  ;

I l e CONDUCT OF OPERATIONS  ;

m i

.. _ - _._ , __. - __. _ _ _ . . _ _ _ . _ _ . ~ . .

l'

~

i l

FUNCTIONAL AREAS (continued) i i

e ENGINEERING i

l e RE-ENGINEERING

• EXPECTATIONS e TRAINING e PERFORMANCE MONITORING

! e PROCESS CONTROL l e EVALUATIONS

• SUPPORT OF OTHER ORGANIZATIONS i

i i

i' .

I

~

i i l

-l l FUNCTIONAL AREAS (continued) i i

l e MAINTENANCE

! i e COMMUNICA TIONS l e EXPECTATIONS AND DECISIONMAKING l e INDUSTRY EXPERIENCE e WORK CONTROL e PROCEDURAL ADHERENCE i

a

- - -. . _ _ - . . . . _ . . _--.\

d i

FUNCTIONAL AREAS (continued)  !'

! e PLANT SUPPORT i

l e OWNERSHIP

e DECISIONMAKING i

e CORRECTIVEACTIONS i e IMPLEMENTATION I

l l  !

I l

i

{

l l

l i

i FUNCTIONAL AREAS (continued) i f

e MANAGEMENT & ORGANIZATION

!

• EXPECTATIONS AND STANDARDS l e INDEPENDENT OVERSIGHT

, e CORRECTIVEACTIONS

, e RESOURCE MANAGEMENT

5 l

! l 1

! l 1  !

i

l .

l l-  !

i l

l l

l l

ROOT CAUSE #1 ,

i 1

1

• PERFORMANCE STANDARDS NOT ESTABLISHED i

e EXPECTATIONS NOTDEFINED  ;

l e NO EFFECTIVE COMMUNICA TIONS OF EXPECTA TIONS e INCONSISTENTACTIONS l e PERFORMANCE NOTMONITORED

e NO PERSONNEL ACCOUNTABILITY

. . i.

i i

) -

i i  :

j j 4

4 l .- ._ . .. . - _ -

4 i

t i

i ROOT CAUSE #2 t l

e PROGRAMS, PROCESSES, & PROCEDURES DID NOTPROVIDE

\ DEFENSEIN DEPTH i

e LACK OFINTEGRATED ACTIVITIES e IMPLEMENTATION NOT EFFECTIVE .

i e LACK OF OWNERSHIP AND ACCOUNTABILITY e EXCESSIVELY CUMBERSOME

• INEFFECTIVEMONITORING AND FEEDBACK

{

i

~

! i l

. . . - u

(

l ROOT CAUSE #3 i i j -

i

! e CORRECTIVE ACTIONS WEREINEFFECTIVE 1

e PROBLEMS NOT CONSISTENTLY CAPTURED e PROBLEM EVALUATION NOTEFFECTIVE e CORRECTIVE A CTIONS NOT EFFECTIVE AND NOT MONITORED i i.

i l

5 1

4._. . _ _ _ _. - - . . - - _ _ _ - __

l .

4 1

4 4

ROOT CAUSE #4 1

e CHANGE MANAGEMENTINEFFECTIVE

e POOR STAFFDEVELOPMENT e LACK OF LATERAL INTEGRATION e INEFFECTIVERESOURCE USE 0

s I

4 i

- . . . . - . ..---.-_-.\