Text

Maint Team Insp Rept 50-298/89-31 on 891113-1221.Violations Noted.Major Areas Inspected:Overall Plant Performance as Affected by Maint,Mgt Support of Maint & Maint Implementation
	ML20011E191
Person / Time
Site:	Cooper
Issue date:	01/29/1990
From:	Cummins J, Gagliardo J; NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:	;
Shared Package
ML20011E189	List: IR 05000298/1989031; ML20011E188; ML20011E190;
References
	50-298-89-31, NUDOCS 9002090010
	Download: ML20011E191 (53)
	v • d • e

p{4 q7

.

]

+

,

My 7.. w= q R.

-

!

'

-

,

,N

'g'

,g s {

Y

!

fgY e i

!

s

,

" ' O,} g,

' _

'!

~ APPENDIX B=-

. U."S. NUCLEAR. REGULATORY COMISSION

'

REGION.IV-

i

TNRC: Inspect' ion Report: 150-298/89-31 Operating License:: DPR-46

-

.

-Docket:

50-298;

'

~

l

.

?~

Licensee:L Nebraska'Public' Power. District'

i

.

P.O. Box 499:

M

'

Columbus,: Nebraska 68602-0499

'l

,

Facility Name:- Cooper. Nuclear Station

_

e

,

f.

,/ Inspection At:

Coo'per Nuclear Station, Brownville, Nebraska-i

'

3,

,

..

~..

.

"

~ "

, Inspection Conducted: ' November 13 through December 21, 1989 d

5.

g '.ed!L%t/J Team Leade JGryl4')1 s

O

'

<

'

J E. Cummins, Reactor Inspector, Operational Ddte

,

.

r

.

j'ProgramsSection,DivisionofReactorSafety;

,'

.

,m

Team Members: ; B.~, L. Bartlett, Senior Resident Inspector, Project Section D

[

+

Division of. Reactor Projects d

D., A. Beckman, Consultant

.

,.

a

.

T. 0. McKernon,- Reactor' Inspector, Operational' Programs Section:

,

Division of Reactor Safety

...

.

.G.~A.: Pick, Resident Inspector, Project Section C, Division of-

>

-

Reactor Projects

.

L. T.sRicketson, Health Physics Inspector, Facilities Radiological.

'

-

Protection Section, Division of Radiation ~ Safety and Safeguards j

'

D. H. Schultz, Consultant

[

}

x a

~

. Approved:-

/)

M

)i 2.f - Y

.

4.. E. Gigliardo, Chief, Operational Programs Date i

f,

\\ Sectmn,. Division of Reactor Safety.

'

{

'

w lt1,

]

,

L

!

9002090010 900133

'

PDR ADOCK 05000298 O

PDC

,

!

x ---

- -

-

.

,

.

t

,

a-

' Inspection Summary

~

~ Inspection Conducted November 13 through December 21, 1989 (Report 50-298/89-31)

>

Areas Inspected:

NRC staff and consultante conducted a special, announced.

maintenance team inspection (MTI) of maintenance programs-and-the performance-of maintenance activities (safety-related and balance _of plant).

This MTI-encompassed overallL plant ' performance as af fected by maintenance, management support of maintenance, and maintenance implementation.. The inspectors used the NRC Maintenance. Inspection Guidance, dated September 1988, and Temporary

'

Instruction 2515/97, Revision 1, dated September 22,_1989.

'

.

Results:

The inspectors concluded that the'11censee's maintenance process had been functioning to maintain plant components, systems, and structures.in a

,

condition such that they were available to perform their intended functions.

However, the inspectors observed.that certain elements in the licensee's

programs and implementation could be strengthened and identified two. apparent-violations:

-

,

Four instances of either a failure to follow procedures or inadequate

"

' procedures:

(1) Valve LV 123W-11 notisealed as required (Section 1.3),

(2) clearance not set prior to performing work (Section 3.1.3), (3) material storage requirements not met (Section 3.3.3), and (4) ~ material in storage

,

not' properly tagged (Section 3.3.3).

~

Failure to perform required quality control inspections-(Section 2.3.3).

~

!

.

I j.

a s

[

ii

.

-

.

-

_

. - -. - - _. -,,,,,...........

n s3

'

s EXECUTIVE SUMMARY-

%

from Noveober 13 through December 21, 1989, a team of Nuclear Regulatory Commission (NRC) staff and consultants conducted a performance-based inspectio?

erNuclearStation(CNS). The purpose of of the maintenance process at Coop (MTI) was to determine whether components, this maintenance. team inspection systems, and' structures at CNS were adequately maintained so that they would perform their intended functions when required. The inspectors evaluated three major areas:- (1).overall plant performance as affected by maintenance, (2) management support of maintenance, and (3) maintenance implementation.

The inspection was conducted in accordance with the NRC Maintenance Inspection Guidance dated September 1988, and the guidance provided in Temporary Instruction (TI) 2515/97, " Maintenance Inspection," Revision 1, dated September 22, 1989. Based on the latter, the team color-coded a " maintenance

' inspection presentation tree" that identified the major elements associated with effective maintenance. The team used this tree (attached to this report)

as a visual aid during the exit meeting to depict the results of the inspection.

The inspectors concluded that the licensee had developed a maintenance process that adequately maintained components, systems, and structures so that they would perform their intended function when required. However, the inspectors were concerned that the licensee's process relied too heavily on individual

"o (especially craft personnel) knowledge to perform activities correctly and to be disciplined enough to do the correct thing in all situations. The team based this conclusion on the lack of detail (e.g., requirements, technical information, and guidance) in the maintenance-related-administrative and control procedures and work instructions used in the field.

The team considered the experienced maintenance personnel's ability to maintain the. plant a strength. Nonetheless, the following findings exacerbated the-team's concern over-inordinate reliance on these individuals:

Instances in which clearances had not been set to ensure personnel safety.

prior to performing maintenance.

Instances in which appropriate post-maintenance testing had not been performed.

Instances in which the work accomplished exceeded the scope of work

specified on the approved maintenance work request (MWR).

Instances in which necessary technical and quality inspection requirements

(such as torque values, internal cleanliness, material controls, quality control inspections, and post-maintenance test requirements) were not included in the work instructions.

iii

-

_

.

..

_

.. _. _.....

. _ _ _.....

.

.._ _

,

E

..

"

,

i i f%

,

"

-

An' apparent. lack of discipline on the part of maintenance' personnel:in

' filling out MWRs, typified by the number of.MWRs identified which lacked

-

- required _information.

The team identified the following strengths and weaknesses which.are documented in the. report and were discussed with the licensee during the exit meeting.

JSTRENGTHS

-

.The material condition of the plant was very good.

"

The' licensee's staff usually demonstrated strong dedication to the organization and-involvement in the successful operation of the facility.

Site management had 'nitiated programs for performance improvement, such as procedure upgrades, trending, root cause analysis, and predictive

maintenance.

These programs had been recently initiated, and their effectiveness could not be evaluated.

Ths licensee exhibited excellent. internal communications, and appropriate

organizations were involved in the erformances of maintenance activities.

Management was well informed of current plant and system status, and the'

craft personnel = consistently kept-discipline ~ foremen apprised of problems and coordinated with support organizations when needed.

The licensee's radiological protection program appeared strong'in its implementation. The plant had relatively low levels of contamination, swaste, and personnel exposures.

WEAKNESSES There was minimal planning in the preparation of work instructions and maintenance procedures.

The team observed numerous instances in which

' work quality was negatively affected because workers lacked instructions and guidance.

The team identified instances where safety precautions were not taken and quality requirements were not met because of poor licensee procedures.

These included cases in which equipment clearances were'not used, in which

,

poor controls were established'for system cleanliness, and where there was t

a lack of an aggressive quality control inspection program.

"

Post-maintenance testing was addressed by the licensee's procedures,_but implementation was inconsistent, and post-maintenance testing sometimes was not performed.

Maintenance personnel appeared to lack discipline in controlling the scope of work and the quality of documentation for the work authorized by MWRs.

~

Jobs were sometimes expanded beyond the initially identified components iv

_ _ _........ _..

g

,

%.

.;

3

,,

and problems,. indicating poor work control.. In addition, jobs were i

sometimes so poorly documented that;it was difficult to establish what had

'

.

L-actually been done.

,

-

The licensee's programs and practices for ' identification and correction of l

"

-

conditions adverse to quality were weak and erratically implemented.

-

Problems were initially identified in work documents, but were not

.,

recognized as requiring identification to licensee. management and further p

followup.

Root cause analyses were not performed with sufficient'

consistency to assure prevention of the recurrence of the identified

- pro'ulems.

.

The licensee lacked an effective planning group for preparation'of maintenance work instructions and requirements.

Craft personnel were

-

responsible for preparing their own work instructions and related-

'

documents, but craft personnel were not acquainted with the compu.ter data.

~!,

'

base located in their shops that were needed to perform effective-planning.

The licensee had not achieved a sufficient balance between adequacy ofL

'

procedures and documentation and reliance upon craft knowledge and work

-

practices.

This weakness was characterized by incomplete work-

instructions, expansions of work scope, work clearance problems, poor e;

lifted lead controls, and insufficient documentation of work activities

-

for record purposes.

The licensee's method for controlling locked / sealed valves was not well I

defined nor understood by the licensee's staff.

The, licensee:'s process for design and configuration control-and the-

-

control of vendor technical information was. weak.

-

The licensee's QA audit criteria did not include a determination of the 1,

effectiveness of maintenance.

<

'*

The licensee's review of. completed. work control documents was not sufficient

-

to. identify many problems found by the team during their review of completed work packages.

The licensee's parts ~and material control efforts were marginal and provided

-[

L11ttle assurance that unqualified and nonconforming materials, parts,:and

'

chemicals would be installed or introduced into the plant.

The licensee's training of craftsmen, who were " grandfathered" under the accredited training program, is significantly behind schedule.

The training of the electrical craft in electrical safety practices was marginal.

,

v

!

~-

--

_

,

a o - i

..f.

'

,

'

,

..

'(

DI

-

b-s 1.Lg.

!?

'~

,

ye

>

,

.

E-3'

"

TABLE-0F CONTENTS-

,

t

,

V:

'

Y

.g

-

,

J LEXECUTIVE SUMMARY,.

...........................:.. -lii:

'

.

. INSPECTION DETAILS...................-..........

-1 l

'

1.

OVERALL PLANT PERFORMANCE RELATED TO MAINTENANCE

.1.

. '.

..

...

,

$

J1.' 1 Scope'.....

1.2 Conclusions'...........-.............

.1...

........,..............

.

l'

'

.1.3 Findings

.

.....:......................

as

.

2.

M NAGEMENT SUPPORT'0F MAINTENANCE..-,-..............

'

>

[2.1/' Management Commitment'and. Involvement.........

..-

a

'

2.1.1l Scope.......................

L3

,-

.

,

.

.2.1. 2 -

Conclusions....................

2.1;3 Findings...................,.-.-

g Lc

'

t

,

2.'2 -Management Organization and Administration............

'.1

2.2.1:

-Scope.........-,..

...<........

.l

.

,

2.' 2. 2.

Conclusions...........<.

1.

4-

~,

.......

c 2. 2. 3 Findings...................

<.. -,

'2.3 Technical. Support.......

6-q

.

...............

12.3~1 Scope.....................

6

'

m;

..

,

2.3.2 Conclusions,...................

L

.

'2.3.3 Findings......................

7-

- " - '

'

.

,.

g.

L3,1 MAINTENANCE-IMPLEMENTATION.................... c16

.

3.1 Maintenance Work Control...................

16'

,

3.1.1 Scope.......................

3.1.2

' Conclusions....................

.

3.1.3 Findings......................

16.

3.2 Plant Maintenance Org'anization Programs...........

30:

i

,

"

3.2.1

-. Scope l.

......................

3.2.2 Conclusions....................

,

.

2. 3 Findings,.....................

3

'

x i

l

..

Vi

-.

h

-,

. $9 '.,.,.}

<

g

,

y..

?

te

~ -e s

13.3 :: Maintenance Facilities, Equipment, and Material Control. :. -.; L36

.

_

. s

.' 'y -

'A

~ 3. 3.1--

Scope.

..................36

'

......

.

23.3.2:

Conclusions..

.t..

~~ '

..................

"/

3.3.3-Fi ndi ngs...f.

.......... ;... :.. -

..

-

!3.4 -Personnel Control'......-.-.-._....-,....._. :..

40-

.-

3.4.1;

Scope.......-.: ................--40

.

- c 3.4.2'

Conclusions..-...................

"

.

3.4.3-Findings..:.._,

..-.

._.....-..

.s.5.

.

-

4.' -

EXIT INTERVIEW.

...-'

............. -.:

....

=44 a

~

APPENDICES A'

. LIST OF~ EXIT MEETING = ATTENDEES

.

B.

ACRONYMS

.

C.

MAINTENANCE TEAM; INSPECTION-PRESENTATION TREE

'-

,

Y i

i I

(

?.s,t-z

, a t; f i.

,

?..I,

.ha.

': E '

+?

,

'

,

g

'

4-

'

.

.1

-

i

.' f '

vii

" c.

l s

b

p-

,

,.

I ([,

'

3; C~

I.

..

INSPECTION DETAILS

-

L t

1.

OVERALL PLANT PERFORMANCE RELATED TO MAINTENANCE

,

1.1 Scope The inspectors reviewed Cooper Nuclear-Station's operating history; data and'

performed system and. plant walkdowns as direct, observable indicators of the i

effectiveness of the station's maintenance implementation.The team selected these areas bur on the premise that overall plant performance with respect to plant operabil ny, equipment availability, and general reliability:could be related directly to the effective implementation of maintenance.

!

l'. 2 Conclusions

,

The inspectors concluded that the licensee had implemented an effective maintenance program.

Radiological exposure and contamination levels were low

~

throughout the plant, and plant walkdowns identified few deficiencies.

.

However, some areas, which were not routinely entered, were found to be in a

.'

'less-than-desirable state of cleanliness.

In addition, the licensee did not

. track historic data and identify trends that would be useful to management.

'

s 1.3 Findings The inspectors selected the core spray system and the essential portions'of_the service water system for visual inspection.

In general,-the condition of the major components of these systems were considered satisfactory, with all components' properly lined up.

However, the inspectors observed the following deficiencies.

On. November 13, 1989, the inspectors observed that_ valve LV 123W-11-(Core

'

Spray Pump 18 suction from the Torus) was in.its required open position,

.but it was not sealed as required by Operations Procedure 2.0.2, " Operations Logs and Reports," Revision 13.

This is'an apparent violation (298/8931-01)

of 10 CFR 50, Appendix B, Criterion V, which requires that activities shall

be accomplished in accordance with approved procedures.

The inspectors observed that valves required to be sealed were labeled-with large', easily visible tags; however, not all valves labeled as sealed valves were, in fact, required to be sealed.

Procedure 2.0.2,-required sealing devices.

However, the inspection revealed valves that wet e not listed in Procedure 2.0.2, which were labeled as sealed, although they were not, in fact, sealed.

Examples included valves CM-86 and CM-88 (Emergency

'

Condensate Storage Tank 1A and IB supply), which were labeled as locked-closed, but were not listed in Procedure 2.0.2 and did not have sealing devices.

The inspectors were concerned that licensee personnel could

'

confuse valves requiring seals with valves not requiring seals and leave a

'

_

._

.

_

<

..

,

.

seal-required valve unsealed.

Indeed, during the walkdown which identified valve CS-11 as unsealed, a member of the licensee's staff confused valve CS-11 with valves not requiring seals and stated that CS-11 was not

.equired to be sealed.

'

On November 15, 1989, the inspector observed moisture in a plastic filter housing on the service air system.

The air filter (SA-F-12) was in the service air supply line (100 psi) to the reactor water cleanup system filter /

demineralizer.

When the filter /demineralizer was in service, 1000 psi reactor water leaked through two manual valves and a check valve into SA-F-12, which was at a low point for that portion of the service air system.

The licensee had placed a caution tag on closed Isolation Valve SA-V-33 in order to prevent primary water from migrating to the rest of the service air system, which was also the licensee's breathing air system. When the caution tag was placed (October 19, 1989), chemistry and health physics personnel verified that the leakage was limited to the portion of the system isolated by Valve SA-V-33; however, the licensee had not established a regular testing program or limited the use of the breathing air.

The licensee stated that the breathing air system would be tested prior to uso (it had not been used since the leak was identified).

The licensee repaired the leaking valves by November 30, 1989.

There were minimal areas of radiological contamination in the auxiliary and turbine buildings, and most areas were accessible in street clothes.

With the exception of the following observations, the general condition of the plant areas and equipment was satisfactory.

In the essential service water pump rooms, floor areas behind pumps and piping were dirty, were littered with trash, and had not been maintained to the standards of the remainder of the facility.

Although the licensee had implemented a painting program and numerous areas were painted, portions of many systems were not painted.

For example, even though "A" train core spray had been painted since mid-1986,

"B" train core spray was mostly unpainted.

All four residual heat removal service water booster pumps had slight oil leaks.

Tools, cleaning equipment, and material were improperly stored throughout the plant.

Examples included the quad rooms in the reactor building and the area around the bus ducts in the turbine building.

-2-l

-

m hi..,

'

of -

e

'2.

MANAGEMENT SUPPORT OF MAINTENANCE l

'

LThe. objective of this part of the inspection.was to assess licensee ~ management's support of the maintenance process with respect to-the establishment, implemen-

"

tation, and control of an effective maintenance program.

The major areas-eval _

i uated were management's commitment to and involvement in the maintenance' process;

the organization, administration, and allocation of resources to maintenance;.

and technical support provided to the mainter.ance process by management.' The-

.-

following: subsections present the scope, condusions, and. findings applicable to

~

l each of these areas.

,

2,1 -Management' Commitment and Involvement

_

.

~

2.1.1~~ Scope.

.i The inspectors reviewed the. licensee's application of industry initiatives, and

,

evaluated the licensee's programs for reviewing, implementing,.and tracking -

i industry operating experience.

The inspectors.also reviewed the licensee's

programs for tracking, trending, and evaluating plant performance, performance

'

of system engineers, and licensee participation with industry groups.

In addition, the inspectors interviewed selected managers and employees to ascertain whether

"

these-individuals understood.their assigned responsibilities, i

2.1.2 Conclusions The inspectors determined that the licensee made effecthe use of the industry

initiatives and programs that were being implemented; however, additional

involvement by the licensee would have been beneficial.

This would include

expanding.the present narrow scope of response to operating experience review information, additional involvement in the Institute:of Nuclear Power Operations (INPO) human evaluation system, and volunteering for an INP0 maintenance assistance review team.

2.1.3 Findings In conjunction with the inspector's review of moisture in the service air system,

[

the licensee's actions in response to INP0 Significant Operating Experience Report (SOER) 88-1, " Instrument Air System Failures," and Generic Letter (GL) 88-14,

" Instrument Air Supply System Problems Affecting Safety-Related Equipment," were reviewed.

50ER 88-1 made several recommendations concerning operations, training, maintenance, and design / analysis; and GL 88-14 requested the licensee to perform certain verifications.

In all actions reviewed by the inspector, the licensee appeared to have adequately followed the recommendations and performed the verifications.

However, as illustrated by the following observations, the licensee tended to limit evaluations and verifications to examples specifically stated in the referenced documents and did not expand the scope to similar systems beyond what was required.

-

Except for the common portions of the systems, the licensee did not apply

,

the SOER information to the service air system.

-3-i

-

.

. -

.

-

. -

-.

r

,

Q

'

,

r L

!

N

.

Inspection and Enforcement Notice 87-28, " Air Systems Problems'at U.S.

s Light WaterLReactors," was issued 1 year prior to SOER 88-1 and GL 88-14.

Internal, licensee' documents stated that air-operated valves were tested:

r

,

L during preoperational testing for failure mode on loss-of air supply and'

,

no other testing was required.

However, through further evaluations associated with SOER 88-1 and GL 88-14, the licensee determined that 33

.'

valves either were not tested during preoperational testing or had been:

<

.

modified and had not received proper post-modification testing.

These

> valves subsequently were tested.

The inspectors evaluated.the licensee's emphasis on industry programs and

. initiatives..This evaluation revealed that the licensee had personnel.on

'

operation and maintenance' subcommittees to some industry panels such as

>

American Society of Mechanical Engineers, Institute cf Electrical and

. Electronics Engineers, and boiling water reactor owners group.

In addition, the-licensee participated'in INPO workshops and the INP0 Maintenance Peer Evaluation

'

Programs.

However there were numerous activities in.which the licensee had-chosennottoparticipate.

Examples included the industry-sponsored maintenance

'!

,

assistance review team,-the Electric Power Research Institute, and the industry-

'

sponsored outage management.

\\

-The inspectors determined that-the licensee did not have an overall maintenance plan that included specific goals.

CNS had a 1 year schedule of major items

and a 5 yearL outage plan (which was not maintained up-to-date).

2.2 Management Organization and Administration 2.2.1 Scope The-inspectors reviewed establishment of policies, goals, and objectives for maintenance; allocation of resources; definition of maintenance requirements; performance measurements; the document control systems for maintenance; and the maintenance decision process.

2.2.2 Conclusions The inspectors determined that licensee management understood the importance of

-

proper plant maintenance and properly supported maintenance activities.

However, the licensee did not have specific long-range maintenance activity plans and

,

appeared to devote a minimal level of resources to maintenance issues.

2.2.3 Findings Establishment of Policies, Goals, and Objectives for Maintenance The licensee implemented performance monitoring of certain measurable attributes

,

and forwarded these to corporate management.

The Nuclear Power Group Manager set annual station goals with input from station personnel.

A monthly operating report provided the group manager a status report and updates.

In addition, the licensee maintaired an industry-derived performance monitoring report, which was-4-

- - -.. - -.

.

updated quarterly.

The parameters measured, tracked, and trended included availability, capacity factor, number of maintenance work request (MWRs) open, and volume of radwaste shipped.

Allocation of Resources The inspectors reviewed records and maintenance item backlogs to evaluate the licensee's control of maintenance activities.

This evaluation revealed that the licensee used contract personnel to implement improvement and special programs and use Nebraska Public Power District (NPPD) personnel for steady state workload.

Personnel worked minimal overtime during nonoutage conditions.

'

The number of nonoutage, open MWRs was low and easily managed.

Definition of Maintenance Requirements The inspectors reviewed selected work packages and determined that the maintenance process implemented regulatory requirements adequately.

The licensee satisfactorily implemented vendor technical manuals, industry information, and other operating experience review information but could nave improved in this area.

Performance Measurements The inspectors interviewed licensee personnel, reviewed programs and records, and determined that CNS did not have a system to conduct performance measure-ments.

While there was some feedback for MWRs, maintenance performance indi-cators were not used and a maintenance activity plan had not been implemented.

While maintenance performance was included in quality assurance (QA) audits, the audits only measured worker compliance with plant procedures and did not address the adequacy of the procedures as related to industry standards nor the overall adequacy of the maintenance activity.

Document Control Systems for Maintenance The inspectors reviewed the document management system used by the licensee to control the preparation, implementation, review, and storage of MWRs.

Reference materials such as procedures, technical manuals, and drawings were also reviewed to ensure that the latest documents were used in the maintenance process.

The computerized system for control of MWRs was used to track the status of the request from inception to storage as a permanent record.

The team reviewed numerous MWRs in various stages of completion, and in all cases, the licensee was able to quickly locate the MWR associated with the requested maintenance activity.

However, the team noted two concerns in this area.

First, most MWR packages did not include the appropriate documents and reference material; this subject is addressed further in detail under the subject of maintenance planning.

-5-

_ _ _

_ _ _..... _ _...

....

,

..

,

.

,-

,

The team's second concern related to the availability of-the latest revision of

'

.

the procedures ' supporting the MWRs.- The team noted that most of the maintenance planning occurred at the craft level at the time the maintenance technicians were preparing to go out on-the job.

Therefore, obtaining the latest procedure and necessary references was the responsibility of the craft person. Maintenance u

procedures and_ technical manuals available to the craft were maintained in the

'

shop by the shop supervisor-and had been delivered after being approved and:

printed by document control personnel.. In most cases, distribution required approximately two weeks.

Although the computer terminal-in the shop' assessed.a

'

database that-reflected the latest maintenance procedure revision numbers, maintenance technicians were not acquainted with the database and did not check with document control-personnel to ensure that the latest revision of-a procedure

.'

was available for use.

Although no instances were observed where technicians

failed to use the latest procedures, the team perceived the situation as a potential problem.

Maintenance Decision Process-The licensee had a daily point of discharge meeting in which a 3-day schedule was maintained.

This meeting included representatives from maintenance, operations,_

health physics (HP), and engineering.

Schedules were issued to ensure HP and operations support and to schedule work efficiently for the next 3 days.

2.3 Technical Support 2.3.1 Scope The. team assessed the licensee's technical support of maintenance by evaluating the preparation and. processing of work packages and inspecting the following-areas:

Internal and corporate communication channels.

'

Root cause evaluation and failure analysis.

Instrument air-dryer failure.

Equipment performance trending program.

'

Engineering support and problem disposition.

-

Configuration control.

-

Vendor manual control.

,

-

Role of probabilistic risk assessment in the maintenance process

'*

_ Involvement of QA and Quality Control (QC) in the maintenance process.

Integration of radiological controls into the maintenance process.

-6-

_ _ _ _. _.....

-

_

l-9 N

,g-

..

m_

'*

Safety review of maintenance activities.

Integration of regulatory documents into the maintenance process.

>

.

The inspection-team reviewed documentation, held discussions with plant engi-neering and management personnel, attended daily plant engineering and manage-

-ment meetings, and examined in progress and completed maintenance activities to-ascertain the adequacy of communication channels.

In addition,'the team conducted-interviews with. selected managers and employees to verify that individuals.under-

.

stood their assigned responsibilities.

2.3.2 Conclusions

The licensee had not fully implemented any formalized performance trending program.

'

The licensee had not fully established or documented its root cause evalua-tion and failure analysis program, and such procedures were only applied to items the licensee considered to be of high significance (e.g., noncon-

,

formance reports, [NCRs]).

'

The' licensee had not established a formalized deficiency identification-

'

and control system to facilitate consistent root cause evaluation and failure analysis.

As such, the failure analysis and root cause evaluation-process appeared disjointed.

There appeared ~to be a lack of technical reference material and formalized disposition of engineering analyses.

2.3.3 Findings Internal and Corporate Communication Channels The inspectors attended daily meetings of plant personnel at the management and technical staff levels, as well as maintenance department meetings at the first-line supervisor level.

The licensee appeared to have strong internal communications.

It'was apparent during the inspection that appropriate plant personnel and key organizational managers were knowledgeable of ongoing-work activities, problems and current status.

Communication between the maintenance department and the technical support organization was also good.

Root Cause Evaluation and Failure Analysis The licensee had not fully established a documented root cause and failure analysis program.

The need for such a program had been recognized and initial training had been provided to systems engineers and the technical staff.

CNS Policy Directive 11, " Root Cause Analysis," dated July 2, 1986, directed that the station " develop the basic logic for finding the solutions to problems and preventing their recurrence," but the actual evaluation methodologies and initiators had not been institutionalized.

Further, the absence of an aggressive deficiency identification and control system made application of-7-

.

_ _ _ _ _

____-

_ _ _ - _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ -_ _ _ _ _-_____

_ _ _ _

-

.

...

-

..

m

,

l h,

y root cause evaluation'and failure analysis inconsistent.

Use of a documented analysis was only ensured for items subject to CNS Procedure 0.5.1, Revision 3,

"Nonconformance and Corrective Action," but all conditions adverse to quality

'were not necessarily subject to this procedure.

Section 3.2.3 of this report-(subsection, " Deficiency Identification and Control") identified four examples in which equipment degradation and/or improper maintenance and modification activities failed to result in a root cause analysis because the licensee lacked a clear " trigger" mechanism beyond that provided in Procedure 0.5.1.

These examples. included improper packing found in a residual heat removal (RHR)

drain valve, a wetted service water solenoid pilot valve,:a failed diesel generator intercooler, and an improper modification of RHR interlock circuits involving a motor-operated valve failure.

Root cause evaluations were reviewed for several NCRs and plant incidents.

In some cases (e.g.,-the November 25, 1989, instrument air dryer failure and NCR 89-068, which dealt with "RHR Valve Wall Thinning"), the evaluations were quite extensive and rigorous.

In other cases, the lack of guidance and method-ologyresultedinsugerficialevaluation(e.g.,NCR89-128,"ControlRoomHVAC.

=l Damper Failure Modes ).

In the other cases (discussed above), no evaluation was performed.

Instrument Air' Dryer Failure On November 25, 1989, the "B" instrument air dryer post-filter housing. ruptured, causing a loss of instrument air pressure. The rupture occurred when dryer temperature controls failed, overheated the filter housing, and ignited the i

filter media.

The loss of air pressure permitted the main steam isolation-valves to drift closed, causing a full reactor scram.

The resident' inspectors-reviewed the. licensee's immediate response and monitored the system repairs.

Upon return to the site, the team reviewed the licensee's. failure analysis and

' '

long-term corrective actions for the filter.

The licensee identified four contributing causes:

(1) absence of preventive maintenance for temperature switches, which failed and overheated the filter; (2) excessive use of sealant and lubricant, which collected in the filter and may have exacerbated combustibility; (3) failure to specify high-temperature filter media; and-(4) inadequate training of maintenance and operations personnel.

The root cause analysis recommended four long-term corrective actions:

(a) closer monitoring of the dryers via operator logs; (b) additional training in the j

details of dryer operation; (c) evaluation of the dryer design for possible

!

-improvements; and (d) reassessment of preventive maintenance and operational L

- checks.

At the end of this inspection, the licensee had tot yet begun these long-term corrective actions; the items were being entered in the action item control system for tracking in conjunction with NCR 89-196, which was issued for the event.

'

Equipment Performance Trending Program Similar to the root cause and failure analysis function, the trending and analysis of system performance data was not fully developed at CNS.

Engineering Department Instruction ED 89-03, " Trend Analysis Program,"

-8-

.

,

.

..-

Revision 0, issued in March 1989 provided only generalized guidance for system

. engineers developing a system-trending plan.

No specific program outputs were-

-

required, and although the procedure provided for reports to management, it did'

not specify the content or frequency of the reports, particularly with respect to adverse equipment trends.

. Review of available trending information fo_r the high pressure coolan't injection (HPCI), core spray (CS), and RHR systems showed great variation in the extent of data analyzed and the extent of the trend analysis performed.

For example, the HPCI system engineer was tabulating the system inservice test

and' surveillance data and looking for visible trends but was not performing any analytical,or graphical evaluation.

The system engineer for the CS and RHR systems was using a personal computer spreadsheet and graphics program to analyze similar-data.- In all cases observed, reports were only informally provided to-management at the system engineer's prerogative.

The licensee advised.that additional procedures were under development and that training was in progress-which would provide additional guidance to the engineers in setting up their

.

respective programs.

Engineerina Support and problem Disposition

The site engineering department was frequently involved in providing on-the-spot support and technical resolutions for maintenance related problems.

Although the department had an instruction for workload management (ED 88-03, " Work Management Instruction"), no guidance or instruction was provided to ensure the quality and acceptability of engineering problem dispositions that were not administered via other established procedures (such as NCRs or modifications).

Some of the systems engineers were relatively inexperienced, and management

. relied upon close supervision by lead personnel to ensure that system engineer performance was acceptable.

,The inspectors identified several instances in which minor engineering disposi-tions lacked rigor and either failed to address or failed to document key safety

. considerations.

For example, the flats of a service water booster pump flange nut were cut down to provide clearance for an adjacent pipe as part of MWR 89-1059.

The associated MWR special instruction sheet provided instructions for the cut,

,

. noting that "this notch will not weaken the nut as cut out is in the area where no stress exists." Although the licensee's conclusions appeared valid, no reference or basis for this conclusion had been documented.

,

'Similarly, the inspectors found an MWR documenting a loose bolt and a slightly misaligned-flange for the core spray pump discharge relief valve outlet line, which connects, without isolation valves, to the containment suppression chamber (MWR 89-2559).

The bolt was retightened and the MWR was closed because the flange did not leak while pressurized to a 10-15 foot standing head of suppression chamber water.

No evaluation was documented for containment boundary operability for the as-found or as-left conditions for post-accident peak' pressures of 58 psig nor was the apparent flange misalignment addressed.

The inspector observed the as-left flange conditions, reviewed the torquing results from the MWR, and discussed the matter with the plant engineering 1;

-9-l l-'

%

_

d

supervisor and engineering manager who acknowledged the. lack of guidance and stated that such instructions were under consideration on'that basis, the inspector concluded that' the current condition of the flange was acceptable.

The engineering manager acknowledged'the lack of guidance and stated that such instructions were under consideration.

Configuration Control

<

MWR 89-3534 corrected a lube oil leak in the No. 1 Emergency Diesel

+

Generator.(EDG) 3 cam bearing.

The bearing housing was supplied lube oil via a

'

3/8-inch diameter hole drilled through a cast boss in the block beneath the bearing.

Porosity in the cast boss (apparently a manufacturing flaw) caused.it to erode over the years and allowed oil to leak out of the 3/8-inch passage before reaching the bearing.

To correct the problem, the licensee sleeved the internally-bored oil passage with 3/8-inch stainless steel tubing as directed

'by. a Cooper-Bessemer Industries letter (W. H. Lambert to J. Flaherty, CNS) of August 4, 1989.

Engineering Manager Memo CNSS897411 " Technical Justification' for Repair of Diesel Generator 13-Right Camshaft Forward Bearing Lube Oil Supply," evaluated

.

'

tubing size, material compatibility, pressure ratings, and restraint.

The licensee administered this activity as a repair and not as a design change or

'i change in the configuration of the engine. Consequently, no design change or i

specification change was processed.

]

The licensee's Procedure 3.4.3, " Design Change," Revision 0, and Procedure 3.4.5,

" Equipment Specification Change," Revision 0, did not address this type of configuration change.

Procedure 3.4.3 inappropriately defined a design change

,

'

as "any permanent change to the facility as described in the updated safety j

analysis report, and requires changes in documentation such as drawings,

procedures, etc."

Equipment specification changes were defined as "... minor i

modifications which are not complex and which have the same form, fit, and fl function as existing equipment..."

In the case of the bearing, the form j

(i.e., configuration) of the engine was changed, although the functional aspects

'

appear unchanged.

However, the licensee apparently did not perform a 10 CFR 50.59 applicability review.

Detailed design review was not performed nor was any action evident to-update the diesel generator (DG) engineering drawings. Although the

licensee's plant engineering sepervisor stated that the vendor had design l

authority and that the licensee relied upon the vendor's quality and engineering

.

assurance programs, the vendor letter did not indicate that the new configuration

was an " engineered" product nor that it had been subjected to the vendor's

!

programs for detailed design review.

This was considered a weakness in the licensee's program for design and configuration control.

l Vendor Manual Control l

During a plant tour, the inspectors found excessive grease leakage from the

-

HPCI turbine / pump shaft coupling.

Licensee investigation subsequently revealed that the pump technical manual contained vendor instruction sheets for another coupling make and model that had been used to develop the pump maintenance l

l-10-l

!

E

,_

,

-s>

.

procedure.

The' licensee concluded that'the coupling had apparently been overfilled with grease as a result of using these incorrect instructions,

,

s-Although the licensee had a program for. verifying of technical manual as-built accuracy, management was unable to determine why the correct vendor information -

was_not included nor whether this had the potential to' affect other components (i.e., no root cause identified).

The coupling was properly serviced on November 27, 1989, during an unrelated plant shutdown.

Similar examples of missing-information were identified for valve technical manuals.

As discussed elsewhere in this report, no centralized policy or guidelines existed for torquing threaded fasteners.

As a result, the

"

body-bonnet studs on an Anchor Darling containment isolation valve were not torqued following a valve repair.

The controlled copy manual for the Anchor Darling valves-included torque: values for some valve designs, but excluded others including the valve in question.

Although uncontrolled Anchor Darling catalog and vendor manual information provided the plant staff with torque values for essentially all applicable valve-styles, the controlled manual was

.

incomplete, contributing to the misunderstanding between engineering and-maintenance and the omitted torquing of the studs.

MWR 88-4642 was prepared to investigate and repair the Westside intercooler on No. 1 DG 1.

The tell-tale drain was dripping water, indicating a possible service -

water tube leak.

The intercooler was replaced based on the high probability of a tube leak..The maintenance was completed in November 1988, and the removed com-ponent was sent to vendor for possible repair.

In March 1989, the vendor advised the licensee that testing and evaluation was completed ano that tube leakage was so bad that the number of leaking tubes could not be determined, blockage was found in numerous tubes, and the tubesheets were corroded to the extent that replacement was war m ?d.

The vendor concluded that the intercooler was-beyond-economic repair a, cid be scrapped.

The inspection teart anpted to determine if adequate corrective and preventive measures were being taken.

A review of DG Technical Manual 0245, "KSV Diesel Engine, Operation, and Maintenance, Nuclear Power Plant Emergency-Standby,"

Section 13, revealed that the "aftercooler" should be flushed and drained every 4 to 6 months.to keep.the coolers clean and prevent clogging.

This preventive maintenance activity was not in the CNS program as of December 1989, since no apparent response action was taken with respect to the intercooler failure.

In addition, the licensee took no action to evaluate the condition of the other three intercoolers.

The licensee advised the team that a significant change in the preventive maintenance requirements, promulgated as a technical manual change in January 1989, revised Section 14 of the manual, but the maintenance activities suggested by Section 13 had been overlooked.

Involvement of QA and (QC) in the Maintenance Process The QC program was specified by procedures and relied upon the inspection of essential activities by " peer" maintenance technicians rather than by independent inspection personnel dedicated to inspection activities.

The procedures and implementation activities appeared generally acceptable, but several weaknesses and problems were identified.

-11-I l

\\

___.___u__.___._-_-__

_. - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _. _ _. _ _ _. _. _. _.. _ _ _ _ - _ _ _ _ _ _ _ _ _. _ _ _ _ _. _ _ _ _ _ _ _. _ _ _ _ _ _ _ _ _ -. _ _ _ _ _ _ _ _ _ - - _ _

m

_m_

p:

[L

.

,

..

,

!

The team considered the uso of maintenance department peer personnel for quality inspections to be a weak practice which could result in a compromise of quality if strong control was not applied.

i Detailed QC steps were specified in bcth the electrical and instrumentation and control (I&C) maintenance procedures, but mechanical maintenance procedures did not consistently include such steps.

The mechanical procedures were being rewritten at the time of the inspection and were expected by the licensee to include appropriate inspection steps.

QC Procedure 12.5, "CNS QC Functions,"

Revision 4.Section V.C. required that prework WR reviews include a determination of which items were " critical" for mandatory performance of QC inspections, including definition of " critical measurements / clearances,"

o

" critical step performance," and " torquing." The p*ocedure further required specification of QC witness or hold points in the work instructions on WR documents for an essential maintenance activity.

WR 89-2344 authorized " repair or replacement as necessary" of the 3R lobe on the EDG cam.

The work was performed on May 16, 1989, under the direction of t% EDG vendor technical representative.

The licensee used excerpts fron the v6ndor manual and a vendor engineering standard as supplemental WR work instruc-tions.

The excerpts had been annotated to include performance signoffs but did not include QC inspection requirements (i.e., witness points or hold points).

The instructions did not include items meeting the inspection criteria of QC Procedure 12.5, such as critical reassembly steps, critical cam shaft measure-ments and clearances, or verification of valve timing and timing clearances.

Criterion X of Appendix B to 10 CFR Part 50 requires that a program for inspec-ting activities affecting quality be established and executed.

failure to accomplish the reouirements of QC Procedure 12.5 by not specifying or implemen-ting QC inspection requirements is an apparent violation (298/8931-02).

A further weakness noted was that neither Procedure 12.5 (nor any other procedure),

required that changes in job scope be reviewed for increased quality inspection requirements.

As discussed elsewhere in this report, the licensee sometimes modified or expanded the work scope initially stated in WRs with little formal review.

The absence of a structured QC applicability review constituted a potential for missing required inspections.

The licensee's QA program specified criteria for inspecting and auditing; however, the inspector identified a asakness in the QA audit criteria.

Specifically, the licensee verified that plant activities were conducted in accordance with plant procedures without determining the effectiveness and adequacy of program implementing procedures.

The QA program included established and documented methods for reporting quality deficiencies to the affected managers.

Additionally, plant employees could g

report nonconformances utilizing the NCR process.

Trending on nonconformance report data was conducted quarterly to provide information to licensee managers about how the system was being managed.

Despite the available information, the

-12-

. _ _ _......

.

,

<

.

licensee had not looked for comon failure mechanisms by system or root cause code in either trend chart. 'ine licensee committed to conduct a one time review for generic issues to see if such an analysis provided any insights.

Because of

the limited number of NCRs generated, this appeared satisfactory.

Integration of Radiological Controls into the Maintenance Process Radiation Protection appeared to be well integrated into the maintenance process.

The two departments appeared to have a good working relationship because of the age of the plant and because many of the radiation protection technicians were former maintenance employees.

Training appeared sufficient, but the licensee was not particularly aggressive in its training program, apparently because of the

,

lack cf manpower.

The "as low as reasonably achievable" coordinator was consistently involved in the maintenance process.

On the surface, the radioactive materials control practices appeared casual, but there had been few specific radiation control incidents and exposures of the staff were generally low.

<

Safety Review of Maintenance Activities The inspectors reviewed safety procedures, evaluated completed MWRs, and observed the performance of MWRs in order to determine the extent to which personnel and industrial safety were integrated into the maintenance process.

In reviewing CNS Procedure 0.9, " Equipment Clearance and Release Orders,"

Revision 9, the inspectors made the following observations.

Step IV.A.2 stated that when a hazard does not exist, a clearance order was not required; however, the procedure failed to define, " hazard" in terms of temperature, pre: n re, or voltage.

Step IV.A.1 stated that if the system configuration was specified by a controlling document, a clearance order was not required.

However, a clearance order should have been utilized whenever personnel safety was involved.

s

Procedure 0.9 did not state who was qualified to request, prepare, or hang a clearance order.

The NOTE to Step V.A.7.e permitted the omitting of independent verification during outages when the reactor was in cold shutdown because applicable system status would be independently verified prior to restart.

It was the team's opinion that many systems posed a significant safety threat even when in cold shutdown; thus, independent verification should not have been completely omitted during those periods, The team reviewed numerous MWRs from the past year and observed maintenance in progress to determine whether equipment was properly tagged out of service.

In several instances, clearances were not used when they appeared appropriate.

-13-

o-

,

O Rather than routinely setting a clearance unless there was an overriding reason not to, the licensee adopted the prevalent practice of using clearances only when absolutely required.

The team identified the following specific examples.

'

MWR 89-2189 was issued to replace the service water flow indicator (SW-FI-385A) because of the inability to calibrate the device properly.

,'

Both mechanical tubing and electrical leads had to be disconnected and

!

reconnected during the instrument replacement.

Neither a clearance nor

!

" lifted lead" controls were used in performing the maintenance, t

MWR 89-2128 was issued to lift and relocate limit switch rotor leads on Drywell Spray Loop A Inboard Isolation Valve RHR-M0-M013A because of a

wiring error discovered during post-modification testing.

No clearance i

was set for the maintenance, and the MWR provided no indication as to why

,

electrical isolation with a clearance could not be set.

[

MWR 89-2876 was issued to remove the motor from steam supply to augmented

off gas downstream shutoff valve (RHR-M0-921MV) to extract stainless steel lockwire-that was jamming the motor drive.

No electrical clearance v.as set for disassembly of the valve actuator, MWR 89-3206 was issued to repair service water solenoid pilot valve

,

r SW-50V-SPV857, including replacement of a wetted solenoid.

In this case,

'

clearance was set that isolated the air supply to the solenoid pilot valve

(Clearance Order 89-1451).

However, no isolation was provided for the electrical circuitry involved.

The supplementary work instructions i

. included with the MWR incorporated American Switch Company (ASCO)

Supplementary Bulletin Number 8320 that stated under the " Maintenance" heading,: " WARNING: Turn off electrical power and line pressure to valve before making repairs."

,

CNS Procedure 0.9,Section IV.A.I., required personnel to "... obtain an Equipment Clearance and Release Order for work or operation of CNS systems that requires that the system (or portion of the system) not be operated or that it be placed in a specific configuration (not normal) for the safe conduct of the work or operation."

.

r

.The failure to provide the appropriate clearances to-protect personnel and equipment for the maintenance activities discussed above is contrary to Procedure 0.9 and is another example of the apparent violation (298/8931-01) of the failure to follow plant procedures.

The team also observed work on MWR 89-4366, which concerned inspection of the reactor feedpump turbine control pedestal.

This required removing an access cover to expose the turbine hydraulic control linkage internals.

The work was accomplished without a clearance for the turbine steam and oil controls which were under the administrative control of the control room operator.

The stated l-reason for not posting a clearance was the simplicity of the job and the need

'

to expedite returning the turbine to jacking gear operation.

-14-L o

L

,

.

,

.

.-

l The inspectors observed performance of MWR 89-3920, replacement of the No., 2

EDG synchroi, cope in the local DG Control Panel DG2-903.

The team perceived this activity as unsafe for six major reasons:

I

To provide adequate isolation conditions, no clearance was set for the

!

repair activity,

i Workers relied on the positioning of the synchroscope selector switch, which was out of the workers' sight during the repair activity.

I

'

The test voltmeter used to check for absence of terminal voltage was not checked for operability immediately before and af ter checking the meter i

terminals, and before touching the meter terminals for disconnection.

One maintenance technician touched the meter terminals with his bare hand

[

before checking for absence of voltage (the action appeared almost-subconscious, as if the technician were identifying the proper terrrinals).

The wcrk was performed in an energized panel, and adjacent terminals that were touched during the maintenance activity were not checked for presence-or absence of voltage (and were not appropriately protected if energized)

before commencing the work.

'

The meter replacement required the technician to stand on a steel support

'

beam in the bottom of the panel in order to reach the work, but no

.

protective rubber insulating pads were used.

Safety requirements for performance of this MWR were found in the NPPD Safety Rule Book,Section V.9.A., which stated, "Up to and Including 250 Volts.

1. Employees shall take all necessary precautions and use all appropriate, necessary protective equipment to protect themselves...." This guidance was perceived by the team as inadequate, considering the lack of craft-specific training on the subject.

(See " Personnel Training," Section 3.4.3 of this

-

report for details.) Based upon the observations discussed above, the team perceived this area of safety to be weak in both the program and its implementation.

Integration of Regulatory Documents into the Maintenance Process

,

The inspectors interviewed licensee personnel and reviewed documents to determine how the licensee integrated regulatory documents into the maintenance process.

l The licensee had designated one group as the primary interface between the plant and the regulators.

This group appeared effective in gathering and disseminating this information to plant management.

However, the inspectors observed instances in which information stopped at some level above the workers, although it could have been of some value to the workers.

-15-

'

U

..

.

w 3.

MAINTENANCE IMPLEMENTATION The objective of this part of the inspection was to determine the extent of control of (1) maintenance work; (2) plant maintenance organization programs; (3) maintenance facilities, equipment, and material; and (4) personnel control and training.

To evaluate these areas, the inspectors obst.rved maintenance jobs in progress, reviewed programs and work packages, interviewed personnel, and observed activities that support maintenance (i.e., material and personnel control).

The inspectors concluded that the licensee had developed work control programs that were implemented by the experienced staff to maintain the plant in good operating condition.

However, the licensee relied too heavily on individual knowledge and skill and had not achieved a sufficient balance between adequacy of procedures (especially work instructions) and reliance upon craft knowledge and work practices.

3.1 Maintenance Work Control 3.1.1 Scope The team accomplished the inspection and assessment of the work control process by viewing maintenance records and maintenance in progress, including work planning processes; work orders, post-maintenance testing, and adherence to procedures; equipment records and history; work prioritization and backlog controls; and completed work control documents.

3.1.2 Conclusions The licensee's work control practices and programs suffered from weaknesses in both program content and implementation.

Identified weaknesses included lack of prejob planning and support of craft activities, use of equipment clearances, work scope control, lack of work instructions and procedures, post-maintenance testing, and technical support and review.

3.1.3 findings The work planning and control process was administered by CNS Procedure 7.0.1,

" Work Item Tracking - Corrective Maintenance," Revision 13. Work item tracking (WIT) forms were prepared by anyone identifying f ailure-related and nonfailure-related maintenance needs.

The station planning staff was responsible for

,

generating maintenance work request (MWR) forms describing and authorizing

'

l accomplishment of the maintenance activity.

In additica, the planning staff provided the principal point of input to the various computer-based systems, such as the equipment data file, equipment history file, etc.

As discussed

. below, neither Procedure 7.0.1 nor any other procedure specifically prescribed

,

the responsibilities and required actions for many prejob functions.

Additional discussion and examples provided in this report illustrate program and implementation weaknesses such as:

-16-

e

-

-~

c

,

l

,

p-

.

Minimal use of preplanned work instructions and maintenance procedures for

.

F essential (safety-related) maintenance activities;

'

Poor control of MWR scope enabling workers to expand the work scope and

'

the physical boundaries with minimal review, authorization, or consideration

,

of expanded collateral requirements such as quality control (QC.) inspection

[

or post-maintenance testing.

u I

Poor work documentation which, in conjunction with minimal use of procedures.

['

instructions, and checklists, resulted in ambiguous or inaccurate records of what was actually accomplished.

l

Inadequate or ambiguous specification of post-maintenance testing resulting

-

from the poor scope control, poor instructions, and poor job documentation.

Poor prejob specification of safety requirements such as the required use I

of equipment clearances (tagouts).

f

'

Poor prejob specification of technical requirements and quality inspection requirements (such as torquing, internal cleanliness, QC witness and hold points, etc.), which, in conjunction with the absence of detailed work

instructions and procedures, failed to ensure the quality of the maintenance

!

activity and presented the potential for omission of critical activities.

The absence of a planning guide, a conduct of maintenance procedure, or a

,

similar checklist (beyond the very general requirements of Procedure 7.0.1),

which would ensure that significant prejob needs were addressed and would provide specific personnel accountability for accomplishment of the task.

In eneral, the alanning function at CNS had little assigned responsibility for pre ob planning aeyond preparing the authorizing documents and tracking the

,

wor status.

The actual planning functions of staging material, procedures

!

andpersonneldefaultedtothecognizantmaintenancedepartmentandwastypically performed by the shop foremen and craftspersons.

Interviews indicated that these functions became secondary considerations and suffered from lack of personnel resources during heavy workload periods such as outages.

The team considered the lack of prejob support and direction for both outage and nonoutage conditions, to be a major program and implementation weakness.

L Many of the problem examples and observations discussed below involved maintenance on systems and components which the licensee categorized as nonessential (nonsafety-related).

These were included in the team's evaluation because:

(1) the ;icensee's administrative controls for maintenance do not discriminate between essential and nonessential items except for spare part usage and (2) the itemsreviewedweretypicallyinthesystemsandtheperformedfunctIonswhich were traditionally considered important to safety (although the licensee's programs did not use this terminology).

-17-

,

,b.

g

-

.

Maintenance Activities frequently Exceeded Scope Approved in WR

Maintenance Procedure 7.0.1, " Work Item Tracking - Corrective Maintenance,"

'

Revision 13, Section 8.1.3.6, required that the scope of work authorized by a

!

maintenance work request be stated in the " FAILURE / WORK" section of the WR form.

However,_the procedure provided no requirements for defining that scope and no guidelines for aoministering changes to the work scope once work had been initiated.

The team reviewed numerous completed and in progress MWRs to determine whether the scope of work was adequately defined and controlled.

The team found that the initial definition of work scope was frequently too broad i

and that the work scope was frequently expanded extensively without a new or revised MWR and without apparent authorization.

In many cases, licensee main-

'

~

tenance and planning supervision stated that the scope of the MWR was intentionally kept broad to permit the " investigate and repair" activities to have the maximum latitude and flexibility.

The licensee supervisors further advised that when

-

suchpracticeswereused,thejourneymenwereexpectedtopreplantheirapproach to the job, accomplish the diagnosis of the problem, and return to the shop to consult with their supervisors regarding specific instructions resulting from the diagnosis.

The team observed this practice only in a fraction of the cases where

,

broadjobscopeoccurred.

'

Further, the team identified the following instances in which specific MWR scope requirements or work instructions had been provided and had been expanded to include other components, subcomponents, or significant activities not i

originally or specifically addressed.

'

MWR 89-2192 authorized the inspection and replacement of specifically listed parts, as necessary, in the actuator for the Core Spray "A" testable check bypass valve (CS-M0-M015A).

The authorized work included replacement of the finger assembly, rotors, gaskets, and 0-rings, and I

greasing of the limit switch. For reasons not documented in the " Work

'

Completed" section of the MWR, the actuator torque switch was also replaced.

No revision was issued to the MWR for the replacement, and work instructions were not included for the added activity.

MWR 89-2728 listed "F.HR-MOT-RHRPIA" (the 1ARHR pump) and "EE-CB-4160F(RHR1A)"

(the pump breaker) as the component identification code (CIC), and residual

heat removal (RHR) Pump 1A as the component description, when RHR Pump 1A tripped during post-modification testing.

The MWR authorized the craft to

!

" investigate and repair as required." Troubleshooting identified the actual failure as an incorrectly wired motor-operated valve (MOV)(RHR-M0-M013A)

interlock in the control circuitry for the pump.

No revision was made to the MWR scope to authorize the wiring change.

,

Valve RHR-V-67 was repacked (after its packing blew out) as part of MWR 89-2403.

The MWR was originally issued to install a new (replacement)

valve that should h.sve been, but was not, properly packed on installation.

"

No change was made to the " FAIL / WORK" description and, although the work-18-

T-W '

'4

..

,

r'

>

,

g.

I was documented by a short statement in the " WORK COMPLETED" section of the MWR,-the system engineer was unaware of the condition, and no additional

!

action was taken to address the root cause of the packing problem.

J P

Inadequate Supplemental Work Instructions l

,

'

The team found that approximately or.e-third to one-half of all essential MWRs

"

reviewed during the inspection authorized activities such as " troubleshoot and

,

repair as required," or " investigate and repair as required," or made similar

,

blanket statements without providing amplifying work instructions or procedures

!

to control the actual work steps performed.

-

t.

(

For maintenance activities observed by the team, " planning" typically occurred l

L at the craft level when the assigned craft was ready to start the job.

"Investi

'

gate and repair" activities accomplished ju:.t that, typically, no revision to the MWR instructions, details, or scope of work was accomplished prior to

proceeding with the " repair."

,

The team also reviewed the use of two types of licensee procedures.

These included standing procedures, which were part of the plant procedures manual, and

" supplemental work instructions" in MWRs, which were prepared as a part of the prejob planning process for unique maintenance activities.

(The team

,

considered these supplemental work instructions to have the force of procedures o

.

when implemented.)

F

.

[

CNS Maintenance Procedure 7.0.1, " Work Item Tracking - Corrective Maintenance,"

!

Revision 13, prescribed the methodology for preparing MWRs and accomplishing i

prejobsupportresponsibilities.

This procedure was considered weak because it

!

lacked several important aspects as detailed below.

,

t

The procedure did not reference or require use of maintenance support procedures (such as clearances, housekeeping, system cleanliness, rigging,

,

and calibration).

Section IV.I outlined general responsibilities of the shop supervisor /

foremen and specifically designated the responsibility of craftsmen assign-

ment for MWR execution.

Although the procedure mentioned craft qualifica-

.

tion for environmentally qualified (EQ) class equipment, it did not address r

craft qualifications before assignment to other tasks.

'~

Section V.A.3.f specifically required the WIT originator to provide a state-

ment of the failure and "... all pertinent information available...."

Section V.B.2.a required the maintenance planning group, upon receipt of

'

the WIT, to determine "... that sufficient detail is provided to perform the work."Section V.B.2.c required the maintenance planning group to determine whether a maintenance or other procedure was required to perform the work and whether the document was available.

l

,

For many maintenance activities observed by the team, the WIT would state, l-

" Item X is failed.

Troubleshoot and repair." This was translated to the i

MWR and, in most cases, became the sole instructions for repair of the l-19-

>

.

,_

w

-

_

l

+'

,

.

component.

In general, no emplifying "procedura; steps" or supplemental

'

work instructions were prepared except for complex repair activities, for

which standing procedures generally existed.

" Planning" was performed at i

the craf t level at the time the repair activity was started.

i p

"

'

After the MWR was prepared, one copy was, ".... sent to the Control Room

>

for advance preparations ~ for work accomplishment..." (Section V.D.1.a.).

'

No other details were provided in Procedure 7.0.1 about preparatory steps

'

for setting clearances, establishing necessary plant conditions, or 6ther functions expected to result from this step.

,

Paragraph IV.D.I b.(4) addressed the subject of "SPECIAL INSTRUCTIONS" but did not indicate when special instruction sheets were mandatory or

!

optional and did not provide guidance for their content. 'As an example, safety considerations such as confined spa :e work permits were otherwise <

'

,

omitted from the MWR procedure and could be included in this section.

In

addition, the paragraph provided guidance on the extent or level of detail of work instructions (specific guidelines should indicate which activity

,

instructions and which require only limited require step-by-step' guidance).

" skill-of-the-craft

'

,The procedure was silent with respect to requirements for prejob craft briefings.

Guidelines for interfacing with other craft departments and resuming work after an extended interruption were also omitted or ambiguous.

,

Examples of inadequate MWR work instructions included the following:

.

'

MWR 89-3306 authorized investigation and repair of Emergency Condensate

'

Storage Tank Test Line Shutoff Valve HPCI-M0-M024 to restore electrical

,

grounds on'the Valve's 250 volt DC bus supply.

During the repair, the

licensee found that the actuator motor insulation was degraded and required replacement.

Neither actuator repair procedures nor post-maintenance test

.

procedures were referenced by the MWR, The maintenance technicians that

'

did the " planning" appended portions of CNS Maintenance Procedure 7.2.50.5,

"Limitorque SMB-0 through SMB-4 Valve Operator Maintenance," Revision 1 u

L to the MWR, Portions of six different pages were actually cited as appli-

!

cable to the MWR, The balance of the pages were-marked "not applicable,"

a including one page that was applicable, but apparently was not used (page 9 of 43 for the motor removal).

Maintenance Procedure 7.3.33: " Electrically Disconnecting and Connecting Limitorque Valve Operators," Ksvision 4, was not used in the maintenance activity, resulting in a failure to perform

'

post-maintenance testing applicable to the scope of work.

ll MWR 89-2876 removed the motor from the actuator of the steam supply to

'

Augment Off-Gas Downstream Shutoff Valve RHR-M0-921MV to clear a jammed l

gear drive.

The work instructions were " investigate and repair as

-

required," and the MWR included no supplemental instructions, procedares, I-20-

-

.

h

.

,

or references.

The inspector could not determine whether electrical disconnection of the motor was required for the maintenance; therefore, the applicability of Maintenance Procedure 7.3.33 for post-maintenance testing could not be established.

MWR 89-4335 authorized maintenance technicians to " investigate and repair" the " Auto Blowdown Valve" (actually the solenoid pilot valve) for the air-operated motorized screen wash strainer backwash valve (CW-AO-2419AV).

No instructions or vendor information were included with the MWR or referenced by the maintenance technicians in their attempts to repair the solenoid pilot valve.

Consequently, the spring washer was incorrectly installed in the solenoid asstmbly of the valve following replacement of the solenoid coil.

Manufacturer's Bulletin 8345 depicted the proper assembly but was not referenced during the activity.

Additionally, maintenance technicians failed to recognize the mechanical failure of the valve af ter initial repair as indicated by air leakage symptoms that were observed and specifically addressed in Bulletin 8345.

Following these repairs, the valve would not stroke its slaved, air-operated valve, and additional work was required.

MWR 89-2192 required the inspection and replacement of specifically listed parts in the actuator for the Core Spray A testable check bypass valve (CS-M0-M015A). The MWR authorized replacement of the finger assembly, rotors, torque switch, gaskets, and 0-rings, and gressing of the limit switch.

Maintenance Procedure 7.3.33, " Electrically Disconnecting and Connecting Limitorque Valve Operators," Revision 2, was not referenced, included, or used, although the purpose of the procedure was "To provide electrical maintenance personnel with instructions for electrically'

connecting and disconnecting all Limitorque brand valve operators.

A series of three repairs on the same MOV in a period of 3 weeks indicated electrical work practice problems.

MWR 89-2179 performed on May 9, 1989, involved replacement of a cracked torque switch in RHR Heat Exchanger B Vent Valve. RHR-M0-M0166B using Special Maintenance Procedure 89-001.

No abnormal conditions were noted in the MWR concerning the condition of the M0V.

Post-maintenance testing was performed on the valve.

Three weeks later (May 30,1989), the valve failed to operate, and the licensee initiated MWR 89-2625 to "investigato and repair as necessary."

Troubleshooting was initiated on June 1, 1989, by stroking the valve.

The valve functioned, so no further action was taken and the MWR " Apparent Cause" section was left blank (indicating no explanation for the failure of May 30, 1989).

The valve was declared " ready for service."

Later on June 1, 1989, "high amps" were experienced when RHR-M0-M0166B was stroked full open.

MWR 89-2662 was initiated to " investigate and repair" the valve again.

Several adverse conditions were noted in the actuator, including:

a limit switch malfunction, a miswired torque switch (possibly from an error in the May 9,1989, replacement of the torque switch), worn gears,-and stiff grease.

The technicians essentially performed an actuator " overhaul" in accordance with Maintenance Procedure 7.2.50.3,

-21-

.

-

.

L.

.

I

"Limitorque SMB-000 Valve Operators Removal, Disassembly, Inspection, Rework, Reassembly, and Installation," Revision 1.

!:

However, noting the following discrepancies, the team concluded that the

!

actuator overhaul was not performed in full accord with the Maintenance Procedure 7.2.50.3.

Step 8.3.15.2, witnessed by QC, required an exacting measurement known as the "X dimension" to be made as follows:

!

"CAVTION" t

'

" Proper tension of the torque spring discs is very important.

The amount...

A measurement known as the 'X dimension' is obtained by measuring the opening from the outer face of the thrust washer to the face of the sleeve.

This dimension must be measured and recorded to within 0.001 inch."

Despite the required precision, the "X dimension" recorded in step 8.3.15.2 was "3/8 inch."

Step 8.5.2.9, again witnessed by QC, reassemLt.ed the torque springs by tightening the stem nut to the "X" dimension of step 8.3.15.2.

No adverse comments were included in the procedure to reflect the absence of the required exact dimension.

Section 8.7 contained final steps for assembling the actuator to the valve.

Section 8.7.1 applied to nonrising stem valves, and Section 8.7.2 applied to rising stem valves.

Both sections were signed off as completed by the technician for the same valve.

Under Restoration, Step 9.1 required, "Have the Electricians initiate

a MWR to set or verify the settings on the limit and torque switches." The MWR number space was left blank with no explanation.

There was no documented evidence in the package that the limit and/or torque switches were set or verified.

The " overhaul work" was completed on June 8,1989.

Based on-documentation in the MWR package, it appeared that torque and limit switches were not set properly because a " Work completed" entry for June 9, 1989, stated, "After reconnecting valve we could not get the valve to full stroke open." Apparently, further undocumented maintenance was then performed that resulted in proper post-maintenance test results obtained on June 10, 1989.

MWR 89-4241 was issued on October 30, 1989, to " investigate and repair as necessary" frequent trouble alarms on the Division III Containment Hydrogen - 0xygen Analyzer (PC-AN-H2/0211), an EQ category unit.

On October 30, 1989, the licensee determined that a low-flow condition caused l

the alarms, and sensing lines were blown down to correct the condition.

Two hours later a low-sensor temperature alarm occurred.

The original MWR-22-l

_ - _ _ _ _ _ _ _.

,

.-

....

.

was left open, and the sensor heat trace was diagnosed as having " bad parts" at the end of work on October 30 No use of troubleshooting instructions, technical manual, or other reference material was evident in

,

the documentation.. Heat trace parts were ordered, arrived, and were installed on November 20-22, 1989, post-maintenance testing was completed on November 22.

As the work crew departed the job, another low-flow condition occurred and the MWR was again left open.

On December 5, 1989, the inspector observed removal and shop overhaul of

,

the unit's sample pump.

No work instructions or other reference material were present or used.

Although the mechanics were quite familiar with the unit, the inspector observed two problems.

First, a Woodruff key on the pump shaft protruded about 1/4-inch from the end of the shaft and the mechanics, without consulting the pump manual, unsuccessfully attempted to t

hammer the key flush. When questioned by the inspector, the crew consulted.

",

the pump manual and concluded that the protrusion conformed to design specifications.

Secondly, the system engineer requested the mechanical,

,

electrical, and I&C shops jointly to bench test the pump in the shop for air pressure and flow.

The test, observed by the inspector, was performtd using an unsigned and unserialized supplemental work instruction that very-

,

'

vaguely described four desired test configurations.

Although the craft and engineer appeared.to exercise due-care in avoiding damage to the pump, no supervisory review and approval of the supplemental work instructions was obtained prior to work, as required by the instruction form.

Maintenance Procedures The team reviewed several maintenance procedures as they related to and were integrated with the MWR process.

The team also interviewed staff personnel responsible for preparing and upgrading procedures.

Approximately 85 percent of the mi w ance procedures have undergone an upgrade that improved the format t.. i mntent.

The remaining 15 percent will be upgraded by a licensee contractor beginning in January 1990.

The CNS Writer's Guide, " Manual for Preparation of Procedures," Revision 2, provided the upgraded format and specified attributes that were consistent with current industry standards.

The coraprehenJive nature of the Guide was considered a licensee strength.

The team noted that maintenance procedure writers were organizationally assigned under the respective craft supervisors.

Based upon interviews with staff personnel, this organizational relationship was not working well, particularly in the mechanical area.

It appeared that department supervision resisted the procedure preparation and upgrade function in three ways:

(1) industry experience was not readily available for incorporation into existing procedures, (2) existing procedures requiring increased detail were not readily changed, and (3) new procedures requiring development were not readily initiated.

This appears to warrant management attention.

Overall, the inspectors deemed this area of the licensee's program adequate, based upon the existing procedures available and the licensee's program for continued procedure upgrades.

I-23-

)

w-__-_____________--____-

________ _

_

=

_ _.

_l

w

,

f

,e

,

.

,

System and Component Cleanliness and Foreign Material Control The licensee had not cormistently implemented QA program controls for system cleanliness and exclusion of foreign material from systems and components.

"The CNS Quality Assuranco Program for Operations, Policy Document" expressed the licensee's commitment to ANSI N45.2-1973, " Cleaning of Fluid Systems and

,

Associated Components During the Construction Phase of Nuclear Power Plants."

Section 2.19 of the policy document required inclusion of cleanliness require-ments be as part of overall work instructions, rather than as separate documents.

In general, licensee procedures included either no cleanliness requirements or

,

only very generalized instructions.

For example, Maintenance Procedure 7.2.26,

,

" General Valve Maintenance," Revision 6, addressed the subject of system clean-i liness at paragraph VIII.A.3. by stating, " Assure that cleanliness is observed

'

during valve maintenance." No specific requirements or references were provided.

Further, Procedure 7.0.1, " Work Item Tracking - Corrective Maintenance," did not address system cleanliness.

MWRs typically did not include supplemental work instruction: that addressed these subjects.

l As another example, MWR 89-1192 outlined the annual diesel engine inspection in accordance with Maintenance Procedure 7.2.53.1, " Die:e1 Generator Engine Mechanical Inspection," Revision 0.

Both the procedure and the MWR omitted consideration of housekeeping zones, system cleanliness, and foreign material control during the inspection activity, although the engine interior was opened for the inspection.

A review of the training curriculum for the craft determined that the subjects were not addressed.

Based upon these findings, the inspectors concluded that the licensee had appar-ently failed to implement the QA program requirements for system cleanliness and material control.

This represented a maintenance program weakness.

Implemen-tation was not observed because of the limited maintenance in progress at the time of the inspection.

Threaded Fastener procedures and Practices The licensee did not have a consolidated policy or standard for torquing require-ments related to threaded fasteners.

Torque values were not consistently avail-able from the origiral construction specifications, vendor manuals, and some standing maintenance procedures and drawings.

However, no written policies or guidelines were available to instruct the crafts regarding when a fastener should, or must be torqued, and how to obtain the torque value if it was not immediately available from an appropriately controlled source.

The lead mechanical system engineer advised that, in general, fasteners in essential systems would be torqued to specific values.

In cases where specific i

values could not be found by the craft, they were expected to consult the system engineer who would obtain the proper value.

,

-24-

!

- - +

~

a.

I

.

However, mechanical maintenance personnel interviewed stated that their general policy was to torque essential fasteners when specific torque values were provided or available in a prucedure or other normally used references (i.e.,

drawing, vendor manual, etc..).

If torque values were not readily available and the application was not special or unique, the essential fasteners would be tightened " wrench-tight" without a specific torque value.

The team identified one example (MWR 89-4389) where the body-bonnet studs on essential Containment Isolation Valve RW-AO-A095) were tightened " wrench-tight" rather than being torqued.

The valve technical manual did not provide torque values for the specific valve style.

Discussions with engineering indicated that they would have expected to have been contacted for torque values and that the valve should have been torqued.

The cognizant mechanical maintenance personnel believed that they were following normal practice in using the " wrench tight" criteria.

The team also reviewed mechanical maintenance training on threaded fasteners and torquing.

The training was extensive, delivered on an appropriate schedule, and appeared to provide the knowledge and skills necessary to perform the general tasks involved in the maintenance activities reviewed by the team.

However, the training, did not include any criteria or guidance for when torquing should be used, nor did it address how to obtain torque values when they could not readily be found.

,

Similarly, the license failed to provide the craft with procedures, guidelines, or training regarding acceptable and unacceptable thread engagement conditions for nuts, bolts, studs, etc.

One-example of inadequate thread engagement identified by the team was associated with MWR 88-4642, "DG #1 Intercooler Repair." In this case, six of eight service water flange bolts were installed with improper thread engagement (as little as 50 percent engagement), and the craft had not recognized the condition as requiring correction.

Craft and training instructor interviews revealed that identification of inadequate thread engagement was considered so fundamental a craft knowledge that no specific training was provided to compensate for the absence of a standard guideline or procedure.

Post-maintenance Testing The inspectors assessed post-maintenance testing by reviewing completed maintenance documents and observing maintenance in progress.

This assessment revealed that the licensee was not, in all cases, properly specifying the scope of post-mainte-

-nance testing work expected and/or performed.

Frequently, post-maintenance testing specifications were left to the craft.

Further, Procedure 7.0.5, " Post-Maintenance Testing," Revision 0, included no provisions for administration of deferred testing and was silent with respect

l to tracking, accountability, and deferred test completion.

Deferred post-maintenance testing was controlled informally by either the on-duty shift 1.

supervisor or the assistant operations supervisor.

i-25-

,

!

-

4M'

y

.

The team identified the following examples cf improper post-maintenance testing specification or administration.

MWR 89-3206 outlined procedures for correcting an air leak on the solenoid pilot valve (SW-SOV-SPV857) for the air-operated service water control valve for the A and C strainers (SW-857AV).

No post-maintenance testing was. specified on the MWR prior to the maintenance activity to " Correct Air Leak." Troubleshooting revealed that an American Switch Company-(ASCO)

solenoid repair kit had to be installed to correct a " blowing air" and

" wetted solenoid" condition.

Maintenance technicians apparently elected

not to perform post-maintenance testing based on the documentation available with the MWR.

The MWR pack ge included portions of Maintenance Procedure 7.2.49, "ASCO Solenoid Valve Maintenance," Revision 2, which stated at paragraph 9.2, " Perform all post-maintenance testing specified on the MWR," but the MWR did not reference Procedure 7.2.49 or-post-maintenance testing.

'

MWR 89-2728 outlined procedures for correcting a wiring error for limit switches on Drywell Spray Loop A Inboard Isolation Valve RHR-MO-M013A.

e The MWR did not specify post-maintenance testing performance, nor did it document that the appropriate post-maintenance testing could have been performed subsequent to the repair by repeating the_ post-modification testing that originally discovered the problem.

MWR 89-4335 outlined repair procedures for the solenoid pilot valve for the air-operated motorized screen wash strainer backwash valve (CW-A0-2419AV).

The MWR did not specify post-maintenance testing for the repeir work, which replaced the solenoid and installed an internal-valve kit.

Documentation in the MWR reflected that the valve was " bench tested," but provided no indication that the system was functionally tested after the solenoid pilot was reinstalled.

MWR 89-3920 outlined replacement of the EDG No. 2 local control panel synchroscope with a new meter.

The MWR did not specify post-maintenance

' testing, and no functional test was specified or performed.

'

MWR 89-4370 outlined repair of the heater timer switch and conduit on the

.

heater loads and purge valve for Instrument Air Dryer B.

The MWR did not specify post-maintenance testing for any of the repair work.

A post-maintenance functional test may have been performed based on comments in the " Work Completed" section of the MWR, which stated, " Monitored

operation of Dryer 3 times daily from 11-20 thru 11-22-89."

Neither the

results of the monitoring nor acceptance criteria were documented.

Review of Completed Work Control Documents The requirements for supervisory and engineering review of work control documents and records were contained in Procedure 7.0.1, " Work Item Tracking -

Corrective Maintenance," Revision 13.

Section 8.7 discussed the various post-work documentation reviews and assigned specific review functions to specific job functions such as fire protection, QA, engineering, etc.

~26-

m.

-

g

'

p h

'

.

..

L

_

,

[-

x-i

'

'

,

4-

However, the procedure relied' heavily on individual discretion for performance

~

'

c, of the reviews.

For example, Section 8.7.7.2 stated.that a copy of the completed m MWR will.be forwarded to the system engineer who will, "._.. evaluate the action b

taken to ensure that the problem was sufficiently addressed...." The team L

identified examples of completely closed MWRs, which had the following problems described more fully elsewhere in this report, that had not been recognized and/

-

or addressed by the reviewer (s):

-

?

The need for highlighting problems identified during maintenance (such as l

-

wiring errors, an unpacked valve, improperly assembled components, 'etc.)

,

for management attention, root cause evaluation, and followup.

.

.:

Missing or in' adequate post-ma'intenance. testing.

Improper documentation of procedure completion.

'

Missing or inadequate QC inspection requirements or performance.

Missing or inadequate work instructions or technical).information (such es P

torque requirements).-

P The large number of such problems identified by the team indicated weaknesses in the licensee's program for post-work review.

l Equipment Records and History -- Inaccuracies Noted in the Equipment Data

'

File (EDF)

The computerized Equipment Data F.ile (EDF) was used for equipment identification and the performance of both safety-related and nonsafety-related activities.

The EDF also functioned as a master equipment list.

The inspectors identified one example (discussed below) in which electrical personnel were unaware of the need to update the EDF when components were identified that were not previously' covered in the file..In addition, CNS Engineering Procedure 3.15, " Equipment Data File Indexing Guidelines,"

Revision 2 Section IV.B, stated, "The Maintenance personnel performing a Work Item are responsible for initiating Component Index Sheets when plant equipment in the EDF has been affected by Maintenarse Work' Items."

Interview results

'

indicated that, for some systems and plant areas, the EDF input had not been

. verified.

Consequently, not all components in the plant were included in the EDF and subject to maintenance history, spare parts coverage, etc.

The team

,

identified the following examples of these concerns:

MWR 89-4335 outlined repair of a solenoid pilot control valve for

'

Air-0perated Valve CW-AO-2419AV.

No Component Identification Code (CIC)

had previously been issued for the solenoid pilot valve, even though the valve had been repaired at least twice prior to the current maintenance activity.

Although it was suspected that the pilot valve was faulty, MWR 89-4335 cited the CIC for the slaved flow valve controlled by the solenoid valve.

Maintenance personnel took no action to initiate a change to the EDF.

-27-

,

a w

-- =

-- -..m

-.

.

-

.r

-

,

-

.,

'.-

.-

L

'

Although listed in the Q-List, the team noted that (Core Spray System "B" i

vent root valve (CS-V-133) was not listed in the EDF.

(Its counterpart on

[

the "A" train was listed, e

l>

'

HPCI DC Motor Starter Time Delay Relays EE-REL-TR1 and EE-REL-TR2 (for HPCI-M025A control circuitry) were listed in the EDF as being installed in

-

the HPCI M025A Starter Control Panel.

A visual inspection of the motor starter determined that the two relays were not in the panel.

It was later determined that the relays had been removed by a design change completed on or about June 1, 1989.

The associated administrative changes for the design change, including updating the EDF, had not been completed

,

as of December 1, 1989.

.

MCC-K REC Pumps A, B, C, and D Starter Time Delay Relays, CIC # REC-REL-TDR(RCC-A, -B, -C, and -D), were listed in the EDF as t

Model 7012 Agastat relays, which meant that they were not EQ.

The actual model was an E7012 time delay relay, which was EQ qualified.

The PALL Trinity Micro Corporation Instrument Air Dryer vendor technical

-

manual was numbered 0143, but the EDF cited manual number E68-24-2,.

(applicable prior to a major vendor manual update program completed in

1985). - Cross-references available to (but unknown and unused by) maintenance and document control personnel would permit the licensee to overcome the

-

error in the EDF.

'

The inspectors perceived Engineering Procedure 3,15 as weak in that it did not specifically address the subject of " discovering" components in the plant that

'

were not previously in the EDF.

Procedure 3.15 was written from the standpoint of updating the EDF to reflect routine changes such as a plant modification.

Section IV.D of Procedure 3.15 required the EDF Coordinator to verify every 6 months that 10 percent of the components have CIC CLASS attributes that-~

'

agree with the Q-List.

The Q-List was generated in part from the evaluation of componenta 1.isted in the EDF; therefore, this requirement accomplished no real

' verification or validation of the EDF.

The absence of component-by-component

-

-checks for each system by walkdown inspection and the. verification that those components were in the.EDF and/or Q-List was considered a weakness.

The

'

licensee initiated a temporary change during the inspection to correct the

program inadequacy.

In addition, the licensee was evaluating the latter consideration at the end of the inspection.

The problems discussed above were well known to many of the plant's staff and-had functionally been overcome by the use of alternate methods of obtaining

,

necessary information.

However, the team was concerned that inaccurate EDF

,

information could be used by maintenance personnel in performing maintenance activities.

,

Improper Component Identification Codes Recorded on MWRs

,

MWRs utilized CICs as the principle tracking number to identify of the " failed"

-plant component.

The team noted several examples where, at the time of the

deficiency report, the originator of the deficiency did not know exactly which

h-28-

'

l-l

P j

s

t'

'

L i

'

component had failed.

Consequently, the originator entered several CICs or the wrong CIC on the MWR.

The significance of entering the wrong CIC was-that it l

engendered errors in failure reporting and material history, which were tracked

by CICs.

Since the MWR planning group was associated with all MWRs from-

inception to record storage, it appeared to the team that they should have i

ensured the accuracy of the CIC.

The following examples were noted.

MWR 89-2728 listed CICs RHR-MOT-RHRPIA and EE-CB-4160F(RHRP1A) (the pump

,

'

and its feeder circuit breaker) as the failed components.

Further investigation revealed that the failed component actually was the motor i

actuator of Valve RHR-M0-921MV.

However, the licensee failed to correct the MWR after determining the proper CIC.

,

s

MWR 89-2708 listed the CIC as RHR-MOT-M015A.

The proper CIC should have been RHR-M0-M015A since the motor actuator did not have a separate CIC

,

from the entire valve assembly, i

MWR 89-4335 reported an improper control function (air) of the auto Blowdown Motorized Screen Wash Strainer Backwash Valve CW-AO-2419AV. The

,

'

failed component actually was the solenoid pilot valve for the CW-A0-2419AV valve.

No separate CIC was assigned to the solenoid pilot valve in the EDF.

t Technical Specification Impact on Deficiency Correction

.

Technical Specification 3/4.5.C.2 and the emergency core cooling system, "HPCI i

Subsystem," and other Technical Specifications for engineered safeguards prohibit

'

" voluntary" entry into the limiting condition for operation action statement for maintenance without extensive surveillance of the remaining systems to demonstrate

-

operability immediately and daily thereafter.

This provided an impediment to scheduling and performing what should have been

',

routine corrective and preventive maintenance.. As a result, unless a maintenance

' item directly and clearly involved the inoperability of the Technical Specifi-cation system or component, the maintenance item would be deferred until a mode-change or shutdown when the Technical Specification action and surveillance

-

1 requirements-did not apply.

This avoided challenging and/or overtesting the systems during the immediate and daily surveillances, but also deferred otherwise practicable maintenance, s

Examples of deferred work items include:

--

HPCI pump coupling inspection for grease problem (discussed previously in this report).

' *

1A-RHR pump shutdown cooling suction isolation valve (RHR-MO-M015A).

A

"

work item issued on June 5,1989, for repair of a bent shaft.

Work on I

this' shaft had been deferred to the next outage with no MWR written.

l^

l

'

-29-L

-

.

..

.

g

,

.

b.

.

,

'

Replacement of an improper cotter pin in MOV HPCI-MO-M024, for which the'

work item was written July 27, 1989, and scheduled for the next refueling outage.

!

Inspection of HPCI Control Signal Box 330 for correct resistor

'

identification.

>

The licensee had not pursued updating their Technical Specification to utilize the wording of the current Standard Technical Specification position which

~

permits 'Yerification" of other systems' operability by confirmation of

,

surveillance data, etc.

l 3.2 Plant Maintenance Organization

.

.3.2.1 Scope The inspection and assessment of the plant maintenance organization included i

i the review of the following:

Control of mechanical maintenance activities.

Control of electrical maintenance activities.

'

Control of instrumentation and control maintenance activities.

Control of contracted maintenance.

.

Deficiency identification and control methods.

Maintenance trending.

Support interfaces.

The team observed maintenance activities of each of the craft disciplines to determine the adequacy of the shop's functional performance.

The team reviewed completed work packages, procedures, and activities in progress.

In addition, the team conducted interviews with shop personnel and key managers to ascertain

,

their knowledge and understanding of organizational interfaces and individual

,

responsibilities.

!

3.2.2 Conclusions The maintenance department demonstrated a strong interdepartmental organization and had strong support from other plant departments.

Craft activities were not

,

always controlled by adequate conduct-of-maintenance guidelines.

Craft percep-tions of their own skills may have contributed to the failure of technicians to recognize consistently the need for procedural usage.

Although requirements for maintenance actions were usually identified, conditions adverse to quality that required evaluation were not readily recognized and acted upon quickly.

Contracted maintenance use was limited, but contractor loyalty to the licensee permitted the licensee to select and use the same contractors on succeeding outages.

The licensee failed to implement maintenance trending programs, in a manner that would yield the greatest benefits.

-30-

..

I'

'

,,

e 3.2.3 Findings Mechanical Maintenance Organization The team reviewed.the activities of the mechanical maintenance department through L

a combination of work observation and review of past maintenance documentation.

The craft appeared knowledgeable of their equipment and had a high average experience. level.

Interaction with other departments appeared adequate.

,

-Interviews and observations of departmental supervision and journeyman craft

'

personnel detected a tendency to shun the use and value of procedures, relying instead on their self perceived personal skills and experience to accomplish L

both essential and nonessential maintenance activities successfully.

Similarly, several department members indicated less than enthusiastic support and

,

[

acceptance of the peer quality control inspection process.

Specific examples of L

workobservationsinvolvingperformancedeficienciesarediscussedextensively.

in the " Work Control" and Technical Support" sections of this report.

Electrical Maintenance Organization The inspectors perceived the personal motivation and pride in responsibility for the plant's proper operation as a strength of the craft in the electrical organization.

Their attitude and commitment to performing their assigned tasks to the best of their ability was readily evident in their daily performance.

Shop supervisors were intimately involved in the day-to-day activities and the welfare of the technicians, lending guidance and support when appropriate.

Although the team perceived work instructions as a weakness as detailed in other sections of this report thecraftdemonstratedskillsprevIouslypracticedon

_

many occasions in obtailling necessary and appropriate references such as on-line

'

electrical diagrams and technical manuals.

When faced with MWRs that instructed the craft to " investigate and repair," shop planning was conducted at the craft level ~using such documents to ensure timely and effective field troubleshooting.

Shop foremen monitored craft qualifications to ensure that only qualified personnel were assigned to maintenance tasks; craft personnel requiring specific on-the-job

. tasks to complete qualification were judiciously assigned to ensure timely

. completion of qualification tasks.

The shop foremen personally managed the electrical measuring and test equipment (M&TE) program and the procurement of special tools to ensure that the craft had available the necessary items to support required maintenance.

i Instrumentation and Control Maintenance (I&C)

l-

'The I&C discipline appeared strong in its ability to perform assigned maintenance and repair functions.

Craft personnel communicated well within the shop and

. among supporting organizations (such as operations, health physics, and system

,

engineering).

The craft appeared knowledgeable of their equipment and plant

!

procedures.

However, the team noted that, in some cases, the maintenance i

_

procedures were marginal in content.

In some instances, the procedures failed l

.

,

l-31-

.

i

'

__

_ _ _ _ _ - _ _ _ _ _ _ _ -.

-+

.

to delineate specific directions for QC, preventive maintenace testing, and cleanliness.

In other cases, the maintenance technicians performed activities without written procedures.

The following examples illustrate the above concerns.

The craft performed preventive maintenance items 02564 and 02565 without formal procedures, using only a preventive maintenance card and a calibration sheet (RR-7A & 78). The preventive maintenance card referenced the calibra-

' tion sheets (rather than a procedure), though the calibration sheets were not formalized.

The craft was responsible for transposing data from previous I

calibration sheets.

It was noted that the calibration sheets referenced the applicable drawings, but these references were not present at the job site during work performance.

Furthermore, the craft did not use a lifted-lead logging system for record and control purposes.

In discussions with the 1&C supervisor, Ra inspector determined that the lifted-lead control process had not been instituted down to the preventive maintenance activity level.

Surveillance Procedure 6.2.5.1.1 contained insufficient guidance for con-ducting the work activity.

During the surveillance, the technician connected

'

the test rig to the high pressure side of the Barton differential pressure meter in conflict with Procedure Steps 8.1.6 and 8.1.7, which required connection to the low pressure side.

The technician pressurized the high pressure side to 20 inches water (trip point was supposed to be 12.5 inches water) before contacting the assisting technician in the control room. The technician corrected the error and continued the surveillance without inci-dent.

During subsequent discussions with the I&C supervisor, the team determined that the in plant system labeling disagreed with the procedure.

The licensee subsequently changed the surveillance procedure to agree with in plant labeling to avoid future confusion on the part of the technicians.

-

'

MWR 89-4550 was written for investigation and repair of a heater level control valve (CD-LT-66).

The team observed that, while performing the work in a high radiation area, the technicians did not have instructions for work on the controller and did not carry with them the MWR or a copy of it.

The inspector noted that the work activity planning for I&C assigned items was the responsibility of the I&C foremen.

There was no_ formalized planners'

guide utilized.

The inspectors noted instances where the MWRs did not reflect the required QC or post-maintenance testing, as well as examples of inconsistent M&TE identification in the work package.

Despite the above observations, the inspectors found that the craft generally performed their maintenance functions well with minor exceptions.

The plant had not experienced many inadvertent engineered safety feature actuations or scrams because of craft error (human error, procedural inadequacy, or procedural nonadherence).

,

The inspectors concluded, however, that the strength of the maintenance function performance resided with the crafts' abilities and knowledge. The-32-s

, -

. - - _ _ _ _ _ _ - _ - _ - _ - - _ _ - _ - _ _ - - _ - - _ - _ _ - - _ _. _ _ _ - _ - _ - _ _ _ _ - _ - - - - _ - - _ - _. - _.

_-

_

- - - - _ - _ - - - - - _ - _ - - - - -. - - - _ - -

.

a

.:-

,

-

.

craft implementation of maintenance activities was satisfactory; however, E

there was a lack of formalized programmatic attributes which would have f

ensured complete and comprehensive documentation.

Control of Contracted Maintenance In general, the team found that the licensee's maintenance department utilized

,

contractor support at the craft level only during refueling outages unless an unusual circumstance arose.

Loyalty and availability on the part of contracted maintenance personnel made it possible for NPPD to retain the same personnel, literally by name, on succeeding outages.

Thus the licensee enjoyed a high

'

l.-

degree of reliability in the performance of the support personnel.

Specific L

qualification requirements were included with contractual arrangements.

All

!

contracted maintenance personnel performed their work under the licensee's programs and procedures, as opposed to approved programs of the contractor.

In addition to general employee training for purposes of site access, contractors l

were required to complete component-specific training, if applicable, to perform

maintenance activities.

The licensee performed direct supervision or worked side-by-side with contractors to ensure adequate monitoring of contractor

,

t performance.

Based on interviews with plant staff responsible for contractor

'

performance, the team concluded that the ability of the licensee to choose contractor personnel resulted in work quality equivalent to that of the plant

craft.

.

Deficiency Identification and Control Title 10 CFR 50, Appendix B, Criterion XVI, provides for two levels of corrective action mechanisms..The regulation requires that less significant conditions

'

adverse to quality (such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and nonconformances) be promptly identified and corrected.

For significant conditions adverse to quality, the regulation also requires that the licensee determine causes, take preventive measures, document those measures, and report them to appropriate levels of management.

The MWR system was the principal mechanism for routine deficiency identification and corrective action for material condition deficiencies.

Procedure 0.5.1, "Nonconformance and Corrective Action," Revision 3, was esta-i blished for identification of significant conditions adverse to quality in accor-dance with the commitments of Sections 2.15 and 2.16 of the " Cooper Nuclear Station Quality Assurance Program For Operation, Policy Document," Revision 5.

The licensee had no mechanism for the identification of deficiencies and conditions adverse to quality that may not clearly meet the significant thresholds of Pro-cedure 0.5.1 and may not.be suitable for processing via the MWR system.

The licensee relied on systems engineer and supervisory reviews of work activi-

' ties and documents to identify conditions requiring escalation to management and/

or additional evaluation and/or corrective and preventive actions.

The team

. identified the following examples where:

(1) conditions adverse to quality were encountered but were not necessarily recognized as requiring further evaluation and corrective or preventive 1ction, or (2) significant conditions adverse to quality were identified, but a nonconformance report was not prepared.

-33-

,

____

'

.

,

+

.

,

Manual' Valve RHR-V-67 was replaced.with a new valve in accordance with

,

MWR 89-2404. The new valve was installed using minimal work instructions

<

(as discussed elsewhere) and, when initially pressurized with reactor coolant, leaked severely from the packing gland. Maintenance personnel

- believed that either the valve had not been packed or the packing was so minimally torqued that it blew out when pressurized. The valve had been drawn from stores and welded into the system by the plant welders.

In past practices, the welders would have requested that the mechanics pack the valve prior to installation.

(This step was not usually specified by missed for this the MWR or work instructions, however, and it was possibl{ replaced packing workitem.) The MWR documented the corrective action as in valve that was blown out..." and provided no additional

'

identification of the condition or followup actions.

The team noted that no systematic method existed to ensure that such valves were packed prior to installation.

The team considered this an example of a condition adverse to quality for which identification and correction were inadequate.

MW 89-3206 repaired an inoperable, leaking service water solenoid pilot valve (SW-50V-SPV857), which controlled an air-operated flow control valve for Service Water Strainers A and C (SW-857AV).

The essential solenoid pilot valve was inoperable because of wetting of the solenoid and required a valve kit and a new solenoid to effect repair.

The MWR offered no explanation of wetting and no further evaluation of the failure was evident.

The team considered this a condition adverse to quality warranting further action.

MWR 88-4642 documented the replacement with a spare of the-leaking No.1 EDG west side intercooler.

Subsequent vendor testing and inspection of the removed unit found severe cooler degradation from corrosion including both tube wastage and clogging as documented in the MWR package.

However, no followup action was initiated to investigate the failure, even though the vendor declared the intercooler beyond economic repair.

The team considered this to be a significant condition adverse to quality warranting issuance of a NCR.

MWR 89-2708 required the replacement of a burned-out motor in essential

,

Motor Operated Valve RHR-M015A (RHR Pump A shutdown cooling suction).

The motor windings were overloaded during testing of Design Change 87-118,

"RHR Pump Interlock Modification," apparently because of unanticipated wiring problems.

This problem was identified at approximately 5:00 a.m.

on June 5, 1989.

The repairs were completed during the day and the shop supervisor reported the MWR complete at 4:49 p.m. on June 5,1989.

The apparent cause listed on the MRW was "... due to incorrect wiring on D.C.87-118 for RHR Interlock installation," but no further action was taken related to the design change.

At approximately 5:00 p.m., while attempting to start RHR Pump 1A to continue testing of the design change, momentary overloads, grounds, and breaker trip alarms were received.

MWR 89-2726 was issued to investigate and repair what was perceived as a problem with RHR Pump 1A. Troubleshooting identified the actual problem as a wiring problem in the limit switches of MOV RHR-M0-H013A, the Drywell-34-l

.

-

i

'

x _

.

L

!

Spray Loop A inboard isolation valve.

The control circuitry of the RHR pump was adversely affected by a wiring error in RHR-M0-M013A, a valve ~ in the interlock circuitry of the pump, and affected by Design Change 87-118.

j No apparent cause was listed on the MWR, and no further action was

,

documented except to change the wiring.

The team considered this example

.

L to be a significant condition adverse to quality for which an NCR should

have been prepared.

This failure of the licensee to establish and implement aggressive programs to

'

identify and document preventive actions and to report conditions adverse to L

quality was a weakness.

The team further noted that, because these examples l

were not recognized as requiring followup, root cause evaluations and/or failure analyses were not performed in all cases.

,

'

Maintenance Trending The maintenance trending program at CNS was in its inf ancy.

Programs such as diagnostic testing and evaluation of performance data, equipment history data, failure history, and industry operating experience were, for the most part, in a developmental phase.

A formalized trending program had not been implemented as discussed in Section 2.3.3.

.

It was difficult to ascertain the overall adequacy of the present preventive maintenance database because insufficient trending data existed (i.e.,

vibrational analysis and deficiency trending).

Supporting data was assimilated by certain system engineers using varying methods and no formalized feedback system existed.

During the inspection, the inspectors identified items which should have been included in the preventive maintenance program, but were not.

Examples included temperature switches on the instrument air system heater dryer, whose failure resulted in a loss of instrument air and subsequent scram.

Additionally, a review of MWR 88-4642, which replaced an EDG intercooler because of the tube leakage, indicated that the program did not include

,

preventive maintenance for periodic flushing of the intercooler.

In addition, a limited review of the licensee's QA audit program indicated that audits of the preventive maintenance program typically did not entail a

,

comprehensive review cf vendor manuals for inclusion of recommended preventive maintenance.

The licensee's audits consisted of only a limited sampling.

Support Interfaces On the basis of direct observation and discussion with shop supervisors and craft personnel, the team concluded that support from other departments was fully implemented.

System engineers and'other personnel willingly and promptly assisted maintenance and troubleshooting efforts.

It appeared to the team that excellent rapport existed between the maintenance group and other support l

personnel who demonstrated an excellent commitment and attitude toward

'

maintenance of plant systems and equipment.

The following examples highlight

.these observations.

-35-

- - -,

p

"

,

!

.

While troubleshooting a radwaste tank pump controller, the electricians determined that it was necessary to bypass specific contacts of a level

[

controller to prove the theory that the pump controller was functioning

!

correctly and that the level controller was faulty.

The operations group radwaste operator was summoned, promptly arrived, verified system line-ups with the appropriate procedures, and assisted with controller operation to verify the theory.

In another instance, electricians required engineering verification of their evaluation of a complex timing device for the instrument air dryer.

The

,

[-

degrees of rotation for the timed operation of the micro-switches that controlled engineering drawing described the operation of the timer as a function of

- the dryer's' heaters.

Consultation with the system engineer was made through a telephone conversation.

The visit, plan review, and confirmation of the L

electrician's evaluation took only a few minutes, and the electricians were soon back at the repair effort.

.

I 3.3 Maintenance Facilities, Equipment, and Material Control 3.3.1 Scope The team assessed the maintenance facilities and controls over parts and materials, maintenance tools and equipment, and metering and test equipment to determine how well these elements support plant maintenance and repair activities.

3.3.2 Conclusions The~ maintenance facilities were generally adequate to support maintenance.

With minor exceptions, the controls for equipment, tools, and metering and test equipment were satisfactorily established and implemented.

The program for material control in the warehouse was well documented and, with minor exceptions, well implemented.

The program for material control in the plant +,001 crib and plant spaces was weak and relied heavily upon the integrity and diligence of plant personnel to ensure that unqualified material were not used in essential applications.

3.3.3 Findings Parts and Material Control-The licensee had established and implemented a program that specified controls for receiving, handling, and storing of materials in the warehouse. Additionally,

the licensee had-established controls for procurement of parts and materials, warehouse issues and returns, control of spare parts (including a reorder L

mechanism), and identification and control of shelf-life items.

,

-36-

_-_;

n~-

t b

The existing warehouse had insufficient controlled storage space.

For example, some of the bays had materials stacked too high creating falling object hazards.

l'

There were eyewash stations near hazardous chemical storage locations; however, there was material in the aisles blocking the path to the eyewash stations.

The licensee was in the process of constructing a second, larger warehouse which should relieve the congest. ion.

Even though-the warehouse had limited space and manpower, good control and protection of stored materials and equipment existed, as evidenced by sleeves protecting bolt stock, coverings over the open ends of pipe fittings and rotating machinery, and clear identification numbers.

However, the following deficiencies in the program were identified by the team.

e

,

The CNS " Quality Assurance Program for Operation, Policy Document," Revision 5, dated June 7, 1989, Section 2.13, committed the licensee to follow the guidance of ANSI N45.2.2-1972, " Packaging, Shipping, Receiving, Storage, and Handling of Items.for Nuclear Power Plants." Section 2.7.1 of that standard defined Level A items as those exceptionally sensitive to environmental conditions and required special measures for protection from one or more of the following effects:

temperature,-humidity and vapors, physical damage, and airborne contamination.

Section 6.4.1 prescribed for items in storage the requirements for periodic inspection and examination and the subsequent documentation of the results.

Section 6.6 required preparation of written records that included

such pertinent information as storage location, inspection results, protection, and personnel access.

The inspector determined on December 4, 1989, that Plant Services Procedure 1.7, " Warehouse Storage," Revision 5, dated December 23, 1987, violated these ANSI requirements since:

No temperature or humidity limits were identified for protection of the Level A storage items.

No provisions were implemented for recording temperature and humidity conditions for Level A items.

Requirements for periodic inspections and examinations were specified, but not implemented.

This is another example of the apparent violation (298/8931-01) of the failure to follow plant procedures.

Plant Service Procedure 1.6, " Warehouse Marking and Tagging," Revision 5, dated June 4,1988,Section V.3.2.a.(2) required that parts scheduled for future troubleshooting or evaluation shall be identified with a " HOLD" tag.

Section V.B.2.a.(2)(a) stated further that items having expired shelf-life shall be identified with a HOLD tag.

Contrary to these requirements, on December 4, 1989, while touring the warehouse on December 4, 1989, the inspectors observed untagged, used electrical parts on a dolly.

Discussions

!

-37-l l

'

-

~

--

- -.

.

-

.

with the design engineer and receipt inspector indicated that the e & trical parts were replaced during the 1989 refueling outage with EQ parts, and the used parts were awaiting dedication for use in non-EQ essential systems.

Similarly, ASCO solenoid valves located in Level A storage had an " ACCEPT" tag upon which was written that the shelf-life of the internal components had-expired and that it was necessary to get a rebuild kit prior to installing the valves in the plant.

The use of the " ACCEPT" tag was improper since Procedure 1.6 requires a " HOLD" tag to communicate this type of problem.

This is another example of the apparent violation (298/8931-01) of the failure to follow plant procedures.

Overall material control in the plant was a weakness.

There were no requirements specified for control of materials that were issued by the warehouse to the-craft.

Formal control over chemicals within the protected area was not evident.

Proper use of chemicals was lef t up to the craf t; however, it should be noted that personnel had been instructed to contact the chemistry department anytime they were unsure about proper use of the chemicals.

The licensee was drafting a procedure that would provide control of chemicals for use in the protected area on safety-related components.

In the electric shop and tool crib, the inspectors found components which did not have tags indicating that they were accepted by the warehouse for essential

'

or nonessential use.

The licensee did not have a program category for these materials.

Based on interviews, the team determined that the issuance of-these components to the craft would only occur when the MWR was for nonessential work item uso. The existence of these noncontrolled parts and materials provided the potential for unauthorized use of nonessential material in essential equipment and systems.

Plant Services Procedure 1.11, "ESPI (equipment spare parts inventory) Program,"

Revision 4, Attachment B, dated September 7, 1989, provided a list of consumables

!

in the plant.

The list was short and incomplete.

The team considered that list a weakness, since it failed to identify many items used in the plant that should have been designated as consumable (such as shim stock and lapping compound).

Maintenance Tool and Equipment Control In general, the control of tools and maintenance equipment in the shop spaces was adequate.

(Control of calibrated tools and equipment is discussed separately under " Control and Calibration of Metering and Test Equipment.")

Some problems (illustrated by the following examples) were observed with

contaminated tools stored for outage use and tools and equipment stored in the

-

plant spaces, especially the radiologically controlled area (RCA).

Wire and nylon rigging slings were abandoned in place, shackled to the ceiling of the HPCI pump room, l

,

.

-38-

!

I

.

-

.

o

,

f

.

t

'

!

Rolling tool bcxes (gang boxes) were staged throughout the RCA containing t-piles of toole, loose hardware, consumable sealants, lubricants, and

solvents.: Tlie tools and materials were typically dirty and disorderly in

,

i and on top d the boxes.

No tools ' requiring calibration or special

storage were observed.

l p

}.

Initial observations of the laydown area in the 903' elevation of the

[

L turbine building (adjacent to the weld shop and water treatment area)

I revealed loose equipment, large tools., rigging equipment, pipe and p

structural steel stock, etc. stored in the open and largely unmarked.

i n

i~

Contaminated tools used during outages were stored in the areas above and i

within contaminated storage drums and boxes in the solid radwaste compactor.

The drums and boxes were stacked nearly to the ceiling and

'

appeared to present a handling hazard in that some stacks appeared only

'

marginally-stable.

,

!

During the first week of the inspection, the licensee performed a major

'

cleanout and reorganization of the area; this activity had apparently been L

planned prior to the beginning of the inspection.

Reinspection following cleanup found conditions substantially improved, but untagged pipe and other i

materials still remained in the area.

t Control and Calibration of Metering and Test Equipment (M&TE)

,

The mechanical shop maintained torque wrenches and other calibrated devices

.

used for torquing and metrology.

The electrical and I&C shops maintained their own orograms and inventories for tools, meters, etc. covered by the

.

E normal preventive maintenance program.

,

The team reviewed the shops' programs for issuance control and device

!

traceability and found the electrical and mechanical s< hops' programs to be L

acceptable.

However, the I&C and electrical shops did not verify-(against'a

,

secondary standard) the calibration of M&TE upon issuance or return.

'!

Identification of-out-of-calibration M&TE was not apparent until the calibration due date period.

This program was identified as a potential

, weakness in'that traceability of M&TE to specific work items was dependent upon

.

the WIT system and identification of M&TE on the MWRs.

The inspectors also noted the following additional observations regarding the I&C department M&TE

'

a activities.

c, I

~

MWRs 89-1612, 89-1059, and 89-1621 inconsistently identified the M&TE used in the performance of the work activity.

Instrumentation numbers were either omitted from the MWR cover sheet or differed from those

.

!

identified on the enclosed M&TE sheet.

,

The I&C shop provided a segregated quarantine area for defective instruments.

However, the team noted that M&TE (both contaminated and

,

noncontaminated) was stored in uncontrolled areas, and no formalized issuance or return methodology was established.

-39-

'

,

pA

~

'

a

_

f'

+

,r

+

L

.I&C department M&TE range and= accuracy had been determined and recorded'

-*

L for each instrument' scale in the test equipment index.

The I&C shop-maintained approximately 250 M&TE items of which 36 items were out of

calibration and overdue (i.e., more than 2 months past.their due date).

'

The team noted that the I&C's ability to perform work activities was not-affected by this backlog.

Furthermore, the backlog numbers were somewhat _-

,"

misleading since identification of M&TE requiring calibration was

,

accomplished 4 months prior to the actual out-of-calibration date.'

In

"

. discussions with the shop personnel, no problems existed in obtaining M&TE-on an-emergent basis and sufficient M&TE spares were on hand.'

:

Based upon these observations, the licensee's program for control of M&TE appeared satisfactory.

However, the identification of the M&TE used in. work activities appeared to be a potential weakness, as discussed above.

Greater licensee attention to detail and complete and accurate documentation of M&TE

.

. identification on MWR cover sheets was warranted.

' 3. 4 Personnel Control 3. 4'.1' Scope

.

'

The. team's inspection and assessment of personnel control included a review of.

>

maintenance staffing, personnel training, and the test and qualification.

process.

3.4;2 ~ Conclusions The licensee's human resource policies had been implemented and had established

~

effective staffing.

Based upon the. low backlog of work items at the craft and g

technical support levels, current and projected staffing appeared adequate.

'

The licensee's supervisor or worker ratios. were adequate to provide effective control and coordination of work activities.

Procedural improvement

,

requirements, particularly at the MWR preparation level, may adversely impact

,

staffing resources.

' Although the team recognized the licensee's extensive commitment to training and' qualification in quantity, the licer.see should improve training quality in

'

L the classroom and on the job to ensure quality craft performar,ce.

,

!

1:

'

3.4.3 Findings The team found.the licensee's maintenance staff to be highly _ experienced in the electrical and mechanical areas, averaging about 16 years experience.

The I&C technicians were not so experienced, having an average of only 5 years experience.

Individuals interviewed attributed the reasons for the low experience level in I&C were related to the licensee's compensation, benefits, and promotion policies.

Continued staff growth in all disciplines, with low attrition, made it probable that plant experience would continue to increase.

The licensee was noted to have enjoyed an ability to retain contract help on a

" named person" basis; this same ability was found in their hiring practices.

-40-

.

ym +_ _.

q f

Q':#

o

-

p,

,

C The ' licensee was able to meet craft increase requirements by hiring only

qualified " journeyman" (entry level at the plant),: who had previously worked at

. th, 9 nt as contractors and, therefore, were known entities.

Cu..

.t, organizational charts were available for the entire plant staff.

They

.

'

7'

reflected a supervisor or worker ratio adequate to maintain eloctive control of the work force.

Worker accountability was handled at the direct supervisor level, and a ready resource of job applicants. encouraged high standards of personal performance.

,

I As discussed in the section concerned with job planning, MWR details needed to be increased.

Since the task of MWR " planning" largely resided at the craft

'

level, both worker and supervisor strength needed to be increased if the current policy was continued.

Emergency maintenance and off-shift coverage was handled on a call-in ba' sis.

On-duty call rosters for craft had not been found necessary because of close proximity of persons available for call-in, and the informal. agreements about

'

who would be available for call-in, if necessary.

During the inspection, a major equipment failure occurred on a weekend back-shift; initial. craft response occurred within 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months , and. full coverage was available and operative in 6 hours6.944444e-5 days 0.00167 hours 9.920635e-6 weeks 2.283e-6 months .

Mechanical Maintenance Staffing Mechanical maintenance department staffing had increased slightly over the past 2 years.

Staffing included 1 supervisor, 2 foremen,1 parts specialist, and 18 mechanics, for a total staff of 22 persons.

During 1989-89, the licensee added 1 foreman, 1 lead man, and 1 mechanic (included in above figures), for a 22 percent growth over the original 18 person staff.

During the same period, the department dedicated two mechanics to the new preventive maintenance requirements, one mechanic to-procedure writing, and one mechanic to training and safety support.

This effectively negated the staff ' increases as far-as direct work load support was concerned.

l

. During 1987-09, training grew from a minimal time commitment to a cycle.

'

' commitment of 1 week out of 4, effectively removing about 25 percent of the staff from the work force.

No compensatory changes in staffing appeared to

'

have been planned, although one additional mechanic slot may be scheduled for 1990.

L I&C Maintenance Staffing

'

The:I&C maintenance staff included approximately 20 technicians, 3 specialists, 3 foremen,'and a parts clerk.

Of the 20 technicians, 9 were fully qualified

~

-(i.e., had Level I or II certification).

The remainder of the technicians were Level III with seven of them being uncertified and in training.

-41-

- -

f-y na

.

F'

-

.

0 9

..

_

L'

'

--This staffing allowed the I&C shop approximately nine technicians available for-

[

routine wor _k activities with three to six technicians in training.

Of the three foremen, two foremen performed prejob planning, in progress work Lcoordination, and maintenance of current procedures and vendor manuals.

The third foreman-was responsible for tracking the work item program and prioritizing work items.

Two of the I&C specialists were responsible for-procedure writing, while the third. logged changes to procedures and acted as an administrative clerk.- The foreman / worker ratio was about 1-to-6.and appeared-sufficient to maintain control of daily work flow.

The inspectors'noted that

'

= the-I&C shop had a minimal work backlog with only a few past due preventive y

maintenance items.

Discussions with the I&C supervisor indicated that the

- present staffing level was adequate, although some overtime was required to

.j meet the discipline work level goals (i.e., 30 work items.per week).

Personnel Training The licensee's training and qualification program had been INPO accredited approximately 2 years prior to this inspection and was well documented.

A

-

two part program was specified by the training program descriptions for each craft.

Position requirements were based upon'the craft discipline, and task i

requirements were based on component-specific training.

The licensee was able to " grandfather" the qualification of nearly all persons at the time of the

,

accreditation by INP0.

The licensee had committed to. complete all position and I

task training requirements for all personnel in 5 years from the date of

'

accreditation.

Only a,small number of new-hires (about two) in each craft had completed all position _and task courses.

Grandfathered technicians in the mechanical group had completed approximately 15 percent of the courses, electricians about 20 percent, and the I&C group about 50 percent.

Thus, many.

of the electrical and mechanical craft relied mostly on their plant experience, rather than formal classroom training, for much of their day-to-day performance.

This had significant implications in several areas of craft performance.

An area of special concern to the team was electrical safety practices in the field._ Clearcces were not routinely set when performing electrical maintenance

!

in many of the MWRs reviewed by the team.

One occasion of observed maintenance j

'

involving exchange of a panel meter resulted in several observations of poor

!

electrical safety practices.

The team learned from staff interviews that

'

electrical safety practices were considered " skill-of-the-trade" knowledge that all personnel should have been aware of.

As discussed in the staffing section, in general, maintenance technicians were hired into NPPD at the " journeyman" level, with approximately 5 or more years experience.

The training courses related to craft safety were, therefore, formulated on the premise that the craft would be safety proficient.

A review of the training curriculum and lesson plans indicated that electrical safety was covered for all craft in the General Employee Training course (GEN 001-04-01),

Industrial Safety,Section V, entitled " Electrical Safety." One page was devoted j

to the subject and addressed the following topics:

L

' *

Critical organ for shock is heart.

p-42-l,

,

'

L.

_

,

- -

.

Y

.

...

t

Moving shock-victims.

-

Grounding hand-held electric power tools.

'

Qualified personnel working on energized equipment should follow the NPPD

-

Safety Rule Book. -

No other electrical, craft-specific safety subjects were provided in the training curriculum (such as use of high-voltage gloves, use of protective rubber. insulating pads, and testing of potentially energized circuitry.before maintenance).

The team also noted that the NPPD safety rule book was vague ~and.

nonspecific.

Discussions with the licensee's staff indicated that such

.

subjects may be presented during monthly safety " stand-up" meetings presented at the craft level.

Craft personnel indicated that many subjects (such as safety and component, courses) required greater detail.

Based on observations of electrical safety implemented in the field, the team

_ perceived that the lack of craft-specific training was a training weakness.

As a result of deficiencies identified in the licensee's programs for control of threaded fastener torquing, thread-engagement, and gasket material application, mechanical maintenance training in these specific subject areas t

was reviewed.

The lesson plans, student text,-and handout material for Lessons INT 020-03-01, " Mechanical Fundamentals - Torquing," Revision 9, and INT 002-02-01, " Mechanical Fundamentals - Fasteners," Revision 2,.were reviewed.

in detail with the-licensee instructors.

These lessons had been delivered to

,

all mechanics within the last year.

The lesson on gaskets and sealants was new

'

and was scheduled for delivery in early 1990.

The content of the training module on-gaskets and-sealants was discussed with the instructors.,The team concluded that although training addressed these subjects, procedural coverage was also required to ensure field implementation.

TehtandQualificationProcess

'

,

Training requirements consisted of performance-standards and measures for each of three different types of' training:

classroom, laboratory,andon-the-job.

Qualification card completion was computerized so that shop supervision could readily. ascertain any individual's qualification status.

The qualifications were monitored by supervision to ensure proper qualification before job assignment and timely completion of outstanding qualification requirements for all individuals.

i-43-

[E

'.

L:, s.

o

s-r t

%;

..

-

4.

EXIT' INTERVIEW-

.The inspectors met with Mr. G. R. Horn-and other members of the licensee's

. staff at the end of this inspection on December 8, 1989. =The inspectors i

summarized the scope of the inspection and briefed the licensee on their-preliminary inspection findings. - The licensee did not identify as proprietary any of the materials provided to, or reviewed by, the inspectors during this inspection.

On December 21, 1989, Mr. J. E. Gagliardo, Mr. G. L. Constable, and

'

_

Mr. J. E. Cummins held'an exit interview with Mr. G. R. Horn and other members of the licensee's staff to discuss the scope and: findings of the' inspection.

Attachment A identifies the. attendees at the exit meeting.

The team contacted other members of the licensee's staff during the inspection to discuss identified issues.

LThe color-coded presentation tree (Attachment C) was used as a visual aid-during the~ exit meeting to depict the results of the inspection, i

i I

.

I j

,,

,

l l

L I

i i

___

,

-w.

'

..

~

j 4 -

)

I

-$

ATTACHMENT A

!

,

EXIT MEETING ATTENDEES

- G. R. Horn, Senior Manager of Operations J. M. Meachum, Division Manager of Nuclear Operations

!

S. M. Peterson, Senior Manager. Technical Support Services E.-M. Mace, Engineering Manager H. T. Hitch,-Plant Services Manager S. C. Woerth, Management Trainee M._E. Unruh, Maintenance Supervisor

'.

G. R. Smith,- Licensing Supervisor

. G. E. Smith, Quality Assurance Manager - Cooper Nuclear Station R. E. Wilbur, Division Manager Nuclear Engineering and Construction G.-A. Trevors, Division Manager Nuclear Support

- V. L. Wolstenholm, Division-Manager Quality Assurance K. C. Walden, Manager Nuclear Licensing and Safety

.

- L.'A. Bray, Regulatory Compliance Specialist R. L. Gardner, Maintenance Manager H. A. Jantzen, Instrumentation and Control Supervisor

.

R. Brungardt, Operations Manager

,

t

.

.{-

r

????%id a

";

' '

'

,

[Wb

.

Les '

o:.

.l

'

[? '

. l:

g*

<~.;

.,

,

L

.,

,

ATTACHMENT B a;

ACRONYMS 1 l

'

ANSI.

American National Standards Institute j

ASC0J American Switch Company

-

CFR Code of Federal Regulations-

CIC..

' Component-Identification Code

.

,

CNS

Cooper Nuclear Station'

'

/

CS

. Core Spray.

.

f

,

L.-

DRP Division of Reactor Projects

,.

EDF Equipment'. Data File.

. EDGE Emergency Diesel Generator (

,

~.

'EQ" Environmentally Qualified;

GL Generic Letter.

'

HP, Health Physics

.

~

h HPCI-HighPrecsureCoolant-Injection

!

^

I&C-Instrumentation and Control.

.

INPO Institute of Nuclear Power Operation i

ce L MOV Motor.0perated Valve

.s o'

.M&TE; Meter and Test Equipment

<;

MTI Maintenance Team Inspection

..

,

v NCRE

_ Maintenance Work Request

'MWR'

,

- Non-conformance' Report i

'

NPPD Nebraska'Public Power. District

.e

.

',

QA

- Quality Assurance

'

.

QC

' Quality Control-

'

RCAR Radiologically Contro11ed' Area v

<

Ri:RL Residual. Heat Removal a

,

50ER Significant Deprating Event Report-

'

', ~

. WIT

' Work Item: Tracking

,

'

Kr

!

,

ah

.

v

e i

_

n

,

o P

._,

,

.--

~

--

v t e Letter Sequence Request
TAC:73989, (Open)
Initiation Request Acceptance... Administration Questions Answers Results Approval... Other: ML20012E833
MONTHYEAR1990-01-29 [Table View]

v t e Inspection Report - Cooper - 1989031
80 \| 81 \| 82 \| 83 \| 84 \| 85 \| 86 \| 87 \| 88 \| 89 \| 90 \| 91 \| 92 \| 93 \| 94 \| 95 \| 96 \| 97 \| 98 \| 99 00 \| 01 \| 02 \| 03 \| 04 \| 05 \| 06 \| 07 \| 08 \| 09 \| 10 \| 11 \| 12 \| 13 \| 14 \| 15 \| 16 \| 17 \| 18 \| 19 \| 20 \| 21 \| 22 \| 23 \| 24 \| 25 \|
1989 Quarterly Integrated 1Q (0) 2Q (0) 3Q (0) 4Q (0) Assessments Mid Cycle End of Cycle 7(0) 8(0) 9(0) 10(0) 11(0) 12(0) 13(0) 14(0) 15(0) 16(0) 17(0) 18(0) 19(0) 20(0) 21(0) 22(0) 23(0) 24(0) 25(0) 26(0) 27(0) 28(0) 29(0) 30(0) 31(0) 32(0) 33(0) 34(0) 35(0) 36(0) 37(0) 38(0) 39(0) 49(0) 90(0) Security Operator Exams Emergency Planning Other

IR 05000298/1989031

Text

Navigation menu

Search