ML16341G003
| ML16341G003 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon  |
| Issue date: | 03/04/1991 |
| From: | Kirsch D, Miller L NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| Shared Package | |
| ML16341G002 | List: |
| References | |
| 50-275-91-04, 50-275-91-4, 50-323-91-04, 50-323-91-4, NUDOCS 9103200125 | |
| Download: ML16341G003 (22) | |
See also: IR 05000275/1991004
Text
U.S.
NUCLEAR REGULATORY COMHISSIOtt
REGION
V .
Report Nos.
Docket Nos.
License
Nos.
Licensee:
50-275/91-04
and 50-323/91-04
50-275
and 50-323
'I
Pacific
Gas
and Electric Company
77 Beale Street
San Francisco, California 94106
Facility Name:
Diablo Canyon Units
1 and
2
Inspection Conducted:
February 11-14,
1991
Inspector:
er,
le ,
peratsons
ect>on
a e
cygne
Approved by:
~Summar:
D.
. Kirsch,
C
e
Reactor Safety Br
ch
ate Signe
Inspection
on February 11-14,
1991 (Report Nos. 50-275/91-04
and 50-523/91-04)
Areas Ins ected:
This was
an announced,
special
inspection to follow-up on the inspection
documented in Inspection Report Nos. 50-275/90-29
and 50-323/90-29
concerning
mechanical
maintenance
measuring
and test equipment
(MH M&TE) issues
at
Diablo Canyon.
Inspection
procedures
30703 and 92700 were used.
Results:
The inspection
made the following general
conclusions:
1.
The licensee's
management
at the site
and corporate offices had not
ensured that the guality Assurance
(gA) and guality Control
(gC)
Departments'ignificant
audit findings relating to the
M&TE program were
substantively followed up and corrected
by the Ma'intenance
Department.
2.
The
gC Department,
and to a lesser degree,
the
gA Department,
were not
.aggressive
in seeking corrective
action for the findings of their audits
of HM M&TE.
q 1 Q3'1QQ l ~>5 9 fQ3Q5
pop
pz)DC/
Q~7QQQ275
f'PR
.
2
3.
The methods
and equipment
used
by the Mechanical
Maintenance
Department
to control
M&TE were still inadequate,
despite
the previous licensee
audits
and
NRC inspection,
(50-275/90-29),
based
on a limited sample of
activities.
These findings appear to represent
a significant safety matter because
they
indicate
a chronic programmatic
weakness
in the control of MM MINUTE, which may
have, or at least
had the potential to adversely
impact installed safety
related equipment.
Further, although .these deficiencies in the control of
NN
NIENTE were identified by the gA and
gC organizations,
these oversight groups,
Mechanical
Maintenance
and
PGIIE management
were ineffective in achieving the
necessary
corrective action.
I'ne
apparent violation (50-275/91-04-02)
was identified:
failure to identify
a breakdown in the program for control of measuring
and test equipment
as
a
nonconformance,
and failure to promptly correct this breakdown.
Unresolved item 50-275/90-29-01
was resolved
by this inspection into violation
50-275/91-04-02,
and is therefore closed.
Three
open items were identified.
i
DETAILS
Persons
Contacted
- J. Townsend,
Vice President,
NPG, Plant Manager
<<M. Barkhuff, guality Control Manager
'T. Bennett, Mechanical
Maintenance
Department
Manager
C. Seward,
Senior
Power Production Engineer
H. O'onnell, Regulatory Compliance Engineer
D. Taggart, Director, Site guality Assurance
J. Strahl, Mechanical
Maintenance
Foreman
<<B. Giffin, Asst. Plant Manager, Maintenance
Services
A. Young, Sr.
gA Supervisor
<<T. Grebel, Regulatory Compliance Supervisor
<<Attended exit.meeting.
The inspectors
also held discussions
with other licensee
and contractor
personnel
during the inspection.
Back round
This inspection
was
a followup inspection to Inspection
Nos. 50-275/90-29
and 50-323/90-29.
The purposes of the latest inspection were threefold:
to review further the existing program for control of measuring
and test
equipment
(HSTE), to review the corrective=actions
which had been taken
for deficiencies identified in earlier versions of that program by the
licensee's
audits
and the previous
NRC inspection,
and to determine whether
enforcement action was appropriate for the unresolved
item identified by
that inspection.
This unresolved
item concerned
what appeared
to be
ineffective corrective action for the
HSTE program weaknesses
previously
identified by the licensee's
audits
and surveillances.
Review of Existin
Pro
ram for Control of MSTE
The inspector
conducted surveillances of work by personnel
in the
Mechanical
Maintenance
(HH) and Instrumentation
and Controls
(ISC)
calibration and tool issuing facilities.
Selected
tool issue logs,
calibration records,
MSTE modules from -the Plant Information Management
System
(PIMS), and personnel
qualification records
were reviewed.
In
addition, tool issuance,
return,
and calibration were observed.
a.
The inspector
observed that calibrated tools were issued
and
calibrated
by the
ISC departnent in a careful
and methodical
~
manner.
Licensee representatives
stated at the beginning of the
inspection that all work performed in the radiologically
controlled area
(RCA) during the refueling outage in progress
would be done using only MSTE issued
by the
ISC department.. This
policy was announced in a memorandum
dated
November 21, 1990,
and
became effective January
14, 1991, shortly after the end of the
previous
NRC inspection.
The inspector determined that because
of this change in policy,
approximately
75K of the calibrated
torque wrenches
were under the
control of the .ISC department,
a higher percentage
than was found
"'
during the previous =inspection.
C
The only potential discrepancy
observed
in the .ISC department control
=-.
program was that calibration personnel
stated that they did not have
an effective method to ensure that issued 'tools were promptly returned
when the job to which the tools had been assigned
had
been completed.
A
b.
The inspector
observed that calibrated tools were issued
and
calibrated
by the
MM department in -a manner which was generally
consistent with the less detailed procedural
guidance
required
by
the licensee for this department.
However, the following
significant discrepancies
were identified:
(1)
On February 12, 1991, the inspector
observed that Tension
Dynamometer
8157
had
been
issued for Mork Order C0078894-01
on January
23,
1991,
a work order which was completed
on
January
29, 1991.
This was not
a safety related job, but was
associated
with safely moving a cask
used for transportation
of radioactive material.
At the inspector's
request,
the
tool was located.
The personnel
using it stated that it had
been in use to check chainfalls for some days,
a different
job than the one for which it,had been
issued.
The licensee
personnel
were not aware of this earlier job.
The inspector
noted that this was
a current
example of a finding
identified in gC Surveillance
Report
PCS 90-0030,
dated April 12,
1990,
and
a similar one re-identified in guality Assurance Audit
90812T, dated
September
6, 1990.
The surveillance report
had
stated= that
28% of the
MSTE usage
sampled
was not recorded
as
required
by guality Assurance
Procedure
(gAP) 12.A, Control
and
Calibration of Measuring
and Test Equipment,
Sections
4.25 and
4.26, Revision 19.
The
gA audit stated that 35K of work orders
reviewed did not contain adequate
MSTE'escriptions,
as required
by AP C-40S3,
Revision 12, Attachment 6.3, Instructions for
Completing
and handling W/0's with PIMS on Line.
Finally,
MM M&TE personnel
stated that personnel
frequently did
not return
MSTE once the job for'hich it had been issued
was
completed,
and that testing personnel
did not have
an effective
.
method to ensure that the equipmei t was returned.
Based
on this
limited sample of one job which was checked,
and the comments
from both
MM M&TE and
the inspector concluded that
the licensee's
controls to ensure that
MSTE was traceable
to the
jobs where it w'as
used
were still ineffective.
This is an open
i tem (50-275/91-04) .
On February
12 and 14,
1991', the inspector requested
that eight
torque wrenches available for issue in the
HH calibrated tool
issue
room be checked at one point of their useable
range for
calibration.
Licensee
personnel
performed
a check on each of.
these
wrenches.
Two of the eight (25%),8 393 and 8381, were
found out of calibration.
Torque'wrench
8393 indicated
240
in-lbs at an actual
value of 218 in-lbs', while torque wrench
1381 indicated
125 in-lbs at an actual
value of 145 in-lbs.
Both wrenches
were indicated by their calibration records to
be in calibration.
The tolerance for torque wrenches
to be
considered. in calibration was four percent of their setpoint
per Step 7.2.7 of Procedure
HP M-53.1.
Licensee
personnel
could not explain this finding.
They
indicated that typically one out of 100 times they would
find a similar discrepancy.
The discrepant
torque wrenches
were
removed from service fot calibration.
The inspector noted that Maintenance
Procedure
H-53.1,
Revision 5, Section 7.4 required. that torque wrenches
be
verified before
and after their use at the setting, or over
the range, at which they were to be used.
The inspector noted
that it was possible that
a torque wrench could be in calibration
at
some points
and out of calibration at others.
In this case,.
a simple one point calibration check such
as that performed at
the inspector's
request
would not necessarily
indicate
a
previous verification error had been
made.
However, this.
relatively high percentage
of discrepancies
suggested
a significant percentage
of verification errors.
The inspector concluded that the verification program for HM
torque wrenches
was potentially failing to ensure that only
calibrated tools were in use.
This is an open item
(50-275/91-04-03).
The inspector
observed
the
MM MSTE personnel
perform
several verification checks of audible indicating (snap-
type) type torque wrenches,
also
known as "click type"
torque wrenches.
These verifications were performed
on
different models of the Milliams Torque Mrench Tester.
Steps
7.2.2
and 7.3. 1 of procedure
h)P M-53.1, referenced
in the
previous paragrapli,
required
ihe calibration of torque
wrenches
in accordance
with the applicable Technical Bulletin.
These
personnel
were not aware of the prominent Note in
Technical Bul'letin No. TB-129, the applicable j'echnical
Bulletin for this activity, which described
the proper
use of this equipment:
"Extreme care must be exercised
when checking audible
indicating (snap-type)
wrenches
so the operator
does
- not pull beyond the "break
away torque."
The inspector
observed
that
MM MSTE personnel
were
una
of this precaution,
and routinely did not use extreme care,
unaware
,'onsequently,
for smaller torque wrenches,
measuring
in
in-lbs, these
personnel
routinely pulled the wrench
significantly beyond the break
away torque.
For example,
for a wrench set to.break
away at 100 in-lbs, the Milliams
Tester typically indicated
a peak torque value of 130-140
in-lbs had been attained during the test.
For larger torque
wrenches,
the
same effect was not noted.
For example, for
a wrench set to break
away at 100 ft-lbs, a peak torque value
of 102 ft-lbs was typically attained.
The person performing
the test attempted
to estimate
the value of the release tor u
t the process
was inherently imprecise given the relative
e
orque,
rapidity with which the wrench passed
through the break
away
.val ue.
The inspector noted that field personnel
could reasonably
be
expected
to be even less meticulous
than the testing
personnel
who routinely exceeded
the setpoint torque values
unintentionally.
The inspector concluded that this testing
had demonstrated
that use of snap-type
torque wrenches
in
the in-lb ranges
could result in significant overtorquing of
fasteners
in those
ranges.
A related,
but broader
conclusion
was independently
reached
by the licensee's
Technical
and Ecological Services
(TES) Division Report
420DC9176,
(January
14,
1991) issued in response
to Action
Request
A0183483
(March 17, 1990).
That report made
one
observation
which was relevant to this discussion:
"Click" type wrenches
may have
a second
Ltorque3 peak
that can significantly over torque
a bolt."
The inspector
determined that the IhC NTE program
had
'ecently
greatly reduced
the number of snap-type
wrenches available for use,
whereas
the
NN MATE program
had
not addressed
the concern.
To the contrary,
on February
14,
1991, the inspector
was advised
by licensee testing personnel
that workman were requesting
snap-type
torque wrenches for
use inside the
RCA from the
MM NTE calibrated tool issue
room, despite
the licensee's
decision discussed
in Section 3.A
above to use only I&C calibrate'd tools in the
RCA beginnin
January
14,
1991.
eginning
'I
The inspector
concluded that, for small torque wrenches,
the
licensee's
program to control torquing of fasteners
had not
been nearly as precise
as required
by HP M-53.1.
The
inspector
noted that the effect of the unknown errors
produced
should
be considered
by the licensee for systematic
evaluation.
(Open Item 50-275/91-04-01)
~
~
4.
Review of Previous Oualit
Assurance
Audits and gualit
Control
urve)
ances,e
ate
to
1
ro ram
ea
nesses
nreso
ve
Item
-
1
The inspector
reviewe'd the status of corrective action for the
surveillances,
audits
and the inspection which preceded this inspection
and which were related to Of MSTE.
Inspection
Report
Nos. 50-275/90-29
and 50-323/90-29 previously detailed
much of this chronology.
It is
summarized
here,
combined with additional information determined during
this inspection.
Four licensee
reports
from December 5, 1989 through September
6, 1990
identified repeatedly that the
NH METE program had significant weaknesses.
These reports
w'ere.:
a.
ualit
Control Surveillance
89-175 dated
December 5,
1989
This report concluded that "a significant problem exists with
fMN] MSTE traceability to specific work activities."
An
expanded surveillance
was recommended,
but no definitive
corrective action which addressed
correcting this problem
was indicated
by the surveillance.
b.
gualit
Control Surveillance
90-030 dated
A ril 12,
1990
This surveillance identified a variety of programmatic
problems.
These included:
"chronic omission of data,
data errors,
missing signatures,
.
.
. logs not maintained;
incomplete
and
no history searches/evaluation
for out-of-
tolerance
MSTE, history searches
ARs exceeding
procedural
time limits;
equipment
usage
not being consistently recorded,
jeopardizing
the accuracy of the calibration data baseline;
"as found" data not recorded
and in some instances
photocopied
and uses (sic) for multiple tools."
inadequate
training and qualification of caIibrators;
extrapolation of calibration data which is not permitted;
use of standards
with less
than the Lprocedurally] required
accuracy;"
0
6
This expanded
surveillance of MM M8TE concluded that:
"Unsatisfactory performance of tool calibration and issue
activities
was primarily due to unclear or incomplete
procedures
and poor practices
by maintenance
personnel.
Many of the program administrator's
corrective actions
during the last year only corrected
the symptoms of the
problems
and not their causes
thus allowing problems to recur..
. . The overall effectiveness
of the Mechanical
Maintenance
calibration program is unsatisfactory."
A follow-up surveillance
was
recommended
by gC to determine whether
the corrective actions identified and scheduled
by the
HM department
were effective.
In discussions
with the inspector,
the
gC Manager stated that
he
had recomnended
to the Mechanical
Maintenance
Department
Manager
that
a nonconformance
report
(NCR) be issued,
but that manager
considered
that an
NCR was inappropriate
because
the surveillance
findings were not significant enough,
and because
that manager
thought that
some of the surveillance findings were invalid.
At the time of the previous inspection,
which began
on November 27,
1990,
some of the Action Requests
initiated by RC for specific findings
of this surveillance
had still not been addressed
by the Mechanical
Maintenance
department
(e.g.
AR A0184108, dealing with rusted standards,
and
AR A0183542, dealing with widespread
r ecordkeeping errors).
1/here
the findings had been addressed,
they were addressed
in piecemeal
fashion
by lower level per'sonnel,
using
comment fields on Action Requests,
rather
than in a coordinated
way which clearly demonstrated
management
oversight
and approval of the response.
Most significantly, licensee
personnel
could not provide any documentation
which addressed
the five
overall conclusions
and seven
recommendations
of this surveillance.
During this inspection, this documentation
was still not available.
The inspector concluded that these
conclusions
and recommendations
had
not been formally addressed.
ualit
Performance
and Assessment
Branch Surveillance
90-126,
dated
a
j
The Site guality Assurance
Manager stated that this surveillance
was
initiated by guality Assurance
because
of concerns .regarding the
validity of gC Surveillance
90-30.
This surveillance
was
a review of
previous surveillances,
audits, corrective action requests
(ARs), and
guality Evaluations
(gEs) related to the program for HM NTE.
This
surveillance clearly identified that
some of the findings identified
by gC were repetitive,
and
recommended
a gA audit of the area.
This surveillance mentioned that
MM MSTE deficiencies
had been
identified as early as
1985.
The inspector confirmed that guality
Assurance Audit 85230P,, dated October 17, 1985,
had identified
failures to properly document
HATE usage.
These
problems
were
documented
to have
been corrected
by procedural
revision and
personnel
training.
The inspector
observed that surveillance
90-126 did not recommend
-'n
NCR for the recurrence'f
MSTE program weaknesses.
d.
ualit
Assurance
Audit 90812T, dated
Se tember 6, 1990
This audit independently
reassessed
the
MSTE programs.
including
MM MSTE, five months after the
MN program deficiencies
were identified
by gC.
Ho comment
was
made regarding the existence of programs with
different procedural
requirements for use of HSTE.
The audit did
reconfirm several of the deficiencies identified in the
gC
Surveillance
90-30.
Generally,. it was less clearly written, and
made
no general
conclusions or recormendations
regarding the
HM MSTE
program.
The audit also identified recurrent deficiencies related to those
identified in 1987 by ACR OCO-8?-gA-NOOl.
That 1987
NCR referred to
the discovery that the calibration accuracy ratios
between calibration
standards
and
MSTE had not, in all cases,
been determined
and documented.
This 1990 audit found recurrent failures to control this ratio,
as
had
gC Surveillance 90-30.
Most of the problems identified by Audit 90812T were limited to
technical
issues.,
The most significant findings of this audit
appear to have
been:
Usage information for HSTE was not recorded
as required
on 35K
of 20 work orders
reviewed.
This was
a repeat finding from gC
Survei'llance
90-30.
The procedurally required accuracy ratios
between calibration
standard
and
HSTE of 4:1 were not attained for several different
types of equipment.
This was
a repeat finding from gC
Surveillance
90-30.
HN did not maintain vendor manuals
for. most of the equipment
they were responsible for calibrating; therefore,
tool
room personnel
could not describe
or refer to recognized
practices
and methods of calibration.
This particular
discrepancy,
as noted in paragraph
3.b.3 above,
was still
evident during this inspection in that tool room personnel
were not familiar with the precaution in the vendor manual for
the use of click type torque wrenches.
This audit did not state clearly any conclusions
regarding the
adequacy of the
NN MSTE program,
nor did it make any clear
recommendations
to management,
unlike gC surveillance
90-30.
However, unlike the
gC surveillance,
which only documented
the
specific findings as Action Requests
to the
HN department,
this
audit did issue
several
Audit Finding Reports
(AFRs), which required
the
NM department
to respond with a root cause
determination for the
problems.
0
Action Requests
were the licensee's
working level request for
information by the requestor of the affected party, in this case,
the
MN department.
At the request of the
MM department,
the due
date fop these detailed corrective actions
was extended..
At the
time of the previous
NRC inspe'ction,
no action
had been taken
on
these findings.
This audit did not clearly identify any nonconformance
or programmatic
breakdown,
nor did it refute
gC Surveillance
90-30.
Ho corrective
action for its findings had
been taken prior to the
NRC inspection
which commenced
on November 27,
1990.
5.
Conclusions
Re ardin
Licensee Corrective Action for MM M&TE Pro ram
e ic>encses
Prior to Ins ection 50-2
9 - 9
The previous
NRC inspection
(50-275/90-29) identified that
a
nonconformance
report had not been initiated to address
the recurrent
failures to follow the established
MM M&TE program.
That inspection
focused
on the statements
in Audit 90812T that
MM had not fully
implemented the corrective actions required
by NCR DCO-G7-gA-HOOl.
As discussed
in paragraph
4d above, that NCR's findings were narrowly
focused
on calibration ratio discrepancies.
The failure to initiate
a
nonconformance
was identified in the previous report as unresolved
item
90-275/90-29-01.
The previous inspection also identified several
examples,
in addition to those discussed
in this report, of specific quality problems
in the
MM M&TE program.,
After review of the material
discussed
above, discussion with licensee
personnel
and managers,
and the observations
discussed
in Paragraph
3,
the inspector concluded that the licensee's
corrective action in response
to the many formal reports of a deficient
MN M&TE program
had been inadequate.
guality Control Surveillance
90-30, in particular, clearly reported
a
significant condition adverse
to quality, namely numerous
examples of
failures to implement the
MM M&TE program procedural
requirements.
The
licensee's
own surveillance
repor t characterized
the
MM M&TE program
as
"unsatisfactory"
and "not acceptable."
Senior licensee
management
was provided with the report of surveillance
90-30,
as
was guality Assurance.
Yet, at. the time of the previous
inspection,
over seven
months later,
a nonconformance
report
had not been
issued for these
programmatic discrepancies.
As
a direct result, the
cause for the weaknesses
had not been determineo,
and corrective actions
to restore
program quality were not defined.
Rather,
an additional audit
and
an additional surveillance
were performed which added additional
examples of specific problems,
but did not integrate
the findings further.
These
subsequent
efforts reduced
the clarity of the
gC surveillance's
conclusions
and recommendations,
and postponed
and diluted effective
corrective action to correct the overall problem.
The inspector,
therefore,
concluded that the scope
and breadth of thes
failures
was substanti'al
enough to indicate
a breakdown in the guality
ese
Assurance
program for the calibration and control of measuring
and test
equipment.
This is an apparent violation of the requirements
of
10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," and guality
Assurance
Procedure
15.B, "Nonconformances,"
Paragraph
2.1.1
and 3.1, in
- that
a nonconformance
report for this breakdown
was not initiated until
after the previous
NRC inspection
again brought this problem to senior
management's
attention
(Enforcement
Item 50-275/91-04-02).
Unresolved
Item 50-275/90-29-01,
which directly referenced
the calibration
ratio discrepancies
NCR, is correspondingly
incorporated
as part of this
apparent violation, and is closed.
As noted abbve, related
issues of potential overtorquing of small
fasteners
due to use of click-type torque wrenches
(Open Item
50-275/91-04-01),
use of uncalibrated
torque wrenches
(Open Item
50-275/91-04-03),
and undocumented
use of calibrated tools
(Open Item 50-275/91-04-04)
were identified.
These
items are
considered
integral parts of the
MM HSTE program breakdown.
They
will be followed up during followup on the .Enforcement
Item.
Review of Licensee Corrective Action Since the Previous
Ins ection
As imnediate corrective action for the findings of Inspection 90-
29/90-29,
the licensee
issued
nonconformance
report
NCR DCO-90-
MN-N089 on December
21, 1990.
The nonconformance
was described
as:
"Previous
gC and
gA audit findings indicated
a significant
number of program implementation deficiencies.
These deficiencies,
and the time involved responding to the deficiencies warrant
further evaluation."
At the conclusion of the latest inspection, this
NCR was still under
revision to determine
the appropriate corrective actions.
Most of the
proposed corrective actions from the most recent
gA audit, 90812T,
had
been delayed
pending the development of a unified corrective action plan
from this
NCR.
Unresolved
Item
An unresolved
item is
a matter about which more information is reauired
to ascertain
whether it is an acceptable
item,
a deviation, or a violation.
Exit Interview
The inspector
met with licensee
management
denoted in Paragraph
1 on
'February 14,
1991.
The scope
and detailed findings of the inspection
were discussed.
Licensee representatives
acknowledged
the findings of the
inspection.
Subsequently,
on February 25,
1991, licensee
representatives
were informed that
an Enforcement
Conference
on the results of this
inspection
would be conducted
on March 8, 1991.
n
r
4k