IR 05000289/1979010
| ML19253C667 | |
| Person / Time | |
|---|---|
| Site: | Crane  |
| Issue date: | 08/28/1979 |
| From: | Robert Carlson, Devlin J, Ebneter S NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML19253C650 | List: |
| References | |
| 50-289-79-10, NUDOCS 7912060664 | |
| Download: ML19253C667 (27) | |
Text
.
.
U.S. NUCLEAR REGULATORY COMMISSION OFFICE OF INSPECTION AND ENFORCEMENT Region I Report No. 50-289/79-10 Docket No. 50-289 License No. DPR-50 Priority:
Category:
C
--
Licensee:
Metropolitan Edison Company Post Office Box 542 Reading, Pennsylvania 19640 Facility Name:
Three Mile Island Nuclear Station, Unit 1 Investigation at:
Middletown, Pa.; Danbury, Conn.; and Reading, Pa.
Investigation conducted:
April 19, 20, 25, 26, 27, May 2, 7, 8, 14, 15, 16, 17, 31, 1979; and June 12, 1979 Investigators)
N M1hv vibf/9f uS. D. Ebnetery h1Af, Engineering d4te s'igned Support Secti a.'b.
2, RC&ES Branch
/'
/ i N $hhvU hiumbi;/N7f
.-.
,@. W. Devlin, Cpief,/ Security &
qate signed Investigation Section, Safeguards Branch
%b <w hRah7,I477
.
E.JtrniganQReactorInspector ddte signed Approved by:
/k 4%
//gn X$/'/
(vR. T. Carlson,/.4Qfyf, Reactor
,dyte sign ~ed Construction & Engineering Support Branch Investigation Summary:
Investigation Conducted on April 19, 20, 25-27, May 2, 7, 8, 14-17, 31, June 12, 1979 (Report No. 50-289/79-10)
Areas Investigated:
An investigation conducted by three regional-based in-vestigators of activities and documentation related to the inservice inspec-tion (ISI) performed at Unit 1 from February 20 through March 10, 1979.
The investigators reviewed ISI procedures, examination data records, health physics 1506 136 7912060 hb
.
.
.
.
records, time logs and quality assurance records.
The investigators inter-viewed Metropolitan Edison personnel, Hartford Steam Boiler Inspection and Insurance Company personnel and personnel employed by Nuclear Energy Services, Inc.
Investigation involved 218 hours0.00252 days <br />0.0606 hours <br />3.604497e-4 weeks <br />8.2949e-5 months <br /> at the site, licensee corporate offices and contractor facilities.
Results:
Of the five areas investigated, no items of noncompliance were identified in four of the areas.
Within the scope of the investigation, one apparent item of noncompliance was identified in one area (Infraction - inad-equate corrective action 50-289/79-10-01).
1506 137
.
.
I.
Backgroup A.
Reason for Investigation On April 18, 1979 IE:HQ notifieo Region I by telephone that several newspapers carried an article written by a Los /.ngeles Times cor-respondent which specified " Irregularities have been found in the X-ray inspections of some of the welds in the containment vessel at the Three Mile Island nuclear power plant's sister reactor, the Los Angeles Times has learned."
1E:HQ obtained a copy of the newspaper article as published in the Washington Post and sent a copy to Region I via facsimile.
Region I inspectors, on-site. n"itacted Metropolitan Edison personnel and a preliminary review indicated that there was some validity to the newspaper report even though it appeared that there were some technical discrepancies in the article.
Region I then initiated an investiga-tion.
The purpose of the investigation was to; determine the validity of the facts as reported in the newspaper a.
article; b.
define the causes of any verified discrepancies in the ISI data; and c.
make an assessment of generic aspects.
B.
Identification of Involved Organizations 1.
Metropolitan Edison Company (Met-Ed) P. O. Box 542 Reading, Pennsylvania 19603 A utility company licensed by the Nuclear Regulatory Commission to operate the Three Mile Island Unit 1 nuclear power plant.
2.
Nuclear Energy Services, Inc. (NES), NES Division Shelter Rock Road, Dar. bury, Connecticut C6810 A supplier of nondestructive examination (NDE) services to the nuclear industry.
1506 138
.
.
-
.
3.
Hartford Steam Boiler Inspection and Insurance Company, Valley Forge, Pennsylvania 19481 Provider of Authorized Inspection Agency Services required by the ASME Boiler and Pressure Vessel Code.
II.
Summary of Findings A.
Investigation Findings An investigation of the Inservice Inspection (ISI) during February-March 1979 by Nuclear Energy Services (NES) for the Three Mile Island Unit 1 components and piping revealed a number of inc.iequacies.
1.
The newspaper article was partially correct.
A major error was that it reported radiography (X-ray) inspections.
No X-ray inspecticn was done.
2.
Metropolitan Edison Quality control had identified and docu-mented the ISI data problem prior to the newspaper article.
3.
Quality assurance activities to provide corrective actions to preclude recurrence of previously identified deficient conditions were inadequate.
4.
The ISI program was apparently not administered adequately by either the licensee or the ISI contractor.
5.
There were numerous examples of nondestructive examiners'
failure to follow procedures.
6.
Several instances of improper certification of calibration / examination times were identified.
7.
The ISI team did not receive adequate indoctrination / orientation in relation to the specific procedures.o be utilized for NDE at TMI.
8.
There were some indications that the licensee radiation protec-tion program may be inadequate.
1506 139
.
.
B.
Conclusion There were numerous errors, inaccuracies and inconsistencies in the ultrasonic (UT) nondestructive examination data sheets.
However, there was no pattern that would clearly indicate a systematic effort to falsify data.
There was one set of data in which there was a clear conflict between recorded on-site times and calibration data sheet times.
The examiner stated he had performed the examinations and readily admitted that the times were wrong.
Supporting documentation provided by the licensee identified the code inspector as witnessing the exam in question.
Other conflicting entries in records were explained as deviations from procedures, or in some cases, records were not retrievable which perhaps would clearly substantiate whether or not falsification of records had occurred.
However, confidence in the suitability of the data for decision making is very low and none of the ultrasonic test data can be accepted for credit toward meeting ASME Section XI requirements.
.
1506 140
.
.
DETAILS A.
Persons Contacted 1.
Metropolitan Edison Company (Met-Ed)
J. Seelinger, Superintender +, Unit 1 W. Potts, Superintendent, technical Support W. Ream, ISI Coordinator J. Potter, QC Engineer T. Mackey, Manager, Site Quality Control G. Troffer, Quality Assurance Manager R. N. Prabhakar, Supervisor, Generation QA Engineering P. A. Levine, lead Audit Engineer T. F. Barbagallo, Lead QA Procurement Engineer 2.
Hartford Steam Boiler Inspection and Insurance Company (HSBIIC)
D. B. Eggleston, Regional Manager W. Rodgers, Authorized Nuclear Inspector (ANI)
3.
Nuclear Energy Services, Inc.
(NES)
William J. Manion, President, NES Division George T. Hamilton, Vice President, NES Division Karl F. Schmidt, Jr., Manager, Inservice Inspection George Oberndorfer, Quality Assurance Manager Frank Carr, Manager, Field Operations NES Nondestructive Examiners B.
Chronology of Events Related to the Investigation A brief chronology of events related to the investigation is as follows:
February 8, 1979 NES examination procedures arrived at TMI February 12, 1979 Outage started February 15, 1979 NRC Inspection of ISI February 20, 1979 NES examiners on site
.
February 20 -
March 10, 1979 ISI examinations performed 1506 141 J
.
.
February 21-23, 1979 NRC ISI inspection February 24, 1979 ANI met with Met-Ed to resolve communi-cations problen.
March 8, 1979 Meeting with ANI, NES, and Met-Ed 29 Exams unacceptable March 10, 1979 NES Examiners left site March 19, 1979 Met-Ed QC wrote NCR 79-18 ANI noted in log possibility of data problems March 22, 1979 ANI, HSBIIC Regional Manager and Met-ED met to discuss concerns about data April 13, 1979 Met-Ed Internal memo defining data problems April 18, 1979 Newspaper article in " Washington Post" NRC Region 1 initiates investigation April 19-20, 1979 NRC Inspector on-site preliminary data review April 25-27, 1979 NRC Inspectors on-site Review program, records, procedures April 27, 1979 Meeting TMI - NRC, NES, Met-Ed.
NES discuss problem of data-April 30, 1979 NRC Inspectors met with Hartford Steam Boiler Inspector and Insurance Company Regional Manager May 2, 1979 Meeting at TMI-2 NES presented preliminary results of NES investi-gation and interviews May 7, 1979 NRC meeting with Authorized Nuclear Inspector and Regional Manager at TMI May 8, 1979 Meeting Discussions with Met-Ed and data review May 14, 1979 Meeting at NES NRC and NES management May 15-17, 1979 NRC Interviewed NES examiners 1506 142
.
May 31, 1979 Meet at Met-Ed Corporate Headquarters Discuss QA and Review records related to ISI June 12, 1979 Exit interview at TMI C.
Metropolitan-Edison 1.
Inservice Inspection Metropolitan Edison (Met-Ed) has committed to an inservice inspec-tion program which meets applicable NRC and ASME Section XI require-ments.
Specific requirements are detailed in Section 4.4 of the FSAR, Technical Specifications and the Operational Quality Assurance Program.
To implement the commitments and requirements, Met-Ed has established an ISI program wnich is described in the TMI Unit One Inservice Inspection Administrative Plan.
This is supplemented by TMI Surveill-ance Test No. 1300-1, Inservice Inspection Program which defines the specific examinations by system weld number to be performed during an outage.
Detailed nondestructive examination (NDE) procedures wer] prepared by NES Level III examiners.
These basic NDE procedures are converted to TMI-1 Surveillance procedures by the addition of clarifying instructions and an official approval sheet.
As an example, NES procedure NIP 007-M, Manual Ultrasonic Examination Procedures for Pressurizer Vessel Welds was converted to TMI-1 Surveillance Procedure 1300-1J.
Met-Ed has contracted with Nuclear Energy Services, Inc. (NES) to provide examination services, procedure development, and technical advice in the ISI area.
NES provides the specified number of certi-fied NDE personnel qualified in accordance with SNT-TC-1A in each applicable discipline.
Examinations of designated welds and components, data recording, and evaluation of indications are performed by NES examiners.
Met-Ed controls the ISI through an ISI coordinator who is the primary interface between NES and Met-Ed.
He provides the direction to the on-site NES supervisor and provides support functions.
Met-Ed on-site quality control acts as an independent reviewer of the ISI data and is responsible for providing surveillance over ISI activities.
The Hartford Steam and Boiler Inspection and insurance Company (HSBIIC) is the authorized inspection agency at the TMI-1.
The Authorized Nuclear Inspector (ANI) monitors the ISI activities and certifies that the examinations are conducted in compliance with applicable ASME Section XI requirements.
1506 143
.
.
2.
ISI Records The ISI records in question were generated as a result of visual (VT), penetrant (PT), and ultrasonic (UT) examinations performed by NES examiners between the dates of February 20, 1979 and March 10, 1979.
The data sheets were prepared by Level II or Level III exam-iners and approved by a Level III examiner.
To be acceptable for credit toward Section XI requirements the data sheets must be accepted by the ANI.
In addition, Met-Ed quality control personnel review and approve the data sheets.
In each of the nondestructive examination procedures prepared by NES there are instructions related to the preparation of calibration data sheets and recording of examination data.
In general, the procedures require the recording of calibration settings, initial calibration time, time of recalibration checks and the final cali-bration time.
In addition, the examiner that performs the exam is required to sign and date the data sheet immediately upon completion of the exam.
The NES ISI Program Quality Assurance Manual lists procedure 25-DC-001, Inservice Inspection Data Control System, as an implementing procedure for control of data generated during an ISI.
This is a comprehensive procedure for maintaining control of the calibration / examination data sheets and requires review of each sheet by a Data Controller and the supervisor.
Paragraph VI.F of the ISI Administrative Plan specifies that Met-Ed QC personnel perform or monitor the performance of the inspections as required by the Supervisor-QC.
The approval signature of the Supervisor-QC is required before an examination is considered complete.
The authorized code inspector reviews each data sheet and upon acceptance, stamps and initials the data sheet.
The Met-Ed QC review identified numerous problems / discrepancies with the ISI data sheets.
These were documented on NCR 79-18 dated March 19, 1979.
Specific problems included NES failure to follow procedure, use of wrong calibration blocks, improper distance amplitude curves (DACs) and data inaccuracies.
Subsequent to this, Met-Ed quality control prepared an internal memorandum addressed to Met-Ed quality assurance which summarized their position in regard to the ISI data.
This memorandum stated that the validity of some examination data sheets was in question and that the Authorized Nuclear Inspector (ANI) refused to attest to 29 data sheets.
It further stated that several data sheets indicated inspections being performed during a period of time that south gate records indicate the examiner was not on site.
1506 144
.
Some of the details that were reported in the newspaper article were substantiated by this memo; i.e., the number of welds (63), irregular-ities in routine inspection reports, and the inspections were per-formed by NES during the refueling outage in February and March 1979.
The investigators reviewed a substantial portion of the examination data sheets and verified that there were numerous examples of failure to follow procedures, errors and inconsistencies in the recorded data.
As noted previously, the Met-Ed QC department had identified and documented these deficiencies.
Data control and review apparently presented a major problem based on the review of data sheets and interviews with personnel of Met-Ed, NES and HSBIIC.
In interviews with NES personnel it was revealed that:
the data control procedure, 25-DC-001 was never implemented during the TMI ISI; there was no orientation of NES personnel at the start of the ISI; demands for prompt submittal of data sheets by Met-Ed led to bypassing of the NES supervisor; NES examiners frequently re-copied data sheets which led to transpositional errors, mixup in signatures and errors in dates and times; NES examiners failed to follow procedural requirements in recording calibration times.
Calibration and examination times were not recorded immediately upon completion of the exam but were left unrecorded until the end of the day; and NES examiners failed to observe procedural requirements in correcting data entries.
Some exam sheets were not voided when new sheets were prepared.
There apparently were other problems with the sequence of data reviews.
For example, during the early phase of the ISI, the ANI reviewed the data sheets before Met-Ed QC.
However, Met-Ed QC rejected come data sheets after they had been endorsed by the ANI.
This hd to a change in the review sequence whereby Met-Ed QC reviewed data sheets before the ANI.
1506 145
.
3.
Health Physics (HP) Records Several HP records were used by the investigators to try to verify that the times recorded on the examination records were valid and radiation exposure records were reviewed in an attempt to establish a correlation with work reportedly done.
All of the records were not available and the HP records themselves contained errors, incon-sistencies and omissions.
Radiation work permits (RWPs) utilized were 21702, 21749, 21509, 21502, 21420, 21592, 21528, and 21591.
In addition, hard copy HP records of containment entry times and exposures were used when available.
Records for only three of the examiners were retrievable, the records for the remainder of the ISI crew were either lost or misplaced.
The ISI crew was essentially divided into two teams, each team consisting of a Level I and a Level II examiner which is consistent with NES procedural requirements.
An additional team consisted of a Level I and a Level III examiner, the latter of which also served as NES supervisor.
For ultrasonic examinations, the Level I examiner did the actual scanning of the weld area and the Level II observed the instrument display.
This means that there is a significant physical difference (up to 6 feet or more) between the team members and the radiation field source.
In addition there may be other physical barriers between the two examiners.
Therefore, it is expected that there would be some differences in radiation exposure even for individuals on the same team.
Some of the welds examined were outside containment and in significantly lower radiation fields so it would be expected that the team performing these exams might receive a lesser exposure.
Some individuals did not perform specific exams as a team but rather as an individual and therefore there would not be a direct correlation of cumulative exposures for team members.
Data from the RWPs appears to be consistent and correlates in most instances with welds examined.
For example, RWP 21702, the team members of one team received 200 and 210 mr and the members of the other team each received 30 mr.
On RWP 21420 the team members of one team each received 25 mr and the members of the other team received 60 mr and 45 mr.
15015 146
.
.
The two examiners who received tne least cumulative exposure per-formed most of the examinations on welds outside of containment.
In addition, during interviews, both expressed more concern about exposures and had significantly more knowledge about radiation protection.
The concern about exposure is also a contributing factor in failure to completely fill out data sheets when an exam was completed.
As one examiner pointed out
"get in and get out".
The RWP data should have been recorded each time a containment entry was made which would allow a correlation of the., umber and times of containment entries with times recorded on exam data sheets.
However, exact entry and exit times were not always recorded and apparently some of the exposures were not recorded.
The examiners, in interviews, related that frequently they would bypass HP and would read and record their own dosimeter readings.
Two of the examiners in interviews made specific comments related to the apparent inadequacies of the radiation protection program at TMI.
In summary, the comments indicated that HP personnel were not always available at their designated stations, surveys were not always made in applicable RWP areas, and containment entry and exit was not adequately controlled.
4.
Site Sign-in Records Site security required that all contractor employees enter and exit by the TMI south gate during the time period in question.
TMI Security provided list of site entry and exit times by individuals for the NES examiners.
This provided the most accurate and consistent set of times for correlation with examination data sheet times.
5.
Falsification of Records The possibility of falsification of records was noted in the ANI record dated March 19, 1979 and in the Met-Ed internal memorandum of April 13, 1979.
This was based on observations by persons associated with the ISI activities and the inconsistencies in the ISI examination data sheets.
A specific question directed to the six NES examiners, the ANI and two of the Met-Ed personnel who had direct interface with the ISI data was:
Do you have any specific knowledge of any falsification of the data, or do you want to make any allegations of falsification of the data? All of the responses were negative but the Met-Ed personnel and the ANI responses indicated that they had little confidence in the data.
NES management, as the result of their investigation, felt that all of the examinatior.;, % d been performed and that no deliberate intent to deceive had occurred.
1506 147
.
The record review performed by the investigators verified that some of the inconsistencies indicated that the records were not valid.
However, there did not appear to be any pattern to the inconsistencies, rather they occurred in a random fashion.
A significant discrepancy of recorded times in relation to calibra-tion / examination occurred on March 8, 1979.
One of the teams recorded on data sheets that calibrations were performed at 11:15 a.m. and 12:15 p.m.
However, in direct conflict with this, the TMI south gate entry-exit logs showed that the team members were not on site between 11:35 a.m. and 12:15 p.m. and the containment logs (from HP records) show that the team left containment at 11:00 a.m. and re-entered at 1:00 p.m.
During interviews, the NES examiners ex-plained that the data sheet times were estimated and entered at the end of the day.
The NES management investigation revealed essen-tially the same results as obtained by NRC interviews.
For the crew in question, on March 8, 1979 there were 13 initial calibration times and corresponding final calibration times.
Based on estimated times to do calibrations and examinations the team filled out the data sheets and found that there were several calibration times remaining to be entered.
The only time period left in which exams could be recorded was the time between 11:00 a.m. and 1:15 p.m.
If the other time periods were selected, then there would be a conflict of time period for examinations.
So, the examiners entered the wrong times on the data sheet.
The examiners stated that all of the examinations had been performed.
Documentation provided by Met-Ed indicates that the weld examinations listed as being performed during the time period in question had been performed and had been witnessed by the ANI.
Thus, even though the wrong times were recorded, the examinations apparently were performed.
Hcwever, the NES procedural requirements which require that examination data be recorded upon completion of the exam were not followed.
6.
Quality Assurance (QA)
The Met-Ed QA program applicable to TMI Unit 1 is described in Appendix 1A of the FSAR.
This has been revised and the current program is designated as Revision 7, Change 3.
The scope of this investigation required that only those aspects of QA relative to the ISI program be investigated which includes vendor selection, ISI audits, and surveillance.
The Quality Assurance Program,Section VIII.1, Vendor Selection, delineates responsibilities related to vendor evaluation and selec-tion.
It spt::ities that vendor evaluations are conducted by a team and that the depth of evaluation depends on complexity and importance to safety.
If the evaluation of a vendor is satisfactory, the vendor is placed on the approved vendor list (AVL).
To remain on the AVL, the vendor must be evaluated at least once every five (5)
years.
1506 148
.
.
The licensee could not provide objective evidence that NES had been subjected to a source selection audit.
The original contract with NES (actually CONAM) was entered into in 1973 (P.O. 44162) and is an open ended type which can be activated without requiring QA review.
Met-Ed QA identified this and other undesirable features of their procurement QA last year and are in the process of correcting it.
Met-Ed conducted an audit of NES-Conam, Rahway, N.J. in 1976 which is documented in audit report number 75-36.
This audit identified several areas of concern and a number of " agreements" were reached regarding NES-Conam inspection procedures and administrative arrange-ments.
Item 5 of the Agreements (Attachment A to the audit) expressed a concern about the NES project manager and the Conam ISI manager which was the result of an audit finding that stated "some confusion exists as to role of NES Project Manager, and Conam personnel at site." This was noted under the audit checklist item which required a verification of the responsibilities of the NES project manager, Level III inspector and Data Controllers.
Item 6 of the Agreement stated that Conam will prepare an addendum to their Inservice Inspection Data Control Procedure (25-DC-001) to indicate the specific responsibilities of Conam personnel for the forthcoming inservice inspection at TMI-1.
There was no objective evidence available to confirm that the agreements had been carried out.
In addition, NES Conam has undergone reorganization, facilities have been relocated and significant personnel changes have occurred since the audit (75-36) but no re-audit has been performed.
Section XVIII, Audits, of the QA program and implementing procedure GP 4016 establish that Inservice Inspection (designated as Item 20)
shall be audited at least once every two years.
The last audit which Met-Ed QA conducted of ISI was Audit 77-18 which was performed in May 1977.
In accordance with procedures, Met-Ed QA had scheduled an ISI audit, 79-03, to be conducted from February 4 through Febru-ary 18, 1979.
This audit was not conducted due to realignment of Met-Ed personnel which caused the QA auditor to be reassigned.
The audit was deferred and t1is was documented.
The audit has not been performed.
Section VII.2 of the QA Program states that the Supervisor-Quality Control is responsible for surveillance of site contractors to assure that they meet all technical and quality assurance / quality control requirements.
Site QC performed surveillance of NES activities during the March-April 1978 ISI and documented the results on Sur-veillance Report 78-69.
This report identified "In general the first submittal of the UT data sheets was unacceptable."
Specific 1506 149
.
discrepancies noted were NES use of procedures other than those approved by Met-Ed, data sheets were not identical to those specified in the procedure, use cf unauthorized calibration blocks.
Other data sheet discrepancies scch as missing information and bad signa-tures were corrected prior t.1 being approved by QC.
However, during the ISI in question, February-March 1979, site QC stated that they performed no surveillance of ISI activities but relied on the authorized nuclear inspector (ANI) to provide surveillance.
Met-Ed site QC did perform the records review as required and identified many defici-encies.
This represents two different audits /surveillances which identified deficiencies in data control and records of NES ISI, the latter one occurring on the previous ISI, with less than one intervening year.
Thus it appears that corrective actions and/or follow-up actions were inadequate to correct the basic situation and prevent recurrence of data control and examination records problems identified in audit 75-36 and surveillance 78-69.
This is contrary to 10 CFR 50 Appendix B, Criterion XVI which requires that measures shall assure that the cause of the condition is determined and corrective action taken to preclude repetition.
The Three Mile Island Unit 1 FSAR, Appendix 1A, Operational Quality Assurance Plan, Section 21, Corrective Action states in part that:
"An evaluation of a nonconformance includes consideration of the cause.... and solutions that reduce the probability of a repetition of the nonconformance....
procedures include provisions for the identification of problems and deficiencies,.... and for reaudits or inspections to assure that corrective action and documentation are accomplished."
This is addressed in Section XVI of the Operational Quality Assurance Plan, Revision 7, which states "The aedit and surveillance finding follow-up activities shall be performed in accordance with a written procedure which ensures verification of the timely and proper imple-mentation of corrective action to preclude repetition of deficiencies."
This apparent item of noncompliance is categorized as an infraction.
Section VII.2, Vendor Survt:llance, places responsibility for sur-veillance of site contractors on the site Supervisor-Quality Control.
The reporting and documenting of the surveillance is managed in a similar manner as vendor surveillance.
The results of these reports are used by Operational QA in determining vendor surveillance fre-quencies and continued vendor qualification status.
No action was taken to perform an NES vendor review even though documentation in the form of surveillance report 78-69 indicated serious deficiencies.
In addition, s~ignificant management and organizational changes had been made et NES since the last audit in May 1977.
,
1506 150
.
Met-Ed QA revoked the approved vendor status of NES on April 19, 1979.
However, the revocation was effective only for the NES-California Division, and did not apply to the NES Division at Danbury, Connecticut.
This latter Division is the one whose work was in question.
The effectiveness of revocation is further in doubt since the type of contract involved did not involve a QA review before activation of the contract.
Further evidence of the ineffectiveness of Quality Assurance was noted during the investigation.
As noted previously, the contract for ISI Services, P. O. 44182, was essentially open ended (blanket order) and therefore could be activate. without QA review.
Met-Ed QA identified procedural 'eficiencim qith this type contract on audit 78-05 conducted in April of 197u.
Corrective action was in process but had not been fully implemented.
D.
Hartford Steam Boiler Inspection and Insurance Company (HSBIIC)
The investigators interviewed the Authorized Nuclear Inspector (ANI) and his supervisor, the HSBIIC Regional Manager, to gather additional facts related to the TMI-1 ISI.
The ANI had received two weeks of formal nondestructive test training and two weeks of on-tne-job training with an experienced ISI inspector prior to participating in this ISI.
This was the first ISI that he had been assigned to.
In relation to the NES crew the ANI stated that one team started slow but was very competent.
One team did not carry a watch and asked him for a time check on several occasions.
The other team did carry a watch.
In his opinion, the NES supervisor knew the procedures well but that at least one of the teams was not familiar with the procedures.
This was recorded in the ANI log.
He felt, based on his observations, that all evaluations of indications were performed by the Level II examiners.
~The ANI was located in the same trailer as the NES examiners.
He verified that data sheet information frequently was transcribed by NES examiners, usually by someone other than the person who performed the exam.
The ANI had several meetings with Met-Ed personnel to discuss problems.
On February 24 he met with the Met-Ed ISI coordinator to discuss commun-ications breakdown.
On March 8, he met with Met-Ed and NES to discuss twenty nine examinations that had been categorized as unacceptable.
The ANI log noted several unacceptable conditions related to procedure inadequacies and failure to follow procedures.
In addition, an entry on March 19, 1979 noted the possibility that certain required re-examinations were not performed but data sheets had been submitted.
This latter entry 1506 151
.
.
led to a meeting on March 22 with the ANI, his supervisor and representa-tives of Met-Ed to discuss the possibility of falsified data.
In response to NRC questions, the ANI stated that he didn't want to make any allegations of falsification of data.
He stated that he didn't feel the data was adequate and he was unccafortable with it, but he did not have any concrete evidence of wrongdoing.
The HSBIIC is an inspection agency and does not perform investigatory activities.
Inadequacies in data reports are considered as unacceptable until ziiscrepancies are satisfactorily explained or the item in question is re-examined to code requirements.
E.
Nuclear Energy Services, Inc.
(NES)
1.
General Nuclear Energy Services, Inc. (NES) provided the procedures, per-sonnel and equipment for the TMI-1 ISI of February-March 1979.
The ISI crew consisting of six nondestructive examiners arrived on site February 20, 1979 and the ISI examinations were performed between February 20, 1979 and March 10, 1979.
The crew was divided into three teams; two of the teams each con-sisted of a Level I and a Level II examiner.
The other team was comprised of a Level I and a Level III examiner.
The Level III examiner also sarved as the NES on-site supervisor.
The NES per-sonnel had beer trained and certified in accordance with the pro-visions of SNT TC-1A.
The crew members were unfamiliar with each other and only one (a Level I) had c_en at TMI previously.
In addition, neither of the Level II examiners had extensive experience with ultrasonic examin-ation of nuclear power plant systems.
Interviews with NES management and the NES examiners revealed that no data control procedure had been implemented, crew members received no orientation or training in relation to TMI systems or procedures, and that supervision was less than adequate.
2.
Management Meetings Upon notification by Met-Ed of problems with the ISI, NES management initiated a preliminary investigation.
NES, Met-Ed and NRC personnel met on April 27, 1979 at TMI-1 to discuss the ISI.
NES emphasized that although their investigation was incomplete, it appeared that all examinations had been performed.
There had been some deviations from procedures.
It was agread that NES would try to provide more definitive results by May 2,1979.
1506 152
.
.
On May 2, 1979, NES management presented the results of their prelim-inary review of the ISI data and examiner interviews.
The meeting was held at TMI-1 with Met-Ed and NRC personnel in attendance.
NES presented a detailed explanation of each of the data sets in question.
In summary, NES concluded:
the data contained a number of procedural ar;d documentation errors which made some data packages unacceptable.
NES internal procedures must be more explicitly defined, personnel must be more thoroughly briefed.
the TMI situation was aggravated by support work probler.3, scheduling and communications.
all examinations had been performed and no defects identified.
At this point, NES management had not reviewed all of the records available at TMI.
NRC investigators met with NES management during the week of May 14, 1979 at the NES facility in Danbury, Connecticut.
NES management was cooperative and respensive to all NRC requests.
NES management stated that job control at TMI had been difficult because of poor communication, failure to implement certain procedures and lack of site orientation / training.
In addition, some personnel conflicts existed.
With regard to the latter, the NES supervisor had reported to NES (Danbury) on one occasion that one of the Level II examiners was giving him difficulty.
This was resolved by the Corporate Level III and the site supervisor.
NES management concluded that no deliberate falsification had occurred.
This was based on management review of the records and interviews with four of the examiners.
NES defined falsification in degrees:
First dagree - involves a person not doing a task but reporting it as being done.
Second degree - individual knows something is incorrect, or changes something from a previous situation based on his judgment.
Third degree - failure to follow procedure.
NES management did not find any situations which could be categor-ized as first degree.
~
1506 153
.
3.
Examiner Interviews On May 15 and 16, 1979 the NRC investigators interviewed individually all of the nondestructive examiners that performed examinations at TMI-1 during the February-March 1979 ISI.
Each examiner provided a thumbnail sketch of his employment history and qualifications.
The interviews were conducted in the NES facility, Danbury, Connecticut.
They were essentially non-structured, with the examiner allowed freedom to comment on any aspect of the ISI he chose.
Specific questions pertinent to this investigation which were directed. in most cases, to each examiner are documented as follows:
Question _:
Did you ever previously work with other members of the ISI crew assigned to TMI-1?
Iniividual A - No Edividual B - No.
Only individual C at another site Individual C - No.
Except B Individual D - No Individual E - Knew some of them.
Worked with two on a pre-service inspection.
Individual F - No Question:
How would you describe this ISI?
Individual A -
Lack of supervision on job.
No control. No preparation by utility.
Some conflict between certain crew members.
At first, needed time to orientate self, but things were slow -
they weren't ready for us.
TMI would say th kgs were recdy but they weren't.
Individual B -
No coordination on the job.
Metropolitan Ed; son personnel didn't provide adequate support for preparing components and welds for inspection.
Personally had conflict with Metropolitan Edison personnel and NES crew member.
Met-Ed coordinator didn't understand ISI program or technical aspects.
He wanted NES to do everything - any request for ISI support was met with response "no budget for it.
In one case, directed NES to deviate from procedure because calibration block was not available.
Coordinator constantly badgered NES for data sheets.
Individual C -
There was an overload of work.
Pressure on the supervisor fron all sides.
Demands from Met-Ed to get work done.
Preparations not ways ready for inspection insulation on.
Hectic.
Had 1506 154
.
.
.
personnel conflict with one crew member in relation to calibra-tion.
There was also some animosity between other crew members.
Also had conflict with ISI coordinator.
Individual D -
This job was quite different from all others - rush, rush.
ISI coordinator wanted data right away.
Concerned with getting work done - didn't do this at other sites.
Didn't like condi-tions of the job.
Individual E -
Not much ready for inspection - not too much done.
Crew was new - takes awhile for things to gel.
Met-Ed wanted paperwork before the paper work was done - a little friction here.
Experienced it at other sites, but much worse at TMI.
Poor communications.
Totally urorganized as far as utility is concerned.
Seemed like one of those offbeat jobs you run into once in awhile.
Individual F -
Only problem we had was communications.
ISI coordinator was in containment but not tending to his business.
Always asking a bunch of silly questions.
Preparations by TMI were all right.
Had to wait sometimes, but no big problem.
Question:
Did you have any orientation sessions or special training related to TMI ISI?
Individual A -
Had some health physics training at TMI.
No special orientation or briefing.
Procedures were available for review at our leisure.
Looked them over.
Individual B -
No.
There wasn't any team briefing or discussion of procedures.
No training was given at TMI - they wanted to start immediately.
Supervisor has to accept part of blame for this.
Individual C -
No.
Didn't look over procedures - know I should have.
Every morning we went over job assignments.
I.1dividual C - No.
Individual E - No.
1506 155
.
.
Individual F -
No.
Formal orientation woJld not have prevented the problems.
Question:
Were the procedures adequate? Clear and s ecific?
Individual 8 - Some are too complex.
Individual C - Didn't look over the procedures.
Individual F - Procedures were adequate.
Could be improved.
Wrong in one case.
Question:
There are several calibration time conflicts in the data sheets and between other site logs.
(Specific examples detailed in section of this report were related to the examiners.)
How do you account for these?
Individual A -
Didn't have a watch.
Some crew members carry a watch.
Couldn' t keep track of times - try to estimate as close as possible.
On some days, when a lot of exams were done, tried to restructure at end of day.
Actually knew that some of the times weren't right.
But all of the work was done.
Had to fit all exams in.
Individual B -
Everyone doesn't carry watch.
In many cases, calibration times are estimated and may be off by 15-20 minutes.
Containment entry was lax - could bypass HP and not log in or out.
Individual C -
Some examiners have a watch.
Some don't.
Usually check time on first calibration guess the remainder.
Didn't sign in or out.
Met-Ed HPs not always at their stations.
Half the time they didn't keep logs.
Individual D -
Didn't know.
Had a watch sometimes.
All exams were done.
Concerned with getting work done, not paperwork.
Individual F -
Carried watch all the time.
Our times may be off 10-15 minutes.
TMI didn't want calibration times to overlap -wanted us to spread them out.
Can't always relate calibration times witi containment times because sometimes calibration was done sutside of containment such as makeup and feed water.
Errors in log times must be a mistake on their part.
1506 156
.
.
Question:
There were other errors on the data sheets involving signatures, data entries, and approvals (specifics were related to the examiners).
Would you explain how these could occur?
Individual A -
There were some things wrong with tne paperwork - but nothing that would invalidate the exam.
Individual B -
There was no data control.
NES procedure 25-DC-001 Inservice Inspection Data Control System was not in use at TMI-1.
Data sheets are frequently recopied and some errors are made in transcribing the data.
On this job, the original examiner may not do the recopying although he would sign off the sheet.
For instance, an examiner not busy in the field may recopy data sheets which will then be signed by the original exsminer.
The Met-Ed ISI Coordinator wanted data sheets immediately and accused NES of hoarding data.
This eventually led to submittal of data sheets before they were reviewed by NES supervision.
Individual C -
Some conditions that usually are not done were done at TMI-1.
As example, if data sheets were to be recopied, the original examiners would sign a blank data sheet and the data would be recopied by someone else.
This is not a normal practice.
It probably caused the situation where the data sheet was signed by the wrong examiners.
Individual D -
They wanted the data right away.
Individual E -
Met-Ed wanted the paperwork before the work was done.
Wanted the data before NES supervisor signed off.
Individual F -
Some sheets that had to be re opied were presigned by examiners.
Work had been done but data necded to be transcribed.
Perhaps in recopying, the data was put N the wrong sheet.
Question:
All ultrasonic examinations were recorded as clear.
Were there not any indications, even geometrical, observed?
Individual A -
There were some reflectors evaluated as geometric.
All were less than 50% DAC.
Discussed these with others.
Didn't feel c..Sifortable in evaluating geometrics.
1506 157
.
.
.
.
Individual B -
Some small reflectors were observed.
Nothing over 50%.
Dis-cussed a few with others.
All damped.
All eva._2 tion of geometrics is done by Level II or Level III.
Individual C -
Some small ones.
Level II does all evaluation.
During examin-ation, if spot indicatian, Level II tells Level I to backup.
Individual F -
Some geometrics were obterved.
None over 20 percent.
Instructed not to record geometrics - but none over 50 percent anyway.
Question (s):
Did you re-examine any welds? Who directed that certain welds be re-examined? Were the re-examines done entirely?
Individuai A -
Yes.
Redid the work of some other teams.
Don't know why the re-exams were required.
During the last week of the ISI, did most.
NES supervisor directed certain welds to be re-examined.
Remember some of reactor coolant long and circumferential scans were redone.
Individual B -
Yes.
All that were designated as to be re-examined were re-examined.
Met-Ed personnel specified which were to be redone and this was relayed to the examiners by the NES supervisors.
It was under-stood that any unacceptable data was to be redone.
They should have been done entirely.
In some cases old settings were used - these were in ballpark and probably okay to redo with these.
Individual C -
Yes.
As far as I can remember we redid all we were told to.
Individual D -
Yes.
Redid all we were told to do.
Did redo work of another team.
Individual F -
Yes.
Did re-examine the welds from scratch.
Question:
There are a lot of errors, deviation from. procedures, and questionable practices in establishing the DAC curves during this ISI.
How do you account for this?
1506 158
.
Individual A -
Used one sheet for DAC curve for both circumferential and axial scans.
Didn't know it was wrong.
Individual B -
We (NES) found some of the DAC problems ourselves.
Maybe due to problem of inexperience at nuclear plants.
Individual C -
Don't know.
Didn't recheck DACs.
Individual E -
Can't recall details of weld exams.
Individual F -
We used wrong DAC on a few - we caught our error and reinspected the part.
Procedure was wrong - did the DAC wrong.
Was an NES mistake - we corrected it.
On circumferential and axial scan -
the customer requested two DACs.
Didn't need two when doing calibration on a flat block.
Question:
Did you, or do you know of anyone who has falsified ISI records, either at TMI or any other site?
Individual A - No.
Individual B - No.
Absolutely not.
Inadequate review of data.
Individual C - No.
Individual D - No.
All exams were done.
Individual E - No.
Individual F - No falsification by me or others.
Question - There are significant differences in the exposures received by various members of the ISI crew.
What is your explanation for this?
Individual A -
Some crew members spent more time in containment - depends on welds assigned.
Also Level I's did actual exam on the pipes, Level II watched UT screen at a distance.
1506 159
.
,
Individual B -
Some records entries were by the examiners and some were by HP.
Not consistent - could bypass HP station easily.
Radiation safety program was inadequate.
Individual C -
Depends on job assignment - inside or outside of containment.
I know the danger of radiation well.
Get in and get out get the exam over with.
Can minimize exposure by using several DACs on the same instrument.
Hurry up on exam sheets - fill out later.
HP logs were not accurate Met-Ed personnel were
not always at their stations.
Couldn't get any help.
Individual D -
Depends on work area.
Individual F -
Shielding, time and distance.
4.
Quality Assurance NES has developed a quality assurance program (QAP) which is docu-mented in Specification No. 80 A 90201, Inservice Inspection Program, Quality Assurance Manual.
The QAP esssentially parallels and addresses the eighteen criteria of 10 CFR 50, Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants.
The QAP specifies that the program is to be implenented by NES for inservice inspection programs and that it appl us to the preparation of the Inservice Inspection Program Plan.
Section 2 of the QAM requires the preparation of a Quality Assurance Plan for each Inservice Inspection Project which includes a specific project audit plan.
Section 9, Control of Special Processes, Para-graph 9.2.4 specifies that the QA manager shall perform audits to assure compliance with this section.
Section 18 of the QAM requires the development of an audit schedule which defines approximate dates of audits and specific checklists to be used.
The performance of audits shall be by the QA Manager or personnel selected by him who have been qualified in accordance with a written procedure.
Section 18 also requires follow-up audits on all corrective action commitments.
A Project Quality Assurance Program Plan, Three Mile Island Nuclear Generating Station, Specification No. 80 A 1995 was prepared by NES.
Section 5 of this document specifies that a minimum of one audit shall be performed by the NES Manager of Quality Assurance or his representative during the accomplishment of the basic tasks comprising the project scope.
1506 160
,
.
.
2s NES management stated that all QA auditors were qualified to ANSI N45.2.23 and that audits were performed in accordance with ANSI N45.2.12.
Auditors do not receive any training in nondestructive examination techniques and audit checklists are developed from NES procedural requirements.
All audits are announced in advance.
In discussions with NES management, documentation was presented which portrayed a scheduled audit for the TMI ISI on March 26, 1979.
The audit was to incorporate checklists 80 A 0537, 80 A 0538, 80 A 0540 and 80 A 0541.
Internal correspondence and notes indicated that the outage schedule had been moved forward and this prevented accomplishment of the TMI audit.
Note that this is in agreement with the actual ISI work at the site which was completed on March 10, 1979.
In any case the audit was scheduled to be accomplished well into the outage.
As a rer, ult of the situation at TMI-1, NES has placed increased emphasis on ISI audits particularly at Nine Mile Point and Millstone 2.
NES is subject to audit by procurers of their services.
In discussing this NES stated that Metropolitan Edison had not performed any audit of the NES program since 1976.
Further, it is rare for procurers of ISI/ PSI services to audit NES although many utilities and some nuclear steam suppliers utilize NES services for ISI and PSI.
In summary, no NES QA audit of the TMI-1 ISI was performed during the February-March 1979 examinations.
In addition, apparently supplier type audits of NES by procurers of ISI services are not performed to the extent that they should be.
F.
S neric Aspects NES is a major supplier of ISI services to the nuclear industry through its NES Division and Conam Division.
The investigators surveyed the five NRC regions by telephone to determine the extent of NES activity at nuclear power plants.
The results are as follows:
1506 161
.
.
.
.
Number Sites Inservice Preservice Region I
3 Region II
0 Region III
2 Region IV
0 Region V
0
5 Some of the crew members assigned to TMI-1 have been assigned to crews performing NDE at other sites in Region 1.
To what extent the problems identified at TMI-1 are existent at other sites is not known.
A primary factor is the basic manner in which the ISI activity is controlled which is different at each site.
Discussions with RI NRC inspectors didn't reveal any major areas of concerns at these other sites.
A review of inspection reports for some of these sites did not reveal any significant problem areas.
NES has conducted site audits at two of the operating reactor sitt-in RI, at least one of these has had ISI activities performed by at least one of the examiners assigned to TMI.
NES management stated that they did not identify any major problem as a result of the audits.
However, they will perform audits at their other sites on a priority basis.
The documented audit results were not available during this investigation.
G.
Exit Interview The investigators met with the Superintendent, Technical Support and the Site Quality Control Manager on June 12, 1979 to discuss the results of the investigation.
1506 162