ML20057C424

NRC Mgt of Misadministration Info Is Inadequate, Audit Rept

ML20057C424
Person / Time
Issue date:	09/07/1993
From:	NRC OFFICE OF THE INSPECTOR GENERAL (OIG)
To:
Shared Package
ML20057C421	List: ML20057C420 ML20057C424
References
NUDOCS 9309280354
Download: ML20057C424 (23)

Text

._-

NRC's Management of Misadministration information is inadequate

[

REPORT SYNOPSIS I*

On November 16,1992, a wire containing highly radioactive material broke and was unknowingly left inside an 82 year old patient receiving radiation L

therapy for cancer, contributing to her death 5 days later. The accident went undiscovered until December 1,1992.

Shortly afterward, a newspaper

[

published a series of articles detailing an 17 year history of other harmful L

mistakes made in radiation therapy. These events focused the attention of Congress and the public on the Nuclear Regulatory Commission's (NRC)

~

regulation of medical licensees.

NRC is responsible for creating rules and programs to protect the public from undue radiation exposure. The agency also informs Congress and the public of its progress in meeting this objective. Inherent in these responsibilities is the need to analyze regulatory data, identify adverse trends, and ensure that q

resources are effectively managed and focused to address problem areas.

In a 1980 rule, NRC recognized the need to collect and analyze information on medical *misadministrations"*. NRC's objective was to more accurately determine their frequency and evaluate problem trends. This Office of the a

Inspector General (OIG) review found that NRC still needs to make important improvements in its management of misadministration information

[

if it is to fully achieve this regulatory objective. For example, reported 5

misadministrations increased nearly three-fold in the last 3 years versus the average of the prior nine years, although NRC staff is unable to fully explain

(

the increase. Also, NRC recently made three significant changes to its reporting criteria, including requiring licensees to report only the misadministrations of greatest magnitude. Even with these changes, the number of reported incidents are rising.

s NRC relies on an estimate published in 1987 of annual therapeutic procedures that use radiation. However, since then, the estimate has not been revised or independently confirmed. In addition, we found weaknesses in NRC's Office for Analysis and Evaluation of Operational Data (AEOD) Annual Reports A misadministration, in its simplest terms, is an over, under, or unintended radiation dose to a patient exceeding NRC's regulatory criteria.

m o/9 w n rw i

[

eao m oss eao m PDR ORG NE ED PDR

1 EE NRC's Management of Misadministration Information is inadequate that are used to identify important emerging trends. Due to the manner in which AEOD prepares key information, NRC did not detect significant inaccuracies in its 1989 and 1990 data. We also found that NRC staff base g

their regulatory decisions on case-by-case reviews and assessments oflicensee 5'

events, not on misadministration trends.

These problems lead OIG to conclude NRC has not fully met its 1980 objective. We believe it is essential for NRC to have accurate data to determine whether broad program adjustments are needed to better protect l

public health and safety. To correct these longstanding weaknesses, we recommend that NRC independently obtain and verify the number and type g

of procedures involving the medical use of byproduct material licensees E

perform annually, and establish performance indicators to strengthen its regulatory oversight.

I I

I I:

I' I

I I

I.

I al

=.

I NRC's Management of Misadministration information is Inadequate l

TABLE OF CONTENTS I-REPORT SYNOPSIS...........................................

i l

INTR ODUCD O N............................................. I Ba ckgrou nd............................................. 2 I

i FIND IN G S.................................................. 5 I

NRCs Misadministration Data Is Incomplete.................... 6 Weaknesses In NRCs Misadministration Management............. 9 f

NMSS Has Relied on Outdated Estimates of Therapy 3

Proced ures................................... 9

!I AEOD Annual Reports Mask Variability in Estimated Procedures and Error Rates.....................

10 j

NMSS Does Not Use Misadministration Error Rates To l

Measure Regulatory Effectiveness.................

12 i

The Need For Performance Measures Is Gaining Momentum 14 Recent Efforts to Develop Better Data Have Weaknesses....

15 i

lI NRCs Data Does Not Provide a Uniform National Perspective.....

17 l

CON CLUSIO N S.............................................

19 RECO MMENDATIONS....................................... 20 AG ENCY CO MMENTS....................................... 21 h

I

E NRC's ManaDement of Misadministration information is inadequate APPENDICES I

Objectives, Scope, and Methodology

^

II Agency Comments on Draft Report III Major Contributors To This Report m

L r4 J

l L

m L

F l%

h i'

L

?

L

[

c L

i NRC's Management of Misadministration Information is inadequate 1

l INTRODUCTION l

l The Nuclear Regulatory Commission (NRC) is responsible for ensuring the safe use of nuclear materials in the United States in order to protect public l

health and safety. To meet this objective, NRC has established a body of l

regulations to govern a wide range of nuclear-related activities, from l

commercial power reactors and other industrial applications, to the use of g

radiation in the practice of medicine. However, recent reports of medical 3

misadministrations' of radiation to patients and their consequences has i

focused the attention of Congress and the public upon NRC's regulation of medical licenses; and raised questions regarding NRC's collection, analyses, and management of misadministration information. For example, y

L On November 16,1992, a wire containing highly radioactive material 7

broke and was unknowingly left inside an 82 year old patient receiving radiation therapy for cancer, contributing to her death five days later.

1 Technicians and the physician monitoring the procedure ignored radiation alarms when the therapy was concluded and released the patient for return to her nursing home.

The lost wire was not discovered until December 1,1992, when personnel of a medical waste r

disposal company identified the source that had set off the radiation a

alarms of its incinerator facility. Before the wire was secured, 94 r~

persons received unnecessary exposures to radiation, including people associated with the medical clinic, nursing home, ambulance service, J

and waste contractor. The patient received about 1,600,000 Rad to 2

l the area of her tumor, an overdose of almost 1,000 percent. The 7

autopsy report stated the cause of death as " Acute Radiation Exposure and the Consequences Thereof".

From December 13 through December 17,1992, The Plain Dealer of 7

Cleveland, Ohio, published a series ofinvestigative articles disclosing

]

mistakes that occurred in the medical use of radiation over the past 17 e

'A misadministration, in its simplest terms, is an over, under, or unintended radiation dose to a patient exceeding NRC's regulatory criteria.

2Rad: Radiation Absorbed Dose. Exposures of more than 2000 Rad to the whole body are fatal.

ma/www r,i rL

_ - _ _ - - _ _ _. _ _ - _ _ _ - _ - _ _ - - _ _ - _ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ^ - ^ ^ - -

I NRC's Management of Misadministration Information is inadequate years. Some of these incidents involved fatal or serious injury to l

persons from procedures not under NRCs direct regulatory responsibility. But, the articles showed NRC did not have adequate g

information on at least two fatal incidents that it should have, or 5

information on other events relevant to NRC's responsibilities.

The Office of the Inspector General (OIG) reviewed NRC's collection and analysis of misadministration data to determine how the agency uses this information to prevent incidents, and strengthen its regulatory oversight.

l Appendix I contains additional information on our objectives, scope, and-methodology.

BACKGROUND The medical community uses radiation for a variety of purposes, including cancer diagnosis and treatment. Diagnostic uses attempt to identify the cause g

of an illness or injury. For example, bones and organs are revealed when 3

special film is exposed by x-rays; or, an organ's health can be assessed by determining the accumulation of a radioactive tracer compound ingested, injected, or inhaled by a patient.

Therapeutic radiation doses are much larger than diagnostic doses, and seek l

to stop cancer or relieve pain. Therapy misadministrations are often more serious to the patient. NRC regulates two uses of radiation for therapy -

g; radiotherapy and nuclear medicine:

3 Radiotherapy includes teletherapy and brachytherapy. Teletherapy g'

machines that use nuclear material (often Cobalt-60) to produce a beam of radiation to destroy the cells of a cancer tumor deep inside a patient. linear accelerators use electricity to produce the same effect.

Brachytherapy involves placing sealed radioactive material sources directly onto or in a tumor to kill cancer cells. Conventional gj brachytherapy uses low activity sources for treatments lasting two or E ;

three days, while remote afterloaders are machines that use stronger radioactive sources for shorter treatment times.

Nuclear medicine procedures include radiopharmaceutical therapy, i

which is the ingestion of a radioactive compound, usually sodium onc/9wn hae 2 g3I Eo,

I NRC's Management of Misadministration information is inadequate l

iodide, that the patient's natural metabolism will direct to the disease location.

This audit focused on NRC's information regarding therapeutic uses of radiation because of the greater risks involved.

Th. >bjective of NRC's regulatory program for medical use is to assure the patient receives the dose of radiation or radioactive material prescribed by the l

physician, while also protecting workers and the general public. NRC does not regulate the appropriateness or effectiveness of the prescribed treatment.

Also, NRC recently estimated that it regulates no more than 25 percent of the I

radiotherapy treatments in the nation. This is because NRC only regulates the use of material made radioactive as a byproduct from a nuclear reaction I

I process and does not regulate x-ray machines, linear accelerators, or materials made radioactive by accelerators.

i i

In 1979 the General Accounting Office (GAO) recommended that NRC j

expand the reporting requirements for its power reactor, radiography, and

!g medical licensees. Regarding medical licensees, GAO found NRC needed l E information to determine the causes of misadministrations, alert other licensees of the hazards associated with certain practices, and modify its medical licenses or regulations. GAO concluded:

"Without this kind of feedback on incidents affecting the public l

health and safety, NRC cannot be sure it is adequately regulating the possession and use of nuclear materials in medical practice."'

fI 1

In 1980 the Commission approved the first rule requiring the reporting of medical misadministrations. NRC believed misadministration reports would allow the agency to investigate the incident, identify the cause, and whera possible, initiate action to minimize the possibility of recurrence. Although there had been some serious misadministrations before the rule was published, NRC did not know how many events had occurred and there was II

'Benorting Unscheduled Events At Commercial Nuclear Facilities Opportunities To Improve Nuclear Reculatory Commission Oversicht j

(EMD-79-16, January 26,1979).

~

oio/www

%3

I

E, a

NRC's Management of Misadministration information is inadequate no staff effort to determine the level of under-reporting. Also, NRC staff did not obtain solid documentation of the total procedures performed annually.

The Commission determined misadministration reports were neede.d to provide a mechanism to determine more accurately the frequency of misadministrations and to evaluate problem trends both at individual medical l

institutions and among all medical licensees. Between 1980 and 1992, the NRC staff proposed several relaxations to the misadministration reporting rule. For example, in 1982 the NRC staff proposed the withdrawal of the reporting requirement because they believed the cost and administrative burden to licensees outweighed the incremental value of this information.

E However, the Commission rejected these proposals.

5 The Atomic Energy Act of 1954, as amended, established a shared regulatory l

responsibility between NRC and states. The Act authorizes NRC to form agreements with states (called Agreement States) to regulate some classes of g

byproduct material licenses.

NRC's Ofnce of State Programs formally 5

evaluates Agreement States' radiological programs every other year to determine their adequacy to protect the public health and safety, and their compatibility with NRC regulations. NRC's Office of Nuclear Material Safety and Safeguards (NMSS) regulates radiation use in 21 states, U.S. territories, and the District of Columbia. State agencies are responsible for regulating l

radiation use in 29 states. NMSS regulates about 2000 medical licenses, and Agreement States regulate about 4500 medical licenses.

Approximately 1,100 NRC medical licensees perform therapeutic procedures, or administer sodium iodide I-125 or I-131 to patients in quantities greater g

than 30 microcuries, which have the potential, if misadministered, to pose some risk. On January 27,1992, NRC began requiring these licensees to establish and maintain a quality management program (QMP) to provide l

"high confidence that the byproduct material or radiation from byproduct material will be administered as directed by the authorized user", thus g

enhancing patient safety and minimizing the occurrence of misadministrations.

5 The QMP also requires medical licensees to review a sample of medical administrations at least annually to identify and report any previously g

undetected misadministrations.

When NRC was developing the quality management rule, it estimated that another 2200 medical licensees regulated by Agreement State programs would need to submit a QMP by January 1995.

l oso/mu Pase 4

NRC's Management of Misadministration Information is inadequate In 1979 NRC established the Office for Analysis and Evaluation of F

Operational Data (AEOD) to " provide, as one of its primary roles, a strong, L

independent capability to analyze operational data". The office produces annual reports on the operating experience of reactor and non-reactor

{

programs, and quarterly reports to Congress on safety significant events called

" Abnormal Occurrences". AEOD has produced annual reports since 1985.

[

Misadministrations are usually caused by inadequate training, inattention to detail, deficient procedures or failure to follow procedures, and insufficient supervision. Although machines may deliver the radiation to patients, most misadministrations are caused by human failure to control the procedure.

Specifically, AEOD's 1991 Annual Report stated only I of 19 therapy

~

misadministrations reported by NRC licensees was probably caused by 1

defective equipment; the rest were primarily attributable to human error.

2 FINDINGS NRC is responsible for creating rules and programs to protect the public from undue radiation exposure. The agency also informs Congress and the public of its progress in meeting this objective. Inherent in these responsibilities is the need to analyze operational data, identify adverse trends, and ensure that resources are effectively managed and focused to address problem areas.

F l

AEOD's 1991 Annual Report noted the number of therapy misadministrations

+

reported by NRC licensees during 1991 was about twice the average number

(

reported annually from 1981 through 1990.

However, our audit found weaknesses in the AEOD Annual Reports, including inadequate analysis to determine trends and inaccurate reporting. We believe it is important that r

L, timely, accurate information on medical licensee performance is collected and analyzed to help evaluate NRC's overall effectiveness in protecting public health and safety.

Also, NRC's misadministration data base is incomplete, and may understate L

the number of misadministrations. NRC learns of misadministrations through medical licensee self-reporting, or through allegations of events. However, p

NRC acknowledges it does not know whether its medical licensees have L

reporte.d all misadministrations.

I L.

OiG/73A-14 Page5 e

F l

l NRC's Managernent of Misadministration Information is inadequate NRC is responsible for marshalling its limited resources to identify and ll correct problems, and helping licensees take aedon to prevent their i

recurrence. While NRC responded in 1980 to a regulatory need to obtain gI better misadministration data, NRC still needs improved information to 5

accurately assess the overall effectiveness of its medical licensee regulatory program. For example, AEOD's 1991 Annual Report stated that despite

" increases in the numbers of reportable events, the error rate for all types of misadministrations remained very low."

However, we found NRC uses outdated information to calculate misadministration error rates. The agency li relies on an estimate developed in 1987 by the medical community to establish the number of procedures and patients which are administered gi radiation treatments each ye.ar. NRC has not independently verified the E) accuracy of therapy procedure estimates.

NRC's recent efforts to develop better misadministration information also have weaknesses. NRC is continuing to rely on the medical community to estimate the number of therapy procedures. Further, NRC is attempting to ll use federal databases of Medicare patients, and patients admitted to hospitals, to determine the number of procedures perforraed annually in the United g!

States. However, NRC acknowledges these databases are also incomplete and u

incompatible. In addition, it will be several years before NRC will be able to provide a national perspective on misadministrations because the agency will not begin collecting compatible data from all Agreement States licensees until early 1995.

NRC's MISADMINISTRATION DATA IS INCOMPIEE Since November,1980, NRC regulations have required licensees to report misadministrations.

We found the annual average of therapy misadministrations reported from 1990 to 1992 were 3 times greater than the l,

annual average number of misadministrations reported from 1981 to 1989, with a significant increase in 1992 to 31 events.d (See Figure 1.) This recent

'Because of changes in misadministration definitions in January 1992, g

NRC is changing the classification of some events. Due to these changes, 3

most diagnostic and therapeutic misadministrations invoking sodium iodide are now classified " sodium iodide misadministrations". For consistency with 3l previous historical data, OIG is including two misadministrations in 1992 51 1

OlG/9M44 Page 6 l

I El

• l

NRC's Management of Misadministration information is inadequate increase raises questions as to whether NRC obtained accurate information

(

regarding all misadministrations that occurred in the 1980's.

L Misadministration Reports Are Increasing Therapy Events 31 30 p

- 30 24

'7 2

20 2

Yrb-20 14

bhjik T

12 g

10

• F 10

1. jfgw}-

d G;" m -j'h--

k 10 QQ

-10 g

jg Qg 4

o h.

h.

, ~ ' ~.

0 i

i i

1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 Test E Therapy Events Figure 1 NRC learns of misadministrations primarily from self-reporting by medical licensees, or from allegations. However, according to a senior NMSS official, r

L licensees probably have under-reported misadministrations because they: (1) lack understanding of NRCs reporting requirements; (2) do not detect the i

misadministration; or (3) choose not to report the misadministration.

Further, NRCs inspection program is not designed to detect misadministrations. Medical licensees are inspected on a regular cycle of 1 to 5 years based on the size of a facility and the hazards associated with the procedures performed. NMSS and regional officials stated the discovery of a misadministration during a routine inspection would be very unlikely because inspectors do not have the time to review the large volume of radiation administrations a facility might perform between visits. As of l

l resulting from therapeutic use of sodium iodide as radiopharmaceutical misadministrations.

03G/93A-14 Page 7

Ea NRC's Management of Misadministration Information isJnadequate January 1992, NRC's OMP rule.qum medical licensees to review a l

representative sample of all medical administrations at least annually to identify and report previou':y undetected misadministrations.

However, g

Agreement States are not required to implement this requirement until 1995.

5 NRC staff said the increase in reported events may result from: (1) improved licensee awareness of reporting requirements, partly attributable to NRC workshops held during the development of the OMP rule; (2) the effects of escalated enforcement actions, including civil penalties, against some medical l

licensees; or (3) increasingly complex medical procedures. However, the staff did not have analyses or data to support their assertions.

The recent increase in reported misadministrations for 1992 is of particular

{

concern because beginning January 27, 1992, NRC made three significant gl changes to its reporting requirements. First, NRC now requires licensees to report only the incidents of greatest magnitude - those in which the l

administered radiation dose is 20 percent more or less than the prescribed dose.

Also, therapy misadministrations of sodium iodide require the administered dose to exceed the prescribed dose by at least 30 microcuries.

g f

From 1981 to 1991, NRC's reporting threshold was an administration of 10 5

l percent more or less than the prescribed radiation dose.

l I1 Second, NRC now requires licensees to report when they administer (A) radiation to the wrong treatment site of a person, and (B) a significant incorrect dose in a series of teletherapy treatments, even though the total prescribed dose for the series may be unaffected. NRC retained its original requirement that licensees report misadministrations if the radiation dose is administered to the wrong patient, or by the wrong means.

Finally, NRC now requires licensees to retain records of administered l

radiation doses in error (A) between 10 and 20 percent for brachytherapy and radiopharmaceutical therapy, (B) between 15 and 20 percent for teletherapy, g

and (C) between 10 and 20 percent for therapy doses of sodium iodide where 3

the difference is more than 15 microcuries. However, NRC does not require licensees to report these " recordable" events, and does not plan to collect or l

analyze information on them.

I 06G/9 w u he s Ea w

L NRC's Management of Misadministration Information is inadequate As a result of these revisions, misadministration data reported by NRC licensees for 1992 and later years is not comparable with data accumulated between 1981 and 1991.

I l

OIG also identified weaknesses in AEOD's quality controls to ensure reported data is accurate. For example, data common for separate tables did not reconcile between the 1990 and 1991 AEOD Annual Reports, which indicates AEOD did not verify the accuracy of misadministration data it published.

Each report contains tables that summarize the data reported to NRC since p

1981. However, the totals for several types of information are not consistent L

between tables. Specifically:

[

The total patients affected by misadministrations presented in separate tables differs by 145 patients for the 1990 report and 201 patients for the 1991 report.

m lH The diagnostic misadministrations presented in separate tables differs

{

by 12 events for the 1990 report and by 53 events for the 1991 report.

Diagnostic misadministrations differ between consecutive reports by 65 events.

AEOD staff were unaware of these inconsistencies.

WEAKNESSES IN NRC's MISADMINISTRATION MANAGEMENT NRC is responsible for ensuring the information it reports is reliable and accurately reflects the activities the agency is charged with regulating. OIG L

anairsts round significant weaknesses in NRes information management that has led to inaccurate reporting.

I4 NMSS Has Relied on Outdated Estimates of Therapy Procedures NMSS has relied on estimates published in 1987 to determine the number of the therapy procedures performed annually in the United States. Since NMSS b

has not independently verified the estimates, this raises questions regarding their accuracy, and the validity of NRCs medical misadministration error rate calculations.

ompwu r.p,

e L

I NRC's Management of Misadministration Inforrnation is inadequate NMSS contacted medical societies, including the American College of Radiology (ACR) and NRC's Advisory Committee on the Medical Uses of Isotopes (ACMUI), to cormnent on the estimated number of therapy g

procedures using radiation during development of a proposed Quality a

Assurance rule in 1987.5 This same estimate was published in the Federal Register in 1990, and in AEOD Annual Reports in July 1990, July 1991, and g

August 1992. NMSS officials stated each time the estimates were published and no comments were received questioning their accuracy, they presumed the figures were "close enough".

l The 1987 estimates predicted each year approximately 30,000 g<

radiopharmaceutical therapy and 50,000 brachytherapy procedures are 5

performed, and about 100,000 patients receive teletherapy treatments.6 Of these 180,000 nationwide procedures, AEOD estimates about 40% (or 72,000 g

procedures) are performed by NRC licensees, and the remainder by Agreement State licensees. According to AEOD officials, the division of procedures between NRC and Agreement State licensees was an estimation based on the number of medical licensees per state populations.

AEOD Annual Reports Mask Variability in Estimated Procedures and Enor Rates OIG analysis found that AEOD Annual Reports mask the number of procedures performed annually by NRC licensees because they state the sum g

of estimated procedures since 1981 through the current year instead of g

reporting annual activity.

Narrative which accompanied data tables in AEOD's 1989 through 1991 Annual Reports estimated NRC licensees would perform about 72,000 therapy procedures annually based on the 1987 estimate. However, when OIG compared annual estimates in supponing tables of those Annual Reports with the 1987 estimate to verify consistency, l

5Basic Quality Assurance Practices in Radiation Therapy; October 2, 1987; 52 FR 36942.

Teletherupy has been reported in " patients" rather than " procedures" because a teletherapy patient usually undergoes a series of 10 to 30 treatments. Radiopharmaceutical therapy and brachytherapy generally 3

involves one procedure per patient.

E om/924-u h p to Ee

I NRC's Management of Misadrninistration Information is inadequate I

we found a drop to 17,000 in the estimated procedures performed by NRC licensees for 1989 and 1990. (See Figure 2.)

Estimated Annual Therapy Procedures Performed by NRC Licensees Thousands 80 40 17 17

'l Aw rws

~

j 1988 1989 1990 1991 Year

+

Teletherapy

+

Brachyt.herapy Rad 2ophe.rmaceuucal E Total Therapy lI AEOD staff reviewed our methodology and analysis, and agreed it was correct. However, AEOD and NMSS staff could not explain why the decrease in reported procedures occurred and questioned the reliability of the 1989 and l

1990 estimates. In addition, the supporting schedules used to prepare the annual reports had been discarded.

In addition, OIG analysis based on AEOD data found that annual misadministration error rates were higher than the cumulative error rates

{

AEOD reports have published. To calculate an annual misadministration rate, OIG used the annual misadministration events reported by AEOD and divided them by the annual procedures totals OIG calculated for individual years. Because AEOD's methodology masks annual changes in estimated procedures, it also masks changes in error rates. Thus, the sharp increase in

^

reported events in the last few years caused very little change in NRCs published error rates.

L i

b kk L

c

?

Es' NRC's lAanagement of lAlsadmin!stration Information is inadequate Comparison of AEOD and OIG Error Rates Event Error Rate per Pmeedure nmn

.cose N

.00ln j

's a

m

's 7

s---

---c' 1938 198G 1990 1991 Year OIG Radiopharmacy

+ OIG Brachytherapy E AEOD Radiopharmacy E AEOD Brachytherapy I

Figure 3 As shown in Figure 3, OIG calculated that the annual error rate for g

brachytherapy and radiopharmaceutical therapy in 1991 were 3 and 2 times, respectively, the rate that AEOD reported for an eleven year average. In addition, OIG calculated that the annual error rates for brachytherapy in 1989 l'

and 1990 were about 5 and 9 times the 1988 average, respectively. OIG is not stating that the OIG rates pictured above were the actual annual error rates g

of medical licensees because of uncertainty in AEOD's data; rather, OIG is 5,

illustrating that rates developed from analyzing data on an annual basis are more sensitive to change than the AEOD method of preparing error rates g

cumulatively.

NhfSS Does Not Use Afisadministration Error Rates To Measure Regulatory Effectiveness NRC"s policy is to minimize the agency's intrusion into the practice of medicine by relying primarily on medical licensees to conduct their operations in a manner that protects public health and safety. NMSS uses its licensing l

and inspection programs to assess the adequacy of licensee programs in meeting this obligation.

onop3A-u Page u E--

I NRC's Management of Misadministration Information is inadequate NMSS officials also stated they do not believe misadministration error rates accurately reflect regulatory effectiveness.

Instead, they review each misadministration to determine its root cause, effect on the patient, and whether the licensee has taken adequate corrective action to prevent recurrence. They also decide whether the cumulative effect of individual I

incidents warrant additional regulatory attention. Consequently, NRC staff do not make regulatory decisions based on misadministration error rates, but rather their review and assessment of the implications ofindividual cases.

I In addition, NMSS officials believe misadministration error rates are not a reliable measure of regulatory effectiveness due in part to the rapid changes that are occurring in the practice of radiation medicine. They noted the recent increase in misadministration reports may not indicate increased I

misadministrations are occurring, but rather improved licensee compliance with NRC's reporting requirements. However, the staff did not have analyses or data to support their assertions.

lI The Chairman, NRC, believes the agency has information on reported

.g misadministrations, but acknowledges data on the number of annual 3

procedures is less credible. During recent Congressional testimony, the Chairman concluded that without reliable data on the number of procedures, NRC cannot accurately determine misadministration trends.

We found NRC has not used its licensing and inspection programs to l

independently verify the estimates of therapy procedures supplied by medical l

i societies. An NMSS official stated the 111 broad scope medical licensees7 lg and 142 teletherapy licensees which are inspected annually perform nearly 103 l3 percent of the radiation therapy procedures completed by NRC licensees.

However, NRC has not surveyed these licensees to ascertain the number of l

procedures performed annually that NRC is responsible for regulating.

Improved information on the amount of procedures performed by NRC licensees could: (1) improve the reliability of the misadministration error rate

,l NRC calculates, and (2) identify licensees that perform many, or risky, procedures and therefore could have a larger potential for misadministrations.

lI 7Generally large hospitals.

I I

I:

NRC's Management of Misadministration Information is inadequate The Need For Perfonnance Measures Is Gaining Momentum l

Several recent initiatives indicate a growing awareness of the role g

performance measures can play in effectively managing large federal e

programs. First, The Chief Financial Officers Act of 1990 requires agencies to develop measures of effectiveness of their regulatory programs. Reliable l

performance measures could help NRC decide how to most effectively distribute its limited resources between programs. For example, NRCs Fiscal Year 1993 budget estimates allocate about $295 million, or 55% ofits budget, lt to commercial power nuclear reactors and about $37 Million. or 7% of its 8

budget, to materials licensees. In September 1992, NRC submitted 19 E

performance indicators to Congress and the Office of Management and B

Budget, four of which pertained to the safety of NRCs materials program.

However, none of these performance indicators specifically address the l

medical license program.

Second, Public Law 103-62, The Government Performance and Results Act of 1993 was signed August 3,1993 by President Clinton. The Act directs agencies to establish strategic plans, and use performance goals to measure and report on agency progress in achieving their objectives.

Finally, in an April 1993 report,' GAO recommended several improvements l

to strengthen NRCs materials program, including the establishment of

^

performance indicators to enable NRC to assess its effectiveness in fulfilling its mission.

We believe analyses of medical misadministration data could indicate the g<

effectiveness of NRCs medical license program. However, we do not beheve NRC will be able to reliably evaluate the significance of changes in medical misadministrations, and thus meet its 1980 objective, until the agency l

establishes an accurate data base. NRC needs this information since it is l'1

• Materials licensees include medical, industrial, academic, radiography, E-and gauge-user licensees.

E i

Nuclear Reculation: Better Criteria and Data Would Heln Ensure g

Safety of Nuclear Materials (GAO/RCED-93-90, April 26,1993).

5 oso/924 u n, u gg<

Bw, i

I NRC's Management of Misadministration Information is inadequate g

provided, in part, as a gauge for the Congress and public to measure its effectiveness in protecting public health and safety.

Recent Effons to Develop Better Data Hm>e Weaknesses g

In response to the heightened awareness of limitations in NRC's 5

misadministration data, the Commission requested that a senior NRC manager perform a management review of the medical use regulatory j'

program. To improve its data on administrations, NMSS staff pursued two paths: (1) contact with a professional medical society, and (2) review of Health Care and Finance Administration (HFCA) and National Center for Health Statistics (NCHS) databases. Both efforts have weaknesses that j

undermine confidence in using results.

In developing its estimates of total annual procedures, a NRC Visiting i

Medical Fellow sought information from the medical community in March l

1993.

The revised estimate contains significant changes from the 1987 estimate for the individual procedure types, as illustrated in Table 1.

Specifically, the:

[

Brachytherapy estimate declined 40 percent from 50,000 to 30,000 annual procedures; Radiopharmaceutical therapy estimate increased 33 percent from

+

l 30,000 to 40,000; and, H

l

~

Teletherapy estimate remained basically unchanged from 1987.

L However, NRC staff and medical specialists said many licensees are

[

replacing NRC-regulated Cobalt-60 teletherapy machines with non-NRC regulated linear accelerators.

!m L-,

L_

Olo/9M44 ray 15 ru

E U,

NRC's Management of Misadministration Information is inadequate Table 1:

Comparison of NRC Estimates of Annual Procedures Ferformed in the United States I

Type of 28 Procedure 1987 Estimate 1993 Estimate 22 Chang g Teletherapy 100,000 100,600

+ 600 Brachytherapy 50,000 30,000

-20,000 Radiopharmacy 30,000 40,000

+ 10,000 NMSS also sought to estimate the number of radiation procedures using information contained in two federal databases. HFCA maintains records of procedures funded by Medicare, which is used by about 80 percent of the l

Americans 65 years or older. NCHS, a part of the Centers for Disease Control, performs statistical surveys of patients admitted to hospitals.

However, NMSS has found that developing an estimate of national radiation procedures is hampered by incompleteness of data and incompatibility between the two databases, specifically the lack ofinformation on outpatients under 65 years of age and outpatients treated by Federal government facilities.

Our analysis also found that the March 1993 estimates for the number of annual misadministrations are inconsistent with historical data because the estimates predict fewer events than the recent three year trend of NRC licensee reports. Table 2 shows the last three years of misadministrations reported to AEOD by NRC licensees, and the annual average of those years.

g Table 2 also shows the Visiting Medical Fellow's March 1993 estimate of national misadministration events for 1992, and the adjustment for the portion of events that would be reported by NRC licensees.

l I

2* Published in Federal Registers of October 1987 and January 1990, and in AEOD Annual Reports published in July 1990, July 1991, and August 1992.

2'NMSS internal memorandum from the NRC Visiting Medical Fellow, g

dated March 8,1993.

5 manwu n,a E.

I NRC's Management of Misadministration information is inadequate I

Table 2:

Reported and Estimated Misadministrations by Procedure NRC AEOD Reports: NRC March 1993 licensee g

Licensee Events National Portion of ma es National 1990 1991 1992 Ave.

Procedure for 1992 Estimate i

Teletherapy 10 3

16 9.7 30 12 l

Brachytherapy 8

11 13 10.7 10 f

Radiopharmacy 6

5 2

43 4

16 OIG found the March 1993 estimate for teletherapy misadministrations in I

NRC-regulated states is approximately 24 percent greater than the average of the last 3 years.

However, the average of brachytherapy and radiopharmaceutical therapy misadministrations reported by NRC licensees

!g has been nearly three-fold the number of events predicted for NRC licensees by the new estimate.

l NRC's DATA DOES NOT PROVIDE A UNIFORM NATIONAL PERSPECTIVE

'g NRC's misadministration data does not provide a national perspective because of the shared regulatory responsibility between NRC and Agreement States.

1 Agreement State licensees were not required to report misadministrations

{

,l when NRC promulgated the original reporting rule in 1980.

In 1987, Agreement States were given until April 1,1990 to require their licensees to report misadministrations using NRC's 1981 criteria.

Even though Agreement States now report misadministrations, NRC will not l

have consistent national misadministration information until 1995 because NRC and Agreement State licensees use different reporting criteria. (See Figure 4.) NRC licensees began using new reporting criteria in January 1992.

I However, because NRC believes it takes about 3 years to incorporate these types of changes in state programs, the agency does not require Agreement States licensees to use the new criteria until January 1995. Nevertheless, 4

B's NRC's Management of Misadministration information is inadequate Agreement States have the option of imposing the new criteria on their licensees sooner than 1995 if they desire.

Therapy Misadministration Reporting Criteria g

Reporting Licensees 11 g

States lllllll l 1111l l l l

en se as a4 as as av as ao so si s2 ss 84 es Year E Original criteria E 1992 criteria

• an.nm ors 1, teso Figure 4 The reliability of Agreement State data is questionable because Agreement State licensees have historically reported fewer events than NRC licensecs, even though Agreement State licensees are twice as numerous. For example, g

Table 3 shows the reporting of misadministrations in 1991 by Agreement State 5

and NRC licensees, indicating that 4524 Agreement State medical licensees reported fewer events than NRC's 2094 licensees. NRC officials acknowledge l

the disparity in the number of reports, and stated it probably results from under-reporting by Agreement State licensees.

I I

l I

I NRC's Management of Misadministration information is inadequate Table 3:

Comparison of NRC and Agreement State Licensci Reported Misadministrations in 1991 Licensee location Agreement States NRC States and Federal Facilities l

Number of Licensees 4524 2094 Type of l

i Misadministration i

l Therapy 18 19 Diagnostic 402 441

I CONCLUSIONS Our review found that after nearly 13 years of collecting data, significant l

weaknesses remain with NRCs management of medical misadministration information.

We recognize that NRC staff base their regulatory decisions on case-by-case reviews and assessments, not misadministration trends. However, we believe it is essential for NRC as a regulator to have accurate data to help determine whether program adjustments are needed to better protect public health and safety. The need for timely, accurate data is even greater today than in 1980, l

because NRC recently changed its criteria so licensees report only the misadministrations of greatest magnitude.

Furthermore, even with this I

change, the number of reported incidents is increasing and NRC staff do not have analyses or data to explain the rise, g

NRC has a history of developing outdated and incomplete misadministration data. To its credit, NRC has recently attempted to retine its methodology, but several significant weaknesses remain, including incompatible data bases and incomplete coverage of all patients.

However, NRC has not sought to independently verify the estimates of therapy procedures supplied by medical l

societics. Also, NRCs data will not provide a uniform national perspective osa/v3A-14

,I P.se 19 I

Ea.

NRC's Management of f/lsadministration information is inadequate r

until after 1995 when Agreement State licensees are required to follow the new reporting criteria.

These problems lead OIG to conclude NRC has not fully met the objective of establishing a mechanism to collect and evaluate data on medical licensees; they also raise questions about relying on NRCs misadministration information to evaluate the agency's overall effectiveness in protecting public health and safety.

RECOMMENDATIONS g

To correct longstanding management weaknesses and improve the reliability of its misadministration data, we reconunend that NRC:

Independently obtain and verify the number and type of procedures involving the medical use of byproduct material licensees perform annually.

To assist in meeting the Chief Financial Officers Act requirement that agencies have reliable measures to determine whether regulatory goals are g

being met, we recommend that NRC:

3 Use misadministration data to establish performance indicators to evaluate the effectiveness of its programs and strengthen its regulatory 1

oversight.

Il To provide a more comprehensive national perspective regarding medical misadministrations, we recommend that NRC:

l Encourage Agreement States to (1) report all misadministrations, and (2) adopt NRCs new reporting criteria before 1995.

g I,

Il I.

Ei si

i I

l NRC's Management of Misadministration information is inadequate l

AGENCY COMMENTS I

On August 23,1993, the Deputy Executive Director for Nuclear Materials Safety, Safeguards, and Operations Support (DEDO) responded to our draft report. His comments are included as Appendix II.

1 I

The DEDO agreed with the intent of our first recommendation regarding the

-g need to independently obtain the number and type of procedures involving the 3

medical use of byproduct material licensees perform annually, but had not made a decision on the best means to obtain the data. The DEDO did not I

agree that inspectors should verify the information licensees provide NRC.

We have modified our recommendation to retain the goal ofindependently obtaining better data, while leaving the DEDO flexibility in achieving this l

objective. We continue to believe that NRC should pursue verification of the data licensees provide NRC so that it can have a high level of confidence that the information supplied by licensees is indeed accurate.

The DEDO agreed with the other two recommendations of our report. He I

indicated that the NMSS and Office of State Programs will begin using medical misadministrations as a perfonnance indicator starting January 1994.

He also noted continuing NRC efforts to encourage Agreement States to l

report all misadministrations and adoption of compatible regulations. Finally, the DEDO enclosed clarifying language for several points in the report, and I

we have made changes where appropriate.

I I

t I

I I

I

I I

Appendbc l NRC's Manaptwoont of Misadministration information is Irndequate I

1 OBJECTIVES, SCOPE, AND METHODOLOGY Due to increasing reports of medical misadministrations, and heightened public and Congressional interest in the effectiveness of NRCs regulation of I

its nuclear medicine program, the Office of the Inspector General reviewed NRCs procedures to develop and utilize misadministration data.

We conducted our work from February,1993 through May,1993.

I To obtain a Commission perspective on the medical misadministration issue, we attended oversight briefings on (1) NRCs initiatives to prevent I

misadministrations, (2) a misadministration that resulted in a patient death, (3) medical community views on misadministrations, and (4) the Agreement State program.

At headquarters, we interviewed officials from NRCs Office of Nuclear l

Material Safety and Safeguards, and the Office for Analysis and Evaluation of Operational Data, who are responsible for directing the medical license I

program and developing operational data. We also met with regional ofEce staff to obtain their perspective on misadministrations and the medicallicense inspection program. We reviewed and annlyzed medical misadministration g

reports and other programmatic information NRC staff provided.

We performed our work in accordance with generally accepted Government l

auditing standards.

I

. I I

I I

r I

I I

Appendbc ll NRC's Management of Misadministration Information is 1,mdequate AGENCY COMMENTS ON DRAFT REPORT I

s..

t S

UNUED STATFS 5

~!

NUCLEAR REGULATORY COMMISSION I

\\.....,/

namoro=, o.c. acessem August 23, 1993 I

MEMORANDJi FOR:

Thomas J. Barchi Assistant Inspector General for Audits Office of the Inspector General I

FRON:

Hugh L. Thompson, Jr.

Deputy Executive Director for Nuclear Materials Safety, Safeguards, and Operations Support SUBXCT:

DRAFT REPORT - NRC'S MANAGEMENT OF MISADMINISTRATION INFORMATION IS INADEQUATE This memorandum is in response to your memorandum dated July 26, 1993, in which you requested review and comment on the draft report, entitlea *NRC's Management of Misadministration Information is Inadequate

• The thorough in-I depth review of the U.S. Nuclear Regulatory Commission's collection and analysis of information on medical misadministrations will be useful in NRC's overall review of its medical use program. The report identifies certain areas that need improvement, to optimize the use of misadministration data.

I With respect to your specific recosmendations, I submit the following:

Pecomendation 1 I

Independently obtain and verify the number and type of nuclear medical procedures licensees perform atinually.

j Resoonse I

Agree in part. I intend to explore two options to obtain the number of annual procedures: 1) a record review of operational data by the inspectors, which may require approval by the Office of Management and Budget; and 2) rulemaking I

to ensure that the licensees collect the information in consistent format, which would then be reported to NRC. The second effort seems warranted if we are to gain a reliable estimate of the number of administrations of radioactive material addressed by the ' Quality Management Program and I

Misadministrations' rule, and thus, the denominator for use in determining the rate nf occurrence of misadministrations per administration. An Advanced Notice of Proposed Rulemaking would be published to obtain maximum comments from the regulated community on the best means to achieve this objective.

We have not identified a need for inspectors to verify the information that would be provided by the licensees since this does not represent a health and safety problem. Verification of such infors& tion is not within the scope of I

training for NRC inspectors nor, in fact, is it necessary since licensees routinely collect and retain this information for financial management and OGG/9 % H Pay 1 cf 3 I

Ea Appendbc il NRC's Management of Misadministration Information is inadequate Thomas J. Barchi 2

tax-related purposes. Therefore, we have a high level of confidence that the l

information supolied by a licensee will be accurate once it is made clear 5

through rulemaking exactly what information is being sought by NRC. Such information would include: 1) number of patients; 2) numeer and type (high dose-rate and low-dose rate brachytherapy; teletherapy; radiopharmaceutical therapy; and diagnostic radiopharmaceutical studies) of procedures; 3) number of acministrations; 4) number of misadministrations; and 5) number of recordable events.

I Recommendation 2 Use misadministration data to establish performance indicators to evaluate the effectiveness of its programs and strengthen its regulatory oversight.

Resconse Agree. The Offices of State Programs (OSP) and Nuclear Material Safety and Safeguards (NMSS) are developing a more integrated national oversight approach, using core perfomance indicators. One of the operational indicators to be evaluated in a meaningful way on a national basis is the use of the rate of medical misadministrations per licensee. Starting date:

January 1994.

Recommendation 3 Encourage Agreement States to (1) report all misadministrations, ano (2) adopt 3

NRC's new reporting criteria before 1995.

3 Resconse Agree. The collection of medical misadministration data is not specifically recuired by an NRC regulation or the terms of the Agreements that tre 29 States have signed with NRC. In the spirit of cooperation with NRC and under the provisions of the exchange-of-information program of their agreements, all 29 Agreement States collected and sent medical misadministration data to NRC for calendar year 1992. We will continue to encourage the Agreement States to report all misadministrations and to adopt compatible regulations, including the following: 1) medical misadministration definitions; 2) medical misaaministration reporting criteria; and 3) the " Quality Management Program and Misaaministrations" rule. In order to foster prompt reporting, a joint materials event report database is being developed to collect and disseminate l

NRC ano Agreement State event report data on a real-time rather than catch basis.

I OlGM H Pym 2 of 3 I

a.

Appendix 11 NRC's Management of Misadministration information is inadequate j

Thomas J. Barchi 3

I The staff identified several points that could be clarified for the reader, in addition to some minor technical inaccuracies. These are addressed in the enclosure.

I s -

C ugh L. Thompson, Jr.

-I De uty Executive Director for E

uclear Materials Safety, Safeguards, -

and Operations Support I

Enclosure:

Comments on Draft Report I

I I

I I

I I

~,,

I I

I Appendix 111 NRC's Management of Misadministration Information is inadequate I

MAJOR CONTRIBUTORS TO THIS REPORT I

Richard Donovan, Team Leader I

William D. McDowell, Jr., Senior Auditor Scott W. Buchan, Management Analys.t I

I I

I I

I I

I I

I I

~,,,

I