ML20199H478

Discusses Empirical Data Reported in Reference 4.1

ML20199H478
Person / Time
Issue date:	11/12/1997
From:	Abramson L, Ryder C NRC
To:	Steven Baggett, John Lubinski NRC
References
SSD, NUDOCS 9711260128
Download: ML20199H478 (7)

Text

-.

~

NOTE _TO:

Steven Haggett.

John Lubinski n/oln FROM:

Christopher Ryder 1ee Abramsen he hN~)

SUlHECT:

Empirical Data When combined with specific infonnation about consequences, the empirical data reported in Reference 4.1 can be used to make a rough estimate of risk as defined by NRC (Ref. 4.2). The constituents of risk are scenarios describing the routes of a radioactive source from a licensee through the recycling stream, the frequency of the scenarios, and the consequences of the scenarios. From Reference 4.1, a few general categories of scenarios can be defined in tenns of endpoints, such as being found before or after at e steel mill, Consequences can be associated with these categories of scenarios. Then, using the NRC definition

'of risk (see Equation [1] on page 2), a rough estima:e of risk can be computed. If the issue is in the seriousness of the situation. hen such a calculation is sufficient; but Reference 4.1 without such a calculation is also sufficient for this purpose.- llowever, if the issue is in making changes to regula:lons. -

such an estimaic is inmfr,clent:

Terms must be defined. An appropriate model must be constructed to make appropriate estimates for decision making.

The situation is complex, reflecting the influence of regulations, radiation monitors, the practices e

of using the monitors, and the practices of responding to alarms. Likewise, the data about the situation are complex and must be carefully modeled, The uncertainty in an empirical risk estimate cannot be readily determined from the data of e

Reference 4.1. Uncertainty is present, w hether or not it is expressed-it is an integral part of an estimate. Uncertainty has a tv aring on regulatory decisions.

Clumges to regulations must be based on changes in risk. An estimate of risk from empirical data appli s only to the situation from which the data are collected. The empirical data in Reference e

4.1 apply only to the status quo (more precisely, only the more recent part of the data apply because circumstances have been changing). Should a change in regulations be made, any estimate of the change in risk based only on the data in Referense 4,1 is speculative.

Each of these points is discussed in turn.

[

l Discussion l

b 1.1 Definition of Tenns

- For the purpose of discussion, only radioactive sources in nuclear gauges are considered.

Assess is to estimate the value, worth or quality of something. This difiers from an analysis, which is to separate into parts to identify or study its structure (Re,4.3),

r i A lost device is not necessarily in the recycling stream. Lost means that the device cannot be taken into

. account. Only gauges that are in the recycling stream may be melted at a steel mill,

+

m 97y12g12eg71112 lll SSD POR

$/l

$~ Sbl

ihe term rh4 has many definitions. Equation [1] is a definition used by NRC is reponed in Reference 4.2.

m n qr;c, el where R = risk f' a hequency of a scenario C = consequence j = scenario (series of events leading to the consequence)

The phrase ut th A has a different mtaning and is taken to be a radioactive source that is in a state for potentially being lost.

1.2 Complexity of the Data Looking at the data of Reference 4.1, the observed hequency of finding radioactive sources has been increasing since the first device was discovered in 1983. 3ut circumstances have been changing:

lhe number of radiation monitors at the scrap yards and steel mills is increasing.

e The capability of the radiation monitors to detect radioactive material is increasing.

The use of scrap metal is increasing.

The population of nuclear gauges at risk may be increasing as the economy grows and increasing demands for high quality products are placed on industries.

The problem may be worsening as might be concluded from looking at empirical data. Or the awareness of the problem may be increasing as might be concluded considering other factors.

Even if the above were constant, empirical data would still be complex. Probabihty is the chance of an event occurring. To compute a probability, the event needs to be carefully defined. A probability estimate is meaningful only when an event has been specifintly defined. Without specifically defining the evem, the meaning of an estimate becomes ambiguous. The probability of finding a radioactive source in the recycling stream is the result of a complex series of events. Many factors that are a part of the circumstances w here a nuclear gauge might be lost (see box, " Factors involved in Losing and Finding Nuclear Gauges" on page 3). As these factors change, circumstances changes, and hence, the above circumstances and estimates made from the data, must be updated. When circumstances change, enough data is needed to adequately represent the new circumstance.

A model must be defined from which understandings can be obtained abou, what quantities are needed to estimate desired probabilities. Figure 1 is the population of devices containing radioactive sources. The total population of devices is represented by Set {T}. Most nuclear gauges are in use on production lines controlling processes. These devices are not at risk of being Lst because they are in place, controlling production. Only a postion of the gauge population is at risk of being lost. In Figure 1, the population at risk is represented by Set ( A). These gauges are at a potential to being lost because they are no longer in use:

2

removed and set aside during renovation on a mothballed process line e

in storage l' actors involved in Losing e

in a defunct or abandoned fac ity

& Finding Nuclear Gauges u

e lhe population at risk is necessarily very smail e rederal because the gauges are used to contrel production.1 hose gauges that are in production implementation and enf.orcement e

are not at risk. Of those gauges at risk, some are lost. Set {ld, hke Set { A) cannot be readily IJteniers estimated Estimating Set II.) has an additional management organuation at industrial facilities e

complication arismg from the definition of /ou e rdations between labor r.id management (see page 1 of Section 1.1.). Some gat'ges will be

• Iken3er control programs and practkes nonornks peculiar to an industry found, either before or afler being melted in a e

furnate; the gauges that are found in the recyclin,',

e demand for products stream are represented by Set {l ). Set ( A) can

• P[]n7'",I]dh$;',rllj chy i

be disided into many regions, such as gaugu that become stranded in the recycling stream, buried Nah age anJ I)cmolnion Prattkes in a landfill. or unkno% n disposal. The purpose scrap Yardi and sted Mini of figure 1 is to illustrate general states of rnonHoring equipment devices, not to delineate all of the possible states e

"*"h"'i"8 P'"k '5 of a nuclear gauge.

demand for scrap metal e

From Figure 1, the annual probability of finoing a nuclew gauge is given by Equation [2].

l'r.1]A} = N' pi N 3 where fr/F/A! = ptabability of finding a nuclear gauge in th(

recycling stream gisen that a gauge is at risk A,

= annual number of gauges at risk N

= annual number of gauges found in the f

recycling stream (both before and aller A]

melting at a steel mill)

The data ofiteference 4.1 must be used with caution to estimate N,.

The data. sere collected over a period w hen circumstances were unquestianably changing. During the period coscred by Reference 4.1, monitors were being installed and the equipment was impros ing. As the data ' indicate, the number of radioactive sources discoscred in the recycling stream %culd be expected to figure 1 Venn Jiagram of ouclear gauges increase simply because more effort was being made to find them, throughout nii inaustrici A - number or Thus, N, is increasing os er time, especially in recent years, saused na f ""umbe' of sausc5 Although N, might also be increasing over time, it is implausible f."[ji n ""*,',o'r((u'g"5 ' '"

l t

that Na is increasing as fast as Nr. This means that Pr{l%} has ban inneasing ova rnent years.

1he denominator, N,, is the number of gauges at risk,.iot the total number of gauges, N,. Using instead the total nurr ber of gauges, N,, will result in an underestimate of the probability of fmding a gauge in the recycling stream. In Figure 1, the total number of gauges is what can most easily be estimated.1he number of gauges at risk and the number lost are both much more difficult to estimate.

1.3 Uncertainty in Risk Estimates A point estimate of risk is computed from a single set ofinputs. Ilut because the practices and phenomena represented in a nsk analysis are only partially understood, inputs cannot be precisely dermed. Accordingly, there is ollen much uncertainty about the inputs. Therefore, a risk estimate will also be uncertain. With uncertainty being an integral part of a risk estimate, a point estimate is only one possible value of the output. A point estimate without an indication of the uncertainty can be taken to imply that the uncertainties are small, which is doubtful. Given that uncertainties are an integrr.1 part of a risk estimate, a point estimate can be misleading. l'igure 2 shows a hypothetical distribution dawn on an arbitrary scale.1he distribution represents a large amount of uncertainty.

Given a point estimate <mdyet himwing the distribution in advance. where would the point valuefsdlin relation to the distribution?

1here is a tendency to try to associate the location of the pdnt estimate with the unknown dktribution by characterizing the input as "best" estimate. This implies that the point estimate output of a mathernatical function is also a best estimate. A problem with the term be.5/ estimate is that it is vague and seldom dermed. Suppose that best means unblated. Then the output would be a best estimate only for a linear function. For example, suppose we have a linear function Y = aX.1hc expected value of Y is simply:

I?(Y) = ap where 4 88 Constant p = E(X1, the expected value of the random Probabmty variable, X 1

v Thus, if ic is a best (unbiased) estimate orp, then 9 = as is a best estimate ofap.

{',vjbg a post Ef Ilut the mathematical functions in a risk analysis are nonlinear.

1 2

If we have the nonlinear function Y = X, then a best estimate of 2 would be an unbiased estimate of 2 Ilovever, the expected value of fis not p. Rather, it is:

0 2

I!(Y) = p, g 2

2 Figure 2 DirTerence in variance off penpccthes ghen by the point where a2

=

estimme and the distributen or a nndom s ariat4e.

4

If a'is large (large uncenainty about X), then p = R' would Random be a highly biased estimate of p and would not be a best Variables 2

Value '

estimate of 2 The rules for propagating point salues through A

B C

the equation differ from the rules for propagating both distributions and the quantities describing distributions (i.e.

mean, median, mode).

%m 1.4 Regulatory Changes

@(

jr

• )y Estimating the risk of the current situation by itselfis only enough to determine w hether or not a problem exists.

P Assessing the effects of changes in regulations requires assessing changes in risk. While a rough estimate of risk can be estimated from the data of Reference 4.1, changes in risk due to changed regulations cannot be estimated from this data.

2"'j, ^ln"["L'[,'",l((",',

While experienced regulators may propose reasonable changes, dirkting from. compehon or such changes still need to be assessed to determine what j'" l#8'"d ' *"'

" reasonable" means, not only in terms of expenditures to implement them, but also in terms of their effects.

Understanding the modeled situation through a risk analwis is as important as the risk estimates themselves. The risk estimates indicate combined effc

'he frequencies and consequences of all 1 o plausible scenarios. A risk analysis shows how the combined elfects arise,'hus provi&g the elements of cogent reasoning discussed in Reference 4.4 (use of all relevant information, valid premises, and correct logic). A risk analysis makes these elements explicit. Data and stated premises from a variety of sources are assembled in i logical manner.

Calculating risk with Equation [1] requires information about the scenarios leading to consequences, the frequencies of the scenarios, and the consequences of the scenarios. The empirical data in Reference 4.1 supplies only some of this information. The scenarios themselves are incompletely defined because the sequence of es ents leading to the outcome are, in most cases, unknown. Many scer,arios could have lead to the observed outcomes (i.e., reported events). The reported events are only those that have occurred and are known, not a complete set of outcomes. The frequency of the scenarios cannot be determined in detail. While a risk analy sis may not fully renect all of the factors involved in losing and finding nuclear gauges (e.g., box on page 3), at least some of them can be made explicit and others can be at least acknowledged. Needed for liquation [1] are all reasonable scenarios, their frequencies, and their cor> sequences. All that is availaMe from Refereace 4.1 is an incomplete list of discoveries from which a few scenarios can be deduced.

Regulatory decisions must account for uncertainty in estimates because uncertainty is an intern.1 part of the estimates. The uncertainties are present, whether or not they are expressed. Expressing the uncertainties can change the way in which regulatory decisions are made. Figure 3 shows hypothetical point estimates and distributions of three quantities, A,13, and C. Suppose that the relative imponance of A,11, and C are to be determined. Using the point estimates, C > B > A and C > A. Ilowever, using distributions is more complicated. Because of the large amount of overlap, one concludes that C a B and 11 A. Ilowever, one cannot conclude that C - A because there is little overlap in these d;stributions. In fact, one could make a case that C > A. Thus, transitivity does not hold when uncertainty is taken into

.S.

i account.1hese problems do not exist with point estimates, but point estimates clearly give a distorted view of the relative importance of the quantitles. Ranking issues and comparing issues is more

[

colnplicated when uncenainties are expressed.

L 2 Use of E.npirical Data r

The empirical data documented in Reference 4.1 is useful in that it indicates a problem of radioactive sources in the recycling stream. The data show that the loss of control is not an unusual event bat is recurring. Though a rough risk estimate may be computed when with this and information about consequence:, Reference 4.1 without a risk estimate clearly indicates that a detailed analysis is warranted.

This is a reason for a systematic analysis of the situation.

1he data are not a statistical sample, chosen at random. The events themselves have a chance element to them, but the data are not randomly chosen.1he sample is one of convenience, developed as a result of what has been reported. Therefore, an unbiased estimate of the probabilities cannot be easily obtained from the data.

lhe empirical data in Reference 4.1 not only show that a problem exists, but confirms an intuitive understanding of the situation. Reference 4.1 shows that the discoveries of radioactive sources in the recycling stream has been increasing since the first discovery. This agrees with the observation that monitoring equipment is becoming more prevalent and sophisticated. If the frequency of discoveries had instead decreased, then further investigation would be needed to reconcile the observations and intuition.

'ihe data are also useful for making statistical inferences with statistical models; this is different from using the data themselves lacking an unde

• standing of the model. For example, given that radioactive sources wers found in one segment of the recycling stream and not in another, the chance of no observations when some observations are expected can be determined; such a calculation leads one to make further inquires with a comprehensive analysis.

3 Summary Three issues should b: assessed to decide on regulatory changes.

1.

Is there a problem?

2.

What should be changed?

3.

To what extent should changes be made?

Reference 4.1 clearly indicates that a problem exists, thereby addressing the first issue. A study such as Reference 4.1 would be necessary for deciding whether or not to perform a risk analysis.

Expuience and judgement suggest that the problem is complicated. Without a systr atic analysis, the effects of clearly warranted regulatory changes remains in question, especially whe. he people proposing changes acknowledge the incompleteness of the proposal. A risk analysis addresses the second issue by delineating the factors and relationship of the factors in a manner that can be reviewed and discussed, in the absence o an explicit standard of risk to meet, the third issue can be addressed by r

considering the marginal changes to the point of diminishing returns.,

I Just as a model, such as l'igure I, is needed to understand the probabilities as discussed above, a model is needed to understand risk. Without a model, the situation of radioactive sources in the recycling stream is murky. Situations will be based on intuition, which is often wrong, or expertjudgement, which is not a i

substitute for, but instead a part of, a systemati antlysis. With an analytical model, the situation changes from murky tojust very complicated.

Reference 4.1 suggests that changes are necessary, it does not say what to change and how much to change it to achieve desired results. Until the appropriate analysis is done, the second and third issues will remain unresolved. Gis en that potentiall) s ery costly decisions are to be made, a rigorous analysis is necessary. One may argue that an analysis, including a collection of useful information, is costly.

Ilowever, the decisions to be made will have consequences that are even more costly.

4 Refe ences 4.1 J.1.ubenau and J. Yusko, " Radioactive Material in Recycled Metals," #calth Phy3/cs, April 1995.

4.2 U.S. Nuclear Regulatory Commission, "A Review of NRC Staff Uses of Probabilistic Risk Assessment" NUREG.1489, March 1994.

4.3.

Oxford American Dictionary,lleald Colleges Edition, Avon llooks, 1986,1350 Avenue of the Americas, New York, New Yo:L,10019.

4.4 Kahane,11., l.ogic and Contemporary Rhetoric Sixth Edition, Wadsw orth Publishing Company, 1992.

r 7

-