ML19309C782

Slide Presentation Entitled, Two Approaches to Treatment of Inventory Differences in Nuclear Matl Accounting

ML19309C782
Person / Time
Issue date:	03/31/1980
From:	Lurie D NRC OFFICE OF MANAGEMENT AND PROGRAM ANALYSIS (MPA)
To:
Shared Package
ML19309C777	List: ML19309C776 ML19309C782
References
REF-10CFR9.7 NUDOCS 8004090265
Download: ML19309C782 (38)

Text

- - - - - - - _ - -

nby 3

0(E-0) g TWO APPROACHES TO THE TREATMENT OF INVENTORY DIFFERENCES IN NUCLEAR MATERIAL ACCOUNTING i

A COMMISSION BRIEFING h

c;

3 i

Prepared By ifi

?

hl Dan Lune cl =

~

'1 mo 3

1:

=

Applied Statistics Branch JJ Office of Management and Program Analysis O

e<

%.-+,>-

m.m

-+m.-.m.-

PURPOSE 1(E-1)

• To Present an Overview of Two Approaches to the Treatment of Inventory Differences in Nuclear Material Accounting:

(1) The Classical (Statistical Hypothesis Testing) Approach (2) The Game Theorf (Strategic Analysis) Approach

• To Compare the Capabilities and Limitations of the Two Approaches l

l Opinion: The Two Approaches Are Complementary, Not Competitive; They l

Address Different Aspects of the Treatment of Inventory Differences k

2(E-2)

OUTLINE e Brief Background e Two Basic Concepts: ID and LEID O The Classical Approach e The Game Theory Approach l

e in Conclusion...

~

BRIEF BACKGROUND 7/76-1/78 NRC Sponsors Studies in Applications of Game Theory Approach to Treatment of Inventory Differences (NUREG-0290 and NUREG/CR-0490) 4/78 MC&A Task Force Recommends to Commission that Role of Game Theory be Reviewed by Peer Review Group (NUREG-0450) 11/78-8/79 Peer Review Group Evaluates Game Theory (NUREG/CR-0950) 9/79 Open Door Memo to Chairman Hendrie from S. Moglewer Expressing Concerns with Material Accounting 11/79 Chairman Hendrie Requests Briefing for Commission

4(E-5) l TWO BASIC CONCEPTS: ID AND LEID i

4 e Inventory Difference (ID) e A Model for ID i

f e Limit of Error of ID (LEID?

i I

e What Does Experience Show?

5(E-6)

INVENTORY DIFFEREN'CE (ID)

WHAT IS ID?

By 10 CFR @ 70.51(a?:

"C6) ' Material Balance' means a determination of material unaccounted for LMUF? by subtracting ending inventory (EI) plus removals (R) from beginning inventory (Bl? plus additions to inventory (A).

t Mathematically, MUF = Bl + A-El-R" Since 1977: ID s MUF~

em-w A MODEL FOR ID ID = True ID + Error e True ID is Not Known e Error is Not Known True ID Term includes:

Error Term includes:

• Loss

• Instrument Errors

• Diversion e Measurement Errors i

e Sampling Errors e Bookkeeping Errors i

Note: If any Component of ID is a Random Variable, then ID is a Random Variable. In Nuclear Material Accounting ID is Treated as a Random Variable.

7(E-8)

LIMIT OF ERROR OF ID (LEIDD WHAT IS LEID?

- Random Variables Usually Have Several Associated Measures of Variability (Variance (o2), Standard Deviation (o), Limit of Error (LE), Range, Etc.)

By ANSI N15.16 Standard (1974):

By 10 CFR @ 70.51(a):

"3.2 Definition. The limit of error

"(5) ' Limit of Error' means the of an estimator T is twice the' uncertainty component used in standard deviation of T; that is, constructing a 95 percent confi-twice the square root of the dence interval associated with a variance of T."

quantity after any recognized bias has been eliminated or its effect accounted for."

Opinion:

Despite the Confusion in Definition, the Spirit of LEID Implies that (when no Loss / Diversion Occurs) the Probability oflIDl Exceeding LEID is about 0.05

i 8(E-10)

WHAT DOES EXPERIENCE SHOW7 l

NRC Inventory Experience l

April 1974 - December 1978 l

Number of Facilities 17 Number of Inventories With LEID Reported:

803 Number of Inventories Where llDl > LEID:

375 - Nearly Half !!!

(When There is No Loss / Diversion, about 40 llDIs Would Be Expected to Exceed LEID)

Opinion: Something is wrong !

Source: Nuclear Material Safeguards Status Report (White Book),

IE, June 1979

9(E-12)

THE CLASSICIAL APPROACH 1

e Testing the Hypothesis that True ID Equals Zero

• A Management Dilemma e

Current NRC Practices.

e Operating Characteristic Curves e

Deficiencies in Current NRC Practices

10(E-18)

CLASSICAL APPROACH: TESTING THE HYPOTHESIS THAT TRUE ID EQUALS ZERO (Model: ID = True ID + Error) in The Absence of Errors (Unreal World) e

- ID s 0 No Loss / Diversion

- ID $ 0 Loss / Diversion

!n The Presence of Errors (Real World)

Result Actual State of Affairs of Inventory True ID = 0 True ID # 0 Undetected ID 5 Threshold Correct Decision i

Loss / Diversion ID > Threshold False Alarm Correct Decision

A MANAGEMENT DILEMMA 11(E-15)

Where to Set the Alarm Threshold?

Probability of False Alarm Threshold too Smail:

Probability of e High Probability of Undotected Loss / Diversion Sounding Alarm in Case of Loss / Diversion Threshold too Large:

e High Probability of False Alarm e Low Probability of False Alarm o Low Probability of Sounding Alarm in Case of Loss / Diversion s

12(E-16)

CURRENT NRC PRACTICES (A Departure From Statistical Principles)

(1) By 10 CFR @ 70.51(e)(5) Upper Limits on LEID Are Established. These Limits Are Given as Percentages of Additions to or Removals from Material in Process, Whichever is Greater.

(2) Denote the Limit on LEID by LEID-LIMIT.

(3) Management Action (Page,1974).

Event Action llDl > LEID Notify NRC llDl 21.5 LEID-LIMIT Notify NRC and Reinventory llDi 2 2 LEID-LIMIT Notify NRC; Shutdown and Search Until llDl < 1.5 LEID-LIMIT or Until NRC Approves Facility Reopening llDl 5 LEID Implies No Loss / Diversion (The Hypothesis of No Loss / Diversion is Accepted)

OPERATING CHARACTERISTIC CURVES (Probability of No Alarm) 1.0

,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ~

.8 g

4

~

6 Z

-Threshold T

o Set At g

~

1.5 LEID Threshold Set

.4 Threshold'

= 3 Kg At 2 LEID = 4 Kg ~

m Set

-8 At LElD = 2 Kg E

~_

.2

~

~

0

!5 ' ' '

.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Magnitude of' Loss / Diversion, in Kgs (assuming a = 1 Kg)

14(E-19)

DEFICIENCIES IN CURRENT PRACTICES

• llDI is Questionable Criterion for Guarding Against Loss /Diveision

• No Uniformity in Definition of LEID

1. Actual Behavior of llDl/LEID Ratio is Very Different from Expected Behavior Based on Spirit of Definition e LEID-LIMITS Are Not Based on Statistical Principles

• Hypothesis Testing Procedure Cannot Make Statement that No Loss / Diversion Occurred e LEID-LIMITS May Be Too High, Resulting in High Probability of Undetected Loss / Diversion Opinion: Statistical /probabilistic aspects of these deficiencies are correctable

-.m-,-

15(E-20)

THE GAME THEORY APPROACH CAccording to NUREG-0290 and NUREG/CR-0490)

• Game Theory Approach to the Treatment of ID e Three Moves of the Game

• A Cost Function e Optimal Solution to the Game e An Extension of the Game:The Random Alarm Threshold e Deficiencies in the NUREG-0290 and NUREG/CR-0490 Studies

16(E-24)

THF_ GAME THEORY APPROACH TO THE TREATMENT OF ID Concept implementation Objective:

Minimize Expected Penalty to Defender Cost:

Given Explicitly in a Utility Function Decision Variables:

(I)

Alarm Threshold (11) Estimators of Amount Diverted Strategy:

Set of All Combinations of Alarm Thresholds and Associated Estimators of Amount Diverted Optimal Strategy:

"Best" Combination of Alarm Threshold and Associated Estimators of Amount Diverted Motivated by: NUREG-0290

3 17(E-25)

THREE MOVES OF THE GAME LAlarm Threshold Set at a Kg of SNM)

Move 1 Diverter Removes x Kg of SNM.

Move 2 Defender, Upon Taking Inventory, Learns that ID Kg of SNM Are Unaccounted for.

Move 3 Defender, Knowing ID, Acts as if y Kg of SNM Have Been Diverted. (The Variable y Provides an Index of intensity of Search; it is also Declared to be the Estimate of Amount Diverted.)

Adapted from: pp.17-18 NUREG-0290 Vol.1

18(E-27)

A COST FUNCTION A Cost Function in Dollars (Penalty to Defender)

Penalty = Cost of Cleanout Inventory

+ Cost of x Kg of SNM Diverted

+ Cost of Search for y Kg of SNM

+ Cost Due to Wrong Assessment of Amount Diverted

-Utility Derived from Recovery of Diverted Material Adapted from: pp. 23-24 NUREG-0290, Vol.1 l

- - - - -... - - - -,. = _.... -. _ -

t 19(E-29)

OPTIMAL SOLUTION TO THE GAME (o = 0.3 Kg)

Maximum Index of Search Effort Expected Alarm Threshold ID 6 Threshold ID > Threshold Penalty to Defender

.002 y2 =

.876 22.3 0.00 y3

=

0.30 (o)

.070

.953 10.6 0.57 (1.90)

.125

.975 7.4 0.60 !LEID)

.130

.975 7.5 0.90

.200

.957 10.1 1.20

.296

.926 14.8 1.50

.410 1.017 20.5 1.80

.534 1.280 26.7 2.40

.790 1.827 39.5 l

3.00 1.052 2.388 52.6 All entries are given in Kgs.

Reconstructed from: Figure 4, p. 22 l

l NUREG/CR-0490 l

l

20(E-32)

AN EXTENSION OF THE GAME: THE HANDOM ALARM THRESHOLD Maximum Expected Penalty to Defender for Alternative Alarm Thresholds Maximum Expected Penalty (in Kgs.)

Random Alarm Strategy 2.5 Fixed Alarm Threshold At 0.30 Kg 10.6 At 0.57 7.4 At 0.60 7.5 At 1.20 14.8 At 1.50 52.6 Reconstructed from: Figure 11. p. 39 and Figuio 4. p. 22 NUREG/CR 0490 (Corrected per S. Moglewer's note of 1/14/80)

21(E-30) 1 DEFICIENCIES IN THE NUREG 0290 AND NUREG/CR-0490 STUDIES l

e The Present Formulation of the Cost Function Does Not Appear to Account for All Eventualities

1. Input into the Cost Function from l

Decision-Makers and Experts in Relevant Disciplines is Lacking Opinion: Studies are incomplete. Deficiencies Could be Alleviated by Further Study.

22(E-33)

IN CONCLUSION...

e Advantages of Game Theory Approach over l

Present Practices

• Disadvantages of Game Theory Approach l

Relative to Present Practices e Peer Review Recommendations from NUREG/CR-0950 i

m

_----__m--.---____------m-*--,-*--w-

- -.w- - -

v-

+ve

=ww-

• ww-

-y=

i I

r 23(E-34;l i

ADVANTAGES OF GAME THEORY APPROACH OVER PRESENT PRACTICES i

1. The Approach Explicitly Considers the Possibility of a Diverter.

2. The Strategy for the Defender is Based on Utility Considerations.

3. The Alarm Thresholds are Derived from Optimality Criteria Rather than by Arbitrary Decision.

1 l

4. For the Appropriate Model, Maximum Assurance (Best Guarantee Against Penalty;l is Achieved.

24(E-35)

DISADVANTAGES OF GAME THEORY APPROACH RELATIVE TO PRESENT PRACTICES

1. There Are inherent Difficulties in Developing a Cost Function; it Cannot be Developed Without Input from Decision-Makers.

2. Continuous Updating of the Cost Function and its Parameters Are Necessary for Maximum Effectiveness.

3. The Game Theory Approach Requires More Information in its Application than the Classical Approach and Would Appear to Require More Resources in Its implementation.

i

4. Generic Requirements for Regulations Concerning the Game Theory Approach are More Difficult to Provide than for the Classical Approach.

l 25(E-36)

PEER REVIEW RECOMMENDATIONS FROM NUREG/CR-0950 It is Recommended that...

1. NRC Take Steps to improve Game Theory Development Applicable to Material Accounting.

2. After (Presumed) Successful Development, NRC Take Steps to implement Game Theory as a Decision-Making Tool.

3. Current Regulations Relative to Action Criteria (as Suggested by NUREG-0450) for Plant ids Not Be Changed until Sufficient Experience with Game Theory or Other Benefit / Cost Analysis Will Have Been Accumulated.

4. Research be Initiated on Use of Game Theory to Multiperiod Data and Use of Repeated Games of Incomplete Information for Application to Nuclear Material Accounting.

5. Research and Development be Conducted to Consider the Utility of Game Theory in Potentially Applicable Safeguards Areas Other than Nuclear i

Material Accounting.

Abstncted from: pp 41 12 NUREG/CR-0950

[1 liliti.10lil(Al'IlY Davis, Morton D (1970), Game. Ilicony, Basic Ilooks Dresher, M. (1961), Cxunes of Staategy, Theoty and Appfications, Prentice-Ilall l

[

Jaech, J. L. (1973). Statiatical Methods in Nuefcast Materlat Contaof, U. S. Atomic Energy Consnission j

publication TID-26298 I

~

Var 1 \\n, S. (1959), Ifathenstical Me thods and Theo%st in Games, Protetanuting.and Tecnomics, Vol.1 j

Ashll;on Wesley f

Lindgren. D. W. (1976), Statisticar Thevay, McMillan Luce, R. D., and Ral f fa, 11. (1957), Gamc4 and l'ccisions, Wiley Page, R. G. (December 6,1974), Letter to Gentlemen, Material and Plant Protection Branch, U. S.

Atomic Energy Coninission Page, R. G. (1975), " Features and requirements of the U. S. Nuclear Regulatory Consnission." Presentation to International Symposium on the Safeguarding of Nucicar Material in Vienna, Austria.

U. S. Atomic Energy Commission:

l

- Regulatory Guide 5.18 (January,1974), " Limit of Error and Principles of Calculation in i

Nuclear Kiterial Control" U. S. Nuclear Regulatory Consnission:

- Code of Fedeant Regufations (1979), Title 10. " Energy"

- NUREG-0290 (June,1977), "A Study of Nuclear Material Accounting," Vol. I-III

- NUREG-0450 (April,1978), " Report of the Material Control and Material Accounting Task Force,"

Vol. 1-IV i

- NUREG/CR-0490 (November,1978), "The Application of Game Theory to Nuclear Material Accounting"

)

(

- NUREG/CR-0950 (August,1979), "A Review of the Application of Strategic Analysis to Material l

Accounting" l-

i t

4 BRIEFING ON TWO APPROACHES TO THE TREATMENT OF INVENTORY DIFFERENCES IN NUCLEAR MATERIAL ACCOUNTING March 20,1980 PURPOSE The purpose of this briefing is to provide the NRC Commissioners with information on two approaches to the treatment of inventory differences in nuclear material accounting. This information should be useful in providing a perspective on statements concerning loss or diversion of special nuclear material.

4 SCOPE The briefing will present an overview of the Classical (Statistical Hypothesis Testing) Approach for evaluating inventory difference data.

It l

will then present the Game Theory (Strategic Analysis) Approach and provide a comparison of the capabilities and the limitations of the approaches.

~ d BPIEFING OUTLINE I.

Introduction A.

Purposes B.

Background

II.

Two Basic Concepts A.

Inventory Difference (ID) i B.

Limit of Error of Inventory Difference (LEID)

III.

Classical (Statistical Hypothesis Testing) Approach 4

IV.

Game Theory (Strategic Analysis) Approach V.

Comparisons and Recommendations 2

1 s

t i

i I

l l-l

.