ML20196H722
| ML20196H722 | |
| Person / Time | |
|---|---|
| Issue date: | 06/05/1998 |
| From: | Greeves J NRC OFFICE OF NUCLEAR MATERIAL SAFETY & SAFEGUARDS (NMSS) |
| To: | Vaughan R COALITION ON WEST VALLEY NUCLEAR WASTES |
| Shared Package | |
| ML20196H668 | List: |
| References | |
| REF-PROJ-M-32 NUDOCS 9907060434 | |
| Download: ML20196H722 (2) | |
Text
O.
W s
f
%'9%
UNITED STATES c
.p j ~
WASHINGTON, D.C. 20086-4001 NUCLEAR REGULATORY COMMISSION
't
.,s*
June 5, 1998 l
Mr. Raymond ~ Vaughan l
Coalition On M: Valley Nuclear Wastes Sharp Street East Concord, NY 14055
Dear Mr. Vaughan:
I am responding to your letter of March 21,1998, regarding the West Valley site. Thank you for
_ providing the technical information on the emplacement of steel pilings through the Lavery Till beneath the main plant and the seismic / tectonic issues. This information will be considered during the U.S. Nuclear Regulatory Commission's (NRC) technical review of any proposed remediation once a plan is provided that addresses the affected facilities.
Your letter also comments on the material provided by Jim Haminelman of Science Applications
)
International Corporation, which were hard copies of a slide presentation given to the NRC on November 17,1997. This material, entitled " Application of NRC Criteria to West Valley Draft EIS Alternatives," was presented to NRC and simultaneously discussed in a conference call with the U. S. Department of Energy (DOE) and the New York State Energy Research and Development Administration (NYSERDA) as part of NRC's informal review and consultative role under the West Valley Demonstration Project (WVDP) Act. This presentation and discussion l
were arranged to give NRC staff an interpretation of the potentialimpact of applying existing NRC waste disposal and decommissioning regulations and policies at the West Valley site.
NRC staff asked questions on the material to help us understand the points being made, but we did not offer any judgment of it. No transcript or recording was made of the discussion.
1 Since West Valley specific decommissioning criteria have not been determined, the discussion was on existing NRC regulations and policies. It is the NRC staff's understanding that the subject matter covered will be addressed as part of a performance assessment and cost-benefit analysis to be submitted by the DOE in the future for NRC staff review and written comments.
As you point out, part of the material dealt with " preliminary performance assessment results (not in the draft EIS)." Requests for the preliminary performance assessment should be directed to the DOE office at West Val!ey.
Although the staff placed the discussion materials in the NRC's Public Document Room, the discussion did not take place in a public meeting. It is the NRC staff's longstanding practice to i
provide the public with the fullest information practicable on its activities and to conduct its l
business in an open manner as set forth in the Commission's Policy Statement on Staff Meetings Open to the Public (59 Federal Register 48340). However, the Memorandum of Understanding between NRC and DOE on West Valley (46 EB 56960), which predates the policy statement, does not apply NRC's open meeting policy to meetings between NRC and l
DOE in connection with the WVDP. A decision to apply the policy in the future would not, in my i
judgment, be inconsistent with the principle that NRC review and consultation are to be conducted informally, and are not to include, or require, any formal NRC procedure or action.
Therefore, it is my intention to obtain DOE's agreement to apply the policy statement to future j
NRC staff meetings with DOE.
9907060434 990528 PDR PROJ M--33 PDR
)
~
. Mr. R. Vaughan -,.
Public notice of such meetings will be given in accordance with the policy statement, and the public will be able to obtain a schedule of staff meetings on West Valley through telephone recording (800-952-9674), electronic bulletin board (800-952-9676), or World Wide Web (http://www.nrc. gov /NRC/PUBLIC/ meet.html).
Sincerely, John T. Greeves, Director Division of Waste Management Office of Nuclear Material Safety and Safeguards cc: - B. Mazurowski, DOE R. Tormey, DOE P. Piciulo, NYSERDA H. Brodie, NYSERDA P. Merges, NYSDEC CTF i
1 COALITION ON WEST VALLEY NUCLEAR WASTES Sharp Street East Concord, NY 14055 *(716) 941-3168 J
l September 3, 1998 John T. Greeves, Director Division of Waste Management Office.of Nuclear Material Safety & Safeguards U.S. Nuclear Regulatory Commission Washington, D.C.
20555 D, ear Mr. Greeves:
i Thank you for your letter of June 5, including the informa-tion you provided on Jim Hammelman's presentation to NRC on November 17, 1997.
With regard to Hammelman's presentation, you advised us that " Requests for the preliminary performance assess-ment should be directed to the DOE office at West Valley."
We
-have requested this assessment from DOE (see my enclosed letter to Dan Sullivan dated July 28) but have not yet received it.
Our understanding of the assessment is based on what Hammelman in-i cluded in his presentation to NRC on November 17, 1997.
Presumably NRC recognizes why we are interested in Jim Hammelman's preliminary performance assessment.
Lest there be any doubt, I.will discuss the matter in this letter.
It is a matter that is of interest to us and should be of interest to any agency that has a regulatory role at West Valley.
Specifically, Jim Hammelman's preliminary performance as-sessment shows the extreme gepsitivity of human health impacts to e
assumptions made about the site and facility.
This degree of sensitivity should serve as a red flag to any regulatory agency.
In other words, predictions of human health impacts from the West Valley site are highly dependent on the assumptions made.
l Such sensitive linkages between assumptions and impacts indicate a need for caution in carrying out performance assessments.
We believe NRC would concur.
If not, please let us know.
The aforementioned preliminary performance assessment is just one example of the very sensitive dependence of human health impacts on assumptions made about the West Valley site and facil-ity.
Others can be found, for example, in Table 3 of SAIC's
" Revised Performance Assessment for the High-Level Waste Tanks and Vitrification Facility," sent to Dan Sullivan (DOE) by Jim Hammelman (SAIC) on January 23, 1998.
Table 3 of the " Revised Performance Assessment" shows huge reductions in predicted impacts as a result of changes in assump-1
- mn.
.en h
() W O 0 0 {
~
_' 1 tions.
Specifically, Table 3 showspredictedpeakannualdoses g
be '. g revised downward from 6.4 x 10 mrem to 0.14 mrem (!), and mrem to 40 mrem (!), and from 1.4 x 105 mrem to 21 J
f rwi 8. 9 x 107 mrem (!), etc., as a result of " revised designs."
According to the accompanying text (pp. 7-8), " Table 3 presents the doses to the agricultural / residential intruder under Alternatives IIIA and IIIB....
With the revised isolation design for the HLW tanks, the peak annual intruder dose from the tanks would be six to eight orders of magnitude lower than the dose calculated assuming the Draft EIS design were implemented....
These results show that the revised designs for the HLW tanks and the vitrification facility would effectively retain the short-lived radionuclides...."
Any intelligent person, and any responsible regulatory agency, must raise questions such as:
How effectively are the short-lived radionuclides retained?
Is the level of confidence 100%?
Have all plausible modes of failure been recognized and
. included in the analysis?
In the face of such enormous dependencies of human health impacts on the assumptions made, it is crucial that steps be taken to ensure a properly prudent methodology.
For example, I believe that performance assessments at other sites use probabil-ity-weighted methods to deal with uncertainty, yet I have seen no application of such methods at West Valley.
Why not?
Has NRC raised this question with DOE?
In NRC's view, can confidence levels of 100% be assigned to West Valley " revised designs" that reduce predicted doses by six to eight orders of magnitude?
In our view, binary choices between " good" and " bad" designs are not protective, given the reliance on naive assumptions and the enormous differences in predicted doses.
A more prudent approach would seek to identify plausible modes of failure and their estimated probabilities.
The task of developing defensible predictions of impacts to j
human health from the West Valley site is a major responsibility.
In our view, it is a responsibility shared by at least three
)
agencies:
If you disagree, please let us 1
know.
In our view, each of the agencies hr >: a responsibility to ensure that the methodology for predicting.mpacts is defensible and prudent.
]
In this regard, I enclose for your information a paper by Oreskes et al. that was published in Science in 1994 ("Verifica-tion, Validation, and Confirmation of Numerical Models in the Earth Sciences").
It cites some NRC sources and draws examples from nuclear waste disposal, and, more generally, refers to various things that seem relevant here:
" Government regulators and agencies may be required by law to establish the trustworthi-ness of models used to determine policy or to attest to public safety...."
(p. 641)
"(A] match between model results and present observations is no guarantee that future conditions will 2
j
t be similar, because natural systems are dynamic and may change in unanticipated ways."
(p. 643)
"Models can also be used for
. sensitivity analysis--for. exploring 'what-if' questions--thereby illuminating which aspects of the system are most in need of i
further study, and where more empirical data are most needed."
(p. 644)
"In areas where public policy and public safety are at stake, the burden is on the modeler to demonstrate the degree of correspondence between the model and the material world it seeks to represent and to delineate the limits of that correspondence."
(p.644)
We hope the above discussion is useful in relation to NRC's role at West Valley.
If you believe we are misrepresenting NRC's j
role, please let us know.
1 In addition to the relatively technical material covered ir.
this letter, we need to review some other aspects of NRC's role at West Valley.
As time permits, I will write again.
Sincerely, A
Raymond C.
Va han cc:
D.
Sullivan, DOE D. Miller, NYSERDA P. Merges, DEC T. Attridge, CTF 3
{
1 I
J
e COALITION ON WEST VAILEY NUCLEAR WASTES 3
Sharp Street East Concord, NY 14055 -(716) 941-3168 l
l July 28, 1998 Dan W..Sullivan, Project Manager U.S.
Department of Energy West Valley Demonstration Project
- P.O. Box 191 West Valley, N.Y.
'14171
Dear Dan:
l We need to discuss a few issues that relate to the EIS.
In
'.that context we-are enclosing a copy of the stipulation of Com-promise' Settlement which, in Slo, specifies the conditions under which we may obtain "those documents requested with reasonable specificity which are reasonably related to the preparation of the EIS..."
First, we request succortina information and documentation on Jim Hammelman's November 17, 1997, presentation entitled
" Application of NRC Criteria to West Valley Draft EIS Alterna-tives."
A copy of Jim's presentation is enclosed.
(You and I have. talked about this item before.
You have tended to say it contains numbers pulled out of thin air, or something of that sort, implying that no supporting information or documentation exists.
I do not believe this is the case.
Please see the second paragraph of the enclosed letter dated June 5 wherein John T.
Greeves of NRC states that Hammelman's." Application of NRC Criteria to West Valley Draft EIS Alternatives" was " presented to NRC and simultaneously discussed in a conference call with (DOE]
and (NYSERDA)'as part of NRC's informal review and consultative role under the West Valley Demonstration Project (WVDP) Act.
.This presentation and discussion were arranged to give NRC staff an interpretation of the potential impact of applying existing NRC waste disposal and decommissioning regulations and policies at-the West Valley site.
NRC staff asked questions on the mate-rial to help us understand the points being made, but we did not offer any judgment of it...."
I find it difficult to believe that Jim Hammelman would make such a presentation, engage in
' discussion, and answer questions, unless he had some basis for his presentation.)
Jim Hammelman's " Application of NRC Criteria to West Valley
. Draft EIS Alternatives" is especially important because it shows
.thg, extreme sensitivity of West Valley performance assessment results to the assumptions made.
See particularly Jim's page 14.
We believe you understand the'importance of sensitivity analyses I
which are intended to identify and quantify sensitive linkages between various' input conditions and results.
It is especially important in'this case because the results show very high human 1
r
~!
.a exposures (expected peak annual radiation doses exceeding 10 to 30 rems).
We need to see how these results are traceable back to the input parameters or assumptions.
Thus, to repeat, our first request is for supporting infor-mation and documentation relating to Jim Hammelman's " Application of NRC Criteria to West Valley Draft EIS Alternatives."
We believe this is material covered by $10 of the Stipulation and we therefore make this request under 510 of the Stipulation.
Please let us know when and how the requested material will be avail-able.
Second, we request a copy of the Statement of Work to which Jim Hammelman refers in his January 23 letter to you (copy en-closed).
We are primarily interested at this point in Task 11G of the Statement of Work.
If Task 11G is reasonably self-explanatory, we would be satisfied for t time being with a copy of 11G alone (but may request the entir, Statement of Work at a later date).
We regard the Statement of Work as material covered by $10 of the Stipulation and we therefore make this request under $10 of the Stipulation.
Third, we need further information on the reassessment of North Plateau erosion rates that you mentioned at the July 7 WVDP Quarterly Public Meeting.
If we understood you correctly, SAIC is reassessing North Plateau erosion rates.
We request (under 510 of-the Stipulation) detailed information gn th_e methodoloav that SAIC is using and ligy it comoares to the methodoloav already used in deriving the erosion rates presented in the DEIS.
We want to be sure the methodology is consistent.
Fourth, we want to remind DOE that the duration of the public comment period following issuance of the Supplemental DEIS is unresolved and will depend in part on the amount of new mate-rial introduced in the Supplemental DEIS.
Using erosion rates as an example, we are likely to need a longer public comment period if new erosion studies by SAIC produce results that are methodo-logically flawed and/or grossly different from the erosion rates already presented in the DEIS.
In such an event, we would need a j
relatively long comment period to give us time to consult with our own erosion experts and possibly to seek permission for our own onsite measurements (by ourselves or our experts).
As you know, the Stipulation (54) calls for a six month public comment period but does not specify whether a six-month period would, or would not, apply to a Supplemental DEIS.
We have already indicated 1) that the question is unresolved and will require mutual agreement and 2) that we are willing to be open-minded and accommodating with regard to the overall sched-ule.
Nevertheless, we also wish to preserve our own rights to have adequate time to review and respond to a potentially large amount of new material in the Supplemental DEIS.
Somehow we need to resolve this, possibly by having the Supplemental DEIS be an open, publicly accessible document while it is being prepared, 1
2
r s'
with no major last-minute changes.
This is just an idea.
We welcome your thoughts on how we can resolve the length of the j
comment period.
In the meantime, DOE should make no presumption l
about the length of the comment period and should be aware.of our interest in having the length of the comment period be propor-i tionate to the amount of new material that DOE will be introduc-ing in the Supplemental DEIS.
j We look forwa ed to hearing from you on these several items.
Sincerely, J
Mbd Raymond C. Vaughan cc:
D. Miller, NYSERDA J. Greeves, NRC 3
3AR.TICLE gi Verification, Validation, and Confirmation of C
Numerical Models in the Earth Sciences Naomi Oreskes,* Kristin Shrader-Frechette, Kenneth Belitz l
Verification and validation of numerical models of natural systems is impossible. This is puter program may be verifiable (12).
because natural systems are never closed and because model results are always non-Mathematical components are subject to unique. Models can be confirmed by the demonstration of agreemera between observation verification because they are part of closed and prediction, but confirmation is irmerently partial. Complete confirmation is logically systems that include claims that are always precluded by the fallacy of affirming the consequent and by incomplete access to natural true as a function of the meanings assigned phenomena. Models can only be evaluated in relative terms, and their predictive value is to the specific symbols used to express them always open to question. The primary value of models is heuristic.
(13). However, the modeh that use these I
components are never closed systems. One h
reason t ey are never closed is that models require $ sput parameters that are incom-In recent yean, there has been a dramatic Verification: The Problem pletely k own. For example, hydrogeologi-
)
increase in the use of numerical simulation of " Truth"
)
modeh in the earth sciences as a means to cal moth require distributed parameten
{
such as hydraulic conductivity, porosity,
{
evaluate large-scale or complex physical pro.
The word venfy (from Latin, vena, meanmg storage coefficient, and dispersivity, which t
cesses. In some cases, the predictions gener-true) means an assertion or establishment of are always characteri:ed by incomplete data ated by these models are considered as a basis truth (9). To say that a model is verified is to sets. Geochemical models require thermo-for public policy decisions: Global circula-say that its truth has been demonstrated, dynamic and kinetic data that are incom-tion modeh are being used to predict the which implies its reliability as a basis for pletely or only approximately known. In-behevior of the Earth's climate in response decision-makmg. However, it is impossible completeness is also introduced when con-to increased COs concentrations; resource to demonstrate the truth of any proposition, tinuum theory is used to represent natural estimation modeh are being used to predict except in a closed system. This conclusion systems. Continuum mechanics necessarily petroleum reserves in ecologically sensitive derives direcdy from the laws of sfmbolic entails a loss of information at the scale areas; and hydrological and geochemical logic. Given a proposition of the form "p" lower than the averaging scale. For exam-1 models are being used to predict the behav-entails "q," we know that if"p"is true, then ple, the Darcian velocity of a porous medi-ior of toxic and radioactive contaminants in "q"is true if and only if the system that this um is never identical to the velocity struc-proposed waste disposal sites. Govemment formalism represents is closed.
ture at the pore scale. Finer scale structure regulaton and agencies may be required by For example, I say, "Ifit rains tomorrow, and process are lost from consideration, a
)
1;w to establish the trusr*dir.w of mod-I will stay home and revise this paper." The loss that is inherent in the continuum els used to determine policy or to attest to next day it rains, but you' fmd that I am not mechanics approach.
public safety (1, 2); scientists may wish to home. Your verification has failed. You Another problem arises from the scal-test the veracity of models used in their conclude that my origmal statement was ing-up of nonadditive properties. The con-investigations. As a result, the notion has false. But in fact, it was my intention to struction of a numerical simulation model emerged that numerical models can be "ver-stay home and work on my paper. The of a ground-water flow system involves the ified" or " validated," and teduuques have formulation was a true statement of my specification of input parameters at some been developed for this purpose (1, 3-5).
intent. Later, you (md that I left the house chosen scale. Typically, the scale of the Clauns about verification,and vhrinn of because my mother died, and you realize model elements is on the order of meters, model results are now routinely found in that my ongmal formulation was not false, tens of meten, or kilometen. In contrast, published literature (6).
but incomplete. It did not allow for the the scale on which input parameters are i
Are claims of validity and verity of numer-possibility of extenuating circumstances measured is typically much smaller, and the ical models leginmate (2, 7)? In this article, (10). Your attempt at verification failed relation between those measurements and we examme the phi 1~nnhal basis of the because the system was not closed.
larger scale model parameters is always un-terms "venfication" and " validation" as ap-This example is trivial, but even an certain and generally unknown. In some plied to numencal simulation models in the apparently trivial proposition can be part of cases, it is possible to obtain input data at earth sciences, using examples from hydrology a complex open system. Indeed, it is didi-the scale chosen by the modeler for the and geochemistry. Because demand for the cult to come up with verbal examples of model elements (for example, pump tests),
assessment of accuracy in numerical modeling closed systems because only purely formal but this is not often done, for practical is reest evident at the interface between pub.
logical structures, such as proofs in symbolic reasons. Even when such measurements are lic policy and scientific usage, we focus on logic and mathematics, can be shown to available, they are never available for all examples relevant to policy (8). The princi-represent closed systems. Purely formal model elements (14).
j ples illustrated, however, are genenc.
structures are verifiable because they can be Another reason hydrological and geo-
}
proved by symbolic manipulations, and the chemical models are never closed systems is N Oreskes is o the Department of Eartn Sciences and meaning of these symboh is fixed and not that the observation and measurement of the Department of History, Dartmouth Coneos. Han-contingent on empirically based input pa-both independent and dependent variables
,',"4 g,H rameten (II).
are laden with inferences and assumptions.
- N r
n e an pa. FL 3362C. K. Beutz is in the Depanmera of Eanh Numerical models may contain closed For example, a common assumptton in scences. Danmoum coneoe, Hanover, NH 03755-mathematical components that may be ver-many geochemical models of water-rock
'To whom conosponcence shouio ce addressed.
iftable, just as an algorithm within a com-interaction is that observable mineral a.,-
SCIENCE
- VOL 263 + 4 FEBRtJARY 1994 641 x
semblages achieve equilibrium with a mod-
'there is no way to know that this cancella-nuclear waste repository at Yucca Moun.
eled fluid phase. Because relevant kinetic tion has occurred. A faulty model may ap-tain, Nevada, Davis and cu-workers (I)
- data are frequently unavailable, kinetic ef-pear to be correct. Hence, venfication is suggest that "[tlhe most common method of fects are assumed to be negligible (15). But only possible in closed systems in which all validation involves a companson of the
}
many rocks contain evidence of disequilibn-th components of the system are estabhshed measured response from in situ testing, lab J
um on some scale, and the degree of disequi-independently and are known to be correct.
testing, or natural analogs with the results librium and its relation to kinetic controls In its application to models of natural sys-of computational models that embody the can rarely, if ever, be quantified. To attempt tems, the term verification is highly mislead-model assumptions that are being tested" to do so would necessanly involve further ing. It sug: ests a demonstration of proof that (31). But the agreement between any of inferences and assumpions. Similarly, the is simply not accessible (26).
these measures and numerical output in no absence of complete thermodynamic data for way demonstrates that the model that pro-mineral solid solutions commonly forces Validation duced the output is an accurate representa-modelers to treat minerals as ideal en:1-tion of the real system. Validation in this members, even when this assumption is in contrast to the term verification, the term context signifies consistency within a sys-known to be erroneous on some level. Mea-validation does not necessarily denote an tem or between systems. Such consistency surement of the chemical cenposition of a estab3shment of truth (although truth is not entails nothing about the reliability of the mineral phase to estimate the activities of precluded). Rather, it denotes the establish-system in representing natural phenomena.
chemical components within it requires in-ment of legitimacy, typically given in terms strumentation with built-in assumptions of contracts, arguments, and methods (27).
" Verification" of Numerical j
about such factors as interference effects and A valid contract is one that has not been Solutions matrix corrections. What we call data are nullified by action or inaction. A valid inference-laden signifiers of natural phenom-argument is one that does not contain obvi-Some workers would take as a starttug point enn to which we have incomplete access ous errors oflogic. By analogy, a model that for their definition of terminology the ana-(16). Many inferences and assumptions can does not contain known or detectable flaws lytical solution to a boundary value or be justified on the basis of experience (and and is intemally consistent can be said to be initial value problem. In this context, they sometimes uncertainties can be estimated),
valid. Therefore, the term valid might be may compare a numerical solution with an but the degree to which our assumptions useful for assertions about a generic comput-analytical one to demonstrate that the two
{
hold in any new study can never be estab-er code but is clearly misleading if used to match over a particular range of conditions lished a priori. The embedded assumptions refer to actual model results in any particular under consideration. This practice is often thus render the system open.
reali:ation (28). Model results may or may referred to as verification (4, pp. 7-8; 32).
The additional assumptions, inferences, not be valid, depending on the quality and The companson of numerical with analyt-and input parameters required to make a quantity of the input parameters and the ical solutions is a critical step in code devel-model work are known as " auxiliary hypoth-accuracy of the auxiliary hypotheses.
opment; the failure of a numerical code to q
eses" (17). He problem of deductive verifi-Common practice is not consistent with reproduce an analytical solution may certainly cation is that if the verification fails, there is this restricted sense of the term. Konikow be cause for concem. However, the congru-often no simple way to know whether the and Bredehoeft (2) have shown that the ence between a numerical and an analyucal pnncipal hypothesis or some auxiliary hy-term validation is commonly used in at least solution entails nothing about the correspon-pothesis is at fault. If we compare a result two diferent senses, both erroneous. In dence of either one to material reality. Fur-predicted by a model with observational data some cases, validation is used interchange-thermore, even if a numerical solution can be and the comparison is unfavorable, then we ably with verification to indicate that model said to be venfied in the realm of the analyt-know that something is wrong, and we may predictions are consistent with observational ical solution, in the extension of the numer-or may not be able to determine what it is data. Thus, modelers misleadingly imply ical solution beyond the range and realm of (18). Typically, we continue to work on the that valid. tion and verification are synony-the analytical solution (for example, time, model until we achieve a fit (19). But if a mous, and that validation establishes the space, and parameter distnbution), the nu-match between the model result and obser-veracity of the model, in other cases, the merical code would no longer be venH.
varional data is obtained, then we have, term validation is used even more mislead-Indeed, the raison d'etre of numerical model-ironically, a worse dilemma. Mate than one ingly to suggest that the model is an accurate ing is to go beyond the range of available model construction can produce the same representation of physical reality. The impli-analyncal solutions. Therefore, in applica-output. His situation is referred to by sci-cation is that validated models tell us how tion, numerical models cannot be venfied.
entists as nonuniqueness and by philosophers the world really is. For example, the U.S.
The practice of comparing numerical and as underdetermination (20, 21). Model results Department of Energy defines validation as analytical solutions is best referred to as are always underdetermined by the available the determination "that the code or model bench-marking. The advantage of this tem >--
data. Two or more constructions that produce indeed reflects the behavior of the real with its cultural association with geodetic the same results may be said to be empirically world" (29). Similarly, the Intemational practice-is that it denotes a reference to an equivalent (22). If two theories (or model Atomic Energy Agency has defined a vali-accepted standa-d whose absolute value can reah:ations) are empirically equivalent, then dated model as one that provides "a good never be known (33).
there is no way to choose between them other representation of the actual processes occur-than to invoke extraevidential considerations ring m a real system" (30). For all the Calibration of Numerical Models like symmetry, simplicity, and elegance, or reasons discussed above, the establishment personal, political, or metaphysical prefer-that a model accurately represents the "ac-In the earth sciences, the modeler is com-ences (19, 23-.25).
tual processes occurring in a real system" is monly faced with the mverse problem: The A subset of the problem of nonunique-not even a theoretical possibility.
distnbution of the dependent variable (for ness is that two or more errors in auxiliary How have scientists attempted to dem-example, the hydraulic head) is the most hypotheses may cancel each other out.
onstrate that a model reflects the behavior well knowri aspect of the system the distri-Whether our assumptions are reasonable is of the real world? In the Performance As-bution of the independent vanable is the not the issue at stake. The issue is that often sessment Plan for the proposed high-level least well known. The process of tuning the t'OL. 263
- 4 FEBRUARY 1994 642 SCIENCE
, p, ma ca.urmm2 w,w usm - aa u mwu.%.
wm uinii model-t. hat is, the manipulation of the Confirmation higNy complex scienti6c hypothesis. Con-
~ '
independent variables to obtain a match 6rmation theory requires us to support nu-S between the observed and simulated distri-If the predicted distribution of dependent merical simulation results with other kinds bution or distributions of a dependent vari-data in a numerical model matches obser-of scienti6c observations and to realize that able or variables-is known as calibration.
varional data, either in the 6 eld or labora-ven6 cation is impossible.
Some hydrologists have suggested a two.
tory, then the modeler may be tempted to step calibration scheme in which the avail-claim that the model was veri 6ed. To do so Numerical Models and Public Policy able dependent data set is divided into two would be to commit a logical fallacy, the parts. In the 6rst step, the independent fallacy of " affirming the consequent." Re-Testing hmotheses is normal scienti6c prac-parameters of the model are adjusted to call our proposition, "Ifit rains tomorrow, I tice, but mcdel evaluation takes on an added reproduce the 6rst part of the data.Then in will stay home and revise this paper." This dimension when public policy is at stake.
the second step the model is run and the time, you 6nd that I am home and busily Numerical models are increasingly being used results are compared with the second part of working on my paper. Therefore you con-in the public arena, in some cases to justify the data. In this scheme, the first step is clude that it is raining. Clearly, this is an highly courovenial decisions. Therefore, the labeled " calibration," and the second step is example of faulty logic. The wenner might implication of truth is a serious matter (43).
labeled " veri 6 cation." If the comparison is be glorious, but I decided that this paper The terms veri 6 cation and validation are now favorable, then the model is said to be was important enough to work on in spite of being used by scientists in ways that are "ven6ed" (3, p. I10; 4, p. 253). The use of the beautiful weather. To claim that a contradictory and misleading. In the earth the term veri 6 cation in this context is proposition (or model) is veri 6ed because sciences-hydrology, geochemistry, meteo-highly misleading, for all the reasons given empirical data match a predicted outcome rology, and oceanography-numerical models above. A match between predicted and is to commit the fallacy of affirming the always represent complex open systems in obtained output does not verify an open consequent. If a model fails te reproduce which the operative processes are incomplete.
system. Furthermore, models almost invari-observed data, then we know that the ly understood and the required empirical in-ably need additional tuning during the so-model is faulty in some way, but the reverse put data are incompletely known. Such mod-called veri 6 cation phase (3, p.110). That is never the case (36).
els can never be ven6ed. No doubt the same is, the comparison is typically unfavorable.
This conclusion, which derives strictly may be said of many biological, economic, and further adjustments to the independent from logic, may seem troubling given how and arri6cial intelligence models.
parameters have to be made. This limita-difficult it can be to make a model or What typically passes for validation and tion indicates that the so-called veri 6 cation develop a hypothesis that reproduces ob.
veri 6 cation is at best confirmation, with all is a failure. The second step is merely a part served data. To account for this discrepan-the limitations that this term suggests. Con-of the calibration.
cy, philosophers have developed a theory of firmation is only possible to the extent that Given the fundamental problems of ver-confirmation, founded on the notion of we have access to natural phenomena, but 16 cation, Bas van Fraassen (22) has argued science as a hypothetico-deductive activity.
complete access is never possible,' not in the that the goal of scienti6c theories is not In this view, science requires that empirical present and certainly not in the future. Ifit truth (because that is unobtainable) but observations be framed as deductive conse-were, it would obviate the need for model-empirical adequacy. Using van Fraassen's quences of a general theory or scientific law ing. The central problem with the language terminology. one could say that a calibrated (37). If these observations can be shown to of validation and verification is that it im-model is empirically adequate. However, be true, then the theory or law is " con.
plies an either-or situation. In practice, few the admission that calibrated models invari-6rmed" by those observations and remains (if any) models are entirely con 6rmed by ably need " additional refinements" (3, p.
in contention for truth (17). The greater observational data, and few are entirely re.
110) suggests that the empirical adequacy of the number and diversity of confirming futed. Typically, some data do agree with numerical models is forced. The availability observations, the more probable it is that predictions and some do not. Con 6rmation of more data requires more adjustments.
the conceptualization embodied in the is a matter.of degree. It is always inherently This necessity has serious consequences for model is not flawed (38). But confirming partial. Furthermore, both verify and vali-the use of any cahbrated model (or group of observations do not demonstrate the verac-date are affirmative terms: They encourage models) for predictive purposes, such as to icy of a model or hypothesis, they only the modeler to claim a positive result (44).
justify the long term safety of a proposed support its probability (39, 40).
And in many cases, a positive result is nuclear or toxic waste disposal site. Consid.
Laboratory tests, in situ tests, and the presupposed. For example, the first step of er the difference between stating that a analysis of natural analogs are all forms of validation has been defined by one group of model is "venfied" and stating that it has model confirmation. But no matter how scientists as developing "a strategy for dem-
" forced empirical adequacy" (34).
many confirming observations we have, any onstrating [ regulatory] compliance" (1, 45).
Finally, even if a model result is consis-conclusion drawn from them is still an Such affirmative language is a roadblock to tent with present and past observational example of the fallacy of affirming the further scrutiny.
data, there is no guarantee that the model consequent. Therefore, no general empiri.
A neutral language is needed for the will perform at an equal level when used to cal proposition about the natural world can evaluation of model performance. A model predict the future. Fint, there may be small ever be certain. No matter how much data can certainly perform wil with respect to enon in input data that do not impact the we have, there will always be the possibility observational data, in which case one can ne of the model under the time frame for that more than one theory can explain the speak of the precision and accuracy of the which historical data are available, but available observations (41). And there will 6t. Judgmental terms such as excellent, which, when extrapolated over much larger always remain the prospect that future ob-good, fair, and poor are useful becaun they nme frarnes, do generate significant devia.
servations may call the theory into question invite, rather than discourage, contextual tions. Second, a match between model (42). We are left with the conclusion that definition. Legitimately, all we can talk results and present observations is no guar-we can never venfy a scientific hypothesis about is the relative performance of a model antee that future conditions will be similar, of any kind. The more complex the hypoth.
with respect to observational data, other because natural systems are dynamic and esis, the more obvious this conclusion be.
models of the same site, and our own may change in unanticipated ways (35).
comes. Numerical models are a form of expectations based on theoretical precon-SCIENCE
- VOL 263
- 4 FEBRUARY 1994 643 e
n.
r,
ceptions and experience of modeling other 5537 (Sandia Natonal Laboramnes. Albuquer and cannot eerebre be omfimed or refuted by Que NM,1991). These workers cite the case of any fact of expsnence" [A. J. Ayer. Language.
stres. None o se can g g Ohio versus EPA, n wNch a federal appeals court Tn#r and toge (Dover. New York,1946. repnnt-in absolute terms.
found the state responsible for testing computer ed.1952), p.16). Analyte statements are wnfL-models used to set emssion tirnets on electne ab6e bemuse "they do not make any assertion Then What Good Are Models?
power pla,nts [U.S. Law week 54. 2494 (1986)]. about the empincal world, but sarply remrd our The cou,1 ound tne government naue becaun a determnmion e use symbus n a certain fasnion-had made no effort to determee the reliabihty of (ite, p 31). Also see A J. Ayer. Ed.. Lopeal Models can corroborate a hypothesis by 8* nodd used. G#wn that the legal nEessity of Positmsm (Free Press. New York.1959).
rnodel testing has been established, what claims
- 14. If it were technically and ow,ui.,My possible to offering evidence to strengthen what may are iustifie on em basis e such tests?
undstaka exnaustiw swnpung on em scme e be aheady partly established through other
- 2. L F. Konikow and J. O. Bredehoeft Adv. Water model elements. then we would run the nsk of means. Models can elucidate discrepancies Resour. 15. 75 (1992L modifyng the connnuum properties we are tryng in other models. Models can be abo be used
- 3. H. F. Wang and M. P. Anderson, introductson to to measure The insertion of closety spaced dnil Groundwater Modeling: Anrre Dreerence and 5-holes into a porous rnedeum may change the tot sensitivity analysts-for exploring,what n,te Bement Methods (Freemen, San Francisco, hydraulic properties of mat medium. Furthermore.
if" questions-thereby illuminating which 1982b the dependent vanables of the systen@aulic asE# cts of the system are most in need of
- 4. M. P. Anderson and W. M. Woessner. Appled head, solute Concentration, and mneral assem.
Groundwater Modelsng: Simulatson of Flow and blages-.cannot be obtained at the model e6e.
further study, and where more empirical ao.ct,w transporr @cademe Pren, New York, ment scat To know these pwameters pwfectfy data are most needed. Thus, the primatY 1992).
mould be to one out ew regon being mode 6ed value of models is heuristic: Models are
- 5. The voume of Adv. water Resour. 15 (1992) is This pont is also made by C. F. Tsang [Grotrid-entirety dedicated to tne discussa of validatan water 29, 825 (1991)).
representations, useful for guiding further and venficaDon of computer rnodels.
- 15. Re:endy geochemistJ have made considerable study but not susceptible to proof.
- e. In our expenence such claims are partcularty progreu on the kinetos of mnwal reactions, but The idea of model as representation has abundant in cases in wbch an otMous public the pcent remains the same: In the absence of k
led the philosopher Nancy Cartwnght to P'*Y'""'""'"*'"""*'"**""'-
"d*
"' d*
"""Y '" d*'"' "e"s""a"se*n*c'e o'f rounding the prnr=d nign-level nuclear repoe-netics can be ignored. Smtarty, o b
the claim that models are "a work of (1c-Itory at Yucca Mountain, Nevada Examples in-complete thermodyname data. trode 4ers neces-tion" (46). In her words, "some properties ciude N. Hayden, "8.Jn6v. NNMSI flow sarily extend available data beyond the ranpe of ascribed to objects in the model will be and transport mdes: cow 1 Reams" (Sandia iaboramry in6nnation. To casi ris bad moenno National Laboratones. Albuquerque, NM,1985).
is to mas tre pont: Data are never complete, genuine properties of the objects modeled, K. Stephens et al., " Methodologies for assessing inferences are always required, and we can never but others will be merely properties of iong-tem weaa of histwievel radoactr.
tm cenan wncn inferences are omd and wnien convenience." Her account, which is no waste packaget" NUREG CR-4477, ATR85 mes are not as gmd.
(581041) 1 ND (U.S. Nuclear Regulatory Com-
- 16. An otmous example from omvapi, ii;; rnodeling doubt deliberately provocative, will strike mason, WasNngun, DC,1986), T. Bnkowsla er is em noton of the mean global termerature. How many scientists as absurd, perhaps even al., " Yucca Mountam prograrn summary of re-do we measure the average temperature of the offensive. While not necessanly accepting search, site nonttmng, and techncal mew ac-Earth 7 Our nost base data can be vwy deeply her viewpoint, we might ponder this aspect tuties.",w(State of Nevada. AQency for Protects-layered.
u,c,,,
,,,, p,,,ect Omce. Carson Cay, Neva-
- 17. C. G. Hempel and P. Oppenheim, PNbs. Sci 15.
ofit: A model, like a novel, may resonate da.19e8); L Costin and S. Bauw, " Thermal and 135 (1948); C. G. Hempel Aspects W Scentr5c with nature, but it is not a "real* thing.
memancal mdes first be,ctwmark exercist Part Expfanatm (Free Press, New York,1965): PNbs-l Like a novel, a model may be convincing-t bannat anaWs," N1221 UC 814dSam W d Nansat Scence (Prenbce-Hat Engle-g dia Natonal Laboratory, Albuquerque. NM,1990).
wood Cliits. PU,1966).
f it may, ring true,, if it is consistent with ouf R. Barnard and H. Dockery " Nominal Configura-
- 18. For tNs reason, C. F. Tsang (14) proposes that experience of the natural world. But just as ton / H%~w parameters and calcutanm model evaluaton should always be a step-by-we may wonder how much the chracters in
'**** ~ # 1 # "T* " 8' "'"m*'Y **
smo omcedura w='-=,e assessmert calculanonal exercises
- 19. This perspectve refutes a simp 6e Poppenan ac-a novel are drawn from real. A ahd how for 1990 (PACE 90)," SAND 90 2727 (Sandia Na-count of falsificabon, wherein we are expected to l
t much is artifice, we might ask te same of a tenal Labonnones Albuquerque, NM,1991).
throw out wiy rnodel whose predictions fail to model: How much is based on observation
- 7. For recent entwes of venncatm and validanm match snpincal data. As many W e s have in hydrology, see (2); J. D. Bredenceft and L F.
emphasced, especially imre Lakatos and Tho-and measurement of accessible phenomena, Konikow. Gromdwarn 31,178 (1993). For a mas Kuhn, scienbsts routinely modify thew rnodets how much is based on informed judgment, smtar enbque b geomemrstry, see D. K. Nord.
to At recalenrant data. The quesbon is, at what and how much is convenience? Fundamen.
stmm, Eos 74, 326 (1993): M v. of te met do scienests decide ther further rnodifica-Mh CEC Nansal Ansegue Wesp Grmp Me*
tons are no bngw acceptabieMhwi.o.s are tally, the reason for modeling is a lack of bg and Asgator %rs Anabgue Propet Anal still debebng this queston [T. S. Kuhn. The Struc-full access, either in time or space, to the atirkshop, To6scb. Spain,5 to 9 0ctober 1992; H.
fire of Scenride Aemlution (Unrv. of Chcago phenomena ofinterest. In areas where pub.
vm Marav6e and J. Snema Edt (EUR 15178 EN.
Press. CNcago. ed. 2,1970); The EssenpaI Ten-i Con msson of to Eumpean Commundy, Brus-sm. Selected Sades in ScerW Tradman and lic policy and public safety are at arde, the sets,1994).
Change (Unrv. of CNcago Press, Chcago.1977),
burden is on the modeler to demonstrate
- 8. Two recent editorials deanng with te interface of I. Lakatos, in Cnteism and tie Growth of Knowd-the degree of correspondence between the modeling and public policy at Yucca Momtam edge, l. Lakatos and A. Musgrave. Eds. (Cam-model and the material world it seeks to are C. R. Mabne, h Scr. Tectnot 23.1452 bndge Univ. Press, Cambodge 1970L pp. 91-(1989). and I. J. Winograd, itxf. 24,1291 (1990).
196; K. R. Popper, The Loge of Scaenti6c Discov-represent and to delineate the limits of that
- 9. For exampie, re Aridam 1buse Chabndged wy (Basic 8mks, New York,1959); Conpcases correspondence.
Dictkna y grves the first defirubon of venty as "to and Aervratons-The Growin of Scentine Kirwv.
Finally, we must admit that a model may pm to truri of" (New York. 1973). 06ctonary edge (Base Omks. New York.1963)).
definrbons of venfy vahdate, and confirm reveal
- 20. Nonumeness may anse on a vanety of levels:
confirm our biases and support incorrect the circulany preisent in common use, thus high-h wid BrMehoen (2) hee WM intuitions. Therefore, models are most use-lighung the imperadve for cor.sistent scenefiC the heterogenedy of the natural worid 6ethke ful when they are used to challenge existing usage.
IGeocnm Cosmoohn Acta Se. 43u (1992))
10 ** * ** same as saying that Fere was an has snphasged te possbildy d multple rmts to formulations, rather than to validate or implicit cciens panbus clause.
Spveming equabans. Also see L N. Plummer, D.
verify them. Any scientist who is asked to
- 11. Godei questioned te posseility of venficaron L Par *hurst. D. C. Thorstenson. end. 47. ces use a model to verify or validate a predeter-even in closed systems (see E. Nagel and J. R.
(1983), L N Plumrner, "Pracbcal Appicabons of mined result should be suspicious.
Newman, Goders Pmd (New York Unrv. Press, Groundwater Geochemistry," First Canaden.
New York,1958)].
Amencan Conference on Hydrogeoogy (Natonal
- 12. On the problem of verifcation in computer pro-Water Well Association, WortNngton OH,1984),
REFERENCES AND NOTES gramming. see J H. Fetzer. Ccr rnun. ACM 31 pp.149-177,L N. Plummer. E. C. Prestemon, D 1048 (1988). ltt. Am. Math. Soc. 36,1352 (1989);
L Parkhurst, U.S. God Sury Water Aesour. In-
- 1. P. A. Davis, N. E. Otague, M. T. Goochch, "Ap.
Mads Mach. 1, 197 (1991).
vest. Asp. 9f-4078 (1991), p.1.
preaches for tte vahdation of modets used for
- 13. This is equivalent to A. J. Ayer s classc definition
- 21. Also referred to as the Duherr>Ouine thesas, after performance assessment of higrWewel nuclear of an ana'ytic statement as one that is "true sole #y Pierre Duhem, who emphastzed the nonunique-waste reposrtones." SAND 904575/NUREG CR-in vv1ue of the meaning of its constituent symbois, ness of scientfc explananon, and W V O Quine.
644 SCIENCE + VOL 263 4 FEBRUARY 1994
+
h,. _
\\
who emphas zed the wnohstc nature of scientifc speCr0c prccess, a partcular site. Cr a gfven got, Soc. Stud. Sci. 23. 263 (1993).
theory. Both perspectives refute any simple ac.
range of applicability. Unfortunatety, even with 34 A good example of van Fraassen's concept is the count of the reiaron between theory and obser.
such a degree of spectfcsty. the elements of the view expressed by G. de Marsity. P. Combes. and vation The classe essays on underdeterminaton rnodel (the conceptualizaton, the site-specrfic P. Goblet (Adv. water Resour. 15. 367 (1992. have been repnnted and cntiQued n S. Harding, empocal input parameters the est*nated tem-who claim that they "do not want certainty. [but] Ed., Can Theones Be Refuted 7 Essays on the perature range) are stdl underdetermined. Fur. will be satisfied with engineenng confcence Dunsm-Ousne Thesis (Redet. Dordrecht, the thermore, he notes that establishing "the range of [W]e are onty (tryng] to do our level best." This is i Nethertands.1976). appicaton" of a rnodel cannot be done odepen-a commendabry honest approach but one that will 22 B. van Fraassen, The Scient#c image (Oxford centty of tne desired performance entena. "There invite a very different pubic reaction than claims Unrv. Press, New York.1980). is the possibehty that a performance entenon about "venfed" rnodels. i 23 H E. Longino (Scence as Socist Knowfedge could be defined in such a way that the cuantay of
- 35. Using post-audits of " validated" rnodels. Konikow (Pnnceton Univ. Press. Pnneeton. N.I.1990)) ex.
interest can never be predcted with sufreient and co-workers have shCwn that even models that amines the role of personal and pohtcal prefer. accuracy because of intnnssc uncertaintes o the produce a good history match of past data often ence o generating sex bias in scientifc reasor> data.. Thus, one has to modify the performance do ternbty when extended into the future (L F. ing Her point is that extraevdentail consder. cntenon to something more plausible yet stdl Konekow and J. O Bredenoeft, water Resour. ations are not restncted to bad science but are acceptable for the protWem at hand" (C. F. Tsan9 Res 10,546 (1974), L F. Konikow. Groundwater charactenstc of all scence, thus makog dsfferen. In (f 4). p. 827). But this concluson begs the 24 173 (1986), and M Person, water tiaton between " legitimate" and " illegitimate" Question, Who decides what is plausible and what Resour. Res 21,1611 (1985). L F. Konikow and i preferences difficult. is acceptabie? L A. Swain. m 26th internaronal Geological Con-I
- 24. For a counterargument, see C. Glymour, in The 29 " Environmental Assessment: Yucca Mountain press Selected Papers on Hydropeobgy. V. H.
Phesophy of Scence. R. Boyd, P Gasper, J 0. Site, Nevada Research and Development Area. Hiese. Ed. (Hanover, West Germany,1990), pp. 1 rout Eds, (Massachusetts institute of Technolo. Nevada." vol. 2 of US. Department of Energy 43S449). Typcally, this occurs either because gy Press. Camundge, MA,1991) pp 485-500 DOEN-0073 (Ofree of Crvilian Radioactive the conceptualization of the system built into the
- 25. Ockham s razor is perhaps the most udely ac.
Waste Management Washogton. DC.1986). This numercal model was incorrect or because mod-cepted exampie of an extraevidential consider, definition conflates the genenc numencal sarnula-eiers failed to antcipate significant changes that atiort Many scentists accept and apply the pnn-tion code with the site-spec 4fic model. A site. subsequently occurred m the system (for exam-copie in their work, even though it is an entrety specrfic model might accurately represent a pie. changes in climate dnving forces). Post audit trvWi:;al assumption. There is scant empin. physcal system. but there is no way to demon-studies by these and other workers have been cal evideru that the world is actually simple or strate that it does. A code is simply a template reviewed by M. P. Anderson and W W Woessner that sirnpie accounts are rnore likety than complex until the parameters of the system are put in, and (Adv water Resour 15. 167 (1992)). Of five ones to be truei. Our commitment to simplicity is therefore could not even m pnnciple, accurately studies reviewed, not one rnodel accurate #y pre-largely an inhentance of 17th-Century theology represent a physcal system. dcted the future. In several cases, models were
- 26. In the earty 1920s, a group of phdosophers and
- 30. " Radioactive waste mii-wo.rw glossary,"
calibrated on the basis of short-duration data sets scientists known as the vienna Circle at empted 1AEA TECOOC-264 (Intematonal Atome Energy that inadequately desenbed the range of natural to create a logeally venfiab6e structure for so-Agency, Venna,1982). A recent summary of conditions possible m the system. This issue of ence. Led by the phdosopher Rudolf Carnap, the European work in thrs area in the context of temporal vanation becomes partcularty impocant "logeal pesitiests wshec to create a theoretcal-radioactive waste management is grven by P. for modeling the long-term disposal of nuclear ty neutral observation language that would form a Bogonnski er at Radochim. Acta 44/45, 367 wastes. Changes in ene geo!ogeal conditions of I basis for purety logcal structures, free of auxdiary (1988). the reposrtory site, whch could lead to charges e assumptions, for all of science. Such logcal con-
- 31. In definog rnodel " validation," these workers use the dynames and structure of the system, are not structions would be venfiable (R. Carnap, repnnt.
the desenptor " adequate" rather than " good.- only possible but. given enough time. almost ed m Logeal PbsrtMsm, A. J Ayer. Ed. (Free presumably because they recognize the dficulty certain. Press. New York,1959). pp. 6?-81. Also see A. J. of definog what constitutes a " good" representa-
- 36. Vanous prviosophers including A. J. Ayer. W V.
Ayer (1946), in (f3). For a hestoncal perspective tion. They propose that a rnodel need only be O Quine, l Lakatos, and T. S. Kuhn have ques-on Camap and logcal positMam, see I. Hacking, " adequate" for a "grven purpose" in this case tioned whether we can m fact prove a %pothesis j Representrng and lntervenrig (Cambndge Univ. compliance with federal regulations. But this def-false. Ayer emphasized that refutations, no less Press, New York,1983); R. Creath, Ed., Dear instion begs the Queston of whether the regula-than confirmations. resuppose certain condi-l Camap, Dear Ourne The Ousne-Carnap Corre-tions are adequate. Furthermore, because these tions [Ayer,1948 (f 3. espec ally p. 38)). Quene, soondence and Related Wbrk (Untv of California workers recognize that rnodels cannot be validat. Lakatos, and Kuhn emphasized the wnohste na-Press, Berkeley.1990), and R. Boyd. in (24). pp ed but refuse to relinquish the term validaton, ture of hypotheses and the flexible options for 3.-35. The mfluence of the Vienna Circle on phe-they end up with an almost <%d#sible modWeatons to "save the phenomena * (19, 2f). losophy of science was profound. W. V. O. Oume statement of the r goals. "[M]odeis can never be However, none of these moves really undermines has called Camap "the dommant 6gure in phdos-validated, therefore validaton is a process of Popper's argument that it is stdl possible o pnn-ophy from the 1930s onward" (in Creath, above. building confidence in models and not providing ciple to prove a theory false. but not possible even op.463-466) But m spite of Camap's stature and ' validated' models" (P. A. Davis et al., in ( f), p. 8). in pnncip6e to prove a theory true (Popper (19)) influence, the philosophical program of "venfica-
- 32. For exampie,in the guidelines of the U.S. Nuclear
- 37. Note that this is just one view. Many phdosophers tiorwsm" collapsed resoundingly in the 1950s [P.
Regulatory Corrrnisson Radcactrve Waste Man-have disputed the hypothetico-deductive modet Galison, Scr. Corirext 2, 197 (1988)', J Rouse, agement Program (NUREG4865) (U.S. Nuclear
- 38. The noton of diversity in confirmation helps to Stud. Hist. Phts. Sci 22,141 (1991)). It was Regulatory Commiscon, Washington, DC,1990),
explain why It is important to test a rnodel in a officially pronounced dead in the EncycJopedia of "venfication" of a code is descnbod as "the wide vaiety of circumstances-mcluding those Phesomy in 1967 [K. R. Popper, Unended provison of an assurance that a code correct!y that rnay appear quite different from the expected Ovest An intellectual AutobiograMy (Collins, performs the operations it speerf4s A common circumstances at the rnodeled site-4espite ap-Glasgow,1978), p. 87]. There now appears to be method of verication is the companson of a parent arguments to the contrary. For example, nothing in the pnilosophy of science that is as code's results with solutons obtamed anatytcal-Davis and co-workers (f) have argued that test-uniformty rejected as the possedity of a logcaJty fy " However, a certain confuson in the literature og the performance of a model in areas not venfiable rnethod for the natural =oences. The over terrmnology is made evioent by companson relevant to regulatory compliance is a waste of reason is clear Natural systems are never closed. of Anderson and Woessner (4) with Wang and resources and can lead to tre needless rejecton
- 27. For example, Webster's Seventh New CotAspiare Anderson (3). Prevounty, /viderson had referred of models that are adequate to the task at hand.
Octonary (Memam, Sonngfield, m,1963) grves to this process as validatori, and more recentty, While this rnay sometimes be the case. confbrma-j the following definition of validatiert to make le-and more misleadingly, as venficaton. tion theory suggests that successful testing of a ga!ty valid, to grant official sancton to, to confirm
- 33. Admittedty, computer programmers engage rou-model in a vanety of domains provides important the validity of (for example, an election). Random tinety in what they call program "venfication "
support for the conceptualizaton embodied in the House sirnilarly cites elections, passports, and however, the use of the term "venf! cation" to rnodel Faded tests help to estabhan the lemets of documents [ Random House Dcticriary of the Er> desenbe this activity has led to extremely con-model adequacy, and may cast legitimate doubt phan language (Random House, New York, tentious debate [see Fetzer (1988), in (12) and on the model conceptualizaton of the pr'ysical or 1973)]. letters in response in Commun. ACM 32 (1989)} ChemeCal processes invoNed. l 23 For examp6e, a widely used and extensively de-One sinking feature of "venficaton" in computer
- 39. In his classic account of the principle of verifica-bugged package such as MODFLOW [M. G.
science is that it appears to be motivated, at ton, A. J. Ayer [(1946), in ( f 3)) opened the door Mcdonald and A W Harbaugh, US. Geol Sury least in part, by the same pressure as in the to undermining his own positen by recognizing Tech., Water Resour. Invest., book 6 (1988), earth science Community a demand for assur-that empincal statements Could never be proved chap A1, p.1) or WATEQ [A. H. Truesdall and B. ance of the safety and rehability of computer certain but only probable. He called this condition J Jones, Nat, Tech int Serv PS2 20464 (1973), programs that protect pubisc safety, in this case, " weak venficaton," an obvious oxymoron. In hind-p.1] might be vahd, but when applied to any those controihng missile guidance systems sight it is easy to see that " weak venficaton" is paricular natural situaton would no longer nec-(6d. p 3r6). For an interesting histoncal paper probabdistic confirmation [Ayers (1946). in (1J), essanly be vahd C. F. Tsang has argued that on the problem of establishing certainty in the pp 99100 and 135136] Popper preferred the rnodels should be vahdated with respect to a manufacture of weapons systems, see G Bu-term "corroboraton" to em@asize that all confir. SCIENCE VOL 26.) 4 FEBRUARY 1994 645 m
%=, l mation ip L nerently weak (Popper.1959 ( f 9)). For rny. wrisch was extrernety well confirmed for cen-4). The task of site sesection, as defirwd in this ~ a recent perspectrve on probabilistic confirma-tunes and then overtumed completely by the report. consisted of evaluat[ing} the potentialty tion, see A. FrankJin and C. Howson. Stud. Het. Copemican revolution. See T. S. Kuhn. The Co-acceptable tries against the disqualifying conde-PMos. Scr. 19. 419 (1980); and C. Howson and P. pemcan Rest:Wuron (Harvard Univ. Press. Cam-tions... " The authors concluded that the Yucca Urbach. Scientific Acasorwy The Bayesaan Ap-bndge. MA 1957). Indeed. every scentific revo-Mountain srte was "not disqualifed." That is. the proach (Open Court. La Sane. It,1989). lution ovolves the overturning of well-confirmed null hypothesis is that the site is safe. the burden 40 Camap therefore argued that all inductne logic theory. See 1. B. Cohen. Revolution in Sesence of proof is on those wrio would argue otherwise was a logic of probability (R Camap, o The (Belknap Press. Cambndge. MA 1985). 46 N Cartwnght. Now the Laws of Physcs Lee (Clar. ProdJem of Jnductive Loge.1. Lakatos. Ed. (North
- 43. Konikow and Bredehoeft (2). on the basrs of their endon Press. Oxford 1983), p.153.
Holland. Ams;erdam,1968). pp. 258-267]. Con. extenssve expenence with both scientists and
- 47. This article was prepared for a session c,n hydro-farming observations gave us warrant for a certain govemment ofricials. emphasize that the lan-icgscal and p.J.i J modeling in honor of degree of belief.
guage of venfied and validated rnodeis is typcany David Crerar at the Amencan Geopnyscal Unon.
- 41. An example is the evidence of faunal homologies interpreted to mean that the models under discus-May 1993. We thank the organizers. A. Maest and in Alnca and South America, before the accept-sion are, in essence. true. It is also clear that this D. K. Nordstrarn, for eviting us to prepare 1tus ance of plate tectanc theory. Thesa data. wnich is the intent of many authors who claim to base arbcie: J. Bredenoeft for stimulating our thinking on were used as an earty argument in favor of cono-results on " validated" models.
the tope: J. H. Fetzer. L Korukow M. Mitchell. K. i nental dnft. were considered to be equaisy well 44 We haw never seen a paper in whch the authors Nordstrom. L Sonder. C. Drake. and two reviewers f explained by the hypothesis of land bndges (N. wrote. "the empencal data invalidate this rnodel." for helpful comments on the manuscnpt. and our { Oreskes. Hist. Stud. Phys. Scr. 18.311 (1988)].
- 45. Another example is found in the enwonmental research assrstant. D. Kaiser. We dedcate this
- 42. An obvious exarnple of this is Ptolemac astrono-assessment overvew for Yucca Mountain (29 p.
Paper in app,m,=,m of the work of David Crerar RESEARCH ARTICLE the fut mnhydmlytic mactiom m be cata-lyzed by an annbod, (3, 4). This concerted miAistion, formally a Claisen rearrange-Routes to Catalysis: Structure of a -m;g'eds: l Catalytic Antibody and Comparison ""#' M'2""'E":"d't mitted step in the biosynthesis of reasine and i with its Natural Counterpart phenylalanine, and the en:yme chonsmate j mutase accelerates this reaction by more than 2 million. Although the precise factors that i Matthew R. Haynes, Enrico A. Stura, contnbute to the e6iciency of the en:yme are Donald Hilvert, Ian A. Wilson Still pwdy undentwd, it is kmwn that the ] uncataly.ed reaction occun through an asym-j The three-dimensional structure of a catalytic antibody (1F7) with chorismate mutase metric chairlike transition state 2 in which t tctivity has been determined to 3.0 A resolution as a complex with a transition state carbon-oxygen bond cleavage precedes car-cnalog. The structural data suggest that the antibody stabilizes the same conforma-bon-cahon bond formation (7,8). In aque-tionalty restricted pericyclic transition state as occurs in the uncatalyzed reaction. Overall ous solution the flexible chorismate molecule j shape and charge complementarity between the combining site and the transition state preferentially adopts the extended pseudo-cnalog dictate preferential binding of the correct substrate enantiomer in a conformation diequatorial conformation la and must be appropriate for reaction. Comparison with the structure of a chorismate mutase enzyme converted to the higher energy pseudo-diaxial indicates an overall similarity between the catalytic mechanism employed by the two conformer Ib on the way to the transition proteins. Differences in the number of specific interactions available for restricting the state (9). Binding sites that are complemen-rotational degrees of freedom in the transition state, and the lack of multiple electrostatic tary to the compact transition state species interactions that might stabilize charge separation in this highly polarized metastable (and the corresponding substrate conformer) species, are likely to account for the observed 10* times lower activity of the antibody would therefore be expected to increase sub-relative to that of the natural enzymes that catalyze this reaction. The structure of the stannally the probabdity of reaction. The 1F7 Fab'-hapten complex provides confirmation that the properties of an antibody favorable entropy of activation (AiS8 = 13 catalyst faithfully reflect the desjgn of the transition state analog. cal K mol-3) of the en:yme-cataly:ed pro-cess compared to the spontaneous thermal rearrangement is consutent with this idea (6), as is the observation of strong en:yme inhibi-The mammalian immune system has been mits systematic exploration of the basic pnn. tion by the conformationally restricted endo-successfully exploited by chemists to create ciples of biological catalysis and, through oxabicyclic dimimy!k acid 4 which approx-antibody molecules with tadored catalytic ac-cc.mpanson with naturally occurring enzymes, imates the structure of 2 (12). Stabilcanon of civities and specificities. Haptens designed to evaluation of altemative catalytic pathways any charge separation in the transition state mimic the key sterecelectronic features of for pamcular reactions. In the absence of through electrostatic or hydrogen bonding transition states can induce antibodies capable structural information, it is dif5 cult to deter-interactions might also conenbute to the po-of cataly:ing various chemical transforma-mine precisely the extent to which the tran-tency of the enzyme (13). tions, ranguy from simple hydrolyses to reac-sition state analog dictates the catalytic char-g tions that lack physiological c..mterparts or actenstics of the induced annbody. Thus, ,, 1 - Os T7 ' c are normally disfavored (l). The ability to detailed knowledge of the mode of transition O create novel active sites in this way (2) per-state analog binding by antibodies should co - s = facilitate the further development, through ,/ s M R Haynes. E A. Stura, and L A witson are in the rational redesign, of both transition state an-D*S*"'"*n' d ** cut soccy. The Senops A alogs and fint-generation catalytic antibodies. search Institute. La Jolla. CA 92037. D. Hilvert is with H() The unimolecular conversion of (-)-cho-ene Depanmente d cnernistry and uncutar Boccy. The Senppe Research Instnute. La Jolla. CA 92037. rismate into prephenate (Fig.1) was one of 4 646 SCIENCE
- VOL. 263 4 FEBRUARY 1994 A
Q [. COALITION ON WEST VALLEY NUCLEAR WASTES Sharp Street East Concord, NY 14055 -(716) 941-3168 l l September 28, 1998 Shirley Ann Jackson, Chairman U.S. Nuclear Regulatory Commission Washington, D.C. 20555
Dear Chairman Jackson:
We are an interested party in matters relating to the West i Valley site. In view of the recent request by the New York State Energy Research & Development Authority (NYSERDA) to review and comment on NRC's decontamination and decommissioning (D&D) crite-I' ria for West Valley in draft form, we likewise request the oppor-tunity to review and comment on the West Valley D&D criteria in draft form. We continue to request 1) that NRC's D&D criteria for West Valley be no less protective than NRC's rule on decommissioning that is applicable elsewhere, and 2) that NRC comply with NEPA in issuing D&D criteria for West Valley. I For NRC to allow NYSERDA and other interested parties to review and comment on such D&D criteria in draft form is not sufficient, in our opinion, to satisfy NRC's NEPA obligation with respect to issuing West-Valley-specific D&D criteria. Neverthe-less, we seek the same opportunity that NYSERDA has requested to { review and comment on the West Valley D&D criteria in draft form. Sincerely, S [U m / l Raymond C. Va'gh n Y f
1 kah COALITION ON WEST VALLEY NUCLEAR WASTES S Sharp Street East Concord, NY 14055 -(716) 941-3168 l December 31, 1998 Shirley Ann Jackson, Chairman U.S. Nuclear Regulatory Commission { Washington, D.C. 20555 { l i 1
Dear Chairman Jackson:
..Thank you for your letter of December 24. I gather from your letter that the Coalition on West Valley. Nuclear Wastes will not be invited to' address the Commission at the January 12 meet-ing in Rockville. As you indicate in your letter, our written statement will receive the same consideration as will the oral presentations at the meeting. This arrangement appears to be satisfactory. We have no ) particular desire to make an oral presentation as long as our 15-page written statement dated December 2 will receive equal consideration. Nevertheless, I want to raise a couple of possi-ble procedural concerns about this arrangement. Let me run them { past you, so to speak, and you and the other Commissioners can I then decide whether anything further needs to be addressed in j i these areas. I our purpose here is to eliminate any possible misunderstand-ing prior to the January 12 Commission meeting. We do not mean to suggest that any misunderstanding exists but believe that it J is.better to say too much-rather than too little prior to this ) important meeting. Thus, please treat the remainder of this letter as a checklist. If you and the other Commissioners see nothing therein that needs further attention, then I think we can agree that there are no substantial procedural misunderstandings. As background, please note the following points we raised in' our letters of December 2 and 3. In our December 2 letter to Bill Hill (sent as a cover letter with the faxed copy of our December 2 statement), we asked, "Do the Commissioners want an opportunity at the January 12 briefing to ask questions in person regarding our enclosed written statement?" In our December 3 letter to you, (sent as a cover letter with the paper copy of our December 2 statement), we noted that we'"are not planning to make an oral presentation at the January 12 briefing unless you think our presence there would be useful to provide further explanation -or answer questions." In your December 24 letter, you do not respond specifically to these points. Instead, you indicate 1) that other stakehold-ers, including the West Valley Citizen Task Force, have been invited to addrens the Commission on January 12, and 2) that written statements such as our own will receive equal considera-10A' h ff
tion. These are the two areas in which we need to make sure that no misunderstanding exists. one possible concern involves the following sentence in your letter: "The West Valley Citizen Task Force, of which you are a member, and other stakeholders, have been invited to address the Commission in that meeting." The phrase "of which you are a member" is factually accurate. If this phrase has no further implications, then we see no problem. If, on the other hand, the phrase implies that the Citizen Task Force 'CTF) can and should present and explain the views of the Coalition on West Valley Nuclear Wastes on January 12, then we disagree. As you know, our 15-page statement delves into issues of greater complexity than those raised by the CTF. We do not mean to suggest any disagree-ment between our views and those of the CTF, but, simply, that we are a much older organization than the CTF and have dealt with certain aspects of the West Valley site in a more detailed or technical way than the CTF has done. The second possible concern involves the equality of oral presentations and written statements. In and of itself, an oral presentation seems generally comparable to a written statement. However, you and the other Commissioners apparently intend to ask questions and/or engage in discussion with thos'e who make oral presentations on January 12, presumably for the purpose of im-proving understanding of various points. This is the area in which we are concerned that a written statement may not receive equal consideration. Granted, the Commissioners would also have the option of using letters or telephone calls to ask questions or engage in discussion of our December 2 statement. In either case, we think it is incumbent on the Commissioners to employ methods at their disposal to ensure a clear understanding of the points raised in our 15-page statement dated December 2. In the absence of oral or written questions, we will assume that all of the points we have raised are clearly understood. In conclusion, we think the January 12 meeting and the pre-and post-meeting deliberations of the Commissioners will provide a welcome opportunity for NRC to gather relevant information and move toward a clear policy for the West Valley site. We look forward to the outcome and wish you well in this and your future endeavors. Sincerely, 0 _JOSA 1L-Raymond C. Vaughan cc: T. Attridge, CTF B. Mazurowski, DOE P. Piciulo, NYSERDA P. Merges, DEC
- I:
? \\ ,r 'e STATEMENT OF POSITION ON COMMISSION PAPER SECY-99-057 submitted by Coalition on West Valley Nuclear Wastes 10734 Sharp Street, East Concord, N.Y. 14055 March 29, 1999 General
- 1. SECY-99-057 properly offers prescription of the License Termi-nation Rule (LTR) as one option now available to NRC for decon-tamination and decommissioning (D&D) criteria for the West Valley Demonstration Project.
We believe the Commission should approve th'is option (labeled 1(a) in SECY-99-057) and thus prescribe the LTR as the D&D criteria for the West Valley Demohstration Project (WVDP). 2 NEPA has clear requirements which are applicable to federal agencies. In previous communications with NRC, including our statement of position on SECY-98-251 dated December 2, 1998, we have outlined the applicability of NEPA to NRC's adoption of D&D criteria for the West Valley site. We see no substantive discus-sion of the applicability of NEPA to NRC decisionmaking in SECY-99-057. We continue to be concerned that NRC is ignoring its own NEPA obligations in this matter. We continue to remind NRC that NEPA imposes certain requirements which must, as a matter of law, ) be addressed and met by NRC. 3. The comparison of pros and cons in SECY-99-057, Attachment 3, fails to show NRC's substantial NEPA obligations under Options B and C. 4. Our statement of position dated December 2, 1998, on SECY 251 is still pertinent. We have previously asked the Commission-ers to let us know if any point therein requires discussion or clarification (e.g., letter from Vaughan to Jackson, December 31, 1998). Except for discussion with Commissioner McGaffigan, we have had no comments, questions, or inquiries from NRC regarding our statement of position dated December 2, 1998. To the extent that any point therein is not fully understood by the Commission-ers, we continue to invite comments and questions. NRC's authority and oblications at the West Valley site 5. See paragraphs 6-17 of our December 2, 1998, statement of position on Commission Paper SECY-98-251. 6. In several respects, SECY-99-057 misrepresents NRC's authority and obligations at the West Valley site. See following para-graphs for details. 1
o
- 7. The last full paragraph on page 2 of SECY-99-057 provides.a list of six items that purportedly-covers NRC's authority and responsibility under the WVDP Act.
The list fails to include or address 52 (a) (4) of the Act (" applicable licensing requirements" ' for disposal of low-level and transuranic wastes). In addition, the list improperly uses the word "may" in describing NRC's obligation to prescribe D&D criteria.
- 8. The meaning of the word "may" in 52 (a) (5) of the WVDP Act has not heretofore been questioned by NRC.
NRC has accepted the clear meaning of 52 (a) (5), viz., that NRC will or shall prescribe D&D criteria for the WVDP and that NRC is given the authority to exercise judgment in choosing those criteria ("such requirements as the Commission may-prescribe"). The word "may" has not been, and should not be, interpreted as an indication that NRC has an op'tional responsibility _to prescribe D&D requirements. For example, see NRC memo from Hurt to Haughney dated December 7, 1990, PDR ACN 9012130314 ("By the terms of the West Valley Demon-stration Project Act, NRC must prescribe decontamination and decommissioning requirements for the WVDP.") and see also NRC memo from Cunningham to Bangart dated July 10, 1991, PDR ACN 9107180119 ("the decontamination and decommissioning (D&D) crite-ria that NRC is obligated to provide the WVDP..."). See also NRC's Safety Evaluation Report Related to Amendment No. 32 to Facility Operating License CSF-1, February 1982 ("It is important to note...that under the West Valley Demonstration Project Act, the Commission will prescribe these decontamination and decommis-sioning requirements."). Prescription of such requirements is not an optional responsibility for NRC.
- 9. The West Valley Demonstration Project Act, 5 2 (a) (4), requires DOE to dispose of low-level and transuranic wastes "in accordance with apn icable licensing requirements."
We have persistently asked NRC ts clarify whether 10 CFR 61 or 6 NYCRR 382-383 are the applicable licensing requirements for disposal of low-level wastes generated by the WVDP. To date, NRC has failed to re-spond. Ne continue to remind NRC that it has an obligation to address this question. The question is not remote or obscure, as the phrase " applicable licensing requirements" is part of a different paragraph in the same law that SECY-99-057 seeks to address. In our view, " applicable licensing requirements" for disposal of low-level wastes are those which have been promulgat-ed either by NRC as 10 CFR 61 or by NYSDEC as 6 NYCRR 382-383. Which of these licensing requirements is applicable?
- 10. SECY-99-057, page 4, refers to "onsite disposal of radioac-tive waste (under 10 CFR 20.2002)" in a vague and offhand manner which fails to acknowledge the word " licensing" and fails to discuss or justify how 10 CFR 20 can be interpreted as the
" applicable licensing requirements" for low-level waste disposal. The same problem occurs in Attachment 3 of SECY-99-057. The legal mandate for " applicable licensing requirements" appears to point to either 10 CFR 61 or 6 NYCRR 382-383. NRC needs to address this matter. 2
y .s ?
- 11. SECY-99-057, page 5,
asserts that "The Commission has the option to omit waste disposal criteria, but that will leave open the question of what ' licensing requirements' apply to any DOE waste disposal proposals at the site." This assertion is hypo-critical. NRC has strenuously ignored the question of the ap-plicability of 10 CFR 61 or 6 NYCRR 382-383, yet expresses con-cern that the question would be left open by the adoption of the LTR. We agree with the concern but, as previously stated, be-lieve that the LTR should be adopted under 52(a) (5) of the WVDP Act and that. applicable licensing requirements, presumably 10 CFR 61 or 6 NYCRR 382-383, should be identified by NRC in accordance with 52 (a) (4). NRC's refusal in recent months to look at 10 CFR 61 in this context is difficult to understand. The idea is not new, nor is it ours alone. See, for example, NRC's comments dated November 27, 1996, on the West Valley DEIS, especially paragraph 5 of NRC's comments and the acco.mpanying report by NRC's contractor, Center for Nuclear Waste Regulatory Analyses, dated August 1996.
- 12. SECY-99-057, at the bottom of page 2 and top of page 3,
asserts and lists three things that NRC is not authorized to do i at West _ Valley. We will address each of these separately in the l next three paragraphs.
- 13. SECY-99-057, page 2,
asserts that NRC is not authorized to regulate DOE activities at the project site. In a limited sense, this is true. Activities per se are conducted in accordance with DOE's judgment and procedures; the role of NRC is advisory. However, the manner in which DOE wraps up its responsibilities under the WVDP Act is not left entirely to DOE's judgment. Authority for project closure decisions and standards (though not necessarily enforcement) is given partly to NRC under the West Valley Demonstration Project Act. The rationale is that these closure (D&D and waste disposal) decisions will affect other parties, including the public, as soon as DOE completes its project responsibilities and leaves the site. See, for example, the quotation from House Committee on Interstate and Foreign Commerce, quoted in SECY-99-057, Attachment 2, page 4, footnote 6.
- 14. SECY-99-057, pp.
2-3, asserts that NRC is not authorized to enforce DOE's compliance with the D&D requirements that NRC has prescribed. This may or may not be true, but the point is irrel-evant to the question of NRC's responsibility for prescribing D&D requirements in acccrdance with its best judgment. Enforcement of laws and regulations (e.g., NEPA) is sometimes left to other parties, but this does not diminish the requirement for govern-ment agencies to set clear, sufficient, and well-reasoned stand-ards in accordance with law.
- 15. SECY-99-057, page 3,
asserts that NRC is not authorized to determine whether or when DOE has completed D&D for the West Valley Demonstration Project. This may or may not be true, but 3
O.- the point-is irrelevant. It does not affect NRC's obligation to set clear, sufficient, and well-reasoned standards in accordance with law. Meanina of decontamination and decommissionina
- 16. The meaning of " decontamination and decommissioning" is not discussed in SECY-99-057 but was questioned by John Greeves of NRC staff during the January 12, 1999, NRC public meeting.
In this context, it should be noted that, more than twenty years ago, Public Law 95-238 directed DOE to prepare and submit to Congress a study which, among other things, considered the avail-able options for " Federal operation of the Western New York Nuclear Service Center for the purposes of decommissioning exist-ing facilities...", and, furthermore, that a " West Valley Tank Debontamination and Decommissioning Task Group" was created in 1978 under the leadership of DOE and the New York State Attorney General's Office; that this Task Group " felt they could not properly consider the tanks divorced from the other radioactive contamination on the site, so the scope of the Task Group's report was expanded to include these areas;" and that the Task Group issued a 94-page report dated August 1978. See DOE report ' TID-28905-2, Western New York Nuclear Service Center Study, 1978, page A-1 onward, for the above-cited law and complete Task Group report. To the extent needed, these documents may help clarify the meaning of " decontamination and decommissioning" at a time immediately prior to the passage of the West Valley Demonstration Project Act. Possible implications of SECY-99-057 for license reinstatement
- 17. SECY-99-057, page 7, does not adequately address the interre-lationship of D&D requirements and license resolution.
Rein-statement of the Part 50 site license must be conducted in ac-cordance with proper procedures, including public hearings. If the license reinstatement process is to begin now, it should be initiated with proper public notice and should be coordinated with the adoption of D&D requirements. If the license reinstate-ment process is to be deferred until a later date, then the D&D requirements should not foreclose or prejudge reasonably foresee-able issues in which NRC will need to exercise judgment in a future license reinstatement process. CooDerative Acreement with NYSDEC
- 18. Any agreement that NRC makes with NYSDEC must 1) not create obligations that are inconsistent with those already made by NRC, and 2) not foreclose or prejudge reasonably foreseeable issues in which NRC will need to exercise judgment in future decisionmak-ing, including a future license reinstatement process.
i 1 1 4 l i
- 19. Any agreement that NRC makes with NYSDEC must be consistent g
with NEPA and its state analog SEQRA. Any such agreement must avoid segmentation and must allow discretionary decisionmaking by either agency to be supported by environmental review under nEPA and/or SEQRA. Such an agreement must not subvert the EIS now being prepared in accordance with the Stipulation of Compromise Settlement that we signed with DOE.
- 20. Any agreement that NRC makes with NYSDEC must address the question of " applicable licensing requirements" for low-level waste disposal under 52 (a) (4) of the WVDP Act.
5
y;;;x f 4 March 29, 1999 l To:' Shirley A. Jackson, Chairman, NRC Annette L..Vietti-Cook, Secretary, NRC . FAX:.(301) 415-1672 l l From:.Raymond C. Vaughan, CWVNW' I
Subject:
. Comments on SECY-99-057 Transmitted with this cover memo are 5 pages, representing the comments of the Coalition on' West Valley' Nuclear Wastes on Commission Paper SECY-99.-057. We would appreciate it if you would distribute copies to all the Commissioners for their con-sideration. Thank you. i
II-W 1.3 ,/ UNITED STATES 2 j 3 NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 g ,,,,, + April 13, 1999 SECRETARY Mr. Raymond C. Vaughan Coalition on West Valley Nuclear l Wastes Sharp Street Esst Concord, NY 14055
Dear Mr. Vaughan:
This is to acknowledge receipt of your letter dated April 7,1999 to Chairman Shirley Ann Jackson concerning NYSERDA's comments on the EnvironmentalImpact Statement. ( A response is under preparation which will be forwarded to you shortly. Sincerely, r ),.m-Annette L. Vietti-Cook l l D T lf
1 F April 20, 1999 To: Steve Abt, Colorado State Paul Bembia, NYSERDA Jon Boothroyd, URI Robert Fakundiny, NYSGS Lisa Hubbard, Corps of Engineers Carol Mongerson, CWVNW Jack Parrott, NRC Ray Pilon, Corps of Engineers Joe Price, SAIC Pat Primi, NYS Office of Atty. General Tim Rice, NYS DEC Rudy Slingerland, Penn State Dan Sullivan, DOE Garry Willgoose Mike Wilson, SUNY Fredonia Eric Wohlers, CTF Michael Woldenberg, SUNY Buffalo From: Ray Vaughan, CWVNW and CTF
Subject:
Comments on West Valley application of SIBERIA model and on sources needed for its evaluation
- 1) Sources needed for evaluation of this SIBERIA aDDlication SAIC's Draft " Landscape Evolution Modeling of the Western New York Nuclear Service Center," Rev.
5, March 8, 1999, lists only Parts 2 and 3 of the Hydrology EID (Environmental Informa-tion Document), WVDP-EIS-009. I strongly recommend that reviewers of the SIBERIA erosion model look at Part 1 as well: West Valley Nuclear Services, Inc., Environmental Informa-tion Document, Volume III, Hydrology, Part 1: Geomorphology of Stream Valleys, WVDP-EIS-009, West Valley Nuclear Serv-ices, Inc., West Valley, New York, January 1993. Those from the Corps of Engineers may have already requested this Part i volume (I talked to Ray Pilon about its importance in a phone conversation on April 14) and others should likewise re-quest it. The Geology EID is very useful for understanding the com-plexly layered soils that underlie the site: West Valley Nuclear Services, Inc., Environmental Informa-tion Document, Volume I, Geology, WVDP-EIS-004, West Valley 1
1 ... 1 j Nuclear Services, Inc., West Valley, New York, March-April 1993. The following sources are less relevant to the specific West Valley application but may be useful for general background information on this type of modeling: ) Ignacio Rodriguez-Iturbe and Andrea Rinaldo j 1 Fractal River Basins: Chance and Self-Oraanization Cambridge University Press, 1997 Rudy Slingerland, John W. Harbaugh, and Kevin P. Furlong Simulatina Clastic Sedimentary Basins: Physical Fundamentals and Computer Procrams for Creatina Dynamic Systems PTR Prentice Hall, 1994 'aomi Oreskes, Kristin Shrader-Frechette, and Kenneth Belitz N " Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences," Science 263, 641-646 (Feb. 4, 1994) 21_ Calibration of model acainst known downcutting Page 13 of SAIC's Draft " Landscape Evolution Modeling of the Western New York Nuclear Service Center," Rev. 5, March 8,
- 1999, compares West Valley erosion rates predicted by SIBERIA to those
" predicted" by other studies. One of these other studies is not strictly a prediction but an extrapolation from a direct measure-ment of downcutting alohg Franks Creek over the ten-year period from 1980 to 1990. Based on this comparison, it appears that the SIBERIA model is currently miscalibrated by about a factor of 6. Page 13 of SAIC's Draft describes the approximate erosion rates predicted by the SIBERIA model: 30 ft per 1000 yr in lower Franks and Quarry Creeks, and 12 ft per 1000 yr in upper Franks Creek and Erdman Brook. One of the comparisons made is to " longitudinal studies." The " longitudinal studies" appear to be the direct measure-ments (by surveying) of the longitudinal profile of Franks Creek. See the Hydrology EID, Part 1, esp. pp. 1-4 and 23-24 and Figure 3-1. (Note that this Figure 3-1 is also reproduced in Part 3 of the Hydrology EID as Figure 5-3.) The results are summarized as follows in Part 1 of the Hydrology EID, page 23: " Initial re-sults from the 1989/1990 profiling indicate that Frank's Creek has downcut about 0.61 meters (2 ft) over the past ten years from j the Quarry Creek confluence to the F48 knickpoint complex. Upstream of this knickpoint, in the floodplain area, downcutting has been minimal (less than.5 ft) over the past ten years." The observed rate of 2 ft per 10 yr can be extrapolated to 200 ft per 1000 yr, essentially equal to the "399 ft per 1,000 years" cited in SAIC's Draft, p. 13, and roughly 6 times greater ) than the 30 ft per 1000 yr predicted by the SIBERIA model. Based 1 1 2
} .h J on this discrepancy, I conclude that the SIBERIA model is seri-ously miscalibrated (by about a factor of 6). Many questions can be raised about extrapolation from 10 years to 1000 years, about urbanization of the watershed, about the balance between erosion and deposition, etc. These are indeed valid questions. I invite discussion but in the meantime think that the observed downcuttina rate of 2 ft per 10 years in lower Franks Creek in the best current 1v available benchmark aaainst which the model can he calibrated. In part, it is best because it effectively intearates erosion over 10 years. Ten years is much less than 1000 years but is far more representative than discrete measurements of water flow and sediment loading. I would add a) that the opportunity exists to re-survey the Franks Creek profile to see what has happened during the past 9 years; b) that long-term net deposjtion in upper Franks Creek is unlikely given the predominantly bedrock watershed upstream and the active downcutting (steepening) in lower Franks Creek; and c) that any idea that reforestation would solve the " urbanization" problem is oversimplified, given the need to keep deep-rooted plants out of waste disposal areas and given the role that trees can play in slumping processes.
- 3) Calibration of model against landform evolution in Buttermilk creek slump area Boothroyd and others placed a surveyed grid of stakes on the Buttermilk Creek slump area.
If their surveyed grid was tied to permanent monuments, then the opportunity exists to re-survey the slump area to determine the backcutting rate of this section of the ravine wall over a period of about 20 years. This may pro-vide an alternative way to check the calibration of the SIBERIA model.
- 4) Omission of Deak flows in current model application Previous work by the West Valley Demonstration Project looked at 2-year, 10-year, 100-year, and PMP storms:
See Hydrol-ogy EID, Part 3, pp. 41-44 and 150-151 (Tables 3-4 and 3-5). This information shows that PMP flow rates and sediment loads are up to 10 times higher than the flow rates and sediment loads associated with a 100-year storm. The current SIBERIA modeling work, as described in SAIC's Draft, pp. 7-9, ignores PMP events and thus loses the high-flow end of the spectrum. This may seriously distort the calibration.
- 5) Role of bed load in sediment transport During the April 15 site walkover, I said that I had seen 3
1 j
1 %1 o- \\ many cobble-sized clasts of clay or till in the bed of the tribu-t tary stream that enters Buttermilk Creek from the east opposite the slump area on Buttermilk. The occasion on which I had seen these cobble-sized clasts of clay or till in the bed of the tributarf stream was a prior walkover on July 15, 1998. This was a few weeks ?fter the in-tense local storm of June 26, 1998, and I assumed on July 15 that the clasts I saw had washed down during the storm. I regret that I took no samples or photographs and that the clasts are now gone (presumably broken up and/or dispersed downstream). A likely source for such clasts would be a slump area that can be seen along this tributary stream. Primatic jointing and irregular desiccation cracks in the slumping till may have al-lowed cobble-sized pieces to be pried loose by the torrential flow of June 26, 1998, and carried along as bed load. I believe I saw evidence of the deposited bed load on July 15, 1998, but recognize that more tangible evidence is lacking at this point. Thus, it is my opinion that high flow rates can and will carry a bed load of sediment (cobble-sized rip-up clasts) which has not been recognized in the model's betal value. Slumps may be needed as the source areas for such clasts. If so, there are many slumps throughout the modeled area. See the Hydrology EID, Part 1, Plate 3, and a smaller but similar map in the 1996 Draft EIS, Vol. II, page L-6 (Figure L-1).
- 6) Lack of distinction amonc different soil types in current model application As currently set up, the model has an adjustable parameter to distinguish between the different erodibilities of bedrock and soil.
A similarly adjustable parameter is needed to distinguish between the different types of soil or unconsolidated material (sand, till, etc.) which are found on the site and which exhibit substantial differences in erodibility.
- 7) Rectilinear bias of model Models of this type tend to have a rectilinear bias imposed by their rectangular coordinate grid.
The SIBERIA model should be re-run with the coordinate grid rotated 45 to see how this would affect results. (A rotation of 45 would, incidentally, provide a better match to the preferred orientation of jointing inferred by Fakundiny et al.) i Another potential concern about the orientation and spacing of the coordinate grid is suggested by a discussion in the Hy-drology EID, Part 2,
- p. 17, where certain recommendations are made with respect to the Corps' HEC-2 model and its ability to "have a sense of stream meander and conveyance."
4
3 .i
- 8) Potential for stream capture The question of whether or when Franks Creek will find a shorter, steeper path to Buttermilk Creek (most likely in the vicinity of the Buttermilk Creek slump area) is very important in any assessment of long-term site integrity.
Part of the question involves the relatively erodible layers (the well-known Kent Recessional and the poorly delineated " Lacustrine or overbank deposits" shown in the Geology EID, cross-sections D-D' and E-E', Figures 4-6 and 4-8) that exist in or behind the. slope now being affected by the Buttermilk Creek slump area. More work needs to be done to characterize the layered units in this slope and to ensure that the model treats them in a realistic manner.
- 9) Likelihood that the entire Cattaraucus Creek watershed will continue to downcut and evolve The current application of the SIBERIA model does not look beyond the Buttermilk Creek watershed into the larger Cattaraugus Creek basin.
A detailed look at the evolution of Cattaraugus Creek is not warranted in this modeling effort, but, at the same time, it should not be assumed that Cattaraugus Creek near Springville has reached a stable base level or profile. Possible removal of the Springville Dam knickpoint is one consideration. Other important considerations are the overall gradient and profile of the creek and the bed material into which the creek is downcutting. The overall gradient of cattaraugus Creek from Springville to Lake Erie is fairly steep (about 530 feet in 40 miles). I have requested but not yet received a profile of the creek from the Corps. It is my understanding that the profile is convex upward between Springville and Gowanda (i.e., steeper in the downstream reach from Zoar Bridge to Gowanda than in the upstream reach from Springville to Zoar Bridge). The bed material is variously glacial fill and bedrock (interbedded shales and thin siltstones) of the Canadaway Formation. The bedrock exhibits many joints and shows some effects of an old shallow thrust fault (Bass Island Trend) that passes through the watershed between Springville and Gowanda. In general, it seems safe to say that Cattaraugus Creek is not a mature stream; it will continue to downcut for several millennia. The ongoing evolution of Cattaraugus Creek implies a gradual steepening of the modeled watershed. In other words, if the West Valley site facilities on the North and South Plateaus remain at constant elevation while downstream channels continue to downcut, the modeled topography will become progressively steeper and water velocities will increase accordingly. The SIBERIA model presumably takes this trend into account. 5
0 ,\\ I c.
- 10) Inability of the'model to conserve flow at closed depression e
SAIC's Dreft, p. 3, states that " Accumulation of water is not modeled and the water balance is not maintained at closed depressions." The following sentence asserts that the Wost Valley study area does not have closed depressions. On the contrary, there are several shallow. closed depressions (wetlands) in critical areas of the site. SAIC's Draft, p. 5 (Figure 2), shows a " Low Wet Area" along upper Franks Creek, and we saw a number of wetland-delineation markers on our April 15 walkover. See also the 1996 Draft EIS, Vol. II, Appendix P. It is important that the model be able to handle closed depressions of the type found on the site. i 1 l 4 1 6 .s}}