ML20058G393
| ML20058G393 | |
| Person / Time | |
|---|---|
| Issue date: | 10/11/1990 |
| From: | NRC - LICENSING SUPPORT SYSTEM ADVISORY REVIEW PANEL (LSSAR |
| To: | |
| Shared Package | |
| ML20058G373 | List: |
| References | |
| NACLSSAR, NUDOCS 9011130115 | |
| Download: ML20058G393 (188) | |
Text
_
OR G NA_
l OFFICIALTRANSCRFr OF PROCEEDINGS s
i Agency:
u.s. nuclear Regulatory commission Tide:
LSS Advisory Review Panel Meeting Docket No.
I' i
U"x"Anm Reno, Nevada cam Thursday, October 11, 1990 PAcn 216 - 353 I
.f ANN RILEY & ASSOCIATES, LTD.
(..
1612 X & N.T. Suha KO Mahlegon, D.C 20006 (202) 293-3950 9011130115 901101 ADVC t1 NACL'jcgf R g
216 1
l 2
UNITED STATES OF AMERICA l
3 NUCLEAR REGULATORY COMMISSION 1
4 l
5 l
1 6
l 7
LSS ADVISORY REVIEW PANEL MEETING 8
9 j
10 l
11 Quality Inn 12 3800 South Virginia Street l
13 Zephry Room 14 Reno, Nevada 15 Thursday, October 11, 1990 16 17 The above~ entitled meeting convened at 9: 00 a.m.,
18 pursuant to notice, John Hoyle, Chairman, presiding.
l 19 20 21 22 23 l
l 24 25
/
317 1
Appearances:
2 Mar'. lee Rood, NRC-LSSA 3
-P<,ter Cunning:, City Of L:: V:g :, M:v:d:
4 Kirk Balcon, State of Nevada 5
Malachi Murphy, State of Nevada 6
5111 ;;esten, -Netier.21 Arrh 4'>an 7
ehed: Merkel, FFI/= =
- a 8
Jey Oilb rg, She'3 Pit *==" (rrt/wante) 9 John Hoyle, NRC (Chairman) 10 Stuart Treby, NRC/OGC 11 Elgie Holstein, Nye County, Nevada 12 Dennis Bechtel, Clark County. Nevada 13 Liza Vibert, D.A.,
Clark County 14 Corinne Macaluso, D.A.,
Clark County 15 Barbara Cerny, D.A.,
Clark County 16 Lenard Smith, Lincoln County, Nevada 17 Boyd L. Alexander, U.d. Patent & Trademark Office l
1 l
18 Li r.d D ::ll, OOS/OCn"M 19 Also Present l
20
-Edw rd A. Tin ::, Jr., SAIC 21 Tom Nartker, UNLV-ISRI 22
- K:::= Tag 5'/2, Uupr yeur 23 Rawley Johnson, SWRI-CNWRA 24 Charles Acree, SWRI-CNWRA
~
25 Avi 0:nder, Cent:1
/
l
.n
218 1
Also Present: (continued) 2 Bruce Foster, SAIC 3
Lynn Scattolin, NRC-LSSA 4
Elizabeth Shelburne, NRC-LSSA 5
M :./ W.
Swin: ten, Stat: Of N:1:d:
6 Chip Cameron, NRC-LSSA 7
Dana Mennella, SAIC 5
8 David Nippert, SAIC 9
10 11 12
(
13 14 15 2
i 16 17 18 19 20 21 22 23 24
\\'
25
/
gr-219 1
PROCEEDINGS r'
s 2
(9:00 a.m.)
3 CHAIRMAN HOYLE:
Thank you for bearing with a late 4
start this morning.
We've -- I've been working with the NRC 5
staff on the letter that we decided that we would draft last 6
night.
I'm getting a xerox made of that letter now.
I 7
don't want to start off talking about that. I'm not ready, 8
for one thing, but I do want you to know that we've been L
9 working on that.
10 Yesterday afternc,n at the time we broke, we were 11 talking about headers, and I want to continue with that 12 discussion now with Kirk Balcom.
After that discussion, we 13 will go onto ths handling of technical data.
Kirk.
14 MR. BALCOM:
I'm going to go ahead and not use the 15 mike.
I think I can talk loud enough.
We left off 16 yesterday with a request by SAIC to add two more fields to 17 the head.er, and:the header working group and I pow-wowed for 18 a little while, and came up-with what we think is a simple, 19 hopefully easily-adoptable position.
20 One of the requests was simply for a field in the 21.
headers to note whether or not the material is coming in 23 from a submitter as copyrighted material, and we suggest 23 that that simply _be a yes or a no kind of a field; one that 24 we could add 1.aw which would certainly take care of the 25 problem.
It certainly makes a lot of sense to go ahead and
/-
220 1
do it now even though we don't know the final disposition of 2
header material.
But at least that will allow us to do 3
something once material comes in without a discussion -- you 4
know, what actually should happen to copyrighted material, 5
how it should be handled.
So we suggest that we adopt that 6
field.
7 The other field was called " package I.D."
What 8
that means is that we and the header working group and SAIC 9
in the course of their development of the system, we're 10 working with the concept of pointers; in other words, the 11 relationship between documents, the parent / child concept 12 being kind of a numerical system where you can point forward 13 or backward to various documents.
But that's kind of a 14 relationship between two documents.
It's not -- it doesn't 15 particularly help with the relationship between whole 16 packages of material, and when I say " packages," we mean i
17 predominantly Q/A packages, j
18 So SAIC's technical recommendation is that the 19 corner field will not suffice for that, so we need to add 20 another field, which will respond to the need to reassemble c
21 the entire package of, let's say, of a QA submission, 22 intact, so that the reader, the user, will is rLle to re-23 assemble the package.
If you remember, a lot of these 24
. materials are going to be dissembled for scanning, and 25 sometimes it doesn't -- sometimes it's difficult to re-d
221 1
assemble them -- Let's put it this way, they can all be re-2 assembled properly, but it may not be cost-effective to re-3 assemble them.
And, of course, we don't know exactly how 4
that's going tu happen in the capture station.
5 So the header working group suggests that the 6
submitter -- we haven't worked out the protocols for 7
Eubmission materials yet, but it will probably go something 8
like this:
For materials which are in hard copy, it's an 9
entire Q/A package, would have some kind of a packing slip l
10 or an invoice with it, which would come to the capture 11 center and say what's in it.
And they would also mark -- it 12 would be the submitter's responsibility to mark the fact 13 that this is a package that needs to stay together for ease 14 of re-assembling on the LSS.
15 So we're suggesting that some kind of protocol be 16 worked out; that the submitter take a responsibility for 17 saying, "This is a QA package, and it needs to stay 18 together."
That will be the -- the tipoff to the to 19 whoever's running the capture station; that they need then J
20 to incorporate some system, a numerical system, and is yet 21 to be decided, which will actually be quite simple, which 22 will allow all those packages to be numbered in separate 23 fields so they can be re-assembled.
So what we're 1
24 suggesting is that you adopt the concept, and that we add 25 another field called " package I.D.",
and there needs to be
/
i l
l
222 1
a discussion on that.
2 MS. CERNY:
We're dealing with packages right_now.
'3 You may want to look at the way -- I don't know how NRC is 4
dealing with their packages, but what we're doing --
5 individual documents that will be part of packages are 6;
collected.
You know, we have our local record centers 7
speeding into a central records facility, one in Nevada and 8
one in Washington.
And the local record center collect j
9 documents that are, after a certain length of time, whether 10 the package is complete or not, so that -- because some 11 packages will take two years of your studies before they're 12 completely finished, and you don't-want to hold them up for 13 that long.
So they're sent on in pieces.
14 Individual -- and that's one way of handling 15-packages, and then they're -- they are part of the package, 16 and when the package is complete then, they are also -- they 17 are resubmitted as a package, so that the package is kept 18
-together.
Now what we're doing with -- for example, we have i
19 a quality record center that just takes quality records.
20 And the individual documents that go into those are sent to 21 the record centers as individual documents, but also they're 22 kept in the quality record center until the package is 23 complete.
And then they are resubmitted to the record 24 center as a package, and microfilmed together, and kept 25 together as a package again.
d
223 i
1 So you have each individual piece coming in as it 2
is. prepared, so you don't wait until the end.
So there are, 3
you know, a number of ways of doing this.
And I think, you 4
know, when you start to talk about well, how do we actually 5
get our documents into packages.
6 MR. BALCOM:
So you're making a suggestion for how j
7 we should do that in the LSS:
In terms of taking incomplete 8
submissions and keeping them until they're complete, or 9
submitting -- assigning an empty slot for that part of the 10 package that hasn't come in yet?
11 MS, CERNY:
That's right, that's right.
11 MR. BALCOM:
Part of the materials would be 13 available, but --
14 MS. CERNYt
'100'ra going to run into exactly the 15 same. problem we're running into with the idea that you don't 16 want to. keep a package until it's complete.
You want to put 17 it in in pieces.
18 MS. SHELBURNE:
.But once we know --
19 MS. CERNY:
Pardon?
But.you want a number --
20 MS. SHELBURNE:
Once we know that it is 21 potentially going to be part of the package, it is assigned 22 a unique package number to it; subsequent documents coming 23 in will get the same number.
24 MS. CERNY:
Exactly.
You --
25 MS. SHELBURNE:
As long as it's (inaudible) is
/
224 1
part of it.
2 (A chorus of laughter) 3 MS. CERNY:
No, you need your package number, 4
absolutely, but I just was saying, once you have your 5
package number, it isn't enough to just say, "Well, we're 6
going to collect it as a package."
You've really got to j
'7 worry about how the pieces come in.
8 MR. BALCOM:
I assume now that will be part of the 9
protocol, the submission protocol, but we haven't touched 10 that. Any other discussion about package I.D. or any 11 opposition to adopt these matters as a field?
Sounds like 12 we've got two new fields.
How would you propose me to 13 respond in writing to that?
Should we document this, 14 describe those fields, and submit them to you?
15 CHAIRMAN HOYLE:
Yes, please.
16 MR. BALCOM:
Okay.
On to Abstracts 3.
I guess we 17 have half an hour, and I'd like to take-just a quick minute 18 to give you my: viewpoint.
My sense of abstracts here at 19 this table is that the real users who would benefit from
^l 20 them probably are here.
This is just an educated guess on 21-my part, because I really don't that for a fact.
22 I'm suggesting that probably the primary people 23 who would benefit from abstracts are going to be the 24 technical users, and I'm also going to guess that their 25 primary value is not in searching but in browsing.
And that
/
225 1
we also -- We have a substantial, very, very significant way 2
of getting to materials.
We have the good fortune of 3
designing a system that will have headers and full-text and 4
descriptors.
And I think the descriptors will -- to the L
5 extent the descriptors are well-done during cataloguing, but 6
that they will be the primary utility for certain kinds of l-7 users to find documents.
ThoseIwho don't use that, will use 1
L 8
either the headers or the full-text, so we have an 9
~ incredibly complete way of finding documents.
And I'm not L
10 making an argument one way or the other for abstract.
I'm l
11 just saying what I think -- what my opinion is about how 12 they'll be used.
l-13 I think probably utility to technical users will 14 be -- and perhaps for the lawyer and the regulatory people -
15
- is that given large sets of documents, you'll use an 16 abstract to quickly decide whether or not you want to read 17.
the rest of this document.
And there are probably other 18 ways -- There are, of course, other ways of looking at the 19 use of 20 abstracts, but that's my guess as to what their primary 21 value would be.
So I guess now we have 27 minutes to talk 22 about abstracts.
1 23 Let's see, SAIC, we were waiting for SAIC to look l
24 over the papers, I guess, and so I'm assuming SAIC has 25 something to say about that.
/,
I
236
-1 MS. MENNELLA:
I, too, have to take pretty much 2
the same tact that you did.
I'm not offering one way or 3
another for the abstracts, but to me the real issue is what 4
the purpose the abstract would serve to the user, either as 5
a search aid or as an aid as you mentioned, for determining 6
the relevance of a document.
7 I think the issue of who would write the abstract 8
or what the standards are, are secondary to determining 9
whether the' abstract has a good purpose to the user.
The 10 other thing I'd like to point out is that we've identified 11 three types of entries in the LSS.
There are header only 12 entries which have no ASCII or images, and these are mostly 4
i a
13 the non-document materials.
Then we have header image only 14 documents.
These are ones that do not have any ASCII but do l
15 have an image.
These are things like handwritten documents, I
16
-maps, tables, and anything that can be scanned but does not
{
17 easily convert to ASCII in the present day technology.
And
)
18 then we have a header in which -
and ASCII documents, and 19 these are the majority of the documents.
j 20 In our experience, those technical documents do 21 have abstracts, or they have materials that can be used as 22 abstracts.
For example, summaries, conclusions, f
23 introductions, material like that which we have been I
^
24 capturing and using as an abstract.
We have checked with 25 NTIS and Healey's Technical Information Center, and they do f
/
227 l
1 pretty much the same thing.
They prepare very few abstracts 2
for their documents, feeling that they can get th7 3
information from the document itself in some way or another.
4 We talked to them because we felt their 5
collection, with the exception of correspondence, their 6
collections are very similar to the collections that we'll 7
have for the 123S.
Getting back to the three types of 8
entries, an abstract or some kind of material will have to 1'
9 be prepared for the documents that do not have any ASCII at l-10 all. And our estimate at this time is probably -- that will 11 be-'about ten percent of the documents that exist.
And 12 looking at the collection in the 130,000 pages that we have 13 just finished processing, about 50 percent or more have had 14 some kind of material that we could use as an ASCII.
15 CHAIRMAN HOYLE:
How many, 50 percent?
16 MS. MENNELLA:
Yes..
17 MS, SHELBURNE:
When you say " technical material,"
18 do you --
19 MS. MENNELLA:
Those are things like technical 20 articles cnr technical reports.
I guess maybe scientifically L
21 that if you're including engineering material --
1 L
22 MS. SHELBURNE:
Is that linked to or is that l
l 23 definable by the current document type codes?
24 MS. MENNELLA:
Yes.
Everything -- that would be
=
25 in the report, you know -- also things that probably the ji 1
l 1
228 j
1 majority of the things that are in the publication of it.
2 (Discussion was held off the record.)
3 MS. MENNELLA:
I just want to state from that I
4 comment to clarify that the paper that we wrote was --
5 really only is a discussion involving that set materials, 6
which is the majority of the materials, which will be in 7
text searchable.
It is a given there should be some 8
description abstract, summary of whatever text -- however, 9~
that's defined in the technical data.
But here, I think 10 we're hopefully limiting that discussion that set of 11 material for which there is surgical text, headers, 12 descriptors.
13 CHAIRMAN HOYLE:
Boyd.
14 MR. ALEXANDER:
This is just sort of in the area-15 for what.it's worth, John -- We have abstracts in the patent 16 office -- currently we do, for all foreign patents.
And the 17 reason why we-have those now -- we didn't have them a few t
18 years ago -- is that we rely on text search to'do a 19-preliminary hit ratio to find patents that may be relevant
,20 to the examining patent activity that's going on.
21 Before that, we had only the titles-and just a 22 brief bibliography, some bibliography-type data for foreign 23 patents: the inventor, the inventor's name, the invention 24 itself, the country, things like that.
And in doing a text
^-
25 search, this was not enough for us to find relevant patents
/
l-L
l l
L 229 l
1 for the kind of work we do.
So we decided to try L
2 abstracting all foreign patents in English to do a full-text l
3 search on those, and just reserved the usage of that 4
database for us.
We couldn't really use effectively foreign 5
patents -- at least our examiners couldn't -- just by the 6
basic header.
7' Now, we-don't have quite the header information 8
you have, so maybe yours is more robust, but now there's a l
l 9
reqtirement by international agreement that all patents in 10 Europe and in Japan and every place that we trade with, 11 we'll provide an abstract of that patent and that search, 12 And it's, as I said, it's just reversed the whole use of l
l 13 foreign patents as a research base.
So in that respect, 1
14 abstracts have helped us use our text-search system in 15 finding the kind of information we want to have.
16 MR. BALCOM:
Boyd, can I comment on that?
l' 17 MR. ALEXANDER:
Yeah.
18 MR. BALCOM:
Can I assume from tbat you do not l
l 19 have the full text of the actual patent subnission available l
'20 L
l 21 MR. ALEXANDER:
Not in formed patents.
Well, we 22 do in about 40 percent of the European patents but none of 23 the other countries -- and, of course, Australia and 24
'anadian patents we --
25 MR. BALCOM:
So for you it's a substitute for the
/:
l
230 1
full-text that's not available in terms of searenes.
-2 MR. ALEXANDER:
Becausc that allows us to use that 3
and then that narrows it to determine -- then we can 4
translate it after that, but -- and it's not quite the same 5
problem I understand, but if you want to use the text-search 6
capability you are building, it does enhance that a little 7
bit if there's an abstract that you can then search where 8
there isn't a full-search document available.
I'm assuming 9
that that's the case.
In some cases you have full-texts in 10 some of your documents, and in others none is available, but 11 you might have to search for that.
12 MR. BALCOM:
Well, we've had full-text in a very, 13 very large percentage of it.
14 MR. ALEXANDER:
Okay.
Well, if you had it 15 available, then it's not a help, but to those where you 16 don't have it, it's good.
17 MR. BALCOM:
And I think also that we will have --
18 we're planning to have abstracts for everything that's not i
19 available full-text.
Is that correct?
20-MR. ALEXANDER:
The second part of that is, is 21' that's for the search, and then as you said for browsing, 22 they can quite often browse that abstract without worrying 23 about getting it translated, and find that it's enough to 24 make a decision as to whether it should be translated.
\\~
25 The third point that we have found out is that the
/
239.
1 quality varies.
If the people who do the abstracting, let's 2
say, in Japan, are the examining -- patent examiner in Japan 3
-- the quality is very good.
If they give it to a clerical 4
person or a, if you would call it a professional abstractor S
.there, the quality goes down, because they don't know the 6
technology.
And knowing the technology is important in the 7
abstracting of those kind of documents, so the quality 8
varies all over the map.
You do worry about that, but it is
)
9 still better than having nothing at all.
But that's just 10 for what it's worth, John.
j 11 CHAIRMAN HOYLE:
Yesterday we talked about whether 12 it could be broken up into classes of documents.
Are there
/
13 any further thoughts on that topic?
We talked about the 14 possibility of page number-as using that as a cutoff.
Does 15 that make any sense?
16
.KR. TREBY:
Well, I guess I'd like to suggest that 17 it doesn't for all the reasons we said yesterday.
18 MR. MURPHY:
I agree wich that.
19 MR. BALCOM:
It does not, is that what you're 20 saying?
21 MR. TREBY:
Does not.
l 22 MR. BALCOM:
How about the classification into 23 classes, types of documents, or --
j 24 CHAIRMAN HOYLE:
Can we rule out memoranda.as a 25 class?
/
232 1
MR. BALCOM:
I would think so.
Do you mean under 2
-- well, memoranda is normally short.
3 CHAIRMAN HOYLE:
Well, some of them are three or 4
four pages long, but they're still --
5 MR. HOLSTEIN:
I think it might make more sense, 6
Mr. Chairman, if we --if the group decides that there is an 7
inclination to pursue abstracts further, while we can 8
certainly discuss these types of cutoffs, it may well be 9
that the staff would want to give us -- present to us some 10 options in greater detail for pros and cons of different 11 ways of segmenting this number of pages versus subject type 12 of document, etc.
We can certainly take a crack at it today 13-but I'd like to, if I could, take three or four minutes and 14 respond to a~ number of points laid out in the staff's paper.
15 I spent some time last night talking to the staff about it 16 and reviewed the document again, and I'd really like to go 17 through_a number of the elements in the paper for the 18 panel's consideration.
19 First, I felt that the cost assumptions in the 20 paper were lacking in'some respects.
First, I don't agree
~
21 that -- or I felt that the paper should have figured in an 22 assumption ~along the lines that we talked about yesterday in 23 which the authors would do the abstracts.
The cost 24 considerations as they're laid out in the paper now suggest l
25
-- assume only that there would be some independent party, pr.
I
233 1
and as we've just heard, there is some question about the 2
quality of that type of work done by people other than the 3
authors.
I realize there would be some sort of cost 4
associated with having the authors -- with having the 5
authors do it.
The paper, however, ignores that as an 6
option in its cost assumption.
7 Second, the paper does not calculate the cost to 8'
the users associated with having to do additional review of 9
documents, which once selected, may or may not be relevant, 10 which I think goes to the heart of one of the major problems 11 I had with the briefing paper; and that is, that it -- as it 12 centers most of the discussion around the debate which it f
13 correctly identifies as being the principal debate derived 14 from the literature, as being one that tries to balance 15 recall versus precision capabilities.
Later on in the 16 paper, however, on page five,_they do mention in passing the 17 point that's been touched on briefly here this morning; and 18 that is, the benefit to users of the relevancy review.
19 Again, that's the point associated with determining whether 20 or not 21
-- how long it takes you to determine whether or not a
'22 document, once' selected, really is of value to you.
23 The -- Where I come out on this then is to suggest 24 that regardless whether or not the panel endorses the notion 25 of incorporating abstracts into the LSS as a designed
/
l
234 1
feature, that we have to in any event considering 2
recommending that abstracting be adopted as some sort of 3
formal format criterion for most, if not all, authors of 4
documents going into this system.
Now, we have heard 5
arguments at the last meetjag and at this meeting that that 6
would impose enormous additional costs on the authors of the 7
documents.
We've also heard arguments that it would be next 8
to impossible to enforce it, next to impossible to 9
incorporate it into their contracts in the case of the 10 contractors.
11' I have real problems with those arguments without 12 going into great detail.
We've just heard how the -- in the 13 case of the patents, international patent discussions, 14' they're moving in the direction of a standardized format.
15 We've also heard how a large percentage of documents already i
16 have these abstracts.
So at a minimum, if we did not 17 incorporate abstracts as a designed feature of the ISS 18 itself at a minimum, if we had a larger percentage of 19 documents containing abstracts-by virtue of making that a 20 recommended, if not a required feature of the performance of l
21 the authors, we could at least assure users of the system 22 the capability of being able, once so having selected the 23 document, to quickly review the abstract of that document in 24 the text itself.
25 That may represent a compromise as opposed to
/I i
235 1
continuing the debate about the cost and utility of 2
incorporating it as a designed feature itself.
3 CHAIRMAN HOYLE:
Any comment on that?
4 MS. CERNY:
- Yeah, I -- You know, I've said this time. number three or four -- but really, the idea of 5
6 putting into every contractor's contract a requirement that 7
they must produce an abstract, I just really don't think is 8
workable.
In the tirst place, there are many people who-put 9
contracts in place to colice that they do it.
I mean, who 10 is going to do that, anC then they would have to make sure 11 that their contractors, the people who are in charge of 12 these contractF, that their contractors really do comply 13 with them.
14
'Well, who's going to be in charge of all the cen-15 tracts?
I mean, we're going through something right now 4
16 with quality assurance to make sure that all contracts-go 17 through-a review so_that they have the proper quality 18 assurance features in them.
And that is really extremely 19
' difficult to do.
Wo have a whole organization that just 20 looks at all procuremeats-and all contracts to make sure L
21 that that happens, and you know, that's something that we as a
22 a' program have to do; it's basic to the program.
But 23 something like this is just very quickly going to get out of 24 hand, and nobody is going to be in charge.
And I just don't 25 think it's really workable in a large bureaucracy.
/.
l
236 1
So, you know, I can't go along with that.
I can't 2
agree to that, because I don't think it's going to work.
3 That being the case, I don't see if we don't want to put the 4
money into doing the abstracting, then I think we really are 5
back to Betsy's -- Betsy's paper -- the NRC's paper, and the 6
recommendation that if abstracts are there, we pick them up, 7
but we don't put them in a separate abstract appeal to 8-search on.
We search on them as part of the full-text.
9 Otherwise, from an information science perspective, you have 10 a really skewed sample if you go in to do a search on the 11 abstracts.
12 So for both, you know, practical reasons and
(
13 information science reasons, when you put them together 14 that's.the recommendation I'd make, that we follow Betsy's 15 recommendation in this paper.
16 CHAIRMAN HOYLE:
Elgie, your proposal was also to 17 leave the abstract with the document, so you first get a hit 18 on the document, and then to find out whether you really 19 want that document, you call up its abstract 20 MR. HOLSTEIN:
Yes, although maybe I've 21 misunderstood the way it actually would occur if we did not 22 have abstracts incorporated as a designed feature in the LSS 23 itself, but that's what I had in mind.
Maybe there's a~
24 different approach.
Essentially, I was trying to move to a 25 middle ground there.
I have a problem, as I said, with the
/
237 1
way in which the debate has centered on this question of 2
whether or not abstracts age you in the selection process 3
when, as far as I'm concerned and others who were here 4
yesterday, that is not the primary value, particularly in a 5
full-text retrieval system, a full-text searchable system.
6 That's not the primary value of abstracts, so it 7
has struck me that to a large extent, the pro and con debate 8
has centered on an evaluation of issues that don't take into 9
consideration the primary benefit to users in this type of 10 system of abstracts.
So I was trying to get around the cost 11 problem as well as the problem that Barbara's described, and 12 others have described, about what you do if you have some t
13 documents that have it and some don't.
This is something
(_
14 that I'm very sympathetic to by saying, let's see if we 15 can't move the authors, contractors, whomever, somehow in 16 the direction of providing these abstracts as part of the 17 documents, so that once retrieved, even though you wouldn't 18 search on the basis of the abstracts, once retrieved you 19 could at least achieve the same goal of review -- ready 20 review ability, reducing the cost at the user's end in so 21 doing, by providing those abstracts.
22 I don't know enough about federal contracting to 23 know how erroneous this would really be.
Barbara has taken 24 a strong position on that.
I'm not sure that I have readily 25
-- that I readily see a parallel with the QA process, though
/
l l
238 1
there may be one in part, because the QA process has 2
incorporated a lot of other cultural changes, if you will --
3 MS. CERNY:
No.
I'm talking strictly about 4
getting requirements into contracts and assuring that those 5
requirements are met.
Oc everyone who writes a contract has 6
to go through a review of those contracts to assure that 7
it's in there, and then that they are, in fact, followed 8
when the deliverables come in.
That's the only way you can 9
guarantee that this is going to happen.
10 Well, if we're going to make this such a 11 requirement that every deliverable has to have an abstract, 12 we're going to have to have somebody who does nothing but 13 look at contracts to make sure that it's written in there, 14 because after all, many, many people write contracts.
15 They're not going to know that this is a requirement.
How 16 do you get them.to know it?
And then 1: hen they get their 17 deliverables in -- we have turnover of ataff all the time 18 who write contracts -- how do you enforce this?
It slips 19 away very easily.
That's why there's a parallel.
We have 20 no QA organization that does that.
21 MR. HOLSTEIN:
Right. -There is a big difference 22 though, and I would not --
23 MS. CERNY:
What's the difference?
24 MR. HOLSTEIN:
And I would not argue in favor of i
25 the degree of policing that you have in the QA document to
/'
l
239 1
process.
I would be willing to simply rely upon an 2
exhortation, if not some stronger directive, on format i
3 issues.
Now, preparation of reports by all kinds of 4
contractors are commonly guided, at least to some extent, by 5
format requirements.
I would never suggest that this would 6
be worth having a whole of f!ce within the LSS set up to make 1
7 sure that was done.
8 In the case of QA docume.nts, however, that sort of 9
enforcement is necessary because of the extreme importance 10 of the quality assurance --
11 MS. CERNY:
Sure.
12 MR. HOLSTEIN:
-- to the licensing process itself.
I' 13 So I don't think that sort of enforcement is necessary in 14 this case to move us from the 50 or 60 percent of documents 15 that currently have abstracts, to the 80 or 90 percent of l
16 documents that I'd like to see have abstracts, understanding 17' that you wouldn't get 100 percent.
But you might get,80 cnr 18 90 percent instead of 50 or 60 percent.
19 MS. CERNY:
Not without policing you're not going 20 to, and people who write the contracts -- and even if you 21 had a big campaign to make sure that everybody who was 22 writing contracts now requires this and gets it into their 23 deliverables, a person leaves.
And I maintain that within a 24 couple of years, the whole thing is going to have trickled
(
l 25 away, because there's going to be nobody left, basically,
/
q t
240 1
who is policing this.
2 MR. HOLSTEIN:
Of the 5 percentage of documents 3
that currently have -- of the high percentage of documents 4
that cunrently have abstracts within them, do those 5
abstracts appear, to your knowledge, as a result of the 6
author's own poetic license, their own editorial judgment, 7
or --
8 MR. EALCOM:
Is this a high percentage?
9 CHAIRMAN HOYLE:
Somebody comment.
I think it's a 10 low -- actually, quite a low percentage.
11 MR. HOLSTEIN:
I thought I heard -- I thought I 12 heard 50 percent, but maybe they were talking about --
(-
13 MS. SCATTOLINI:
No, it isn't 50 percent at all.
14 I would refer you to page seven of the paper, where there's 15 a table (inaudible) 16 MR. HOLSTEIN:
Could you speak up a little?
I 17 can't quite hear you.
18 MS. SCATTOLINI:
If you look at this table, 64 19 percent of the documents are correspondence.
There are no 20 abstracts that I know of being created for correspondence 21 tode.y.
The same with legal documents, which comprise 22 another 13 percent of the collection.
Other reports and 23 publications I believe we can say about 50 percent.
Do you 24 think that's true, about 50 percent of the reports and
\\)
25 publications are abstracts?
/
l l
241 1
MR. NIPPERT:
Not higher.
2 MS. SCATTOLINI:
A little higher on those.
3 MR. HOLSTEIN:
Of the technical reports.
That is 4
the figure that I thought I heard.
5 MS. SHELBURNE:
You're limiting your discussion to 6
these technical reports.
7 MR. HOLSTEIN:
What I'm really trying to do is 8
make the case that documents are prepared every day of the 9
week in accordance with format criteria that are established 10 by the people who are requiring those documents to be 11 produced.
But Barbara doesn't agree with that.
12 MS. CERNY:
That's not even true.
There's a 13 deliverable that says -- that it's written there will be a 14 final report produced, and often that's all you have.
There 15 aren't1for every requirement.
You'll produce a final report 16 on such a date.
That's how these contracts are written.
17 MS. SCATTOLINI:
Well, of course, a great many of 18 these documents were not being created under a contract.
19 MS. CERNY:
Ours are.
20 MS. SCATTOLINI:
All of them?
21 MS. / ~ERNY :
Most of them.
22 MS. SCATTOLINI:
Not correspondence, but 23 correspondence isn't the issue, because abstracting 24 correspondence doesn't make much sense to -- I think anybody
~
25
-- we're talking about reports, and they're done almost
/
(
~w
242 1
totally by contractors.
2 MS. CERNY:
Well, if we're limiting the_ universal 3
reports, that's a different issue.
If we're talking about 4
acquiring all offers, documents, that are being placed in 5
the LSS, to create an abstract, that is --
6 KR. BALCOM:
I don't think that's ever been a l
7 requirement.
8 MS. SCATTOLINI:
Let's define what the universe of l-9 our discussion is.
10 MR. HOLSTEIN:
Is the principal objection to 11 providing abstracts one of, as you see it, enforcement, or 12 is the principal objection one of cost?' Or is really both Ji 13 equally?
14 MS. CERNY:
Well, it's a question of cost, but the 15 enforcement issue, it depends on how -- if you decide you 16 want to do abstracts, how you want to do it.
To me, it is-17 unenforceable to have the authors do it, which puts you into
~18 having abstractors do it.
And then,_to me, it becomes an 19 issue of cost.
20 MS. SCATTOLINI:
Well, I would like to clarify the 21 cost issue as well, and that is that this paper assumed that 22 a contractor was doing it, and -- a rate of $66 per hour.
23 That makes him very high.
But actually, my understanding is l
24 that the NRC techni:al staff rate is $86 per hour, so to l
l 25 have the NRC technical staff do it, or a contractor's staff
/.
243 1
being paid at a comparable rate, it's going to be as 2
expennive as, or more expensive than, costs here.
3 MR. HOLSTEIN:
The protocols for submission to l
4 which you were addressing yourself earlier, and I think you 5
gave us one example, some kind of packing slip saying what's 6
in hard copy documents -- I don't know how expensive those 7
protocols are being -- are ultimately going to be -- but how 8
are they going to be enforced?
9 MR. BALCOM:
Well, there's a compliance 10 requirement in the rule for one thing, and I think it's the 11 consensus of everybody here that that will be a substantial, 12 you know, undertaking, and part of the agreement to bill.the
~13 LSS, and part of the capture station protocol.
14-MR. HOLSTEIN:
Just so that I understand these 15 protocols:
So the documents would not be accepted for 16 loading into the system if they did not conform with these 17 protocols.
Is that right?
18 MR. BALCOM:
Right.
j 19 MS. SHELBURNE:
I'd just like to focus on 20 something that.Elgie said in terms of if -- and we all 21 acknowledge they have lot of different benefits of abstracts 22
-- but Elgie has pointed out the one irrelevancy with 23 people.
I think it would benefit.
It's just a question of 24 the cost versus the benefit.
I'd like to point to the
("'
25 option of C.3 which we did recommend and does acknowledge --
/
244 1.
it also acknowledges the other problem that not all 2.
documents have it -- and C.3 is limited to technical 3
documents.
So given that we could define a universe, and I 4
would like to try to tie it to a document type code scheme 5
so that searchers would have some feeling for that.
6 If the documents did have an abstract, or as John 7
has said, something like executive summaries or -- you know 8
a lot of times it doesn't say " abstract," but it can be 9
used.
If that text can be grabbed, pulled out of her tubal 10 field, called abstract, in the header, without significant 11 standards of how it was done, but something that would allow 12 the benefits of the summary of the document for review, 13 unless they have created a hit list, that they're scanning r
14 the headers to say, "Ah, yeah, that's on my topic, but it's 15 really not on that order."
2t's too deep or too shallow on 16 the topic.
17 I would like to point to'C-3 to see if that could 18 be discussed and maybe with some modifications, and what I'm 19.
hearing is Elgie's recommendation.
20 CRAIRMAN HOYLE:
If you would -- C.3, the abstract 21 or the executive summary would be grabbed from the document, 22 and put in a separate field.
23 MS. SHELBURNE:
It would be.
24 CHAIRMAN HOYLE:
It would disappear from the 25 document.
It would disappear from the full-text.
/
245 1
MS. SHELBURNE:
No.
It would still be in 2
fulltext.
We're just going to copy it over.
3 CHAIRMAN HOYLE:
You're copying it >ver.
4 MS. SHELBURNE:
We're certainly not going to 5
destroy the full-text.
6 CHAIRMAN HOYLE:
Yeah.
But it says, "Put in 7
header but not allow this abstract field to be searched."
8 What does that mean?
9 MS. SHELBURNE:
That means that you could not 10 limit -- you couldn't say, "I want all the documents with, 11 you know "X" terms," in the abstract.
Because my concern 12 which is in bold print above C.3, which is the same options
[
13 effecting.the searching, this would give the benefit of 14' relevancy review.
15 MS. SCATTOLINI:
You would be able to browse it i
16 and read it as a separate field, and it would correct -- and j
i 17 bibliographies.
It's just that you couldn't use it as a 18' search parameter.
And the reason we're recommending you not L
19 use it as a search parameter, is unless we can clearly l
(
20 define exactly the collection of documents that are going to, l
l 21 be abstracted, the user would really have no idea what he is 22 searching.
It would be a random collection of documents of 23 the database, so when he got this hit list, he wouldn't know 24 what it represented.
25 KR. BALCOM:
So, do I understand that the abstract
/^
L l
246
'I would actually be there twice: once in the abstract field; i
i 2
once in the body of the document.
It would be searchable in i
3 the body of the document as part of the full-text, not 4
searchable in the abstract.
So you wouldn't lose 5
searchability.
6 MS. SHELBURNE:
No.
I mean, you can still get the 7'
words and the phrases from the text search.
You just 8
wouldn't officially be creating a universe of documents.
9 MR. HOLSTEIN:
And you would meet what I see is 10 the need for enhancing relevancy review capabilities in the 11 system, and' acknowledging that abstracts in this type of 12 system would not be necessary for the actual identifice. tion 13.
'of documents and retrieval of documents.
I certainly --
14 MS, SCATTOLINI:
.That was our -- was to pick them 15-up if they exist, and put them in an abstract field for the 16 purpose of browsing, and begin to print mttt bibliographies.
17 I think the only other practical alternative is to say we 18 will abstract all of the certain-document plans, such as 19 reports. And then we have a very practical issue of how do 20 we implement and enforce.that,'particularly,-if you want to L
21' require the author to do it, because if the authors, for a l
22 variety of reasons don't do it, then the documents come to 23 the LSS.
It's going to bd a real key to the user of the 24-community, that these documents be loaded right away; that
/
25 they be available in a timely manner.
And I'm concerned l
1
/
4
247 1
about having to send those documents back to the authors who 2
may be DOE contractors, and trying to get them to do it.
3 MR. HOLSTEIN:
I don't want to --
4 MS. SCATTOLINI Then we're kind of placed in a 5
role wnere, if we get the document and it doesn't have an 6
abstract, we need to create it, because otherwise you're 7
going to have this random collection of documents that have 8
abstracts.
9 MR. HOLSTEIN:
Okay.
I'm going to -- our 30 10 minutes is probably up, if not almost up.
I'm going to 11 yield to our -- to the experts here on this and withdraw n1 12 urging that we somehow put the onus on the authors to 13 produce thic.
I would like to say, however, that I'm going 14 to be very interested to see how the protocols and the g-15 enforcement of those protocols for submission will be 16 developed in light of the comments that enforcing anything j
17 on any submi.tter to the system is extraordinarily difficult.
18 A and B requires some cer. tral station quality control 19 people.
20 MS. CERNY:
You see, I don't --
21 MR. CAMERON:
I don't th.nk Barbara said that.
22 MS. CERNY:
I didn't say that.
No, not at all.
23 MR. CAMERON:
She didn't say that enforcing any of 24 the requirements on the submitters is going to be difficult.
25 I'm going to be talking a litt?e bit later on about our
/
s l
I l
248 1
compliance evaluation program, and I can explain a little 2
bit more about that.
But I didn't want Barbara's comments 3
to be construed as saying that this enforcement would be 4
difficult I think she's talking about snmethit.s 5
specializeo.
6 MS. CERNY:
I'm talking about contract management, 7
which is really very different.
As far as the submitter's 8
protocols go, we have an organization now that handles all 9
the records of the program.
We put them in a system.
It's 10 going to -- They're going to move from there to the LSS.
I 11 don't see that as an issue, because that's under the control 12 of the people who do records management.
That's their job.
l 13 That's quite dif ferent than telling everyone in the program i
14 every time you write a contract, you have to write into the 15 contract.
It's very different.
16 M*s SCATTOLINI What Barbara is saying is that j
17 she currontly has a records manager contractor that would be 18 doLag the descriptive cataloging on the field terms.
19 MS. CERNY:
That's right.
20 MS. SCATTOLINIt And that contractor is directly j
21 under her control.
22 MS. CERNY:
That's right.
23 MS. SCATTOLINI But if you say abstracts have to 24 be created by the authors, you're then opening up the 25 window.
Why do they have to be --
/
249 1
MS. CERNY:
I'm not concerned about the protocols 2
to the --
3 MR. BALCOM:
I've got one thing, John; that is, 4
that Elgie be -- system design does include abstracts, 5
whether -- no matter what we decide here about abstracts.
6 This is all going to be included as part of the design; 7
there will be a separate field, and as a separate field, 8
you'll be able to manipulate for that field as it appears on 9
your display.
So that's -- we should probably that here.
10 It's not being part of the discussion.
Does Dave have time 11 for a comment, John, a quick one?
12 CHAIRMAN HOYLE:
Okay, sure.
13 MR. NIPPERT:
I guess es I hear this proposal, it 14 says we're going to have an abstract field that is not 15 searchable, but yet there's document types for which there 16 is no full-text for which we had to make the abstract be l
17 unsearchable.
Now, that :o me is kind of a dichotomy.
I 18 guess one of the solutions is if you don't want the abstracc 19 field to be searchable for some type of document, make it 20 not searchable of all types of documents.
And then these 21 documents which have no full-text, we take whatever the 22 abstract is -- or the summary of that document is going to 23 be -- and actually move it down into the full-text area, so 24 there's a piece of it in the full-text -- so if you want to 25 find if one was in the abstract, you're going to find it in
/)
250 1
the full-text portion of the document database.
2 MS. SHELBURNE:
And I think that has another 3
benefit, tool so that when people search full-text, they get 4
all documents, you know, whatever words or phrases, for all 5.
documents, whether they have text or not, because they are 6
not arbitrarily excluding all documents that are on the 7
abstract.
So I think that has -- I'll agree with this.
8 MR. BALCOM:
But what I'm hearing is that we can't 9
settle this today.
Do you think that's the bottom line?
10 CHAIRMAN H0YLE:
The consensus seems to be going 11 in the direction of not doing, you know, any kind of bulk 12 abstracting --
13 MR. BALCOM:
But I think we did settle it.
14 CHAIRMAN H0YLE:
-- but going with C.3.
15 MS. CERNY:
I think we can settle it.
16 MR. BALCOM:
Well, I'm not sure I agree with not 17 searching the abstract field.
18 CHAIRMAN H0YLE:
Okay.
19 MS. CERNY:
Kirk, you can do --
20 MR. BALCOM:
I mean, I'd be willing to --
21 MS. CERNY:
Listen, you can always -- you can 22 always do a search on abstract and --
23 MS. SHELBURNE:
In the same paragraph.
24 MS. CERNY:
Yeah, in the same paragraph, and you 25 would be able to do it.
j.
l 1
l
251 1
MR. BALCOM:
Let me ask this -- this may resolve 2
it quickly.
For the documents which don't -- image only 3
documents and non-technical documents, you would have to 4
maAe the abstract searchable -- or whatever you want to call 1
I 5
that field, if you call it a description -- but you're 6
saying from a design standpoint, Dave, we could --
i 7
MR. NIPPERT:
Move that. down and actually make a 8
full-text page as only the abstract of it.
i 9
MR. BALCOM:
Even in a non -- even in a submission l
10 that doesn't have any ASCII text with it, would appear 11 twice; once in the abstract field, so it would be a search -
12
- so, oksy.
That's the long way around the barn.
13 MR. NIPPERT:
-- abstract is sometimes searchable l
I 14 or not.
l l
15 MS. SCATTOLINI Well, the other alternative is l-l 16 not to call that description of technical data an abstract.
]
l 17 MR. BALCOM:
Yeah, call it a description.
18 MS. SCATTOLINI Call it a description, and then 1
i 19 you dor 'c have this problem of intermixing the two --
20 MR. BALCOM:
Do we have yet another field mhybe?
l l
21 MS. SCATTOLINI That would be pretty clean-cut.
22 MR. NIPPERT:
Well, except on the display sight 23 now, when you say, "I want to see it."
Does it show the 24 abstract or the description, whichever side of the headers 25 it got.
i w
l 252 l
1 MR. BALCOM:
Or we could put it in a title, " slash l
2 description field," unless there's a length limitation on l
l 3
that.
4 KR. NIPPERT:
The titles in general are going to 5
be using a pretty short list -- When you start sticking in a 6
title (inaudible).
(
7 MR. BALCOM:
Now, this is the kind of fascinating 8
stuff that the header working group --
9 (A chorus of laughter) l 10 CHAIRMAN HOYLE:
Is it important for us to resolve 11 everything having to do with abstracts today?
Can that be 12
-- can we come back and summarize it at the next meeting, 13 and nail it down?
l 14 MR. BALCOM:
I think you can come back next time.
I mean, there's a couple of things that are coming out of 15 l
16 this.
I don't see a great demand clamoring for abstracts --
17 you know, for large amounts -- even with the technical l
18 documents, there's a split it seems to me on the abstracts 19 and the technical documents which we have a lot of consensus 20 on that.
From a design standpoint, I think it would be 21 useful for the working group of some of the design people to 22 get together again and do a go-around.
What really makes 23 sense in terms of whatever you want to call the description 24 of the technical document or something that has an image 25 only or a core sample, or something like that, maybe
/
253 1
separats that out from the whole business of abstracting.
2 CHAIRMAN HOYLI:
Elgie, would you want to be on 3
the working group that --
4 (A chorus of laughter) 5 MR. HOLSTEIN:
A committee of one?
6 MR. BALCOM:
The working representatives from 7
Nevada, from the Department of Energy, NRC, and fron the 8
LSSA.
l 9
MR. HOLSTEIN:
I'd be happy to, Mr. Chairman, 10 although I -- I think we have agreement on about 90 percent 11 of this.
And, frankly, on the issues that the working group i
12 is going to be struggling with, I don't have any -- I don't 13 have any real strong feelings.
So unless I misunderstood 14 what remaining issues the working group is going to try to 15 come to closure on, it probably isn't necessary.
So I'd 16 probably turn -- but thanks for the offer.
17 CHAIRMAN HOYLE:
Okay.
What's left --
18 MR. BALCOM:
I think there's an absence of 19 consensus of whether the abstract --
20 MS. CERNY:
I don't think so.
21 MR. BALCOM:
Hum?
1 22 MS. CERNY:
I don't think there's an absence of 23 consensus at all.
24 MR. BALCOM:
Well, just for the option met --
i 25 MS. CERNY:
Basically, C.3, and the question comes d
i 254 1
then, which is really a design issue, is to how you search, 2
whether you move the abstract down into the text fatld, or 3
what you do with it.
But that's a design issue, but ocos 4
anyone not agree to C.37 5
MR. BALCOM:
C.3 says no -- it says no abstract, o
no additional --
7 MS. CERNY:
No abstracting.
8 MR. BALCOM:
Abstracting of technical documents by 9
the -- either by the author or by the LSSA.
10 MR. TREBY:
No additional.
11 MS. CERNY:
No additional.
12 MR. BALCOM:
Other than what exists already.
13 MS. CERNY:
And we pick that up, and it's search-i 14 able as part of -- as part of the full-text field.
And you 15 can always hoke it by doing abstract and word in the same 16 paragraph, if you really care.
Then you can always search 17 on it.
And then the question is just a design issue of then I
18 how we set it up for these documents that don't have -- what 19 we were colling abstracts, what we do with them now.
20 MR. BALCOM:
Do we just save about how many 21 million dollars?
22 MS. CERNY:
Yeah.
23 CHAIRMAN HOYLE:
I believe that C.3 doesn't really 24 speak to whether there should be additional or not.
It just 25 says, "If there are abstracts, they will be grabbed, etc."
/
255 1
But then it becomes a design issue as to whether that field 2
should be searchable or not.
3 MR. TREBY:
Well, I guess I disagree, because 4
Option C.1 would require abstracts prepared if one doesn't 5
already exist, and that's what we are -- I believe the 6
consensus is, that we don't need --
7 MS. CERNY:
Right.
8 MR. TREBY:
So there will be abstracts prepared in 9
the future by those authors who normally prepare abstracts.
10 MS. CERNY:
Prepare abstracts.
That's right.
11 MR. TREBY:
But there's not going to be a l
12 requirement that if, for some reason, a document is created 13 that does not have an abstract, that someone has to go and
(
l 14 create an abstract for it.
1 15 MS. CERNY:
That's right.
16 CHAIRMAN HOYLE:
Okay.
17 MS. CERNY:
I think we --
18 CHAIRMAN HOYLE:
Is there a consensus on that?
l 19 MS. CERNY:
I think so.
20 CHAIRMAN HOYLE:
All right.
21 MR. BALCOM:
No strong dissents.
22 CHAIRMAN HOYLE:
No strong dissents.
All right, l
23 where does that leave us then with header group recommenda-24 tions?
25 MR. BALCOM:
Well, the -- I think the other thing
/
l l
l
256 1
you mean to cover -- at least was in the original agenda --
2 was editing of materials.
And though I understand that 3
that's --that the recommendation has been to move a 4
discussion to another meeting.
5 CHAIRMAN HOYLI:
To another meeting.
I think the 6
issue of the role of LSSA in the capture of documents and 7
altering headers should be discussed as part of the role of 8
LSSA, and I would like that next time.
9 KR. BALCOM:
Okay.
And then the non-technical 10 document, we talked about moving any discussion of 11 additional fields to that -- to the discussion and after the 12 presentation by Rawlee and --
13 CHAIRMAN HOYLE:
I would propose then to take a 14 short break, no more than 10 minutes, please.
And then 15 we'll get on with the centers briefing.
16 (A brief recess was held.)-
17 CHAIRMAN HOYLE:
I'd like to get started again.
I 18 wanted to do two things before I ask the center to go ahead 19 with t'.neir presentation.
The first thing I'd like to do is 20 try to summarize quickly what I believe we reached consensus 21 on in the abstract area.
22 We have consensused that there will not be a 23 requirement for documents to have abstracts, and by 24 documents I'm talking about are the technical documents, and 25 those that are available in ASCII format.
We're putting
/
257 1
aside the non-document types from this discussion.
The 2
abstract, if it exists, will be put into a -- into a 3
separate category, and there might be some design issues in l
4 there.
But as far as our consensus is concerned, we will 5
utilize the abstracts if they exist.
We will not require 6
abstracts to be written where they don't exist on this 7
limited type of data we're talking about.
8 Okay.
The second thing is that I propose that we 9
try to work right all through, perhaps until about 1:00, and l
l 10 break for lunch at that time, and maybe be finished with the i
11 day's work.
What we have left is the briefing by t -
ater 12 and a discussion by Chip Cameron of the status j
l 13 Compliance Evaluation Program, and priority do".'.nt 14 production schedule.
Chip assures me he only ne.eds three
)
15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> to do that, but was willing to do it in a little less l
16 time.
And I think growing out of his discussion will be 17 some items for a future agenda, which I'll talk about very 18 briefly.
So I do think we can do all this by 1 00 if that's l
19 okay with every-one.
Let's try it.
Chip.
20 MR. CAMERON:
I just wanted to introduce the 21 people from the center.
One of the things that the LSS 22 Administrator's Office that we thought was important to work 23 on was a subset of the information universe that's going to 24 be accessible through the licensing support system.
And 25 that's what we called technical data, and we thought that it
/
258 1
needed some further clarification and amplification An terms 2
of implementation.
So we hired the Center for Nuclear Waste 3
Regulatory Analysis to do this work for us.
The center does 4
all the technical work for the Nuclear Regulatory 5
Commission, and they're going to be talking a little bit 6
more about what they do.
But I thought that I would 7
introduce these folks.
Rawlee Johnson, right here.
Chuck 8
Acree; and Steve Young, right over here.
So I'll just turn 9
it over to them.
10 MR. JOHNSON:
Thank you, Chip.
I'm going to 11 start, and I'll move fairly quickly through an outline of 12 what we're going to talk about here today, to give a 13 progress report on our work so far.
The background on the 14 Center for Nuclear Waste, I'll give a brief overview of 15 that, and the project that we're working on, the access 16 protocols to technical data.
Then Chuck will give a status 17 report on the visits that we've made to the primary 18 participants to date.
And he'll share some observations 19 that we've come up with from those visits.
20 The NSS header fields for technical data will be 21 covered by Steve Young, and I'll close by moderating a 22-discussion on the issues that we've identified at this time.
23 The Center for Nuclear Waste and Regulatory Analysis was 24 established by the NRC to really be their technical adviser 25 in the High Level Waste Program.
It's their first federally
/
259 1
funded research and development center, and we began 2
operation in 1987, and had five-year options on our 3
contract.
4 The center is a part of Southwest Research 5
Institute whic's is a not-for-profit research and development 6
in engineering and science.
There's over 2,400 staff 7
members, and in Fiscal Year '90, around 200 million, $190 8
million annual income.
The 10 technical divisions at the 9
institute have probably over 200 computer scientists, 10 electronic engineers, various other disciplines, developing 11 complete computer hardware / software systems; serve NASA, 12 General Motors, General Electric, quite a bit of work for 13 the Air Force; in artificial intelligence, computer ingraded 14 manufacturing, distributed database systems, and other areas 15 of information science, information systems.
16 The center itself in its program is involved in 17 really the engineering and systems engineering, and the 18 integration of the regulatory, the technical and 19 institutional issues and uncertainties that we're dealing 20 with in the High Level Waste Program, based on the 21 regulations.
There's technical activities and research 22 programs in the area of --four program areas:
geologic 23 siting, the engineering barrier system, the repository i
l 24 design for construction and operations, and performance 25 assessment.
/
l l
260 1
We also provide complete quality insurance program 2
and services to the NRC.
We are developing management and 3
technical data systems to support all of those four major 4
programs, and they will interface to the NUDoCS system at 5
the NRC as some interim document reference system, and 6
ultimately, the LSS.
7 The center began to work on this access protocols 8
for technical data t0sk in June of this year, and you may 9
recall receiving a letter from the LSSA identifying what --
10 where it said about to do in asking you to participate in 11 their visits and meetings.
And actually, this work fulfills l
l 12 the main thing of the LSS rule regarding access protocols 13 and the categories of documents dealing with technical data.
14 Our main text objectives are:
one, to define the 15 technical date by category; two, to identify the organiza-16 tion's generating technical date; three, to document 17 existing and planned procedures for providing access to the 18 technical data and to recommend a plan to ensure access 19 which includes the submission requirements and the 20 recommended header content.
And finally, to identify the 21 impacts of the suggested plan to encourage early problems of 22 resolution.
23 So our approach so far has been to examine the 24 background references and talk and make these visits with 25 the primary participants, and we'll go ahead and get into a
/
261 1
status report on the initial work here.
We've heard in our 2
discussions technical data referred to in many, many 1
3 different ways, different people that we've talked to.
For 4
purposes of this task, we'd use the term technical data to 5
mean documentary material which cannot be entered into the 6
LSS in text searchable form.
In other words, it's the LSS 7
material which can be found only with the help of a header.
l 8
And if we refer to it in terms of media rather than subject, 9
some are imageable, like graphics and tables and handwritten l
10 notes, and some aren't, like tapes, disks, cassettes, and so l
l 11 on.
l 12 Looking at the categories of documents in the 13 material as it's described, in the LSS Rule we find both
(
14 categories.
You have your full-text that had -- or index 15 and image, and also have a header in category one.
In 16 category two, you have graphic-oriented documentary material 17 that's imaged and has a header.
In category three, you have 18 those documentary materials not suitable for entry that have l
19 only a header.
And then number four speaks to a package of 20 information, and you can find these cited in the sections of 21 the room.
22 Now, technical data is pervasive throughout all 1
23 four categories.
In fact, a lot of the technicaA data is l
24 really in category one, but for purposes of our work, we're l
25 addressing category two, three, and four in dealing with the j-l l
262 1
access protocols, just as John mentioned at the start.
2 One other area in category four, the package 3
concept, is I think needs a little definition and l
(
4 illustration.
But when we speak of a data record package, l
5 we're suggesting that there's a header that has certain data 1
6 fields completed, and it would have a -- we use the term 7
" table of contents" just as it states in the rule to 8
describe the contents of that package.
Now, here in the i
9 example, we have a Weather Data Dump Record for a weather 10 station.
And we show the handwritten weather data that was 11 taken there just as you'd find it as an item in that package 12 as it's described in the table of contents.
Does that make 13 that clear?
l 14 MR. ALEXANDER:
Well, just a question, John.
This 1
15 is very interesting, and we're going through it rather 16 quickly.
Can I ask that these slides be made a part of the 17 record of the meeting?
18 CHAIRMAN HOYLE:
You certainly may.
And we intend 19 to get a copy.
20 MR. JOHNSON:
Okay.
I have a clean copy that --
21 MR. ALEXANDER:
Thank you.
I just want to -- so 22 we don't have to scramble right now.
23 MR. JOHNSON:
And at this point, I'd like to turn l
24 it over to Chuck Acree.
Chuck is a former Director of 25 Information Systems with the CIA, and he's a consultant to
/1 l
l l
l
263 1
us at the center and is working on this project.
2 MR. ACREE Don't open up the next slide yet.
The 3
next slide gets kind of complicated.
What we did was, the 4
first visit we made was to the Department of Energy's Yucca 5
Mountain Project Of fice in Las Vegas, Nevada, and they 6
provided us one-and-a-half days of briefings and a good six 7
or seven issues of materials to wade through after the 8
briefings.
And we've, of course, done all that.
And what 9
I'm going to now is try to present to you in a very few 10 minutes, given time constraints and also maybe interest l
11 constraints, on what the Yucca Mountain Project Office is l
12 doing with its technical data and how it's packaging that.
13 So it goes into a little bit more about this packaging l
14 concept.
It's necessary tc do that because unless you I
l 15 understand this, it's really hard to understand how the 16 packaging is done, and that's what's important.
17 Now, first off, Yucca Mountain Project Office is 18 the first place that we went into difficulty with the term 19
" technical data."
The Yucca Mountain Project Office does 20 not technical data just the stuff that's non-text l
21 searchable; in other words, the graphics and the tapes and 22 disks and those kind of things.
No.
They say technical l
23 data is all data relating to the technical activities; in i
24 other words, just about everything they do.
And I'm going 25 to be using that definition when I talk about the Department
[
r, s
264 1
of Energy and what it's doing.
2 First of all, the -- you have -- let's just talk 3
about the components.
There are nine local record ce:
..s.
4 I'm going to show you a graphic of this in a minute, so I 5
think it will, hopefully, put it in a little bit better 6
perspective.
There are nine -- Can you hear me all right by 7
the way?
I'm not using the mike.
There are nine local 8
record centers, and they're collecting the data, but we're 9
talking about prime contractors here, including SAIC, The 10 Geologic Survey, S ANDIA, Los Alamos.
They all have local 11 record centers, and the technical data feeds into those 12 local record centers.
Then the local record centers sent it 13 quarterly down to what's called the Central Records Facility 14 in Las Vegas, run by the project office, and they do that 15 within these packages that Rawlee mentioned to you.
16 What that combines, their raw data, so-to-speak, 17 with the analysis and the descriptions of how that data was 18 produced.
It also includes numeric data from a thing 19 called, "The Site and Engineering" -- excuse me.
20 MS. CERNY:
" Properties."
21 MR. ACREE:
" Properties Database," and results 22 from computer modeling graphic display systems.
So all this 23 goes into the package.
Then the Central Records Facility 24 uses two information systems: one for its records, and one 25 to track the data all the way back to its origins.
- Again,
/
269 1
it forms the command to do that, at which time more data 2
comes in, and you fill out a new form.
3 MS. CERNY:
Could I -- excuse me.
4 MR. ACREE:
Yes.
5 MS. CERNY:
If you don't mind.
6 MR. ACREE:
Please.
i 7
MS. CERNY:
Those two -- those two systems, 8
they're really one system.
9 MR. ACREE:
Yes.
10 MS. CERNY:
And they're not two separate systems.
11 The data tracking system remodified the record system, and 12 they are -- so they are tying together.
13 MR. ACREE:
Barbara certainly is more expert than 14 I.
They call it a supplemental system.
15 MS. CERNY:
Right.
But they're really --
16 MR. JOHNSON:
The subsystem of the --
17 MS. CERNY:
That's right.
That's what it is.
18 It's a subsystem of the records system.
19 MR. ACREE:
Then I want to point out, there are 20 three important documents that are produced quarterly.
One 21 is a Technical Data Catalogue, which describes all the 22 technical data within the Central Records Facility; another 23 is a report on the SEPDB, this numeric base, which tells 24 what tapes are available to people should they wish to ord'er s
25 them; another is the Reference Information Base, RIB, which
/
l i
l
266 1
is a summary description of analyzed technical data, the 2
current state of knowledge.
3 If you show them the second slide down, please, 4
the graphic.
Yes.
Okay.
This puts a little bit -- This 5
shows the flow of the technical data, In other words, 6
you've got the -- you've got the acquisition activities and 7
the principal investigators.
The raw data comes down to the 8
local records center.
There are nine of these; prime 9
contracters each have one.
Then, this acquired data goes 10 over to data reduction interpretation; in other words, 11 analysis.
And with the help of the controlled databases, 12 these are the compute; systems that model it and show it 13 graphically, and help this analytical activity.
And that 14 all goes back here in the form of data record package 15 segments where they put it altogether.
They use participant 16 here -- this is a, by the way, a Yucca Mountain Project 17 slide.
It isn't mine, so I didn't change anything.
But we 18 would call this the prime contractors, not a participant.
19 That's their term.
20 So we put this altogether and they, of course, j
l 21 send this down quarterly to the central Records Facility, 22 and they also send an index to all the data which updates 23 this computer system.
Now there's also, as I mentioned, the 24 summary document over here in the green and purple -- This
(
25 is a synthesis of fully-interpreted technical data.
It's
/
4
267 1
kind of high-level information where you put all the 2
technical data together, and it's a notebook.
It isn't a 3
computer system.
It's kept up-to-date page-by-page.
They 4
may send a new page in for a new piece of the information.
5 So that is kept -- that's kept out at SANDIA.
This is kept 6
out at SANDIA.
7 Then we have over here a data catalogue, which 8
keeps track of all the data that's held here.
And it tells 9
where it's located or where it came from, time and date of 10 acquisition, and all kinds of description for them.
That is 11
-- This information is on line with this Automated Data 12 Record System, and it's in hard copy form with the data 13 catalogue in accordance with the site's specific agreement
(
14 with the NRC.
Are there any questions on that before I go 15 on?
16 MS. CERNY:
Could I just ask --
17 MR. ACREE Sure.
18 MS. CERNY:
I mean, I don't know if you'll get 19 into this or not, but I think it's a, for this group, an 20 important concept -- that's the -- that is Nevada.
That's 21 what they do.
We have parallel.
You can take a parallel, 22 Washington Headquarters, where I have a central records 23 facility.
And feeding into it are what's produced at 24 headquarters : Chicago, Idaho Falls, or which any other 25 contractors that would be coming in to headquarters.
J'
268 1
Then we have a wide-area network connecting Nevada 2
and Washington, so -- and in fact, I'm in the process of 3
rewriting the records system so that we check duplicates 4
across the network.
And this is the foundation for tho 5
information that will go into the LSS, and so we will be 6
doing it program-wide, so when we get to LSS access 7
protocols and procedures that we must follow, we will have 8
one department-wide system, though we have two locations.
9 Right now the records of the computer systems are 10
-- they're replicated.
They're identical, but of course, 11 they're separate now until I integrate them all.
And that's 12 happening within the next year, so we should be well set up 13 by the time you finish, you know, your work, to be able to 14 follow protocol's procedures of the LSS.
15 MR. ACREE Okay.
Thank you.
I want to mention 16 one thing:
They've been doing this now with this framework 17
-- they've had a framework for this nearly over the past 18 year, and then formalized their procedures at the beginning 19 of this year.
There is a huge backlog of material of these 20 local record centers, and it hasn't all found its way down 21 here to the Central Facility Data.
But they're working on 22 that.
There's no -- They didn't give us a time and a date 23 yet when that will all be completed, but that will happen.
24 So we.9?ve that.
25 Now, let's go onto the next slide, Steve, to push
/
269 1
this on a little bit.
That's right.
That's the backlog.
2 Now, the technical data that's currently available there at 3
the project office, is available through a written request 4
to the project officer manager, who promises a timely 5
response.
In other words, it's done centrally.
You don't 6
go out to the local record centers and ask for information.
7 You go to the Central Records Center, and they will go out 8
and get the information for you much as a central librarian 9
would do.
10 Now, when LSS loading begins, it will be a rather 11 simple process, conceptually at least.
The project office 12 will scan its packages to submit the images, and it will 13 create a header for each package from its existing computer 14 database headers.
Okay.
15 After we talked with DOE a couple of weeks later 16
-- that was in August; this is getting into September now --
17 we went out and we talked with Mal Murphy, representing the 18 State of Nevada, and we talked about what Nevada does with 19 regard to technical data or non-text-searchable data.
And 20 we came up really with this term " raw data," so we were at 21 that stage at that point.
And we talked with Mal about raw 22 data.
And the Nevada people and its contractors, primarily 23 the University of Nevada and the Nevada Bureau of Mines and 24 Geology, haven't produced any raw data.
That's our 25 understanding, but they may do so in the future.
/
270 1
Now the Nevada people publish formal technical 2
reports containing graphic material, the kind of material 3
we're talking about.
And that primarily analyses DOE 4
acquired raw data.
It safeguards within associated packages 5
the handwritten backup data on which those reports were 6
based.
Then when LSS loading begins, just like DOE, Nevada 7
will scan its reports and its associated backup packages, E
submit the images, and zicate hedders for them.
9 With regard to needs, Nevada considers that the 10 individual items of technical data, which are not text-11 searchable, but are contained within a data record package, 12 don't require their own headers.
In other words, the way I
13 DOE is doing things, is all right as far as Nevada is 14 concerned.
The header for the package as a whole is 15 sufficient.
Furthermore, Nevada feels that technical --
16 maintains that technical which can't be scanned for entry 17 into the LSS -- this is the stuff that, like the tapes and 18 disks and things 19
-- will be identified in the LSS through a header.
They 20 must be transferred from the current storage locations to an 21 LSSA Control Records Center several months before the 22 highlevel waste repository licensing proceedings begins, in 23 order to assure timely, centralized access for everyone.
24 MS. CERNY:
Can I ask a question?
25 MR. ACREE:
Sure.
/
271 1
MS. CERNY:
This starts to get into -- When you're j
2 calking about transferring computer tapes, archiving 1
3 computer tapes, are you going to come up with 4
recommendations for how that is to be done?
After all, 5
after a certain period of time, operating system changes.
6 You can't even read it one month to the next.
Hardware 7
platforms change.
The people who generated it no longer 8
have them.
I mean, to keep up a computer tape is really --
9 so that 10 years down the road you will still be able to 10 read it -- is not trivial.
And the idea of transferring to 11 a central location assumes that something there is going to 12 be able to read it, and yet who at that central location is 13 going to have anything there?
14 MR. MURPHY:
Well, let me explain for you -- if we 15 can revert bsck to the days of our negotiations over the 16 term, if you recall we used on negotiations: raw data.
It 17 was our position -- and I think we all agreed to this, you 18 and Nevada and the NRC staff -- that on this small pointy 19 that at that point, at some point in time, and probably at 20 the point in time where most logically at the point when the 21 LSS system is completed and loaded and certified by DOE to 22 the LSS administrator, and control of the system is 23 transferred to the LSS administrator, this data, these 24 packages, which are part of the LSS, must likewise be 25 transferred to the LSS administrator.
/
272 1
Now, this is a little bit misleading, and it's 2
partly my fault because of my discussion with Rawlee and 3
Chuck.
I don't think that we necessarily insist that there 4
be a physical transfer in each case of a computer tape from 5
a building in Las Vegas to another building in Las Vegas.
6 That isn't essential to us.
What is essential is that 7
custody and control over all LSS material, all LSS material, 8
including the non-searchable -- what I will still call raw 9
data -- at some point in time be turned over to the LSS 10 administrator.
And, you know, for want of a better period 11 of time, I used in my discussion that six-month period 12 before the submission of a license application, where the 13 rule calls for the system to be turned over, certified and 14 turned over.
But that's the point.
15 The point is that this is no different than any 16 other LSS administrator -- LSS information -- and it must be 17 placed in the custody of the LSS administrator.
And if that 18 means that, as I said, if that means that on your Central 19 Record Center in Las Vegas, the guard on the door changes at 20 some point in time and the stuff stays right where it is, 21 but it's under -- it's under the LSS administrator's 22 custody, same room, rather than under Carl Giertz's custody 23
-- that's probably satisfactory.
That's what I'm talking 24 about.
25 MR. ACREE:
Well, --
/
273 1
MR. MUPPHY:
And it could be two or three central 2
locations.
I don't think Nevada is going to insist on --
3 MS. CERNY:
But you see the problem.
4 MR. MURPHY:
Oh, I see the problem.
I understand 5
the problem, but I thought we had worked all that out in the 6
negotiations.
7 MS. CERNY:
Well, but that just raised the 8
question.
9 MR. ACREE:
Barbara.
10 MS. CERNYt Yeah.
11 MR. ACREE:
I think it's important on the computer 12 tapes that we always stay with the standard, you know, the 13 ASCII, and so on, as much as we can for the long term --
14 MS. CERNY:
Yeah, but you know as well as I that 15 you can stay with ASCII -- you change operating systems.
16 You can't read some of this stuff.
If you change hardware, 17 you can't even read it.
I mean, it's --
18 MR. MURPHY:
The header information will be raw 19 data package, because it's supposed to tell us --
20 MS. CERNY:
Well, and the --
21 MR. MURPHY:
-- give us all of the information 22 necessary to use it so --
23 MS. CERNY:
Sure.
But you're telling -- You talk 24 10 years down the road, nobody may have that piece of 25 hardware.
You've got to have procedures in place and --
/
274 1
MR. MURPHY:
Get it from Smithsonian.
2 MS. CERNY:
Yes.
Smithsonian has some tapes, and 3
the Vietnam war is I think the famous example:
There were 4
two computers that could read it; they don't have any 5
anymore, and nobody could read the stuff.
So that you have 6
to have procedures internally for how you're going to update 7
your tapes, and I'm not talking about just refreshing them 8
by rewriting them, but in fact, to assure that they are read 9
by whatever software and hardware that you bring in.
10 So that's the other part of this question I was 11 asking was:
Are you going to have those kinds of 12 recommendations in this for participants who may not be 13 doing that?
14 MR. MURPHY:
But remember this is a discovery 15 system.
This system is to facilitate the discovery of 16 information.
If Nevada wants to use, or the Environmental 17 Defense Fund or the Department of Energy wants to use, a 18 computer tape generated by the State of Nevada 10 year's 19 ago, it's incumbent upon us to be able to do that, not you.
20 MS. CERNY:
Who?
"Us" who?
21 MR. MURPHY:
Nevada.
22 MS. CERNY:
You mean you generated it, therefore -
23 24 MR. MURPHY:
No.
If you generated -- if the 25 Department of Energy generated a computer tape on weather
[
1 1
275 1
stations --
2 MS. CERNY:
Uh-huh.
3 MR MURPHY:
-- 1984, and in 2004 the State of 4
Nevada in preparation for the licensing proceedings wants to 5
read that computer tape, we need to have the 6
hardware / software and technical expertise and be able to do 7
that.
If we don't, that's our problem.
Your problem is to 8
-- is to provide that tape to the LSS administrator, and the 9
LSS administrator's problem is to give us access to it.
We 10 need to be able to use it.
If we can't use it, that's our 11 problem.
We haven't done our job.
12 MS. CERNY:
But see, I just -- And that's why I'm 13 coming from the completely opposite perspe :nive on this.
14 This is supposed to be a discovery system, and we cre all 15-supposed to be using the information that's in it.
So it is 16 i::cumbent upon the person who generates it to assure that it 17 will be read.
18 MR. CAMERON:
Let me amplify on this.
19 MR. MURPHY:
To assure that it's provided to the 20 system in a readurle way.
21 MS. CERNY:
No.
They're different issues.
22 MR. CAMERON:
You're crossing -- Yeah, but you're 23 crossing the boundarv from the LSS to what, for example, the 24 NRC as the licensing authority wants to enforce on par-
\\-
25 ticipants in the licensing process in terms of how this
/
l
276 I
1 technical data should be produced.
Mal is absolutely right i
2 about the fact that it is a discovery system, and just like I
3 we don't *; ant to be at the LSS in the business of telling 4
participants that drta has tc be able to be manipulated in 5
such-and-such a manner, that's for the technical side of the j
6 NRC, not for the LSS side.
So that if these tapes -- it 7
is incumbent on the people who'd want to use these tapes to-8 figure out how they're going to be able to use them, just i
9 like in typical discovery.
10 MS. CERNY:
I really disagree with this.
I really 11 do, because I think your --
12 MR. CAMERON:
From the LSS standpoint, I think 13 that's the way it has to be.
14 MS. CERNY:
You're not going to be able to read 15
.the tapes.
You're just not.
You've got to have certain a
16 standards of documentation.
You have to have -- or you're 17 never going to be able to use the stuff.
18 MR. MURPHY:
Well, what's that got to do with --
19 MS. SHELBUINE:
Isn't that part of the package 20 though?
Would that not be part of the package?
21 MS. CERNY:
Not if you don't -- Not if there are 22 not specifications and procedures for how this has to be 23 done.
People can give tapes.
I've generated more garbage 24 tapes.
I couldn't go back and read those things.
25 MR. JOHNSON:
I think it's a matter of degree, and
/
277 1
I say, you know, inherent in some of the work, some 2
standards that, you know, would allow them to convert and 3
read most of what you -- like 500 and 400 diskettes were 4
popular, or still are used, but anyone can make that 5
conversion today, and you know, it's provided for in various 6
ways.
7 MR. BALCOM:
What's more critical than the tape, I 8
think, is if there is an understanding that the raw data, 9
for example, isn't transferred into ASCII so it's non-usable 10 by a subsequent modeling system, but to preserve the format, 11 the formatting of the data and to describe the protocols for 12 the submission to a library somewhere that say, " Don't 13 change the information that you fed into your model but 14 leave it, and simply describe to us.
We'll figure out how 15 to load it into our model."
But there may be some 16 misunderstanding about converting it into ASCII which really 17 would destroy its utility and using it in a subsequent 18 modeling for a simulation system.
19 I think those are the things that, you know, we 20.
need to sort out and describe in some of the protocols that 21 are to come.
You need probably some technichl-help to do 22 that.
23 MS. CERNY:
You know, I really think this is --
24 the technical data is going to be the key.
It's going-to be 25 the key to the whole crisis process.
/
878 1
MR. MURPHY:
Of course it is.
2 MS. CERNY:
Rather than all the documents that --
3 MR. MURPHY:
It's going to be the whole -- it's 4
the key to the license application.
5 MS. CERNY:
Exactly.
And so we focus on the 6
documents, but really this is the heart of the issue.
And I
?
think how you -- if you -- you should make it as simple --
8 You know, we spend all this time on: do we abstract, and 9
what header fields do we heve, but the real issue is going 10 to be: can anybody read those data tapes?
And that's a far 11 more difficult problem than the one we've just attacked.
12 MR. BALCOM:
I would say -- I would take the 1
-13 responsibility representing Nevada to be able to do that as 9
14 long as I had complete information as to the computer system 15 that you use to generate it, plus, its hardware platforming; 16 plus, what version of the software you use, what kind of 17 tape station you used, and so on; that I would think I could 18 figure out a way to read that tape.
As long as you provide 19 that information and don't change it, convert it into some 20 other format.before you send it to the library.
21 CHAIRMAN HOYLE:
Okay.
Let's go on.
22 MS. CERNY:
Sorry, but --
-23 MR. ACREE:
Well, it's something we have to 24 address to answer the original question.
25 CHAIRMAN HOYLE:
Yes.
/
279 1
MR. ACREE:
Okay.
The NRC Washington.
By the 2
time we got to Washington and talked with Stu Treby and 3
others there, the term " technical data" had become very 1
4 problematic.
So we talked in terms of non-text-searchable 5
material.
We just confined ourselves to that, and we hope j
6 to continue doing that.
So, what the NRC is saying that, l
)
7 yes, they do produce this. Currently they have a NUDOCS 8
Program, to make reference to its record holdings.
9 When the LSS becomes operational, the NRC will 10 submit its documents to the LSS through its existing 11 document control center.
And more specifically, when 12 loading begins, they'll centrally submit their non-text-13 searchable material to the LSSA for entry into the LSS in 14 accordance with procedures to be established by the LsSA.
j 15 Well, of course, that's us.
That's what we're -- that's 16 what we about.
17 They'll create headers for the non-text-searchable 18 material, again, in accordance with LSSA guidance with which 19 we're in the process of formulating as suggested guidance.
20 Now, the location of non-text-searchable material, which is 21 also non-imageable -- and again, we're back to the tapes and i
22 things -- must be identified in the header.
And they would 23 provide a central NRC contact, much as tbc Yucca Mountain 24 Project Office has.
l 25 As far as needs are concerned, we discussed the
/j
280 1
Yucca Mountain Project Office's use of data record packag-2 ings, and they hadn't focused on that to-date, but they were 3
saying if the NRC is going to comment on that,-they need 1
4 some more iniormation which we were not, frankly, in a 5
position to provide yet.
We've gone out there and heard the 6
briefings.
We've read the materials, and we had determined 7
that we had to go back and take some hard looks at the data 8
record packages to see how they're constructed, how the 9
timing of all these is accomplished, if it's one or two 10 years, and how this is all going to work out -- the kind of 11 things Barbara was discussing a little while ago.
But we 12 have to take a look at that again.
13 Anyway, NRC reserved judgment on that until we s
14-find more out about those packages.
15 MR. MURPHY:
The NRC, I might 16 ject at this point 16 in time, specifically agreed with Barbara's team and 17 Nevada's team when we put together this compromise on raw 18 data, if you' recall.
Avi and Phil Altomare were there, back 19 in the corner of the room over at the Airport Plaza, and 20 agreed to this procedure.
And for the NRC staff to now say 21 that they can't comment on this process is to me a little 22 bit disingenuous.
23 MR. TREBY:
No.
I think that what we were saying 24 is that Nevada and DOE got togetner to discuss the packages, 25 how they were going to be made up, and agreed that that
/.
i 281 1
would be fine.
And we agreed, too, about the packages.
2 MR. MURPHY:
Stu, these words -- these words in 3
this rule were written by Barbara and Je.' fry, my team on l
4 behalf of Nevada, and Phil and Avi.
They were written --
5 this language was drafted in part by the NRC staff.
The NRC 6
staff agreed to this part of the rule, and to now say they 7
can't comment on it; they can't agree to keeping c package 8
together --
9 (Phone rings, followed by a chorus of laughter) 10 MR. MURPHY:
-- to now say that they cannot agree-11 on one of the essences of the rule; and that is, to keep the 12 package together and put a single header on a single 13 package, is to me disingenuous.
t-14 MR. TREBY:
Just fck the record, I think all we 15 were saying is that sc'd like to have the package described 16 more fully, which apparently, Barbara has done it.
I'm not 17 saying we disagree with the rule or anything.
It was just 18 that when we were discussing this with the people from the 19 center,.we were unable to describe exactly what this package 20 would consist of.
21 MR. ACREE:
Well, I think it serves again to show 22 that the package is critical here in discussing technical 23 data, and what we're about is looking further into this, and 24 making surc it all works out well and in accordance with 25 everybody's understanding.
/
883 1
When we talked with Barbara Cerny in Washington --
2 really she's already said this, I think, to everyone, in 3
other words -- that the DOE doesn't expect to produce any 4
significant amount of non-text-searchable material -- again, 5
using that terminology -- from its Washington, D.C.
6 headquarters.
And such material that it may have produced 7
in the past or might produce in the future, they again, 8
would enter it into the LSS through the DOE capture station.
9 And DOE doesn't anticipate any extraordinary access needs.
10 Do you want to add anything to that, Barbara?
11 MS. CERNY:
No.
12 MR. ACREE:
Okay; observations.
Putting this 13 altogether, what have we learned so far?
I emphasize we're 14 in a kind of a " data gathering mode" here.
We're defining 15 problems not solutions at this point, and we're, of course, 16 discussing the problems with you.
17 The first part -- The first nart of our task, as 18 Rawlee showed you, was to define technical data.
And we've 19 found that, first of all, that the term " technical data" is 20 a poor one to use for what we're talking about:
the non-21 text-searchable material.
There is no ambiguity in the rule 22 because it doesn't even mention technical data in this way.
23 But certain issues certainly must be resolved, which we've 24 been talking about, and we're going to talk about a little 25 bit further after I'm through.
And we need clear implement-
/
l l
883 1
ing procedures so that everybody knows what we're doing.
2-How much do we have to define technical data, or 3
the non-text-searchable material, as we're now calling it?
4 Well, some categorization is required in order to be sure 5
that none escapes proper LSS entry, and we don't find some 6
handwritten note that's important off in left field some-7 place.
And also, to enable LSS users to find what they need 8
through consistent entry / search terminology.
That has to be 9
done.
10 The second part of our task was to define who's 11 producing technical data, again, a non-text-searchable 12 material.
The primary producers, no surprise to anyone, are 13 the DOE-and the NRC aed their contractors.
Nevada may or.
14 may not produce it.
We think they haven't found anybody 15 else that's producing it vet.
If anyone knows of anybody 16 that's producing it, we'd 17ve to hear about it.
17 MR. MURPHY:
Yeah, let me just make that clear, 1:
18-because we're having a problem with terminology here, 1
19 because the different partie.s use different terms.
When I 20 say Nevada hasn't produced raw data, I mean that we have not 21 been permitted to either through funding constraints or 22 problematic constraints, we have -- our contractors have not 23 been permitted to go into the field to penetrate the surface 24 of the earth, to put down boreholes, to bring water out, and 25 you know, core samples, and things of that nature.
/
284 1
Well, I'm sure that it's all Barbara's fault.
I'm 2
sure that there is some laboratory-type work, and you know, 3.
a scientist would describe as " original product" or 4
something, that might fit DOE's definition of technical data 5
or NRC's definition.
But when I say Nevada hasn't produced i
6 raw data, I mean we have not produced any field-generated 7
data which is not -- which we have not gotten from some 8
other source, primarily DOE.
9 MR. ACREE:
All right.
Looking ahead to our 10 production of a plan to ascure access to this material, we 11.,
think some changes may be necessary in the plans and 12 procedures and systems of the DOE and NRC, but it's far too 13 early to say that yet.
But we're just as a kind of a 14 prewarning of the obvious, perhaps, and we, of course, aim i
15
.to keep that at a minimum.
We don't like to disrupt 16
- anything, 17:
MR. HOLSTEIN:
Chuck, this may be useful for
-18 future reference, and that is that -- I'll just speak for 19-Nye County right now.
Nye County is developing a pretty 20 sophisticated program of socioeconomic data gathering.
- Now, 21 depending on how this issue that we discussed yesterday when 22.
the Draft Regulatory Guide comes out, we may, in fact, be 23.
submitting a great deal of raw data.
24 We may, in fact, also be doing some of this data 25-collection work that DOE had previously planned to do l
(
285 l
l 1
itself.
That's currently being negotiated.
Whether that's 2
the case or not, we may be conducting doing our own L
3 analysis, because we have disagreed with some elements of 4
DOE's methodology.
5 So in that oversight capacity -- so it's possible l
6 that depending on the outcome of those discussions, as well l.
7 as the outcome of the debate over this regulatory guide, 8
that Nye County, as well as other effective units of local 9
government, may be producers as well, as you're defining 1
1 10 them.
l l'
11 MR. MURPHY:
Yeah, I should probably -- I think 12 that's a good point, Elgie.
I should probably make that i
13 point myself. As far as the State of Nevada is concerned,
\\
14 when I say we haven't produced raw data, I mean we haven't 15 produced -- You know, I don't want to insult any 16 sociologists in the roor..
We haven't produced any hard I
17 science raw data.
I think we have generated independently l
18 and original socioeconomic raw data.
That's a good point.
19 MR. ACREE:
Well, those are good clarifications.
p 20 I appreciate that help, because we'll have to deal with l
21
-that.
22' MR. MURPHY:
As well as transportation as well.
I 23 think that applies to --
24 MR. ACREE:
Okay.
Now, in access to raw data --
25 this is, again, looking ahead to our plan for access --
/
286 1
envision this as being reasonably straightforward.
The e 2
will be a bibliographic header; image data will be viewed 3
directly on the screen; the non-imaged data can be 4
requested, or it will be centrally J ocated.- And each deta 5
record package will have several things:
It will have a 6
descriptive bibliographic header, with an abstract, which 7
you -- which has been mentioned in the abstract discussion; 8
a viewable table of contents, not only viewable but text-9 searchable.
Most pages will be viewable as images, and 10 there will be a lot of text-searchable pages within these 11 packages.
12 Can you find that diagram again, Steve, maybe just 13 to emphasize those points.
Let me mention while he's doing 14 that that we also have those three documents that I 15-mentioned of the Yucca Mountain Project was produced, and 16 the technical date of summary, the overall summary that's 17 updated and current information.
We have the technical data 18 catalogue-telling you how to get to the tapes, and we have 19 the listing of available digital data on tape -- excuse me, 20 the technical data catalogue, which tells you the data 21 that's available to CRF; then the-SEPDB which tells you what 22 tapes are available.
23 All that will be available on line after all,-
)
24 again, available for searching.
okay.
Can we go back to 25 that one.
Just to emphasize here kind of how it's laid up,
/
287 1
you have a header over here.
You have a header leading to a 2
package, with a table of contents, searchable, viewable, 3
t*lls what's in the package.
And we have pages that are 4
images, all wf them, and we have also -- a lot of those will 5
be text-searchable, because it will be in readable form.
6 Yes.
'7 MS. SHELBURNE:
I just need some clarification 8
because of what we talked about this morning with Kirk and 9
this idea of' data packages or records packages.
10 MR. ACREE:
We're going to have to relate to that.
11 MS. SHELBURNE:
Yeah.
That has got to be sort of 12 cleared'tp.- Basically, what you're saying those following 13-pieces of-paper are just data sheets that's depicted there.
14 Could there not also be reports that individually stood 15 alone and could be indexed in-text-searchable within a 16 package?
17 MS. CERNY:- We do that, Betsy.
Each individual 18 piece within a package is submitted.
19 MS. SHELBURNE:
So the table of contents could
.20 talk-about --
21 MS. CERNY:
Well, the table of contents, yea, it l
22 will have a report listed.
But there's some things that 23 aren't appropriate; for example, data sheets with data taken 24 in_some borehole, the table of contents will say there are a j
(.-
25 number of sheets, and that isn't separate, y
288 1
MS. SHELBURNE:
I just wanted to clear that up, is 2
the package contains more than just *:' hat may be imageable.
3 There may be individual documents.
4 MS. CERNY:
Yes, absolutely.
5 MR. MURPHY:
Let me ask a question, because I 6
sense that there's been some departure from what we agreed j
7 to in this rule.
I'm not sure that it's of a critical
-8 nature or not, but are you saying now that DOE -- and I'm 9
talking specifically about, you know, the package I'm 10 referring to is that individual principal investigator's 11 pile of documents, whether it's in his three-ring binder 12.
like I have here, cn: his file cabinet, or whatever -- are 13 you saying that that package is going to be separated and 14
-the pages in it -- okay,~we're still talking about keeping 15 the package together, 16 MS. CERNY:
No.
We're talking about keeping the j
17 package together, or when the package is complete, it exists
-18 just like that, as a data record package.
The trouble is, 19 it can take two years, and so-every--- quarterly, a piece is 20 submitted to the -- from the local record center to the 21 Central Records Facility as a part of that package.
l 22 Okay.
The package keeps building, so it's kept 23 together as a package.
When the package is closed out, it 24 is then indexed and microfilmed again as a package.
So it's 25 always -- we always have it as a package.
But, as Betsy was l'.
l l
l
289 1
saying, within that package we can also have a report.
That 2
report will be separately available.
You can just search 3
and get that-report.
You don't have to go into a package.
4 MR. MURPHY:
The report which that package of that 5
raw data resulted in.
6 MS. CERNY:
Right.
It's part of the packaqo, and 7
it's also separate, so you will have your full-text of that 8
report with the authors on the report.
You can get it 9
either through the package.
You can get it just searching 10 the system.
11 MR. CAMERON:
That's a good point.
12 MR. ACREE:
Are you saying,-Barbara, that every --
13 everything though would be available with a separate heading 14 within the package?
15 MS. CERNY:
No.
That's what I'm saying that 16 everything is relevant.
I mean, everything isn't 17 appropriate.
18 MR. ACREE:
Some will be; some will not.
19 MS. CERNY:
Some will be; some will not be.
20 That's why I'm saying you can have just data sheets, you 21 know, a stack of data sheets.
And that they'll just be
~22 within a package, because what are you going to do with 12 3 them?
24 MR. ACREE:
Well, this is the kind of thing that
\\
25 was troubling NRC when we talked about it was just this s
290 1
point, and we,' frankly, couldn't answer it because we have 2
to go back and look at some more of his packages, drag them 3
down from the shelves, and roll up our sleeves and actually 4
look through a few of these to see how they're put together.
1 5
Are they all in yet?
Has it taken one or two years? --
J 6
that kind of thing.
7 MR. TREBY:
And we also didn't know whether a 8
package was done by the -- just by the principal 9
investigator, or whether it was a particular -- like a 10 borehole, where you may have more than one investigator.
We -
11 didn't know what constituted a package.
12 MR. MURPHY:
Well, from our thinking, the package 13 was the information that related to a study.
I mean, it's 14 defined in the rule.
What we talked about is defined in the 15 rule: relates to and supports, or whatever.
16
.MS. CERNY:
Yes.
17 MR. MURPHY:
And do you remember when we started 18 this whole process, our ine.cinctive reaction to the LSS,
~19 Nevada's, was to demand that everything which was 20 searchable, full-text-searchable, be included in searchable 21 full-text.
I.mean, that was the fundamental underpinning of 22 the whole system; that we were going to have a scarchable 23 full-text system.
And everything tb.at was written, 24 handwritten or typewritten or any other fashion, be put in i
25 searchable full-text.
And when we came to this kind of a
/
291 1
roadblock on raw data -- because we've always at least from 2
my individual perspective, as Barbara said -- I've always 3
recognized that this is the stuff whien is going to 4
determine the license application, you know, the site 5
characterization plan.
6 If that thing never gets into the LSS, it's not 7
the end of the world.
Who cares?
But then I went to -- you 8
know, when I sensed that maybe there was something here that 9
I wasn't, you know, fully informed on -- and I went to the 10 State of Nevada scientists, to a representative sample of 11 folks from UNR and UNLV and Bureau of Mines and Mifflin and 12 Associates, and subcontractors that -- I don't know, I think 13 I may have talked to a guy in the University of Toronto.
14 And they all, every one of them, universally, to a person, 15 said, " Christ, don't take the package apart.
Keep that 16 package together.
We don't want to go searching all over 17 hell and high water for a piece of.information here and a 18 piece of information there.
When we want to -- When 19 we're reviewing the adequacy of the conclusions of a 20 geotechnical report of any nature: tectonics or 21 geochemistry, or whatever, we want all of the information 22 the authors of that. report produced, starting from, you 23 know, the borehole up through whatever, and supported that 24 together.
We want to be able to go to someplace and look at-
'f 25 that in one package.
" Don't take it apart," they all said.
f i
292 1
"We never take ours apart.
It's a dreadful sin, and we'd 2.
never do it."
So we said, "Okay, that's great.
We'll agree 3
with DOE that these packages stay together."
4 MS. CERNY:
But, you know, we're just putting this 5
in place.
6 MR. TREBY:
I can recall that you, and I believe 7
Jim Davenport, actually went off to these sites, and you all 8
came back to negotiating --
9 MR. MURPHY:
No.
We all had a big meeting in Las 10 Vegas, remember?
We didn't -- We proposed to go.and visit 11 all these sites, and -- and the department for the one and 12 only time'in the history of this project, came up with a 13 better idea than I had.
14 (Chorus of laughter) 15 MR. MURPHY:
And they brought all ---They brought 16 all the, you know, scientists from all over the place 17 together in a room in Las Vegas.
And we sat down, and they 18 showed us, "Here's my notebook.
Here's what I have."
You 19-
- know, "I'm a rock," whatever they call it, "and here's my 20 pages with thin slices, Scotch-taped to a paper."
You can't 21 search that thin slice of rock.
And our people all said, 22
" God, don't take that page out of the book.
Leave-it there.
23 Keep that package together."
24 MR. TREBY:
I believe they honestly agree the 25 package ought to stay together, but it seems to -- it also
/1
293 1
seemed to us when we were talking about it that -- and we 2
didn't recall how it had been resolved -- that you might be 3
duplicating things, because there might be a report --
4 MS. CERNY:
Yes, we are.
5 MR. TREBY:
-- indicated in that package which 6
these people would have sent to you, and so you would have a 7
report both in that package and somewhere else, in the 8
fulltext-searchable portion of this thing with a cover 9
letter that says, "Here is our report on our study of 10 boreholes."
11 M'J. CERNY:
That's in there, not to worry.
12 MR. TREBY:
So that will be duplicated.
It will 13 be in two places.
14 MS. CERNY:
Yes.
15 MR. TREBY:
It will be in this package, and it 16 will be separate.
17 MS. CERNY:
You've got it.
18 MR. CAMERON:
The duplication will only be -- The 19 report that's in the package will be somehow referenced in a 20 header or a table of contents.
Okay.
In terms of it being 21-in the system twice, will it be in the system twice, or will 22 it only be in-the system once?
You'll submit.that document 23 to us through the normal document submission process.
24 Right?
25 MS. CERNY:
That's right.
And it will be in there
~ - -
/
l
294 1
with its cover letter.
2 MR. CAMERON:
And that will go in.
It will be 3
ASCII in the system.
4 MS. CERNY:
The whole thing.
That's right.
5 MR. CAMERON:
When you look at the table of 6
contents for the package, it will be listed there.
But that 7
actual, physical package will have the report in it, but 8
there will not be two copies of that document that's in the 9
package entered into the system in ASCII.
Right?
10.
MS. CERNY:
That's right.
11 MR. CAMERON:
Because you have to do it that way.
12 MR. TREBY:
There will-be --
(
13 MR. CAMERON:
It would be too confusing to try
\\
14 to --
15 MS. CERNY:
Yeah, but now --
16 CHAIRMAN FOYLE:
There will be a hard copy in the 17 package and a table of contents.
18 MR. MURPHY:
Well, there won't necessarily always 19 be.
I mean, there's some of these packages that --
20 MS. CERNY:
Okay.
But if there is, but then 21 you'll have a pointer.
But what we are doing now actually 22 with packages is microfilming the whole package even though 23 we have the microfilm of each individual document, because 24 we want the package together with continuous microfilm 25 numbers.
That may be an issue for when we scan them, we put i
295 1
them on optical disk two so they're_not spread all over the f
2 place.
But that's a design issue, and we've chosen to 3
handle it one-way with the microfilm, and that's something 4
that will have to be talked about -- you know, how we 5
actually do it when we scan it, and do we scan the whole 6
package when it comes.
But that's later down the road.
7 This is not -- I'm sorry.
8 MR. ACREE:
No.
We appreciate the discussion.
9 MS. CERNY:
I'm taking time from your 10 presentation.
11 MR. ACREE:
I hate to add just one further 12 complication to this, but there are non-imageable items of 13 information that aren't tapes and disks.
Among them are:
\\:
14 extra large maps, photos, or maps that are colored --
15 colorcoded so the color is important, and of course, the 16 image wouldn't capture that -- and they might be admittes 17 back hera someplace.
But then the issue becomes, well, do 18 you image that, let's.say, the color photos, and what do you 19 do about the extra-large things?
Do they get extra headers 20
-- separate headers, excuse me -- that kind of thing.
21 That's an issue which we're going to have to resolve also.
22 23 MR. MURPHY:
Let me ask you a question here:
24 What's the difference between the abstract and the header Y-25 and the table of contents?
What does the abstract say that
/
896 1
the table of contents doesn't give me up there?
2 MR. ACREE:
Well, the abstract could, for 3
instance, be the table of contents.
4 MS. MURPHY:
That's what I thought, yeah.
Okay.
5 MR. BALCOM:
I think it would -- In answer to that 6
now, I would suggest that the table of content doesn't 7
ordinarily totally describe what that item is in there.
For 8
example, if it's a five-and-a-quarter inch diskette of lab 9
data from a analogue device, that a table of contentc 10 wouldn't really tell you what the name of the program was or i
11 whc the -
the question was, and the level of detail I think 12 we're going to look for.
And an abstract or a description 13 field is that level of data which may or may nct be in the 14 table of contents.
15 MR. ACREE:
Well, that's again, an issue we'll 16 have to look at.
All right.
Now, don't be frightened by 17 this.
All this is is a list of the fields that you've been 18 discussing, and I didn't take.out copying -- I mean, this 19 was done early. -And what we're trying to show here -- this 20 is a lead-in to what Steve-Young will be presenting.
He's 21 going to be talking about several of these fields which are 22 particularly applicable to the non-text-searchable material.
23 Those are the ones marked with an "x" or marked here.
These 24 are the kind of things that have been discussed, and I 25 should use a pointer.
/
i
297 1
For i:istance, we've been talking about access 2
codes and storage locations, and people -- names of content, 3
form of data.
This means -- this is just suggesting 4
relationships.
We haven't determined anything.
We're not 5
suggesting anything.
We're just kind of looking and looking 6
at these fields, and what they might contain in terms of 7
technical data.
8 Many of the fields that are particularly 9
applicable to technical data are also text-searchable.
10 You'll notice there's a relationship there between 11 something in a bit more important fields, and some of them 12 free form or uncontrolled, so we can put down whatever we 13 care to put down, so they're available to us for technical 14 data, and that's what Steve's going to be talking about.
15 Steve.
16 MR. BALCOM:
Free form is like handwritten --
17 MR..ACREE:
Uncontrolled, no controlled entry.
18 CHAIRMAN HOYLE:
Excuse me a second.
Elgie?'
19-MR. HOLSTEIN:
Before -- before Steve starts, I 20 unfortunately have to rui. and catch a plane.
I wanted to 21 clarify my understanding of one particular issue.
22 What is your plan with regard to the letter that 23 was drafted yesterday on the issue of expressing the panel's 24 views on the draft reg. guide on topical guidelines.
25 CHAIRMAN HOYLE:
My plan for the letter is as
/
l
.r 298 1
follows:
Jay Silberg gave me a handwritten draft of a 2
couple paragraphs yesterday afternoon which I noodled a bit.
3 Stu Treby has offered a comment or two.
Dennis has 4
commented on that.
And I have had that typed, with the 5
exception of Dennis' comments, which I have handwritten in, 6
and I would like to give all of you before you leave today 7
anoth6r typewritten version o'f that.
I would ask you to 8
comment, say, back to me within the next ten days any 9
further comment you have on that.
I will try to get a 10 second draft out to you, which will be a more complete 11 draft.
It would have a little introductory language and 12 would have the NRC position stated in it.
Since we're 13 going to be the only exception I think here.
Send that back 14 out to you for approval by the membership before I send it 15 to NRC.
I want to be sure everybody has seen the final 16 letter and agrees with it before I send anything to the 17 agency.
18 I would ask you to comment to me if you care to on 19 what level of the agency you think we ought-to ce writing to 20
-- to the Chairman, to the Executive Director for 21 Operations, or to the head of the Nuclear Material 22 Management Office, Bob Bernero who was the one that sent me 23 the guidelines to begin with and I sent them to you, so you 24 could comment on the level at which you think we should 25 write to the agency.
/
299 1
All right.
So I can hand that out right now.
2 Well, go ahead.
3 MR. HOLSTEIN:
I'd just ask if I could have a copy 4
because I'm going to have to dash for the airport.
Thank 5
you.
6 CHAIRMAN HOYLE:
Uh-huh.
7 MR. TREBY:
If I could elaborate a little bit on 8
what some of the introductory material is, I think that 9
there has to be some discussion in this letter as to the 10 reasons why people are taking their positions as opposed to 11' just stating, "These are our positions" because this isn't 12 going to mean a whole lot to whoever they've sent it to, and 13 so what I anticipate the staff adding to it is the. reason 14 why it recommended in its proposed topical guidelines the 15 exclusion of those things-or' underlying assumption in that, 16 the LSS should limit itself to information necessary to 17 reach a licensablo decision based on 10 CFR, Part 60.
In 18 fact, at Part 51, rulemaking has been done, maybe something 19 like that.
But'I think that -- and I would expect-that some 20 of the rationale from the other parties would be that they 21 think such underlying assumptions are too narrow a few and 22 that the LSS should have a broader function than the staff LL 23 is suggesting-or whatever reasons.
I think the industry has 24 a reason that they believe that it shouldn't be left, you 25 know, to some date ten years from now to be resolved.
1
/
300 1
Anyway, I guess in sum my point is that I think 2
the letter needs to be flushed out a little bit to explain
~
3 some of the reasons for --
4 CHAIRMAN HOYLE:
Yes, thanks.
And if anyone wants 5
an additional position by their representative, be pleased 6
to include that as additional rationale for their position.
7 8
MR. HOLSTEIN:
You mean to be forwarded --
9 CHAIRMAN HOYLE:
To be included --
10 MR. HOLSTEIN:
-- under their name?
11 CHAIRMAN HOYLE:
-- in the letter or as an 12 attachment to the letter if you want to send me something to 13 be attached.
Yes, sir.
s 14 MR. HOLSTEIN:
Thank you.
15 CHAIRMAN HOYLE:
Okay.
16 MR. MURPHY:
And I would urge everybody here and 17 those who aren't here but we're going to communicate it to 18 them to get their views known on this issue at this stage, 19 rather than wait till it gets published for comment in the 20 Federal _ Register because it's too late then.
)
21 CHAIRMAN HOYLE:
Yes.
Because as I take it, the 22 bottom line --
23 MR. MURPHY:
Head this baby off at the pass before 24 it gets to the pass, Stuart.
.l 2L CHAIRMAN HOYLE:
I take it the bottom line that
[
301 1
I'm hearing is that it shouldn't be just put out as a reg.
2 guide for public comment but that you want a more formal 3
step at this stage.
4 MR. MURPHY:
Well, that's -- that's what Jay 5
- Silberg wants and I agree with that as a matter of prudence.
6 What I want, what Nevada wants, is that the thing be 7
ashcanned, throw it away.
It's a bad approach.
Don't put 8
it out as a more formal attempt and let us litigate it.
9 Just forget it.
It's wrong.
It's legally unsupportable.
10 It's indefensible.
That's Nevada's position.
11 CHAIRMAN HOYLE:
And stay with the interim reg.
12 guide?
13 MR. MURPHY:
Well, no.
The interim -- the interim y
14 topical guidelines need refinement.
We all agree with that.
15 I mean, there's duplication in there.
There's minutia.
16
.There's -- you know, we put those together in a hurried 17 fashion and they need refinement.- There's no question about 18 that.
What we disagree with is this attempt to -- to 19 reargue Kelbert_ Cliffs (phonetic) that the--- I mean,.you 20 know, you lost that case, Stuart.
It's been more than 14 21 days.
The petition of reconsideration is untimely.
22 CHAIRMAN HOYLE:
The extent to which you want to 23 add any emphasis along that line, please do so in the next 24 ten days.
25 All right.
Fine.
Please --
/
r.
n 302 1
MR. JOHNSON:
John, if you're ready to cantinue, I 2
was just going to point out Steve Young's background.
He's 3
a geo-scientist in the Center and so he's a producer and 4
user of technical data and he acts as our liaison on this 5
work to other technical staff in -- that I mentioned in our 6
program elements, and he also has a pretty extensive 7
background in developing computer applications, so --
8 KR. YOUNG:
So basically I have kind of a user's 9
perspective on this, primarily from the standpoint of 10 technical review which really brings me to discussing thc 11 issue of technical data and what it means with respect to 12 the categories.of documentary material that are actually in 13 the rule.
i 14 The rule makes allowances for basically four 15 relatively easily defined categories of documentary 16 material.
It actually speaks about'or makes references to a 17 lot of other categories of documentary me'.c
.al, but these 18 categories will contain the mass bulk of what is really 19 technical' data.
And it needs to be made fairly clear right 20 up front that of the major categories of documentary 21 material that are allowed for under the-rule, all of those 22 categories contain what is actually technical data, what's 23 being generated as technical data.
24 And as Rawlee pointed out, there's an incredible 25 amount of technical data that'r actually up here in this
/)
A f.h r___'Q
- w
+ ~-
~
303 1
category.
We are prie.arily concerned at this stage with the 2
technical dats that's in these categories down here, the 3
categories that won't be fully text indexed :te search.
So 4
what we've done is examined the use of this type of material 5
in the technical review process.
6 This is a listing and primarily it's just an 7
extraction of header fields out of the array of fields that are already available and it's A list of header fields that 9
at this p 4 we see as being a kind of an optimal array for 10 finding t..inical data within a systematic framework that 11 really is designed to find documentary material.
12 The bibliographic approach to document and 13 informatiGn handling isn't inherently ideal for technical 14 dattbase management.
Technical database management and 15 document and information system bibliographic systems really 16 aren't the same thing, but it turns out that it's not too 17 difficult to find technical data to discover that certain 18 types of technical data are available and where it's at, 19 what it consists of, and what's required to get at.
20 And so if we look at this list of the headers, 21 this list essentially will give you pretty full access or 22 allow you to find or discover the existence of practically 23 any kind of technical data, no matter what documentary 24 material category that it was in.
~
25 There's a couple of header -- potential header
/
m 304 1
fields up here that ara mainly -- this one's mainly an idea.
2 This one I think we see as being pretty much & re quirement.
3 There is -- in going through the exercise of looking at this 4
problem from the point of view of a user, what I teally did 1
5 is try to examine what the -- what different ways ur tacts G
that you could take to enter the system and try to find
[.
7 something, and it turns out that there are a lot of 8
opportunities to search for technical data or to search for 9
graphic-oriented material that's not text indexed by the
[
10 type of media that it was on.
As a scientist or as an 11 engineer, it's not uncommon to want to find everything that 12 is on tape, for example, and you may be -- and this issue 13 that came up earlier between Barbara and Mal about the 14 readability of tapes, there oftentimes are instances where 15 an investigator may actually have some special capacity to 16 read a certain kind of tape.
The guy may have a PDP-11, 17 which is an old VACS computer that will read seven track, 18 1650 BPI tapes and he says, "Well, gee, let me see 19 everything that's on that kind of tape, so it's probably not 20 unreasonable to expect that a media type category might --
21 might be a way that somebody might want to search for 22 technical data.
23 The document type list, and this may actually be i
L 24 fixed -- I probably haven't seen an up to date listing of i
25 document types -- but there needs to be a specific type code d
305 1
for database catalogs, data catalogs, and basically data 2
listings which may be hard copy data dumps that are nothing 3
but just great long listings.
Those things will exist in some cases.
There's certainly no requirement to make them, e
but many people do make them for editing purposes and they 6
may be in packages.
They may be in data -- data record 7
packages.
They may be attached to certain technical 8
reports.
The -- the category of documentary material which 9
currently is planned to be fully text indexed for search and 10 to have headers and whatnot, much of that material actually 11 is data listings, and you can kind of -- it wouldn't be 12 unreasonable in a lot of cases to expect people to be
(
13 actually looking for the listings that's in that material.
14 What I wanted to illustrate here is when you go to 15 search for technical data in its broadest rense, and this is 16 technical data that could be in any category but technical 17 data that certainly will be heavily represented in the 18 nontext searchable area, you first need to look at how you 19 categorize technical data.
And historically technical 20 subject is by far the most common way to do it.
You break 21 it down by the subject area, the subject matter, and you use 22 primarily descriptors to go after that.
23 But for the purposes of this particular project, 24 there are a nuni.er of ways that we currently look for
~
25 technical data.
We normally look for it based on the
/.
i l
306 i
1 technical subject because we're interested perhaps at 2
faultiny and groundwater travel time, petrophysical charac-3 teristics of rocks, et cetera, so we go after it on that
)
4 basis.
5 It's not too unusual to go after it based on 6
document type either or based on who generated it.
7 Oftentimes you will want to look at a certain array of v
l 8
laboratory -- lab technical reports.
Say you wanted to --
l l
9 you -- the -- a lot of the national laboratory 10 subcontractors for the DOE are actually fairly well j
11 partitioned with respect to the type of work that they do 12 and, if you know what a certajn lab has been working on 13 historically, you could ask 14
-- actually ask for everythis from that -- from, say, Los l
15 Alamos, for instance.
If you're interested strictly in 16 geologic mapping, you could ask for everything that the USGS 17 did, as the USGS is the primary generator of geologic maps.
18 But document type is -- is a pretty popular way to break l
l 19 down technical data.
20 Media type as well.
These are pretty minor 21 compared to this.
This is sort of new.
I don't know to l
22 what extent you guys talked about this with respect to the 23 topical guidelines yesterday, but it's not unusual for us to 24 go in and look for technical data based on the 10 CFR 60 25 requirement that that particular data addresses.
This field
/
307 1
would be problematic in practice and it may be difficult for 2
even a principal investigator to categorize this material 3
based on its relationship to 10 CFR 60.
4 Here are those header fields pretty much broken 5
down into the -- into a set of primary examples of the P'.nds 6
of material or categories that fit in that.
It's very easy 7
to search based on title and description.
As everybody 8
knows, the abstract, whether that's a header field or not I 9
see, is -- it's problematic.
It certainly isn't required.
10 I think I heard Betsy say it's sort of a cost benefit thing.
11 That's probably exactly the way to look at it.
12 MS. SHELBURNEt Well, let me clarify before you go 13 any further on that.
The discussions that we had in which
(
14 we decided about abstracts was liraited only to text 15 searching of documents.
16 MR. YOUNG:
One thing that --
17 MS. SHELBURNE:
Any discussion that you want to 18 have about summaries or abstracts, or table of contents 19 needs to be done based on the needs of this type of category 20 information.
l 21 MR. YOUNG Yeah.
22 MS. SHELBURNE:
Totally independent of what's done 23 for text searching.
24 KR. YOUNG:
Right.
1 l
25 MS. SHELBURNE:
Don't -- no distinction can be
/l t
t
308 1
made.
2 MR. YOUNG:
The abstract -- there -- with respect 3
to data record packages, and it may be that for the purposes 4
of practical handling of the material that the vast majority 5
of the non-text searchable stuff actually goes into a 6
package.
It probably would be a good idea to have some sort i
7 of a field in the package header in addition to the title 8
field and I guess I understand that you do not want long 9
descriptions in this title / description field.
10 It probably wouldn't be a bad idea to have j
11 something like an abstract or a summary field for the data 12 record packages to oescribe exactly what's -- what sort of 13 project that the material that's in there is associated 14 with.
It may be material that's associated with a specific 15 study plan.
It may be material associated with a project 16 that's not a formal study plan in the sense that the site 17 characterization plan outlines them, but it's a specific 18 project and you want to know what the purpose of the project i
19 was, what the objectives were, who worked on it, where was 20 it done, et cetera, so it's probably not a bad idea to 21 consider a field like that for the data record packages.
22 The author and author organization, it's in -- and 23 submitter oftentimes you want to locate this material based 24 on who did it and where it came from and basically all of 25 these~ header fields I think are either already agreed on or
/
309 1
are being discussed, which there's not much new about that.
2 Subject descriptors and -- somebody -- Kirk, you 3
mentioned earlier on that you thought that subject 4
descriptors, which are primarily going to be technical 5
subject, will probably be the most powerful search vehicle.
6 Historically, that's certainly been true and probably will 7
be for this system as well, so a lot of attention probably 8
should be paid to the -- the source of technical subject 9
descriptors.
10 Sponsoring organization, identifiers, and comments 11 aren't absolutely necessary but certainly would be helpful.
12 Again, this issue of regulatory category comes up.
We look 13 for material on this basis.
I do not know to what extent 14 other organizations search for material based on what 15 section of 10 CFR 60 that it refers to, but 10 CFR 60 16 certainly has an enormous impact on the type of data that's 17.
being taken, the way it's handled, the kind of analytical 18 processes and the analytical procedures that are being used, 19 and you could probably find just about any kind of technical 20 data you wanted based on what area of 10 CFR 60 that -- that 21 is directing that that material be actually accumulated or 22 measured or analyzed.
23 Pointers are not a bad idea to point to associated 24 materials or even back to a package that the same material 25 might be it..
/
l
310 1
It's not unreasonable to expect to be able to 2
search based on media type, so it's --
3 MR. NIPPERT:
I've got a question on your media 4
type.
5 MR. YOUNG:
Yeah?
6 MR. NIPPERT:
It seems to me back in the earlier 7
packages that we showed, there was one package in which 8
you'd have a whole variety of media types.
9 MR. YOUNG:
Yes.
10 MR. NIPPERT:
I mean, media types would be down at 11 that table of contents level instead of that general header 12 because otherwise you're likely to have a whole variety in 13 there and it can be --
14 MR. YOUNG:
It's -- there's -- some thought needs 15 to be put into that so that a person that is searching for 16 everything on diskette, everything on a three and a half or 17 a five and a quarter or everything on seven track or eight 18 track or nine track tapes, there needs to be some -- some 19 allowance for people to find out what's available on certain 20 media, especially the -- you know, the computer readable, 21 the digital stuff.
22 MR. ACREE Can I say something?
The Yucca 23-Mountain Project Office has a media field as well as the 24 document typed in and it helps further define these things.
25 Now, you may have -- you may or may not want to search on it
/-
311 1
but it's handy there in the header, but that's the reason we 2
suggest it as a possible field.
3 MR. NIPPERT:
Well, but again I think it depends 4
on how much you're going to unitize what's in a package.
5 CHAIRMAN HOYLE:
One package may have everything 6
-- every media type in it.
7 MK. NIPPERT:
I'm not arguing about its utility.
8 It's just the level at which it ought to be associated.
9 MR. JOHNSON:
Yeah, I think -- I think -- yeah, 10 I've made a note of that and it has to be at a certain level 11 to match the particular item in the package.
12 MR. BALCOM:
So that could be, as I see it, done 13 one of two ways.
Probably the direction we're not going in 14 is to have a header for each item and the direction we are 15 going in is some more complete doscription than the table of 16 contents that wouldn't be easily searchable from a media 17 type field that would be kind of a compromise but a more 18 complete description.
If -- you know, I don't know if we're 19 going to need to get into that more or not.
The field --
20 the media field, by the way, was deleted by the header 21 working group a few months ago and it looks like -- it looks 22 like we 23 shouldn't have deleted that and it might have to go back in 24 although I don't see that it would be a tremendous help here 25 in finding a specific --
/
l
I 312 1
MR. YOUNG:
I can give you a specific example.
If I
2 you were interested in, say, the well logs, the geophysical 3
logs from a specific borehole and you vent to go locate l
4 those, so you go look for those data by the borehole name --
l l
5 UE25P-1, say "Show me all the geophysical logs that were 6
done in that borehole."
You come across and all you come up 7
with is a whole set of processed tapes that are way beyond 8
first pass edits, but you knsw that the original field 9
recordings have to be somewhere but, for some reason, you 10 don't find them so you go back and say, "Well, look, I know 11 a little bit about how thosa things were recorded, so I'm I
12 going to go back and I just want a list.
Just dump me a 13 list of all the 1650 BPI eight-track tapes that you have and 14 I'll look through that list and see if the original field 15 recordings were actually in there."
16 It's -- it's a little bit a redundancy that just l
17 by experience we found is helpful in the event that L
18 something is missing that you know should be there that 1
19 perhaps hasn't gotten into the right category, the right 20 stream, something else.
l 21 MR. BALCOM:
We're not making it the way it's 22 designed now that we're not making it real easy to narrow in i
23 on a specific tape or we'll be, as I say, yes, there are l
24 some tapes associated with this package and they should be 25 in such and such a place.
?
l
313
)
1 MR. YOUNG:
It's -- some consideration should be 1
2 giten to, you know, treatment of that material.
I know that 3
it's very likely that there will be data on tape that is 4
associated with a specific project, all of the hard material 5
of which would be in a data record package.
And these data 6
record packages physically are vertical file folders and 7
you're not going to dump a tape in there.
That's going to 8
be sitting over on the shelf somcwhere in somebody's tape 9
libra ry.
10 Now, within that data record package, there 11 certainly can be an item, a pointer, a header, something, 12 even in the table of contents that says that there's 150 13 megabytes of geophysical log data, resistivity on tape and 14 here's the tape, a session number.
For example -- I mean, 15 this is just an example.
I don't know how to handle that 16 specifically.
There's lots of ways to do it.
But this 17 business of data on tape or data that is in somebody's 18 database management system on somebody's either work station 19 sc&ewhere or perhaps the SEPDB is not a bad example to -- of 20 an example of a situation where you might want to point to a 21 certain batch of technical data, a certain batch of raw or 22 processed data that exists in a location that's on -- you 23 know, on a hard disk or on -- that's tape accessible through 24 somebody's computer data management system.
25 So it's -- it isn't an absolute requirement
/
314 1
because most people are going to hit everything they want 2
through --through those -- through those descriptors but it 3
cuts down on the detective work that's required to find 4
things that you may need or that might be useful.
5 And document type if a really useful way to fle.d 6
things, especially if you know a little bit about the 7
project and how things are done within the project.
It's 8
very easy to try to get away and look for technical reports, 9
for example, especially technical reports that were 10 generated by some author or some author organization since 11 it's pretty clear who's doing what work and what national 12 lab that it's being done in, et cetera, and most of the 13 people who are probably going to use this thing on a routine 14 basis are going to have some little bit of knowledge or --
15 either that or are going to discover information and 16 knowledge about where certain projects are being done.
17 Again, it just sort of helps to cut down on the 18 amount of technical detective legwork that you actually have 19 to do to find things.
20-So we think this is a pretty good array.
It's not 21 an exhaustive -- it's sort of a -- it's more of a --
22 currently kind of an optimized set.
This is the biggest --
23 this thing is -- the regulatory category as far as I can 24 tell isn't really anticipated by any of the -- by the header 25 framework that's set up and if -- other than the topical S
315 1
guidelines, which are very -- have a lot of wording in there 2
that is very 10 CFR 60 like, I mean, a lot of the verbiage 3
that's in some of the topical guidelines is definitely 4
regulatory in nature.
Some thought needs to be given to 5
this.
I'7 not sure --
i 6
MR. BALCOM:
Steve, we have a category called 7
special class and I think it just might be possible that 8
that's -- this is the kind of --
9 MR. YOUNG:
I'm not sure that you want to elevate 10 that to some sort of really large important, full-blevn 11 header field.
But there are a substantial --
12 MR. BALCOM:
Well, I mean, no.
I think we already 13 have a header field which would suffice for this.
14 MR. ACREE:
What special class?
15 MR. BALCOM:
Special class is simply a category 16 within which you can classify -- you can set up your own 17 classification scheme.
It's kind of a blanket field for --
18 MR. YOUNG:
In order for something like this 19 to --
20 MR. BALCOM:
-- Dona may have.
Since you used it 21 in the prototype, you might be able to say more about that 22
-- correct me if I'm wrong, too.
23 MS. MENNELLA:
It was originally envisioned as a 24 P.O.
-- given the fact that documents can be classified in l
25 various ways, and we chose in one way, which is using title,
/
l l
316 1
author, whatnot.
There are other ways you can classify 2
documents and special class field was envisioned to capture 3
that.
For example, all documents that are part of the SCP 4
administrative record or all documents that are part of the 5
reference to the SCP record.
So -- also we have in there 6
things like excerpts or if it's a foreign language document, 7
if it's a header only document, sort of a catchall field but 8
it certainly would lend itself to something like this.
9 MR. YOUNG:
Yeah, I would try to name it something 10 that points to this purpose.
There certainly are a 11 substantial number of people who are capable of searching on 12 this basis and what's more, all the -- since -- there's a 13 tremendous amount of data that are being generated that are 14 targeted directly on these subject reas -- siting criteria, 15 design criterion, geologic repository operating area.
16 It wouldn't surprise me a bit if somebody would --
17 if particularly engineering people that aren't really 18 familiar with the geoscience literature would want to pull 19 up all the geoscience stuff that referred -- that has some 20 sort of bearing on the design of either the geologic aspects 21 of the design, the waste package design, so it's a category 22 that would be difficult to have properly filled out with 23 descriptors.
24 People that are well qualified to do this, it's --
25 like I said, it's even difficult for P.I.s to sit down and
/
317 1
say, "Well, this is..." -- you know, there may be four, 2
five, or -- four or five of these categories that it's 3
really directed to or maybe just one or two.
But it's -- it 4
would be a usefni exercise to look into that.
5 MR. JOHNSON:
Thank you, Steve.
I guess to close 6
out our discussion with you, I'de like to identify the 7
issues that we'll continue to work on as we go through the 3
next steps in our project.
9 We've got three categories that we've put them in.
10
!!aader content is the first category.
And how will 11 currently approved fields be used?
These are the things 12 that Steve was just talking about.
13 What fjelds must be added, if any?
Under that, we 14 would, you know, look at are the submitter / sponsor fields 15 sufficient for storage location of non-imaged material?
Can 16 the document-type field.'ncorporate media?
Is the 17 regulatory category field appropriate, necessary?
Is a 18 qualified data indication needed?
19 You know, it struck me this morning when we talked 20 about QA and QA record packages that you almost certai.ly 21 vant to identify certain QA records that are in the LSS and, 22 whether that's done in a separate field or is part of a 23 table of codes you use in another field, I think it needs --
24 needs that kind of indication, s
25 MR. MURPHY:
You know, in a way that's almost a
/
318 1
legal conclusion in a licensing proceeding.
2 MR. JOHNSON:
That's what?
3 MR. MURPHY:
It's a legal conclusion whether or 4
not particular data is QA-qualified.
5 MR. JOHNSON:
That's --
6 MR. MURPHY:
So all that's going to tell you is 7
whether or not the submitter thinks it's QA-qualified.
8 MR. JOHNSON:
Okay.
9 MR. MURPHY:
And we still may -- there still could
]
10 conceivably be an argument over the licensing proceeding.
I 11 think it would be helpful to know that.
12 MR. JOHNSON:
Well, I'm looking at it as someone 13 wanting to retrieve certain records, simply what someone 14 thought were QA.
15 MR. MURPHY:
Right.
I think it would be helpful.
16 MR. ACREE:
And, again, this is a field that the 17 Yucca Mountain Project Office does use in its record systems 18 and it seems like the majority of data --
19 MR. MURPHY:
Yeah, but all I'm saying is that 20 because it's in a -- it says in the header this is good 21 stuff, this is QA-qualified data, isn't going to persuade me 22 or anybody else, but it would be a useful piece of 23 information I think.
24 MR. JOHNSON:
Any -- any further comments on these 25 issues before we go on to the next?
Okay.
/
i 319 1
Under data record packages, we've got issues 2
regarding the timing of submission.
That needs to be 3
defined.
How will the non-imageable portions be 4
individually stored?
Steve talked some about that and 5
sugg rit%i that physical tapes and disks would be stored 6
separately from the package itself.
7 Table of contents must be sufficiently descriptive 8
and the question should it be text-searchable?
We talked 9
about that and I think moved beyond some of the thinking as 10 a group here on -- on that issue when we discussed the 11 abstract.
And, depending on where the table of contents or 12 some descriptive summary of these packages actually fits in 13 the header, I think some ideas have come together here.
14 The header must be thorough.
We talked about 15 that.
16 Should the packages be made text-searchable 17 insofar as possible?
And of course Barbara alluded to the 18 fact that DOE is essentially doing that, so -- any comments 19 on that before we go to the last set of issues?
20 Which we've grouped under non-imageable classes of 21 technical data.
And these classes, the classes suggested 22 here are magnetic media, film, colored graphs and photos and 23 extra large maps.
There may be some others that we don't 24 have in there right now, but those are the sorts of things 25 that would be on the shelf, so to speak, and not viewable in
/
320 l
1 the system.
2 MR. MURPHY:
Well, the other two obvious classical 3
categories, which are handled separately, as I talked to you i
l 4
guys about, are core samples and water samples.
5 MR. JOHNSON:
Yeah, core samples which DOE prefers 6
to exclude, you know, submissible in the LSS context.
7 They're there and they prefer to have access to those, you 8
know, in their location.
9 MR. MURPHY:
Well, access to those is going to be 10 through the sample management facility.
11 MR. JOHNSON:
Right.
12 MR. ACREE:
They're saying that samples aren't 13 data.
The data is figured from the samples.
14 MR. MURPHY:
That's bullshit.
Of course they're 15 data.
But they'll be handled -- I mean, presumably people 16 will have access to -- there's a whole separate cataloging 17 system.
I mean, it's -- but that's not to say that 18 technically they aren't LSS data.
You know, a core sample 19 which is part of the raw data that supports the report is 20 LSS information as far as we're concerned, but you're going 21 to go look at it and have access to it through the sample 22 management facility, not through an LSS facility.
23 MR. JOHNSON:
Of course the information is in the 24 LSS in great quantity on test and measurements and analysis 25 of those core samples.
l d'
l 321 1
MR. MURPHY:
Right.
But if you want to go and --
2 MR. JOHNSON:
The second area, the criteria in 3
this area of non-imageable classes of technical data I think 4
ultimately, you know, is the practicality of what we do 5
there as far as access protocols, the cost-effectiveness.
6 Those are the criteria I think that drive the decisions made 7
in this area.
8 Access protocols of course are needed and so 9
that's where we're headed, to develop those.
10 We appreciate your feedback and your discussion.
11 We'll go on about our business.
12 MR. MURPHY:
Where do you go from here?
What's 13 next?
14 MR. JOHNSON:
Well, we're going to produce two 15 reports on tne technical data definition and the 16 infrastructures, and that will reflect observations and the 17 current conditions of the main participants in their ability 18 to submit documents to the LSS and assess the -- you know, 19 where there may be potential problems or changes that need 20 to be addressed for a specific participant.
21 MR. CAMERON:
Yeah, I think the Center has made 22 substantial progress on this issue in just the few months 23 that they've been working on it, but I would want to 1
24 emphasize to the panel that there are some reports that are 25 going to be coming in from the Center to our office on this
/
u
322 1
issue and that all of it is going to be sent to the panel 2
for review and comment before anything is final, and I'll go 3
through some of the schedule for that in the next few 4
minutes.
5 MR. JOHNSON:
Thank you.
6 CHAIRMAN HOYLE:
Okay.
Thank you very much, 7
Rawlee and Chuck and Steve for your presentation.
8 The next item, Chip, is yours, status of 9
compliance _ evaluation program, status or priority document 10 production schedule, and anything else you want to say.
Il MR. CAMERON:
Okay.
12 CHAIRMAN HOYLE:
In 15 minutes.
No, I don't want 13 to rush you.
14 MR. CA!!ERON:
No, I'll just -- I'll be brief.
I 15 have a handout for you that basically is a time line of 16 what's happening on other areas of the LSS project.
And 17 this is the time for Murphy to read his eewspaper.
He knew 18 that it was convenient.
19 (Pause.)
20 MR. CAMERON:
But I'll just quickly go through 21 this time line.
The purposes of this are to show you the 22 relationship of other activities that are going on in the 23 LSS project, the relationship of those activities to the 24 schedule for design and development that Barbara talked 25 about yesterday.
I think it will also help you to flag key
/!
323 1
issues and to also do some planning for future ARP meetings 2
and activities, and it also sort of givas you a summary of 3
what our office is doing.
4 Now, I put the LSS design and development schedule 5
on the top line, and you can see going across the top that 6
in '90 we're going to have the SAIC design documents done.
7 In August of '91, I listed acquisition support documents.
8 This is -- these are documents that the FEDSIM contractor is 9
going to develop and some of those documents are, for 10 example, alternative aralysis, functional specifications, 11 things that feed into the request for proposal for the LSS 12 pilot system.
13 In November of '91, there's going to be a request 14 for comment on the LSS procurement.
15 Then in April of '92, we have the request for 16 proposals.
November, contractor award.
And in 1993, in 17 August, the installation of equipment, DOE test and 18 acceptance.
19 We're also developing guidance and standards for 20 document identification, submission, and preparation.
The 21 header is a good example of that and I think right now we 22 have March, '91 set as a final date or as a date for the 23 final header for the LSS.
24 I also put the technical data recommendations in 25 this category and right now we have September slated for the sf
324 1
final report to come in from the Center to us on technical 2
data, and we would like to issue that in March of '92.
So 3
that time period between September of '91 and March of '92 4
would be a period that we would want to reserve for ARP 5
review, for example.
6 Alco in this guidance and standards area, we're 7
going to be working on such things as what the standards 8
should be for the submission of ASCII to -- to the LSS 9
Administrator, what the standards would be for the 10 submission of hard copy insges, and things like that.
11 There's also going to be some work done on 12 facility planning and development because we have to be sure 13 that the facility is ready at UNLV to accept the equipment
(
l 14 from the LSS contractor.
And SAIC is working on a generic 15 facility design document.
Lynn Scattolini of our staff is 16 working on s facility planning issues paper, and both of l
17 these are going to be done in the next few months.
And what 18 we'll be doing from then on is trying to develop a schedule 19 of different activities that need to be accomplished and 20 who's going to be responsible for accomplishing those 21 activities so that we can get down to the UNLV facility 22 being ready, which you'll see down in '93 under compliance 23 evaluation, but it should be up under facility planning and 24 development.
25 Another area that we're focusing on is access
/
l
325 1
planning and some of the issues here -- I mean, you've hea d r
2 some of the -- some of these issues come up in the context 3
of the SAIC presentation.
But in terms of implementation of 4
- access, the number of work stations that will be needed by 5
each party who will have accest to the LSS, the number of 6
users that they're going to have, the assignment of access 7
passwords, telecommunication needs, all of these things need 8
to be addressed.
9 The next category of activity is a production 10 schedule and this basically is -- concerns the submission 11 capture, and loading of documents into the LSS.
What we 12 plan to do along these lines is to -- to let another 13 contract with the center to take a look at this question of
(
14 priority document categories which we talked about at our 15 first meeting.
The whole idea is try to provide the most 16 useful datt for technical review to LSS participants when 17 the system is first loaded.
18 Now, originally we were going to try to put the 19 onus on the panel to come up with these document categorie 20 s,
but we thought it would be better to have someone with 21 expertise in a technical side of the high level waste 22 management business take a look at the document categorie s
23 and come up with some priorities and then we would come ba k 24 c
to the ARP for review of those -- those categories.
25 One thing the Center is also going to be looking
/
326 1
at is when will useful data and information be available?
2 In other words, when -- some people argue that -- that you 3
don't need to have anything into the LSS until two or three 4
years after DOE gets onto the site.
Other people say that, 5
"Well, there's.some generic documents out there that can be 6
loaded into the system now that would be -- would be 7
useful."
And as a threshold issue, what we want to have the 8
Center do on this contract is to take a look at -- at the 9
timing, the availability of useful data and then go on from 10 there and categorize those different documents into priority 11 categories.
12 The Commission has required us to go to them with
(
13 a cost benefit analysis of the document loading schedule for 14
- the LSS, including this first pilot system, so what we plan 15 to do is to have a draft recommendation on priority loading 16 categories available from the Center in March of '91.
Ne 17 would then come to the ARP to take a look at those 1
18 categories and then we would go to the Commission for 19 approval of a document loading schedule.
20 MR. MURPHY:
That -- I was going to ask you that 21 question when I saw this -- this flow chart.
Where does 22 that -- why is the Commission concerned about a cost benefit 23 analysis with respect to the loading?
Where does that come 24 from?
25 MR. CAMERON:
That concern comes from --
I g
a~ < -
327 1
MR. MURPHY:
And is that the Commission or a 2
member of the commission?
3 MR. CAMERON:
Well, it originated with a member of 4
the Gummission, but it is a Commisrion viewpoint.
5 MR. MURPHY:
Okay.
6 MR. CAMERON:
And it was expressed in the usual 7
Commission manner, which is I think it was attached to a 8
staf f requirements memorandum to us to go back to the 9
Commission and the ccncern was that we not get too far ahead 10 of the repository schedule, that -- that they take a careful 11 look at -- at how many documents are going to be loaded that 12 are site specifir to see if this is cost beneficial.
That's 13 the way we understand the direction.
'14 But it is something that we're going to have to 15 do.
16 MR. MURPHY:
I'm glad somebody understands it 17 because-I --
18 MR. CAMERON:
Well, what we are -- what we're 19 going to do is we -- we think that there should be priority 20
-- we think that the system should be loaded as soon as it's 21 feasible to do it.
I mean, feasible from the standpoint of 22 the equipment has been debugged and we think that it is 23 useful to establish these priority categories and start 24 loading that data, and that's basically what we're going to 25 precent to the Commission.
And we're going to really
/
328 1
emphasize the benefit side of that in terms of this is 2
useful information for LSS participants.
3 I'm not sure what the cost side of it is going to 4
be at this point.
I think that the words the Commission 5
used was not cost benefit but risk benefit, so I don't know 6
what the risk is, but we're going to have to address that 7
issue.
8 But, you know, from the viewpoint of our office, 9
if we can identify useful priority categories, that's where 10 our emphasis is going to lie.
11 MR. MURPHY:
Well, are you saying that the 12 Commission wants to -- is the Commission somehow reserving 13 to itself the decision as to whether or not to begin loading 14 at all any documents because it's too far in advance of the 15 license application?
16 MR. CAMERON:
I don't --
17 MR. MURPHY:
I guess I don't understand what the 18 concern is.
19 MR. CAMERON:
-- have a real good feel for -- for 20 whether the Commission would say that.
I don't think that 21 the Commission --
22 MR. MURPHY:
Well, they're --
23 MR. CAMERON:
-- would say that.
24 MR. MURPHY:
-- not proposing to get into the cost 25 benefit relationships between loading a certain category of es'
329 1
documents as the number one priority versus another category 2
of documents, are they?
3 MR. CAMERON:
No, no.
It's just really to -- it's 4
just really to go to the commission with information on, 5
okay, the system is -- has been debugged.
We're ready to 6
start loading documents or this will happen well before 7
that, but the idea would be is that we're ready to start 8
loading and, for your information, here is what we're going 9
to load and I don't think tnat the Commission is getting 10 that involved in, well, it shouldn't be this category.
It 11 shouldn't be that category.
It's more to -- to give them a 12 comfortable feeling of what we're going to be doing before 13 we embark on the project.
And that I think is the -- the
(
14 sense of the Commission, at least most of the Commissioners 15 on this, so we don't anticipate that it's going to be a big 16 problem, but certainly it's going to be important for the 17 advisory panel to weigh in with an emphasis on an importance 18 of beginning to load these categories.
I think that will be 19 influential with the Commission at that time.
20 MR. MURPHY:
And also at what point in time do we 21 get to -- to have some input on prioritizing document 22 loading?
That as a priority I think have changed since the 23 negotiation.
Remember, we all submitted certain generalized 24 ideas about what priority voting, at least for backlogged 25
- data,
/
I 330 1
KR. CAMERON:
Right.
I think at that time you 2
were -- was going to be we'll do all contractor documents or 3
something like that.
4 MR. MURPHY:
Right.
5 MR. CAMERON:
But what -- what we're doing is 6
instead of having the panel sort of come up with priority 7
categories, we're going to have the Center as our contracter 8
do a report on priority categories and present that to the r
9 panel for review so that you'll have a straw man document to 10 work on.
11 KR. MURPHY:
Well, should we submit cur views then to the Center as to what the priority --
12 13 KR. CAMERON:
What the Center will be doing, much 7
(
14 akin to what they have been doing with the technical data 15 project, is to go around and talk to the different 16 participants, get an idea from them what they think the 17 priorities are, and then they'll -- they'll integrate all of 18 those views along with, you know, their analysis of it and 19 present a document to us which we'll submit to the ARP for 20 review, so you will be contacted as individual members as to 21 your ideas on priority categories.
22 And, again, you know, there's a lot of ways to cut 23 the -- to cut the pie on that.
24 CHAIRMAN HOYLE:
And you're looking to about March
\\-
25 of '91 as the time frame for that to be available to us as a
/>
331 1
group?
2 MR. CAMERON:
That's right.
3 CHAIRMAN HOYLE:
Okay, so that's something we i
4 could pick up on as -- as a next meeting item for the first j
5 meeting in '91?
6 MR. CAMERON:
No, I think that that would be 7
appropriate.
8 CHAIRMAN H0YLE:
Yeah.
'9 MR. CAMERON:
And what we'll do is we'll try to 10 make sure-that we coordinate our -- our contract, the 11 development of those categories, so that we can have that as 12 an agenda item for the first meeting of the panel in '91.
13 CHAIRMAN H0YLE:
Okay.
Let me just add.
I concur s
14 in what Chip saidLin responding to your questions about the
'15-Commission's involvement at this' stage.
I believe it's just 16 a -- it's'certainly a major milestone to begin loading the 17 system and'I think that's a point at which the Commission 18 would like to be able to have another look at it and be I
-19 briefed on it and if'any policy guidance is necessary, they 20.
c 71d give it at that time.
.t 21-MR.. CAMERON:
That's right.
And another category
'22 on here is compliance' evaluation program and here we're also 23 going to look-for input from the panel.
24 As you know, the Administrator is responsiblo for 25 evaluating the compliance of the participants in terms of
/;
r 4
332 1
the document identification, preparation, and submission b
2 requirements in the rule.
And, furthermore, to periodically 3
report on DOE's compliance with the rule and to ultimately 4
make a certifi.ation decision on whether DOE is in 5
substantial vmpliance with the document submission --
6 identification and submission requirements, that being 7
geared to whether they can file the license application 8
under the Commission's rules and 10 CFR, Part 2, Subpart J.
9 What we're doing is we have a contractor, Lavogt 10 Anderson and working in conjunction with Price Waterhouse,
'll-working on developing a compliance evaluatien strategy for 12 us.
And in terms of a strategy, we're talking about hrw
-13 should we approach the compliance evaluation process?
19 For example, we could have the pP-eticipants,ubmit 15 the procedures that they're going tc use to implement the LSS rule for review, to' ensure that procedures are ir. place.
16 17 Therefore, giving one greater assurance that the 18 requirements would be -- would be met.
We want to develop a 19 comprehensive guidance-manual for all LSS participants on 20 what the document identification,-preparation, and 21:
submission requiren.unts are so that they can look to one 22-place for --'for guidance on what they'ra supposed to do and 23 to amplify on some of the' areas-of the rule that may need 24_
further amplification.
^
25 What would be the approach to actually going out
/l F
k h
333 1
and evaluating compliance in terms of, for example, audits?
2 But we're developing as part of this contract that will 3-actually include the implementation of the compliance 4
evaluation, the first step is to come up with an overall 5
strategy and we want to have that ready for -- for panel 6
review and we're eventually going to go to the -- to the 7
Commission with it.
8 And I know that on this particular sheet, the way 9
we were thinking -- John and I were talking about what might 10 be a good date for the next canel meeting -- we were 11 thinking about -- about June and this shows that there would 12 be.some.
l 13
-- there would be Commission approval, therefore needing ARP 14 review of this issue before June, and I think that what 15 we'll need to do, since it is so important, is to perhaps 16-move that'back a little so that that would be something that
~
'17
.would be on the June agenda also.
18 ~
CHAIRMAN HOYLE:
Or meet earlier than June if you 19 think it would be --
20-MR. CAMERON:
Or meet earlier than June.
..actly.
21 And the final category here is O & M planning, and 22:
the key here "is that we 'have to -- the LSS Administrator has 23 to operate and maintain the syctem.
Now, we have DOE.
24
. designing and developing the system and the problems.of 25 transitioning from DOE and DOE contractors te LSSA and LSSA
/-
334 1
contractors ~is going to be extremely complicated and has to 2
be well thought out in advance, and we are working on that 3
now, and Lynn scattolini from our office is -- that's her 1
4 area, is operation and maintenance and planning for that and j
5 planning for the transition.
And we're in the process of L
l 6
looking for contractors now to -- that would help us do that l
7 O & M planning.
But one thing that Lynn is working on is an t-l l
8.
O & M planning strategy and identifying some of the issues l
l 9
that need to be addressed.
L l
l 10 But basically what I wanted to do is to -- to give L
11 you an idea of all these other layers that all have ":o come 12 together in 1993 so that we can successfully operate and 13 implement the system, and we are fully committed-and -- and l
'14
-welcome the panel's advice on any number of these issues and 15 so we want to make sure that you're -- you're cued in to the i
16
'whole process where we develop some of these products.
l 17 CHAIRMAN'HCYLI:~ All right, Chip.
Thank you very l.
1 l
18 much.
I think we do want as much guidance.as we can from 1
19-the Administrator's office as to what are those items that 20
'are -- are coming due in the next three, four, five, six 21 months so that we spend our time productively on the items j
L 22 that are most important to you from a time standpoint, so l:
23 let's pick out two or three or ;v and try to narrow down a lc i
24
. time frame for the next meeting.
From a planning 25'
' standpoint, the next meeting would be in Washington area,
/;
335 1
Bethesda or Rockville or wherever we can set it up.
It 2
looks like the draft recommendation by the Center and from 3
the Administrator's office on priority loading categories 4
would be available in March.
The paper on compliance 5
evaluation strategy would be available by March also, but 6
perhaps earlier.
7 Everything else looks like it's a little further 8
out.
How about access issues?
9 MR. CAMERON:
We are going to have a access issues 10 resolution plan --
11 CILAIRMAN HOYLE:
Later this year?
12 MR. CAMERON:
-- developed, and I think that it 13 would be worthwhile to -- to have the panel bak at that.
14 KR.-MURPHY:
Access to the system or accees to 15 data are we talking about?
i
?
16 MR. CAMERON:- Access to the system.
In other words, the -- all the practical problems of what a 17-18 participant has to prepare for on access.
19 MR.~BALCOM:
Submission of documents as well or 20 no?
21' MR. CAMERON:
No. ' Access means participants 22 getting-into the system and, you know, the term -- such 23 things as number of work stations needed,. number of users, 24 passwords', all these things, we'd like to start fleshing 25 out.
This paper would be more -- the discussion on it would
/
i
336 1
be to make sure that we had all the issues identified and 2
that we were heading in the right direction, but I think 3
that that,would be -- that would also be a useful -- useful 4
document.
5
-And, Betsy, I do have the right date on the 6
technical data, right?
That's not going to be done till --
7 till later on; right?
8 MS. SHELBURNE:
Right.
9 MR. CAMERON:
Okay.
10 MS. SHELBURNE:
But there will be draft forms in 11 February, federal reports, but the final protocol plan is 12 not due until the end of the fiscal year.
13 MR. CAMERON:
0kay, so it looks like, John, that 14 this access planning, the priority loading, the compliance 15 evaluation strategy would be three issues at least for now 16 that we could - -we could flag for that first meeting.
And 17 besides updates on various thl'ngs.
18 Very good point.
We did say earlier on that we 19 would talk about the whole project management _ aspects.
20 CHAIRMAIT. HOYLE:
Yes, we-did.
21 MR. CAMERON:
Okay.
And including the work 22
-breakdown structure that we're developing and insuring 23 consistency with DOE, so that would be another "- another
-24 issue that we could address.
25 I think that from our point of view, one thing
/:
4 337 j
1 that the panel should think about is -- and I'm sorry that 2
9arbara isn't here right now -- but the SAIC design 3
documents and the SAIC has made their design sessions open 4
to whoever wanted to participate and some people have taken 5
the opportunity to do that -- the SAIC design documents are 6
going to be the foundation for proceeding with these 7
' functional specs that are going to be done in the August of 8
'91 time frame.
So the panel should probably think about 9
how it wants to comment on the SAIC documents, and this is 10 not in the sense of going to SAIC and having them redo 11 anything because I think the contract is -- is closing up.
12 But in terms of proceeding onto the next step of 13 using this SAIC documents, there were a-number of issues 14 that came up yesterday related to this.
I think that it 15 would be useful for all of us that are' involved in design 16-and development to get some feeling from the panel on -- on 17 various aspects of the design.
For example, are th'ere 18
' issues when you go through there that they could be flagged 3
for-further rev - because it might be abla -- we might be 20-able to achieve the LSS functionalities in a more cost 21 effective manner, whatever.
But I think that there's an
\\
22 issue that needs to be addressed there.
23 CHAIRMAN HOYLE:
I think Barbara volunteered to 24 send us all a. copy of the -- the design package.
I guess
\\
25 Maybe it's a very sizeable e mument.
-l l
L i-s i
338 1
MS. MACALUSO:
Yeah, I think she said that, too, 2
but it's cue by the end of November to Barbara.
3 CHAIRMAN HOYLE:
Okay.
It will come to Barbara by 4
the end of November.
5 MS. MACALUSO:
If I remember correctly, yeah.
6 MR. MURPHY:
I thought she said she was going to 1
j 7
send us whatever she got last week.
8 MS. MACALUSO:
The draft?
Okay.
I 9-MR. MURPHY:
I shouldn't put words in her mouth.
l l.
10 I thought that's what she said.
11 MS. MACALUSO:
I don't remember for sure.
I l
12 thought she said the final -- okay.
l p
13 MR. BALCOM:
I think they're going to be available j.
14 sooner and final approval would be the end of November.
(
15' MR. MURPHY:
Yeah, that's what I thought.
16 MS. MACALUSO:
Okay.
l 17 CHAIRMAN-HOYLE:
All right.
We-would ask Barbara 18;
'then to send to each member of the panel a copy of.that L
19 document and --
1 20 MS. SCATTOLINI:
Well, I don't know.if the panel 21
. members want to spend a lot of time reviewing the draft L
22' document because it's being substantially-revised by SAIC:at L
e 23 this point.
1 i
i..
24, MR. MURPHY:
Well, panel members should decide I
25 that'for themselves.
i 1
pf ;
339 1
MS. SCATTOLINI:
Well, that's a good point.
I 2
guess what I'm trying to do is just give you a heads up that 3
the content of the document will be changing a great deal 4
between the draft and the final document which is due next 5
month.
6 MR. BALCOM:
Between what's available now and 7
November 31st?
8 MS. SCATTOLINI:
Well, what's available -- what's 9
available now is the document that you've reviewed in the 10 design reviewing index.
Our comments are being incorporated 11 into the final deliverable, which is due next month.
12 Certainly, and you're *1cht, h a )., I apologize.
I didn't
~
13 mean.to say that you
.2 d nave access to it.
You've all q
14 been invited to the design review meetings.
It's just-that 15 it took-us a week to' read it and I'm just giving you a heads 16 up that it:is being substantially revised at this point.
17' MR.' L BALCOM: ' Chip, how would you propose that that (18 analysis take place?
Did I hear that'you're suggesting some
-19
-- some group get-together and it may be somewhat different 20-than the group that is involved in the' design, at a meeting i
211 perhaps of more broad interest in cost issues or something 22 like that and --
l 23 MR. CAMERON:
Well, I guess.that that's sort of 24 for the panel to figure out how they want to structure that.
25; I,can -- I can-.say that we would like to.get the panel's m;,
340 1
input on the design documents and, Dave, I believe you're 2
doing a -- you're looking at doingt a cost benefit also, 3
another look at cost --
4 MR. NIPPERT:
That's correct, yeah.
We're 5
revising the design documents to address some issues that we 6
had in terms of design and we have a few open issues from 7
earlier that we'll come to closure on.
8 But some of the big things, we're actually going 9
to do a cost benefit study and look at the cost analysis, so i
10 all of that's not going to be done until just about the end 11' of November.
12 MR. CAMERON:
Yeah, I --
13 MR. NIPPERT:
And I think you need to see both of 14 those together really.
15
.MR. CAMERON:
I think you're -- I think you're
'16 right about'that and I -- I think.when that is done, if the-17
' panel could decide how they want to -- to review those i
la documents,.they may.just want to focus on the cost benefit.
'19 They.may want to go in and take a detailed look at the 20 design documents.
I realize that some'pa el members have 21
'more of a capability to do a detailed analysis than others, 22 but there could be a working group of the panel, for
.23 example, to take a look at all of those documents and report.
24 back to the panel in the -- the June time frame again.-
I
.25
.mean, that's a possibility.
~
341 1-CHAIRMAN HOYLE:
Is that timing sufficient for DOE 2
to keep moving on?
3 MR. CAMERON:
Well, Dave, when do you think that i
4 DOE would be able to release -- DOE accepts in November and 5
after that I suppose that --
6 MR. NIPPERT:
It's just a question of, you know, 7
reproduction and distribution.
Documents -- we're going to 8
turn them over -- it will be in notebooks with dividers.
9 There are substantial documents.
There's the capture system 10 document which originally was completed almost 18 months 11 ago.
There's some revisions that will be made in that to 12 reflect decisions that have been made in the capture or the 13 search and image system, document -- the search and the 14 image is in two volumes and then the communications i
~15 do'cuments and'those will be'-- the cost benefit report I 16 guess is. going'to'be on'the order of a hundred page 17
. document.-.And by tha time-we doLall the alternatives in the 18
~ costs, 19 MR. CAMERON:
That's a critical -- you know,.
20 that's a critical docume:it' because I think as we know during.
21' the negotiated rulemaking, the industry was very concerned 22' about costs,p and not to say that 'everybody -- I think 23:
everybody was concerned about cost,. but the industry's --
1 1
24' the results:of their cost' benefit equation came out 25 different'from -- from the rest'of us.
/
I
342 1
But Barbara' was making some commer.tn yesterday 2
about, well, the costs have greatly increased and didn't 3
really explain what the increase was due to, so I think that 4
the cost analysis is going to be a critical document to look 5
at, but it sounds like sometime early next year perhaps l
6 these documents could be made available to the panel and the 7
panel'could take a look at -- we would welcome any comment -
8
- any comments from the panel on any aspect of the system 9
but, in order to be_most efficient, you may want to -- you 10 may want to take a particular focus on this set of l
-11 documents, or'you may want to focus on one particular 12 document.
l
[
13 But it would be important for us to get some 14 feedback from tha panel because this is going to be uses, ao 15 the basis for the RFP that eventually goes out and-we want 1
1 16 to make sure that everything we want in the LSS or 17 everything we need is in that RFP.
18 And, of course, you know, the questions are:down 19 the road'about what's the best strategy for developing the 20 functional specifications in the RFP to -- to achieve that, 21 to get the type of innovation and cost effective proposals 22 ~
that'Boyd was talking about yesterday.
23 So we need to -- in terms of issues coming up in-24 the future,-you need to really make sure that.you don't lose
\\
25 that -- that design and development focus as well as this
/
L
343 1
compliance evaluation, access planning, et cetera, et 2
cetera.
3 CHAIRMAN HOYLE:
All right.
Well, Barbara said 4
that DOE would be accepting the SAIC documents in November 5
but the FEDSIM process is already beginning even this month, 6
and she's going to have an acquisition support contractor 7
beginning work in April.
So I'm wondering whether we need 8
to '.ook at the cost report that's going to come out earlier 9
than May or June time frame which I thought would be 10 appropriate for our next meeting.
-11 MR. CAMERON:
Well, the critical date, the date 12 that DOE was using for the development of the acquisition 13
' support documents, was August of
'91.
Now, that.was based i-14 on installation of the system in January of
'93.
Barbara --
'15 Barbara's schedule Nas using August and September,of '93.
16 So I don't know.
We haven't sat down and talked i
17.
with her since that schedule change'has been made,.but -- so a
18 I don't know whether.the August, '911date for these 19' acquisition support documents gets pushed out further into 20l the future or not at this point, but even if you kept it.at 21 August of '91, I thlak that there's -- there's time to start 22 gettinglthe panel organized towards having something done in 23 the-June time-frame and I think that the way this schedule 24.
is now,'that that should be -- should be timely.
' 25-And I think -- Boyd,,our comments on the schedule
/c
't
l 344 1
in terms of, you know, adding some more time in I take it 2
would probably go across the board in the schedule, not just 3
pushing --
l 4
MR. ALEX'4 CER'.
That's right.
5 MR. CAMERON:
-- the 'atimate installation date 6
off but pushing everything back.
7 MR. ALEXANDER:
It would seem that that would be 8
the case, Chip.
_I think that's the optimum schedule you have now which may be difficult to achieve.
So I think it 9
10 would be everything, my view.
11 MR. CAMERON:
Okay.
Oh, and that would mean that 1
12 there wc'uld be -- if the panel got something together in the 13' June time frame for discussion that that would be plenty of l
l 14 time to be able to' feed that into the FEDSIM contractor's 15~
work on the development of the.RFP.
l 16 CHAIRMAN HOYLE:
Okay, so let me ask Mal.
Mal, do-l-
17 you want some of this draft material or, Kirk, do you 18 alreadv have some of this?
19 MR. BALCOM:
I already have it.
l/
20-MR. MURPHY:
Well, the. design document stuff-21 sticuld be sent to Kirk, not to me.
22 MS. SCATTOLINI:
He has it.
23 MR. BALCOM:
Yeah, I have that.
24 MR. MURPHY:
Other -- other members of the panel 25 don't necessarily have -- Dennis' client needs to look at sr-
345 1
it.
2 CHAIRMAN HOYLE:
But are we saying we war
- see 3
some of the draft material that's being now changed and will 4
become final in November or do you want to see the final 5
material when the additional cost material is available?
6 MR. BECHTEL:
I think I would like to see the 7
draft and the redrafts that come out of it.
8 CHAIRMAN HOYLE:
Okay.
Okay.
And then we will 9
give everybody the finals when they're available, too.
By 10 seeing the drafts, I guess I must ask you to -- if you have 11 any comment on those, they should -- you should be 12 commenting as an individual and get your comments in to the 13 LSSA office I would think.
Is that --
t 14 MR. CAMERON:
Well, I don't -- I don't know what i
15
-- what our input is going to be to the SAIC design 16 documents past this' point other than what we gave when Betsy 17
'and Lynn participated in the design sessions.,
I think that 18 having a draft-document would be good. advance preparation to 15V get familiar with'what.the final documents were going to --
20 were going to look like.
I'm not.trying to say, " Don't 21 comment;on the draft," but I just am not -- you know, this 22 is a contract between DOE.and SAIC.
It is-coming to a close 23 and I don't-know whether comments from us or anybody else 24 are going to bo.able to be accounted for at this point.
25 But I would emphasize the fact that the SAIC j.
346 1
documents are a' foundation, but it doesn't mean that panel 2
comments can't be accommodated in the future before the 3
request for proposal goes out.
4 And, David, I don't know if you can say, you know, 5
-- if it's kosher for you to say anything about what the 6
DOE /SAIC process is in terms of where you -- you are in 7
terms of responding to comments or anything like that now.
8 MR. NIPPERT:
Well, we've maintain a list of 9
action on it and gone through each of the reviews and we'll 10 consider do we want to close all those out, I mean, that we 11 talked about basically and we want to resolve all those and 12 in terms of the final document, we will have a list of those 13 issues that have come up and their resolution in terms of
'~.
14 what we decided, but basically based on the four-day review i
15~
that we completed.last week, there's a_whole lot of other j
16 documents that were,also produced besides the design 17 documents.
I don't think we've got time toladd a whole 18 brand new list of items and put them in there.
There's just 19-
'no'way to do;that within the time tnat's allowed.
1 L
l 20 MR. CAMERON:
Yeah, I think -- I think that what 1
l-21:
it's coming down~to is to focus on the future use of those 22; documents in the preparation of the RFP.
L
}-
-23 CHAIRIIAN HOYLE:
All right.
Well, I agree with o
24 your comments then that we could use the draft documents as f-25 a preparation for the final.
We should have the final to us
/i P1'
347
~1 perhaps in the January time frame or so, maybe earlier, but i
2 then we would discuss that -- I would circulate that to 3
everyone and we would have that as a June meeting discussion 4
item.
5 Any comment that anyone would want to offer to us?
MR. CAMERON:
Betsy, did you --
7 MS. SHELBURNE:
Well, I just -- are we also 8
' calking about having a March meeting?
9 CHAIRMAN HOYLE:
That's what I'm trying to narrow 10 down here.
'll MS. SHELBURNE:
I was just thinking in-terms --
12 CHAIRMAN HOYLE:
Does it sound like that's what we t
13 need?
14 MS. SHELBURNE:
I don't know because I was 15
. thinking if there was a March'one then one agenda item will 16f be after people have received the material around Christmas-
-17 time to look'at it and then come to the-March meeting ~
18 prepared to figure out what they feel can be done at the
'19 June reeting, but1I -- maybe I'm confused.
Maybe we aren't i
-20
'having a March. meeting.
21 MR. CAMERON:
Yeah, it's -- you know, that could
'22' be one function of a March meeting.
I know that travel-23 funds are -- are getting more limited for evarybody and'even 24' though most of us or a lot of us are going to be in D.C.
for 25 the meetit.g, it would -- people would have to -- to travel.
sf 4
348 1
But I think, John, you're going to definitely have to come 2
up with -- if you sent the documents -- you're going to have 3
to come up with a suggestion Foout how the panel would work 4
to review those documents.
Essentially, the step that Betsy 5
is saying could be discussed at a March meeting.
You 6
wouldn't necessarily have to do that at a March meeting 7
then.
We could get the consensus of the panel now on how 8
that should work or you could come up with a suggestion.
1 o
9 CHAIRMAN HOYLE:
I'm flat out of suggestions at
-10 the moment.
Would any of the panel members care to offer 11' one that we can talk about now?
i<
L 12 (No response.)
I q
13 CHAIRMAN HOYLE:
The -- what is your view on a o
L 14 March meeting in the Washington area?
l L
15 MR. TREBY:
I guess I don't see a whole lot of 16
' items to be' brought up,-other than this one which would just
-17 be preparatory to a. June meeting, so --
i L
L 18-CHAIRMAN HOYLE:
We could probably have the access l
19 issues resolution.in --
20 MR. CAMERON:
Yeah, there could be a-couple h"
L 21 things.
It's just a question of -- it's-not like you could-
-22
'have:a March meeting -
you have to think abcut if you have p
a March meeting, does that mean that there will be some-kr 24
' critical' things coming up within the next couple months that 25 you would need to have another meeting to discuss, so you
/;
s
349 1
sort c h*Ve to weigh the costs and benefits of that.
2 Whereas if you had one meeting in June, and I'm 3
not -- you know, whatever you guys want to do, you know, is '
4 fine with our arrice-but there may be a whole list of 5
things to be accomplished at the June meeting so that -- at 6
a June meeting so that, you know, we could be assured that, 7
you know, there would be a full two days or a day and a half 8
of meeting time.
9 CHAIRMAN HOYLE:
Well, it sounds like there in certainly would be.
I think what I need to do is do a 11 little homework on the subject of the design documents and 12:
call each of you on the phone and talk about whether we need 13 an interim meeting, perhaps one in the March time frame, so y
14 let me defer on that.
But I think we found -- we do know we 15 have a number of items on our plate to take up by June, and 16 if we could_get a coupie of those out of the way in Mcrch, 17
_maybe~we could do that, but I will' contact each of you 18 separately on that.
19, MR. BALCOM:
John, minor point on the header
.20 guidance being finalized in March.
It seems to me-that the
'211 presentation about the technical documents has opened header
'22-design up.just.a little bit and it looks-like we need to add 23
- a. field -- another field or two, and normally the process 24 for approving those additional ones is by the ARP.
So it's i
25 possible we could do something like that by phone.
I'mean,
/
350 1
we certainly wouldn't need a meeting to do it, but it seems j
2~
to me that the working group ought to meet again sometime in 1
3 the next two months and meet with a representative of 4
Southwest Center and make a proposal for a new field or two 5
or why we don't need any more and be done with it.
It needs 6
to be done by-March, but I don't see that it needs a i
7 meeting.
8 CHAIRMAN HOYLE:
Okay.
Good.
9 MR. BALCOM:
I think it could be done by phone.
10 It seems to me it could be done by phone.
1 11 CHAIRMAN H0YLE:
It sounds like to me that it 12 could.
By the_ working group, you mean?
.{
13 MR. BALCOM:
By the working group.
14 CHAIRMAN HOYLE:
Okay.,And then --
15 MR. BALCOM:
And the working group will tell you 16 what our recommendations are and then -- but that has to be 17 in Chip's hands -- I mean, that has'to be approved by the 18
.ARP by March according to the. schedule.
19 CHAIRMAN HOYLE:
Okay..'So I will' take note of 20 that and do that by correspondence or included in a March 21_
- meeting.if we have'such.
22 Anything else about the future meeting?
okay.
23 Going back to then the draft letter to the -- to NRC, I'll 24 ask Marilee-to circulate the draft that I gave Elgie before U
25l he left and repeat what'I said then, and'that was that I S
j
351
-l would ask-you to comment to me within the next ten days, so 2
that would be by October the 22nd.
I will try to turn 3
around another draft to get it back to you'in the following 4
week which would be the week of October the 29th for 5
comment.
It could be that we will need an additional round 6
of drafts after that.
I'm not sure.
7 And finally I'll get it to the -- to the agency.
~
8 In commenting, I would like you to give me your thoughts on 9
whether -- at what level of the. agency the letter should be 10 sent and as much rationale for your own position as_you can 11 so that we can put some rationale in the letter.
I think
-12 whatever level we write to would appreciate the thinking of
. (j 13 the committee and'not just its vie. or its recommendation.
14 MR. MURPHY:
How soon do you want them?
15 CHAIRMAN H0YLE:
I'd l'ike that by the 22nd of'the 16' month so that I can crank that into'a real useful draft.
17 MR. BECHTEL:
This is just two pages; right?
18 CHAIRMAN HOYLE:
It's just two pages.
19 MR. BECHTEL:
Okay.
It says one and then three.
20 CHAIRMAN H0YLE:
I --
'21
- MS. ROOD
There was another page but it was
.22 dropped..
,23 CHAIRMAN;HOYLd:- Dennis points out we gave you a L24 sheetLF.iat has no number and then a page three. 'The second L25 sheet is page two.
If there's no further business of the --
/:
T
- jl
l 352 1
okay..
2 Marilee reminds me if we could set a date for the 3
June meeting, we could zero in on a conference room
-4 somewhere in the NRC area.
Our meetings have generally 5
started on Wednesday and are a Wednesday / Thursday meeting, j
6 Does anybody have a -- and it looks like we'll have enough 7
for tWo days, I do believe, so we're talking about June 5 8
and 6, 12 and 13, 19 and 20, or 26 and 27.
I would I.think 9
tend to go for the. middle of the month.
It reminds me that 10 maybe Lenara Smith has his commissioner meetings early in 11 the month, I believe, so I would go for 12 and 13 or 19 and 12 20.
Nobody has a preference?
(
13 MR. TREBY:
J'll suggest the 12th and 13th.
14-MR. BECHTEL:
That's good.
That's good-for us.
15 CHAIRMAN ~ HOYLE:
Okay.
Let's try for-that, 12th 11 6.
and lJth.
' ^ '
17 Before anybody else gets away, I do want to'thank 18 the LSSA office for its participation here with us, Chip and 19f Lynn and Betsy.
-I appreciate that very much.
Oh, and I
.2'O Marilee, an enormous help getting the maintenance man to 21 jump:through the window, and I thank you all for your 22 participation and your help in keeping this moving along and 23 I'll look forward to your comments on the draft letter in 24 the next ten days.
.i 25 Thank you very much.
J' l
353 1
(Whereupon, at 12:45 p.m.,
the hearing in the 2
above-entitled matter was adjourned.)
3 4
5 6
7 8
9 10 11 12 13 14 15 16 17
=18 19 20 21' I
22-23-
.24 25
/'
!l i
REPORTER'S CERTIFICATE 1
i This is to certify that the attached proceed-ings before the United States Nuclear Regulatory Commission i
in the matter oft l
l NAME OF PROCEEDINC:
Ap.%ig(,
, q 3, y v.'(
h.,
.y q
' ]
I ' W'
'*' L ^>
J'* w y
[
DOCKET NUMBER:
\\
PLACE OF PROCEEDING:
- .';\\;Ne., N, L-vere held as herein appears, and that this is l
the original transcript thereof for the file of l
the United States Nuclear Regulatorf Commission i
taken by me and thereaftar reduced tc typewriting j
by me or under the direction of the e.ourt report-i ing._ company, and that the transcrip'c is a true and accurate record of the foregoing proceedings.
'D, I
,c p
",a s,
i4 Officia'l Reporter Ann Riley
&' Associates, Ltd.
4..
l
,,l
.l 4
e
~
~
~
~
g.
~
3 e
s._
OUTLINE TOE ; PRESENTATION BACKGROUND ON CENTER FOR NUCLEAR WASTE REGULATORY.
ANALYSES..(CNWRA)
OVERVIEW OF CNWRA PROJECT FOR LSSA-STATUS REORT e Visits made to:
- Yucca Mountain Project Office.(DOE)
- State of' Nevada-
- NRC (Washington, D. C.).
-DOE (Washin'gton, D. C.)
e -Obseivations e LSS Header Fields for. Technical Data
- e. Issues
+
- N
2
==
~.
i
.u c
v CENTER FOR: NUCLEAR. WASTE REGULATORY-ANALYSES e CNWRA ESTABLISHED BY:THE USNRC
- Federally: Funded Research and Development Center
- Initial contract in (ktober,1987 with 'five year options e PART OF SOUTHWEST RESEARCH INSTITUTE (SwRI),
SAN ANTONIO, TEXAS-
- Not-for-profit Research and Developreent in Engineering and. Science
- Over.2400 staff /$190M gross annual income
- includes over 200 computer scientists and multiple computer labs w
I.
.N-.s__.-s-
- +.
i v-CENTER FOR NUCLEAR WASTE.
REGULATORY: ANALYSES (CONT'D) e-CNWRA PROVIDES TECHNICAL ASSISTANCE /RESEARCH FOR NRC'HLW-PROGRAM
- Systems Engineering-and Integration
- Geologic Setting, Engineered Barrier System, Repository Design for Construction and Operation, and Performance Assessment -
- Quality; Assurance Program
, Information Management and Technical Data Systems Development and Operations, including interface to NUDOCS and LGS
_... _ =.
. ~.,
- ~ ~ ~ ' " ^ ^ - ~
~
- 2.
- ,=
y
.N a
l-l; l
i l
-CNWRA. PROJECT FOR LSSA e Development of access protocols to LSS technical data e Began work in June 1990 l
[
e'- LSSA letter to LSSARP members e Fulfilling mandate of:LSS Rule 2.1011 (d)(10) and 2.1003 (c)(1-3) l 4
l i
i I
i
~.
e
~-
-**p A'%
\\
,y TASK 1 OBJECTIVES e Define technical data by category e identify organizations generating technical data-e Document existing / planned procedures for-providing access to technical data e Recommend a. plan to assure access -including submission requirements. and ~ recommended header content e Identify impacts ofL suggested plan - to encourage early problem resolution L
- m-
rp' 4
..m 4
9 4
I l
APPROACH-1 I
Examine background references ared talk with knowledgeable
[
LSS participants.
-i i
Status report on initial work-1 l
T i
5 l
i I
I
-,i s.
. ~...
s n.
T:
(
i l,
l TECHNICAL DATA e For purposes.'of this Task, we've:used the term " technical data" L
to mean documentary material which cannot-be entered into the LSS'in text-searchable form.
i i
[
- e. in other words,-.it's-the LSS material which can be found only
[
with the help of'a. bibliographic header.
e if we refer to it in terms of media (rather than subject):
l
!~
- Some are imageable: graphics, tables, handwritten notes 1
l
- Some are not: tapes, disks, cassettes, film, colored maps j
l
& photos, extra large maps.
i i:
r
.~
- ~,
~ ~..-
6 CATEGORIESiOF DOCUMENTARY MATERIAL DESCRIBED IN THE LSS RULE
.m
[l] DOCUMENTARY MATERIAL-g TEXT INDEXED IMAGED l-4 BIBLIOGRAPHIC HEADER
-2 c-.
LC n;gg u
.]
-~
((ll][! GRAPHIC ORIENTEDb$i~
gqDOCUMENTARK MATERIAL}^
DOCUMENTARY 2
-,s 2 ilMAGEDi %.3@E m - y TECHNICAL 3
MATERIAL o
ITEM o
J*i
~a ~1-E' DATA O
~~ iBIBLIOGRAPHIC HEADERL m.
_gw
,n. ~.
u.
,~
=
o
....?.
. 7Y.*
, W.
I ?US:' ~$ -
?.
W
[W];DOCUMENTARYLMATERIAir
. j~!NOTSUITABLE FOR ENTRYJ w
w ~ -
n pu ;=
W
- z ? W
- f';
r.
e.n:::BIBL.IOGRAPHIC HEADER. W ^
1-
-3 4
, _a. ? ";.s g o
..n :a....,.
- [lV) PACKAGE OF INI ORMATION
....w..-
BIBLIOGRAPHIC HEADERM: }.
==,
f DATA RECORD PACKAGE LSU Bbliographic Header Submitter Title Abstrad -
Descripters-IdentNiers Comments etc.
l-Table of Cornents Weather Data Dump Record for:WX STA 2
/
/
/
1
(' '
i i
o
. INDIVIDUAL ITEM OF TECHNICAL DATA 3
1s Weather Data Dump Record for:WX STA 2 m
- 3. -
- I t.E.. o.=, e e s i.
n...,. t,i
+
i v
, n e,,,, :....,
V
_:,.r
..w og
- u. w,. n wi.
~...
,, r,... - wg:
2,,
...s IIJ fG# '
- 4L 9 i a f.e 9iPO 0
t 4
i 3
l
- A/:
e,.
~...
u re s 1
'O b! l19.Wh 'Ws$
d**
- n
- In
' s'a
?
I * * *
- ?
<r:
- sc -
- r inn t
<s
w
< I[
d Ad / ef-.94ed '
k N" s' i d t Oi ' f l . d rgfl * $ u '
.u se s,ee-as13 i..
i s
J ' 't fed 4.m a l Es\\L me.'
- 4 I'*w Y
a
{
IJ MM u r i e rre u ca
- cd b,
i a v-a
!.. /.. t
. t
,Jr.
2 e '.. 1 e.
71 g s I
.7
^
s l,W r fL l tit
- N *MW
\\
e n ef t igg.d e t s
<and I
,*ess '
.'e!.se
- ms
'a b#M in c h s e e m mf sta 4
2e 6
l l
t I
I l
,1
~
~
n.
I?
i -
,u
~' -
~
~ ~~
SYSTEM OVERVIEW
~
(OATA RECORO DATA ACQUISITION
,,,any, pACKAos sEoa8EMYse ACTfvfTIES (OA CHECKED) ~
PARTWANT APPROVED DATA I
e C
J ACORARED DATA LOCAL RECORDS DATA REDUCTIONr CONTROLLED DATA FOR m
~
CENTER INTERPRETATION DATA BASES Aco m oseAL ACTIVITIES '
(eg SEPDB) l (PARTICIPA007)
A88AI N C
V (OATA N.OPACKAGE DATA ftECORD SFGaIENTS)
PACKAGES SEDER TO (OUARTEsti.Y)
DATA m.
APPROVED -
RECOftOS DATA suppostyggs CoseTRonEO DATABASE f
I Asso suo suomasseoses 1 f
/ CENTRAL 1
AUToseATED
. RtB -
(CONTftOMED)
RECORDS
/
OATA m
FACittTV TRACamma r--i l tss Q __. _._ _ _
/
sysTEas
- t. ___ _ ;
(PROJECT)
/ N DATA CATALOG 4
esoEx TO oATA r \\
-f \\
TDMGMT5P A06416 90
. -. -.. _..:2..._ : _ :.. -.. i -. - _.. -
l l
1 l
YUCCA MOUNTAIN PROJECT OFFICE (DOE) l l
TECHNICAL DATA = ALL DATA RELATING TO ITS TECHNICAL ACTIVITIES l
i l
e YMPO project organizations submit their data within 45 days to nine local records center's (prime contractors).
e They forward it quarterly to-YMPO Central Records Facility l
(CRF) in Data-Record Packages.
i l
e Packages may include numeric data (e.g., SEPDB) and results l
from computer modeling and graphic-display systems.
j e CRF uses automated Records Information System (RIS) and Automated Data Tracking System (ADTS) to monitor holdings.
i j
e A comprehensive technical Data Catalog, a SEPDB report, and l
a summary description of analyzed technical data (RIB) are all l
produced quarterly.
i l
i l
=
a
~ _
,s YUCCA MOUNTAIN l 1
3 PROJECTEOFFICEqDOE) (CONT'D) t e The. backlog of technical. data at local centers is now being i
incorporated--into. CRF holdings.
)
l
'e YMPO technical data is currently available via written request to the YMPO,~which~ promises timely response.
i l
e When LSS loading begins,.YMPO will:
- -Scan its Data-Record--Packages to submit LSS images.
l l
- Create LSS headers for packages from existing RIS/ADTS headers.
i
~
1 4
l
- ~ -
-~
r-1
- STATE OF NEVADAL PRODUCTION:
e The Nuclear Waste-Project Office (NWPO) and its contractors
^(University of Nevada and Nevada Bureau of Mines & Geology) have:not produced any raw data, but may do so in the future.
e NWPO publishes formal technical reports containing graphic material.
e it safeguards, within associated packages, the handwritten / numeric / graphic " backup" data on which those l
reports were based.
e-When'LSS loading begins, NWPO will:
- Scan its reports anE= associated backup-data packages to-submit LSS images.
i Create LSS headers for them.
I 1
l
y
-=
__ ;- ~
1 i
STATE LOF NEVADA. (CONT'D)-
]
ti i
NEEDS:
e Individual items of-technical' data which are not text searchable L
but are contained within a-Data-Record Package do not require L
their own headers. A header for the package as a whole is sufficient.
e Technical data.which~cannot be scanned for entry into LSS (but
.will~ be identified in the.LSS through a header) must be j
~
transferred from current storage locations to an LSSA-i
~ controlled records-center - several months before HLW-repository licensing ~ proceeding begins, to assure timely access.
l i
t
l
+
~
..L.
~
l 1
NRC 4 WASHINGTON, D.C.)
L o
l PRODUCTION:
1 l
e The?NRC.and its contractors have-produced non-text-searchable material and'will do so in the future..
I i
o Currently, the NRCLemploys an-automated-records index l
(NUDOCS) to. provide reference to its central record holdings i
j o-When the LSS becomes' operational, the NRC will submit its i
documents to the LSS through its Document Control Center.
I i
l e When LSS-loading begins, the NRC will-centrally:
- Submit.its non-text-searchable material to the LSSA for l
entry into the LSS, in accordance with procedures to be l
l established.by; the LSSA.
l
- Create LSS headers for its non-text-searchable material, in j
accordance'with LSSA guidance.
l l
e The location of non-text-searchable material which is also 1
l non-imageable will be identified in the header.
A central NRC contact will be provided.
l i
t 3
i
-.._._,._-,.....J m._
_. -..._ -. _w~.L...,,
-~
m, -
= _,
~
e i.
.i
.l t.
I NRC4(WASHINGTON, D.C.)XCONT'Dy j
NEEDS:-
i i
6 e The NRC is unable to comment on the adequacy of using a single: header for each Data-Record Package as a whole without j
l clarification of DOE procedures for creating a package,
]
i including composition and timing for submittal to the LSS.
i i
(
.i i
y -
3; :. :e
=
~
s DOE l(WASHINGTON, D.C.)
PRODUCTION:
e The DOE does not. expect to ' produce any significant amount of
~
non-text-searchable material from its Washington, D.C.
headquarters.-
e Any.such: material that may be produced or that has been produced in'the past will be entered-into the LSS using a DOE capture station.
NEEDS:
e The DOE anticipates no extraordinary LSS access needs.
t
=
~
^ reg ;
~
=1 j
i OBSERVATIONS q
.:j DEFINITION:OF TECHNICALL DATA:
1 e No ambiguity perceived in the LSS Rule, but:
- Certain issues must be resolved..
1 r
- Clear implementing procedures are needed.
e Categorization is. required:
- To be sure none of it escapes proper LSS entry
- To enable LSS users to find needed items through
. consistent entry / search terminology t
i
?
I
+
i 1
ll t
L i'
2,
_ _ - ~
g t'
'~'
1 1
OBSERVATIONS (CONT'D) j y
I 1
4 l
PRODUCTION OF TECHNICAL DATA:
l
\\
\\
L
.e Primary producers:-
)
- DOE-& contractors I
- NRC;& contractors 1
i l
j.
e Some1 changes.in existing plans / procedures / systems of the DOE j
and:NRC may be necessary to accommodate LSS requirements l
with. respect to technical data.
l 1
4 i
l c
1
..u..
l.
- jl
- l!
l
!!!I' t
)B c
)
a s
D r
c P
t n
s h
E 4
i
)
e b
p S
D e
r a
a
(
'T c
r s
h g
e t
N p
i t
e w
u a
O h
m t
t o
C r,
h n
n e
t o
d si
(
o w
a e
a e
v e
)
t S
y d t
e ah s
g ~e B
a l
t a
s
~
N c
t h
n mo R
d I
e s
c eih O
(
l l
i r
e ih a
l t
r.
i n
s (t
e y
t d
u wp I
i T
e q
a o
a l
r d
d e
e r C s
b a
g g A
oi g g e
e a
ml d
a e
r V
A e
g h mt e
a l
we a ofob a
c T
h R
A e
b k
a p
r u
al i
l b
cb e
w a sC i
E D
c v
n ail a
i e
a P b b e
e e
+
h l
s a
ai S
L p
l aivb t
t a
b c
e B
A a
d vT a
t a
a v
C r
a r
sh xd D a
l i
l t
ot e
e c
e O
g I
i a
l N
o wd c
p b.
g r t
l l
f a
a o
i l
e r
a a
a d
c c
i HC d
R c wp e b
a nii g
i t
s
- s n
n n
E b.
a e
a eiet a
st h h i
d g
t T
h a
ad v
o x
e c
ct s e
e ei l
O g d mD AAMT b TTL u
e a
T o
g h
i r
a n
c w-S e
h o
a S
T mN E
V l.
i i
EC e
e e
e e
C
~
A
\\
l1Ll<<ll[
l f
1
=
' ^
~
' **^^w y,
s i:
4 4
BIBLIOGRAPHICLHEADER FIELDS REQUIRED FOR; DESCRIPTION *AND RETRIEVAL OF TECHNICAL DATA /INFORMATION i-
-
- TITLE / DESCRIPTION '-
j-
- ABSTRACT i
i.
- o AUTHOR-
- AUTHOR ORGANIZATION i
1 i
- EVENT DATE/ CODE l
DESCRIPTORS (SUBJECT)
- SEE LSS THESAURUS '
j i-
- ' SPONSORING ORGANIZATION
- IDENTIFIERS j.
i
- COMMENTS i
l FeTRE8ULIATORY CATE50RY l f
~
- DOCUMENTTYPE I
- SEE DOCUMENTTYPE CODE LIST (DOE) -
- - ADDITIONSTO DOCUMENTTYPE LIST DATA BASE CATALOG DAT5 CATALOG; l
DATA LISTING / TABLE i L*?j MEDIA TYPE?
[
i
- . SEE MEDIA TYPE LIST (DOE) -
1
- POINTERS -
j
- SUBMITTER CENTER j-4 l
+-
-.v.-
=
-s F
^
f g
BIBLIOGRAPHIC HEADER FIELDS REQUIRED
- FOR DESCRIPTION AND-RETRIEVAL OF TECHNICAL: DATA /INFORMATION TITLE / DESCRIPTION SANDIA.
l ABSTRACT DOE-SAIC
-AUTHOR NRC USGS I
- AUTHOR ORGANIZATION NEV LANL i
SUBMITTER LLNL f
SUBJECT DESCRIPTORS i
SPONSORING ORGANIZATION ATMOSPHERIC SCIENCE 1
INDENTIFIERS BIOLOGY / ECOLOGY l
- COMMENTS ENGINEERING / CONSTRUCTION
{
l ~ REGULATORY CATEGORY !
PREAPPLICATION REVIEWJ,
EQUIPMENT I
- POINTERS SITE CHARACTERIZATION i
. GEOSCIENCE l MEDIA' TYPE l l APERTURE 1 CARD 935MM LICENSE' APPLICATIONS 3d?
GOVERNMENTILEGAL AFFAIRS
- MAG DISK FLOPPYI" ~
CONSTlAUTHORIZATION)
HYDROLOGY t
' MAG DISKl HARD (SYS ID)
LICENSE ISSUANCE AND[ _
MANAGEMENT i
- MAG TAPE 2 REElf' M
^ AMMENDMENTy
- MATERIALS
. MAG TAPEICASSETTEt PERMANENTCLOSURE2h,
PLACES.
'.. MICROFORM [ l 9318 PERFORMANCE OBJECTIVES $
PROCESSES / PROPERTIES MYLARt n PERFORMANCE ASSESSMENT SAFETY, SECURITY, ACCIDENTS
. OPTICAL 2 DISK}SYS ID)f PAPERP dss " 17 ~
E.AND OWNERSHIP ANif n SOCIAL / ECONOMIC FACTORS
~
i"E CONTROL c TRANSPORTATION i
i PHOTOIealN' % 'N SITING' CRITERIA [W $$
WASTES / POLLUTION SLIDES $3sMMMk 1 DESIGN' CRITERIA FOR'GROA 2 STRIP CHART VELLUM %)@p % gf[.
DESIGN CRITERIA-WASTE PACKAGEJ.
DATA SHEET i
-VIDEO CASSETTEi F ~ ' "
_, PERF. CONFIRMATION REQUIREMENTS MAP IIS ABE TECHNICAL
- DOCUMENT TYPE CORHESPONDENCE DRAWINGS STATUS REPORTS HOTOS PRESENTATIONS.
PROCUREMENT LOGS COMPUTER PRINTOUTS GOVERNING DOCS NOTEBOOKS DATA SOURCE CODE DATA CATALOG SEISMIC DATA LEGAL DOCS SITE SURVEYS RECORDS PACKAGE VIDEO
~ COMPUTER SOFTWARE CAllBRATIONS NOTES INSTRUMENT RECORDINGS i
O
9; x
.~ -
- RELATIONSHIP BETWEEN TECHNICAL DATA l
CATEGORIZATION AND LSS HEADER FIELDSL CATEGORIZATION OF.
RELATED LSS TEXT i
TECHNICAL DATA HEADER FIELDS '
INDEXED i
1-
- 1. TECHNICAL SUBKCT
.1. TITLE / DESCRIPTION Y
l l
- 2. DESCRIPTORS (TECH. SUBJECT)
Y i
- 3. IDENTIFIERS Y
t i
- 4. COMMENTS Y
l
- 5. ABSTRACT Y
- 6. DOCUMENT TYPE Y
-i 22REGULITORY CATEGORY:-
12 REGULATORY.CATE, GORY.
Y
{
2
- u.
s. :
. :.... u.
l AREA OF REGULATORY CONCERN
!(10 CFR 60) F.
m JN j
I PER 10 CFR 60
- 2. TITLE / DESCRIPTION Y
- 3. DESCRIPTORS Y
i
- 4. IDENTIFIERS Y
-- 5. COMMENTS Y
- 6. ABSTRACT Y
f 4
i i
- 3. DOCUMENTTYPE
- 1. DOCUMENT TYPE Y
}
- 2. MEDIA TYPE:
Y..
l 41MEDIATYPE lilMEDIA'TYPEl Y
j
- e DOCUMENTTYPE Y
i
'i t
.....,c
.a..
p.
~
~
=
i j
i L
ISSUES HEADER CONTENT:
j i
l
- o -How:will currently approved. fields be used?.(title, abstract,
[
descriptors,; identifiers, comments, etc.)
o What fields must.be added, if..any?
E i
- Are submitter / sponsor fields sufficient for storage location of non-imaged material?
3 l
~
j
- Can document-type: field incorporate media?
l l
-- Is a " regulatory category" field feasible?
- Is a " qualified data" indication needed?
l i
i a
._,. = -. -
,3 7__
_ l}.
^
.y.,x
.lSSUES fCONT'D)
DATA-RECORD. PACKAGES:
e Timing of submission must be defined.
e How will non-imageable portions be individually stored?
e Table of Contents must.be sufficiently descriptive.
- Should-it be text-searchable?
e Header must be thorough (abstract, descriptors, etc.)
e Should packages be made text-searchable insofar as possible?
3, - ~ ~ -
.-i-----
l.
a E'
\\,
%. 2.
m ISSUES (CONT'D)
.NON-lMAGEABLE CLASSES OF TECHNICAL DATA:
e Classes suggested:
- Magnetic media
- : Film.
- Colored graphs, photos
- Extra large maps e Criteria: practicality, cost-effectiveness,
e -Access protocols needed o,
>.._.,,,.,,.,...,7.i,,._
__m.1
.._.______._.,..__.,..,.L___...____.._.._
ee
.p -
\\,
a
.g
+
3
- c. s~
t_
1990 (1991.
ACTIVITY
-Jan Mar
'Aug Sep Nov.
2 Acquisition Request LSS Design SA'IC Support' For 6'. Development Design Documents Comment Documents Guidance Header Technical
& Standards Guidance Data Recommendations t
Facility LSSA SAIC l
Planning &
Facility-Generic Development Planning Facility j Issues; Design l
Paper l
Access Planning LSSA' Access
.t Issues ResolutionI' Plan j
i Production Draft Commission l
Recommendation Approval:
Schedule on Priority Document Loading Loading-~ Categories Schedule-t ISSA Position Commission Approval Paper: -Compli-f Compliance Evalu-E hnce Evaluatlon pr gram ation Strategy Strategy.
O&M Planning i-I l
l l
=
7 5.:
- f. -
s.
-+
l : '~'_ ~
m d-
-'f
~
1993 1992
, ACTIVITY:
. March April-
-November.
~ August'
' September
~
- I LSS Design Request' Contract-Install DOE Test 6 De 7elopment-For.
. Award' Equipment 1& Acceptance.
Proposals'.
First ! Node--
l h
Guidaume Technical-
& Standards Data Guidance j
Facility
]
+
Planning &
t I
Development Access Planning r,
Production j
Schedule I
l
~ i t
UNLV i
Compliance
. Facility Ready i,
Evaluation Program i
1 t-O&M Planning 1
k
p2 4
g, *Y ' e se s,%
h t
1.
M embers of the Panel express t,ncern that revision of the interim topical guidelines to exclude from the,LSS environmental (including socio-economic) and transportation related documents was improper because it would thereby exclude information that might be relevant to issues in the NRC licensing proceeding.
They are par-ticularly concerned that, without the availability of environmental information in the LSS, they will be unable to provide independent comment.to the NRC in the future on whether the NRC should adopt DOE's EIS.
To exclude such information at this point in the process would be based on the presumption that it would not later be relevant NRC's adoption of the EIS.
- e 2.
All members of the Panel (except NRC) strongly urged that if LFS,(he the NRC proposed to exclude su'ch documents from the
" C,
1-pland" min
-ets!!
y.. yare.
nunoortino n et e n c l e c i e --- e
- A * ']
e the decision to exclude such documents 4 e made in a formal rulemaking b
proceeding rather than in a Reg Guide change so that a judicial determination can be obtained on the legality of such exclusion if appropriate, j
3.
Although the Panel did not attempt to reach a consensus on whether such exclusions were appropriate or legally supportable (and indeed some categories might be justifiable while others might not), all members of the Panel (except perhaps NRC)believe that a final agency 4
decision, challengeable in court, should be made now so that the i
validity of the exclusions will not become an issue at a time when t,
.I
_;#...n, ' % ; a <.
- W '.t.
--@ 7r.
. g i
m
-. c
[
w,,. j.
y f-w,s'.t
- . e u_4 [. '
E
, a 1
p m.,-
,ef,'
g g '
-O.-
a, -j 8
s g
',6.m -6 i.
.x {' 3 4
1 'i
[
s V
'f 4, t1
-a y
r 4V
)
~n,
,q'.
i g.
n3
-(.
+
- 5j i4 a'
x m c, W
1 e
e a
n n.
f i
m7x a.
o 4
-s v:,,,ts I
, j,
- j eu 1.a..
If [ D[id theilicensing _ process: could be adversely. aHected by. 'a determination-i n,n s
.k
- _'W'1 4 g
y r.',ithattthe:enclusions_were. erroneous.
-g.
.t*
j) y--
, a 5
- +
f_ r p>, g -c
.3-
?sI
,.i P "'
.T',
,_E e
i 4
(*
b
- 15 ph.l/ t
,.3'k
,;3
( f f # ?l1, ;
r-.
.j-.
e m.
,sil piy f.,
4
-g cf y, N
F q
1
- \\
--jp
'r
. q1 1
1 t-i
.r p
4 E
,.I.
L y',/
1 v_.-en y
'z g,U,
'4
?,,.1..
t
--).
(
,..).l
'b-
/1' I--
..,-(-.-j, 1
( }'
-l ['
6
....,,y-.
4
}-.
3 n
- y....
- 's '^3 7*
L 4
i_
+ r,
I j(,,
..n r
r Y-t t
l > _-:'
G,5,,
r4 i
g-4,'
j
.#,} a..' 'd : - ( d-4, e..
- a,q 7-3
,.[--
e L
- f - f v
i t.
,fl 1
.(
l l
r,
t y,
i*- < g h ' '
.{
,+
%-f-3 e
~-.5
' -n ?
.l? j '
g. b-/ T
_ j
' T g -(.-) k.'h..
y i
4-s.
i""' i r
3 ai ; h- -- i.4
' ' },l -
n a
1
- + s
-y{
.,(.<'
- ,j j
3
- 1,
' ?
!T J
s, 9f
- 6
)
d
. fJ,
jgy ' 41.
9.,_
v
> \\
a <-
nA,,,
8 U
f'v. y, ( 'n.-'h'[ _i.)
\\j
[
't bn 4
(._
=g.
o p,.
g 3.j 3 } 1
?!,
fi i
I i
t 1,
.t.h'.-' n.
is. m, i. J) t
/
"4.
l zt <
2 i 1
(,a i I
'k'.
i.
.I-~
n n N'ti>->a e
r
+
)
, (k)
- u r.: ca.. -
,f r..
- p s
ps n.
2 t
s
- e
?
i
)\\.
k N '
- h j'
l] ',,, C t sf
.t y
> ;; L i
6
- ) ?
~
i 3
Mj ',.,
- 9..
.,1
' f ;' q'
- 3 3
3
,lu t-j<-<g-e c.
1
+
', m,, w ll \\ % 4. ' y y'*
\\
t 6
f
.'.'},
b l
- }l y %
,y t1
=
c
,n I
y y1 1 -
i i
4-y' e.
'g y o.
M '-
-m.~
7g<
4 -
s c
3 i
.Ay
,i.;.
A
'.\\ E '
i
4 n
9, l i ty
-.J
(
4 i qw",.a:
.46 - <
m 4
<t
."^
> ~
~
ux-m, -
r
,'.c.
4
'"I e{
gr ;
{;$,3
,l W I' ?
s F
i v *._ i h:
,}
i, m.
1
))
n.
k,; -
f
)
I;[p f.',, )
3 r'
f t
.b'.
i :
..m..l.,..o k
[
g b
-[p i
'il',
a,C v s,'g,,.
m
- 9..
- Wi 1
- b. - ).1 ~ 1 E-J.J.
M"A L-Fi.
m g
it..
y 1
l'
., as :-
1
'i.
}
b l; '-
-4 j
=*
g, y
J I
5E y lf4 i.
t-
,+. g
'i.
"')!
.w
% v Olb.Y L _
t s
,+.
% y(3 e
- z. n.
7 3
~
~
+g m i,#
- 8. ( y '.-
- ._k -; <
i t
,>s
.re 9
1 4 s..,
,{
, -- } V t
- .f n
- 7 k
ii e
g
',., p; i
t
.?.'f.
4 i
- a. 3,
.-,4',.
( N:.,; j y.p i - "
< 4 g3-1.73 a,
.$ g l7 $$
h,'
6-e
=5-'-'_.
, g t.4,],-
s p
p
-_p=0_.
-,..;-4 R
i.
.g p-u g. )
4
_t,g.
Y
.' f.3.' i
^
1 e.'
,'hm y
"h
)f p
L
-e s c
't,_
h 4
x p1 k
)
. " t_.\\
jI
%\\
(,,.
'l'
\\
..> c J
n n.
f
'F s
}
r M.i ~ i
,e
- p-p-
4s x,
%ts.
F %4:
.g
,, 'l
+ - ly % >
..-j[.
f k
_zg g
v-t
+
a
,-a
o***"*%
UMTED STATES r,/\\
NUCLEAR REGULATORY COMMISSION j
1 wAsHINQTON, D.C. 20666 l
s,.... ',
INFORMATION PAPER ON ABSTRACTING IN THE UCENSING SUPPORT SYSTE.M Office of the Licensing Support System Administrator
. September 12,1990 l
1
'k s
4 Septerber 12, 1990 LSSA INFORMATION PAPER ON ABSTRACTING IN THE UCENSING SUPPORT SYSTEM
!. PURPOSE OF THIS PAPER:
At the upcoming October, 1990 meeting of the }tRC Licensing Support System Advisory Review Panel (LSSARP), the members are scheduled to continue the discubsion on their recommendation to the LSS Adainistrator-(LSSA) on the content of the LSS }!eader.
One open item was the extent to which documents in the LSS should be abstracted.
The purpose of this paper is to lay out information about abstracting which the LSSA believes should be taken into consideration by the LSSARP members as they examine this issue.
L II. BACKGROUND:
During the March 1990 meeting of the LSSARP, a Technical Working i
Group was formed to prepare a draft recommendation for the fields
- f or ~ the LSS Bibliographic Header and Full Header.
The Working Group met several times and prepared a report to the full LSSARP.
The report recommended that abstracts be recuired only for documents. and' non-documents that will.not be available in searchable full-text (i.e.,
those with either header. only or header.and image only).
The report further recommended that the abstract field be cetional for documents that will be available in searchable full-text.
The Technical Working Group determined that the LSSARP should discuss the issue as to which LSS document types or groupings should be abstracted.
During the June 7, 1990 meeting, the LSSARP members agreed that abstracts were required for materials that will not be available in searchable full-text.
They then discussed at length the need for an abstract for LSS documents that.will be stored in searchable full-text.
These discussions centered around cost versus benefit considerations.
Differing points were made about the need for any-abstract in the
- header, given availability of full text, the sizable cost of abstracting, and whether only selected sets of documents might need to be abstracted and, if so, which sets.
1
P No firm recommendation evolved.
To focus the issue and to provide more definitive information about the cost implications of alternative abstracting scenarios, the LSSA offered to prepare an i
issue paper for the members to consider prior to the next LSSARP meeting in October.
Since the June LSSARP meeting, the LSSA staf f has reviewed existing information science studies related to this issue and gatheted industry data on the costs of abstracting.
The following is the result of that investigation, including a discussion of abstracting options and some alternatives to abstracting.
III. ABSTRACNNG - WilAT IS IT?
A.
TYPES OF ABS'ntACnNT In the Library /Information Science discipline, three types of abstracts have evolved.
All are based on the human review and summarization of the content of a document.
In order of increasing depth and coverage, they are:
ANNOTATIVE --
A short description of ' the document which briefly describes the subject, usually limited to a few lines in length.
This type of abstracting can be done by the same staf f doing the bibliographic or descriptive cataloging.
INDICATIVE --
A longer description than the annotative abstract, giving a more detailed summary of the document scope and content.
These abstracts are traditionally about 200 words in length.
This type of abstracting is usually done' by professional indexers / abstracters having subject matter background and/or experience. The documents are usually reviewed once both for the assignment of subject terms and for the development of the abstract.
INFORMATIVE -- The most substantive type of abstracting which I
includes not only indicative information but also summarizes the findings, answers, or data in the document.
Such abstracts often eliminate the need to obtain or read the entire document.
The length varies based on depth of l
document content.
As with the indicative abstract, this type of abstracting is also done by professional indexers / abstracters having subject matter background and/or experience.
2
However, unlike the Indicative Abstracts, this type of abstracting may or may not be done by the same staff that are subject indexing the documents. If not, then another staf f resource is required.
K-is obviously more expensive as one moves from annotative to It informative abstracting because of the additional time and higher level of expertise involved in reviewing the document and composing the abstract.
Section IV and Appendix A. contain more information on the cost of abstracting.
B. ABSTRACTING IN THE LSS ENVIRONMPRT Given that the LSS Title / Description field is intended to contain (a) the titles of formal publications or (b) a brief description of less formal or untitled documents, all LSS documents will have annotative-type abstracts.
This makes the assumption that titles L_
of publications are somewhat descriptive of content.
Therefore, annotative abstracting is not considered from a benefit-costs perspective in this issue paper.
Also, in the opinion of the LSSA, the LSS should not attempt under any scenario to provide informative abstracts because (1) the costs are excessively high and (2) such treatment of LSS documents is
' unwarranted given the availability of the document text on-line.
The LSS abstract would only be intended as a search aid, not as a surrogate for the document itself, which is often the case with systems providing informative abstracts.
Therefore, in discussing the pros and cons of abstracts in the LSS environment,.this paper assumes that any abstracts would be of the indicative type.
C.
BENEFITS OF INDICATIVE ABSTRACTS The-following is a list of the potential or reputed benefits of having an abstract field in a
full-text database.
Where applicable, we have included a summary of the information gained from relevant research ' studies.
It should be noted that no specifically applicable research has been found that directly speaks-to the benefits / costs of abstracts in a full-text database having keyterms and header data, such as will be the case with the LSS.
1.
IMPROVED PRECISION --
The presence and use of abstracts may improve the precision of subject / content searches because it is assumed that if a word or phrase is in the abstract, then it is probably a primary topic of the document.
This 3
a
-m mm m mmmes ie
=m='--=='====='*1=='M8='h=m'
F
+
precision is gained by limiting word / phrase searches to the abstract
- field, either initially or after retrieving a
document set via search of full-text or other parameters.
There is a current on-going debate in the information science literature about the benefits and power of full-text database sof tware as compared to traditional systems that have only bibliographic (fielded)
- data, subject
- indexing, and abstracting.
Most of this debate centers around the balance of " recall" versus " precision" capabilities.
The attached articles are representative of the discussions and data surrounding this debate (see Attachments #1 through $5).
It is known that in striving to achieve the greatest recall (retrieval of all relevant documents),
the Drecision (retrieval of only relevant documents) of search results suffers. This axion is applicable to all types of information systems, ranging from bibliographic only to full-text systems.
- However, the degradation of precision to assure greatest collections on e narrow and/or homogeneous topic,pecially for recall is magnified in larce full-text systems, es such as the HLW LSS.
This problem will be further exacerbated in the LSS environment of decision support and litigation support where knowledge of all relevant materials appears more to be essential.
l In a 1986 article (Attachment #1), Gerald Salton summarizes the results of several related studies.
Simplistically l
I presented, the precision / recall. performance of different J
access methods can be drawn from two of the studies.
These L
data support the belief that se'rching the abstracts can significantly improve recall (as compured to. searching the l
full-text alone without) a significant loss in precision.
L Recall Ratios
- Precision Ratios
- l Searching the:
- a. Text of Abstract 0.78 0.63
- b. Controlled Descriptors Subject Indexing 0.56 0.74
- c. Full Document Text 0.20 0.75
- Etq_all Ratio is number of retrieved relevant documents as l
percentage of all of the relevant documents in the database.
Precision Ratio is the number of retrieved relevant documents as percentage of all retrieved documents 4
i m.
As indicated in line b. above, the recall ratior-are better if one has controlled subject terms to search as well as the full-text, without any significant loss of precision.
Subject indexing will be done in the LSS.
2.
RELEVANCY REVIEW -- Abstracts provide a summary of the entire document.
Therefore, browsing the abstracts of a retrieved set of documents can aid in determining the usefulness of the document and the context in which the subject is treated without having to roam around in the text.
Also, abstracts can be very helpful when reviewing document listings or bibliographies in hardcopy away from the LSS workstation.
This would be the case when LSS search spe :ialists or intermediaries, e.g.
librarians, research as'41stants, and paralegals, are performing searches in re sponse to " client" requests.
In one study, the presence of an abstract reduced the number of " hissed documents" dscuments judged as not relevant by a review of the titles but which were subsequently determined as relevant af ter e,nly,iew of the abstracts (Attachment #6).
a rev 3.
COST SAVINGS -- Abstracts can potentially reduce the need for printing hardcopy of documents if a review of the abstract is sufficient for the searcher to determine the relevancy of the document for his/her needs.
4 '.
TIME SAVINGS Abstracts can reduce on-line time if, as above, review of the abstracts negates the need to browse / read the full-text.
D.
UMITATIONS:
1.
Abstracts are only as good as the abstracter.
They are subjective, whether it be the author's characterization of his/her work or tne abstracter's interpretation o f.
the author's work.
.2.
Abstracts do not !mprove recall of, subject / content searches in a full-text database if the abstract does not contTin different terminology from the text.
Different terminology that could improve recall might be more generic, more specific, synonyms, or the translation of jargon.
3.
Abstracting only certain document types / categories places a burden on the user to know when abstracting was done and when it was not.
Otherwise, users could unknowingly formulate search strategies that would provide false results.
For 5
i v
example, if all documents in a collection are not abstracted, then searches limited to the abstract field will automatically exclude non-abstracted documents and thereby possibly exclude relevant materials from the resulting hitlist.
t i
IV. COSTS OF ABSTRACTTNG i
A.
AVERAGE COST PER ABSTRACT The LSSA collected abstracting cost and productivity information from six companies that perform abstracting servicos.
The information provided by respondents varied in terms of assumptions, such as variations in the size of documents, the QC reviewers / supervision ratios, and scope of abstracting.
It was therefore difficult to normalize the data.
However, there was not 1
such a disparity in the data that some useful figures could not be compiled.
The assumptions used for this paper are listed in the.
Table below and Appendix A.
Dets was also provided by SAIC, based on their experience in the LSS prototype cataloging ef forts.
Their data show abstracting times of about seven (7) minuteA per document based on a sample of 47 documents, each averaging 48 pages.
Unfortunatsly, - the SAIC timing estimates did not include a quality control review.
- Also, it was uncertain whether these times consistently included the actual review and analysis of the document scope and content before the composition and keying of the abstract.
B.
ESTIMATED COSTS IN THE LSS The following table presents the estimated costs of abstracting LSS documents by document type.
The figures on the number of documents are extrapolations from recent SAIC re-evaluations of the size of the LSS database (see Attachment #7). The estimated number of pages in this SAIC report was divided by nine (9) to develop an estimated number of documents.
The figure of nine (9) pages per document was selected because-this was the size of the average document in the DOE Nevada RIS collection, which will contribute the vast majority of documents to the LSS.
The distribution of the estimated number of documents by major document types is based on recent figures from the three major ALW document collection; DOE's RIS systems in Las Vegas and at DOE Headquarters and the NRC's NUDOCS system.
6
~
Even though the figures in the table below are just gross estimates and may dif fer from the actual volume / costs experienced in the futures these figures are based on the best available data.
For the purposes of this paper, they do provide the LSSARP memLers with a significantly irproved basis for decision making.
Table 1. ESTIMATED COSTS OF ABSTRACIING IN THE LSS (Numbers of Documents & Dollars in thousands)
Cumulative Document Counts and Costs by Specified Year LSS DOCUMENT BY 1995 BY 2000 BY 2005 COLLECTION BY DOCUMENT TYPE NO. OF EST.
NO. OF EST.
NO. OF EST.
DOCMNTS COSTS DOCMNTS COSTS DOCMNTS COSTS TOTAL 1,278
$33,179 2,296
$59,595 3,759
$97,581 CORRESPONDENCE (64%)
3 doc / hour 818
$17,996 1,469
$32,318 2,406
$52,932 PUBLICATIONS /
REPORTS (23%)
2 doc / hour 294
$9,700 528
$17,427 864
$28,512 LEGAL. fr OTHER' DOCUMENTS (13%)
2 doc / hour 166
$5,483 299
$9,850 489
$16,137 Assumptions
.1. A fully loaded rate of $66.00 per hour. His includes the cosa of labor (alstracters, quahry control redewen, and supenisors),
GM overhead, and fee. Atstracting work actidtics include reading documents, composing atstracts, keying in the abstracu, and performing quality control and supervision.
- 2. A production rate of two atstracts developed and reviewed per hour ($66.00 divided by 2 = $33/atstract) was used for the Publications / Rep >rts and legatrother Document categories. 31s is the production figure used by the National Federation of Indesers and Atstracters for 200 word Indicative atstraca. For correspondence with typicaDy fewer pages than the other two categories, a production rate of three per hour was uxd ($66.00 divided by 3 = 3:2/atstract).
- 3. While it is icknowledged that a pt.rtion of the LSS documenu, particularly formal publications, will have an etstract or summary within the tody of the document, no cost reduction was factored into this table. his decision was based on resporucs of the suntyed atstracting companics. Dey were reluctant to reduce istimates even if documenu contained atstracts, due to the time required to verify the quality of the existing atstract and to edit as required for consistency of coverage with other atstracts. This decision was also supported in the timing tesu perfortned by SAIC in their prototype. Also, no adjustment was made to acknowledge that some docume.ts, such as transmittal correspondence, would not warrant atstracting, ghtn that an annotative summary would te contained in the Title /Dmeription field.
i 7
)
4 V.
6LTERNATIVES TO AB!TTRACriNG Section III.C presented the potential benefits of having abstracts in.the LSS.
This section highlights some of the LSS features currently specified in the SAIC draft LSS Search and Image Design Document which will provide some of the saine benefits of abstracting without the continuing costs of abstracting.
These software features, if not part of the off-the-shelf database package, can be developed at a finite, one time cost.
This section also discusses some other features that could increase precision and recall.
A. CURRENT DOE LSS DESIGN FEATURES 1.
Header Field Analysist After a searcher has developed a hitlist of documents based on his/her search statement, this optional feature, if invoked, would present to the user a computed table of the frequency of occurrences of values for any specified contzslied Vocabulary Header Field.
This shows the distribution of Descriptors, Sponsoring organizations, Author organizations,.
etc. within their hitlist.
For exarnple, given the best known search strategy, the user creates a hitlist of 230 documents on boreholes and volcanic rocks.
The user then requests the Header Analysis
- feature, using the Descriptor field.
The LSS system would then present a listing of all Descriptors used to. describe the 230 and show the number of documents having each descriptor, in decreasing frequency order.
The table would look something liket i
This query found 22,Q units.
Header Analysis on Descriotor Field!
Descriotors Frecuency Fractures 47 i
Fractures (Geologic) 43 s
l Topopah Springs Member 39 Boreholes 36 L
Drill Cores 30 l
Stratigraphy 25 I
I Volcanic Rocks 11 1
Structural Geology 10 Strain (Geology) 4 The user could use this information about their hitlist to select parameters of greatest or least interest to refine the search statement and create a query with greater precision.
For example, 8
the searcher might now want to broaden the search to include all documents on Topopah Springs Member while also excluding documents on Stratigraphy and Strain.
2.
Rankinc Retrieved Documents Based on Selected Term Frecruency:
This LSS feature will allow the user to rank and display the documents in his/her hitlist in decreasing order according to density of selected ASCII-text words in the text.
Density is defined as the number of times a relevant words or phrases appear in the document as a percentage of the total number of words in the document.
For example, the words abstracts, abstracted, abstracting, and abstracters are repeated about 140 times in this 4,000 word paper.
This represents 3.5% of all words in this paper.
The percentage would be even greater if "stop" words (such as a, the, were, most, in, etc.) were excluded from the total word count.
This process will present the hitlist in an order which provides the most relevant documents first on the assumption that if the specified words are repeated frequently in the document, that is a major topic covered in the document.
2 B. POTENTIAL LSS DESIGN FEATURES l
The following are search and retrieval software features that are not currently in the doe design.
These features may warrant further investigation, given the costs of abstracting, the concern of excessively large hitlists, and the problems of low recall and low precision in large text datab, es.
1.a.
Automatic Abstractina -- There are current software packages that purport to scan existing text and present the contents into l:
a ; abstract-like summary.
Such a software feature could be used to add a summary to the LSS header record for presentation to searchers and reviewers of bibliographies to enhance their
. determination of the relevance of documents retrieved.
This would potentially provide the benefits oft (a) reducing the. orders for non-relevant documents or (b) finding relevant documents that might have judged non-relevant upon review of the bibliographic i
information only.
1.b.
ontional Extensive Bibliocranhv Format -- LSS users could the have option of ordering the "first" ASCII page of each document in their hitlist'to be printed along with a header bibliographic listing.
Such a feature would have the same benefits as Automatic Abstracting, described above.
2.
Sonhisticated Rankino Alcorithms
-- over the past several
- years, the information science literature has contained many articles about research to improve text search results using a variety of statistical and lexical analysis methods.
Basically, these are centered on the clustering of related or synonymous terms 9
and word patterns.
Attachments 44 and $8 are examples of such techniques.
The capabilities of such software enhancements to improve recall and precision will be carefully monitored.
As features become proven, they could be incorporated into the LSS design over the life of the system.
E PROS & CONS OF DiFFERENT ONIONS FOR ABSTRACITNG:
A. ALL DOCUMENTS Consistency and simplicity PROS:
Prohibitively Expensive CONS:
Not warranted for traditional ' correspondence' given:
use.of Title / Description Field which will provide short annotative summary for relevancy review.
full-text search capability multiple other access points in the header fields for content /sabject searches of all documents, such as descriptors, i
identifier, project /special class fields etc.
B ALL NON CORRESPONDENCE. TYPE DOCUMENTS "everything but Exclude letters, memos, telephone conversation reports...
l B.1 Abstract all non-correscondence recardless of how lona or short the document.
Less expensive than Option VI.A.
PROS:
- CONS:
- Somewhat wasteful given that some "short" documents do not warrant such treatment.
I B.2 Abstract oniv non-corrosnondence over a certain cace count.
Less expensive than VI.B.1.
PROS:
10 l
~
P o
.t Increased benefits of relevancy review and erseision Selection of document size cutof f is arbitrary CONS:
and subject to debate.
Searchers are very unlikely to keep this arbitrary rule in mind.
Therefore, if they limit their searches to the Abstract Tield for precision, then they could unknowingly exclude whole sets of documents and get erroneous search results.
C.
ABSTRACT ONLY SPECIFIC DOCUMENT TYPES.
C.1 For All Documents Codad as Soecified Document Tvoes --
Pick un Abstracts /Sbmmaries as available within documents or comoose and add if not.
Less Subjective or arbitrary in the selected PROS:
universe than VI.B.2.
Much less expensive because of smaller universe of documents to be abstracted.
Most understandable alternative to most, if not all, searchers.
Therefore least likely to be misused in searching.
CONS:
Still somewhat subjective in that the assignment of Document Type codes is somewhat subjective.
Inconsistent treatment of abstracts and therefore varying quality if abstracts drawn from the text are not strictly reviewed for consistency with LSS abstracting standards.
C.2 Oniv Store Abstracts in Headers for Documents which have author-cenerated Abstracts / Summaries available in ' the text which can be
" crabbed" and out in header as g_garchable full-tex b PROS:
The least expensive alternative while still allowing scarching of this text because submitter's preparation staff and/or LSSA staff do not have to compose and enter the abstract.
11 1
n.
e n
- - ~
The abstract listed in bibliographies will assist the reviewor in determining the potential relevance of documents retrieved.
Universe of documents which contain abstracts CONS:
for searching and for presentation is totally randon.
This does not appear to be a viable option because searchers could not use these randomly existing abstracts with any reliability for identifying relevant documents.
I Subjective in determining if document contains -
text which could be used as an abstract.
Inconsistent treatment of abstracts and
=
therefore varying quality if abstracts ' drawn from the text are not strictly reviewed for consistency.
C.3 Oniv Store Abstracts in Headers for Documents which have author-cenerated Abstracts / Summaries available in the text which can be " crabbed" and out in header but not allow this Abstract field to be searchable.
PROS:
The least expensive alternative.
A. minimal cost to transfer and store the pre-existing text in the header in a non-searchable field.
The abstract listed in bibliographies will assist the reviewer in determining the potential relevance of documents retrieved.
By not allowing searches to be limited to Abstract Field in this option, it prevents users from unknowingly eliminating potentially relevant sets of documents.
This option presents a design issue to be CONS:
solved because the abstracts in LSS header records that desc;ibe documents or data that are Dgi stored 1.i searchable full-text would have to be made searchable.
i
- VII. GU. RRENT 1.SSA STAFF VIEW-i The LSSA staff believes strongly that manually prepared abstracts should not be created for inclusion in the Licensing Support System n
12
in searchable text for those documents that are already stored in searchable full-text due to the substantial costs projected for abstracting in comparison to the benefits.
Although there is the potential for low recall and precision ratios in large text databases, abstracting is not the only remedy.
The other access points in the LSS header fields and the software features specified in the current LSS design will greatly enhance to searchers ability to create useful sets of documents.
Also, the LSSA staff will continue to work with DOE in investigating additional software tools to increase performance and will recommend the development of such software if it is a cost-effective approach.
The LSSA staf f does believe that the text of abstracts that already exist in documents should be captured in the Full LSS Header.
This would be in a non-searchable field to be used for presentation and relevance review only, (option C.3) above. This assumes the design issue can be solved related to the need to search abstracts for those documents / data not stored in searchable text.
l l
l 13 l
l
I
SUMMARY
OF INDUSTRY SURYEY OF ABSTRACTING COSTS APPENDIX A DIRECT HOURLY COMPANY COMPANY COMPANY COMPANY-
' COMPANY-COMPANY LABOR RATES A
B C
D E
F NFAIS ABSTRACTERS
$13.50 -
$25.00
$10.00 -
Unit nr
$12.00
$13.50 -
-18.00 15.00 Charge QUALITY CONTROL REVIEWERS nr
$25.00 nr nr nr nr SUPERVISORS
$30.00
$25.00 nr nr nr nr 4
RATIO OF QC PERSONNEL TO ABSTRACTERS 1:2 1:5 nr 1:3 1:4 nr 1:4 5
RATIO OF Same SUPERVISORS TO 1:20 1:15 nr 1:15 Person nr nr ABSTRACTERS as QC UNIT CHARGE nr
$58.50 nr
$33.29
$16.77 nr nr PER ABSTRACT TIME TO PRODUCE 20 Pages 135 mins /
nr 49 mins /
37 mins /
nr 30 mins /
i AN INDICATIVE of doc.
document 35 page 12.5 page document ABSTRACT per hour document document NOTES:
nr = not reported NFAIS = National Federation of Abstracters and Indexers 1
O
+
~,.
e-.,-
i e
APPENDIX A cont.
CALCULATIONS OF FULLY LOADED NOURLY RATE Average Direct Hourly Rate:
Abstractere
=
$15.75 QC Personnel
=
20.00 Supervisors
=
27.00 Ratio of OC Personnel to 1:3.5 Abstracters
=
Ratio of Supervisors to 1:15 Abstracters
=
Abstractor's hourly rate
$15.75
+
portion of QC rate 5.71
($20 hourly rate for QC personnel divided by 3.5)
$21.46
+ portion of Sup. rate 1.80
($27 hourly rate for Supervisors divided by 15)
$23.26
+ Overhead (120%)
27.91
$51.17
+ G&A (20%)
10.23
$61.40
+
Fee / profit (8%)
4.91 Fully loaded hourly rate for abstracting services.
$66.31
=
2
p o
l APPENDIX B i
ATTACKMENTS
- 1 Salton, Gerald.
"Another Look at Automatic Text-Retrieval Systems."
Communication of the ACL, 29(7).
648-656.
July 1986.
- 2 Blair, David C.
and M.E.
P' Yon. "An Evaluation of Retrieval Effectiveness for a Fu Text Document-Retrieval System. "
communications of th. M 28(3). 289-299.
March 1985.
- 3 Tenopir, Carol.
" Contributions of Value Added Fields and Full-Text Searching in Full-Text Databses. " Proceedinos of the National On-Line Meetino - -1985.
Medford NJ:
i Learned Information, Inc., 1985.
pp. 463-470.
- 4 Ro, Jung Soon.
"An Evaluation of the Applicability of Ranking Algorithms to Improve the Effectiveness of Full-Text L
Retrieval.
I.
On the Effectiveness of Full-Text l-Retrieval."
Journal of the American Society for Information Science.
39 (2),
73-78.
1988.
- 5 A.
Jordan, John S.
Letter to the Editor, Journal of the l-American Society for Information Science (JASIS1 40(3),
i ll 362-363.
1989
+
1.
B.
Lancaster, F.W.
Letter to the Editor, JASIS 40(3), 362.
1989.
r o
- 6-Saracevic, Tefko,
" Comparative Effects of Titles, Abstracts, I
and Full Texts on Relevance Judgements." Proceedinas of the American Society for Information Sciench, Vol. 6 Oct.1-4, 1969. pp. 293-299.
- 7 Science Applications International Corporation.
Licensina l
Eqpport System, Revised Data Scoce Analysis.
Dra f t.
dated August 28, 1990.
- 8 Deerwater, Scott et.
al.
" Indexing by Latent Semantic Analysis" Journal of the American Society for Information Science 41(6): 391-407.
1990.
J
e 1
3 APPLICABILITY OF APPROVED HEADER FIELDS TO THE SUBMISSION AND RETRIEVAL OF TECHNICAL DATA Free Form Particularly Text-(Unco lled Appilcable Searchable BY PARTICIPANT Accession Number Access Code Submitter Center Storage Location Submitter Page Count Title / Description X
X X
Author Name of Contact Author Organization X
X Addressee Addressee Organization X
Document Date
(,
Document / Report Number Document Condition l
Edition / Version X
l Event Date, Code X
Protected Status Related Documents X
Special Class X
Abstract / Summary X
X X
l BY PARTICIPANT OR LSSA:
Document Type Form of Data X
Sponsoring Organization X
X l
Copyee Copyee Organization X
Publication Data X
Descriptors (Thesaurus)
X X
OPTIONAL:
Identifiers X
X X
Comments X
X X
BY LSSA:
LSS Accession Number Number of Images Pointers X
~,
-