POS

[BRIAN SPILSBURY]

> Paul Prescod writes:
>  > I understood that, but I don't understand what happens in the meantime.
>  > Let's say I'm writing an email program that stays "up" for days at a
>  > time. Presumably the local objects get stored to disk once in a while so
>  > that if that machine crashes the mail program can resume itself at the
>  > last "check point". After all, this app may NEVER close so presumably
>  > information on the stack gets saved every once in a while. Obviously we
>  > want this automatic check-pointing to occur fairly often.

Application oriented check-pointing is going to cause pain and agony
in the universe at large unless you want to have unix-style apps with
minimal crossover. Imagine I have my 'mail' app which relies on the
state of some 'queue' app. Now this queue app is being used by a bunch
of things, so we can't just throw it into the 'mail' app.

Off we go, our mail app checkpoints. Question. Does it save information
in the 'queue' app? If so, where do we stop? If not, why do we bother?

If the system explodes before the queue app also checkpoints then we have
now got two applications with disjoint views of the universe.

This really bites, not to mention that you now have a special notion
of 'application'.

>  > Now imagine that I'm ray tracing. I'm using a lot of RAM over several
>  > days. I do *not* want my local data to take up space. I do *not* want
>  > them to use disk space even while the program is running. I do *not*
>  > want to waste time saving these objects to disk every so often.

Well, these days with the wonders of DMA, etc, etc, it doesn't really
chew much cpu time up to write a block of memory to disk.

I don't know how many milliseconds of user time you'll lose, but I'd guess
that it won't be significant, noting that before we write the memory to disk
we can copy it elsewhere, tell the disk-writer-thingy to please write this
chunk, mark that page we've copied as 'probably clean', and if its
not dirty when we've finished writing our copy, we can mark it as
'really clean'. While we're writing to disk, your ray tracer can happily
chug along.

>  > Maybe I misunderstood the proposal, but I don't see how to reconcile
>  > these two apps without giving the programme control over the
>  > checkpointing process. In otherwords, a "commit" function.

A program oriented commit will cause more problems than not unless you
put your programs in boxes and don't let them talk or share data very
much, which kinda spoils the point of a lispos :)

Checkpointing can be painless and asynchronous especially given a fairly
fine grain, like 4k blocks.

>  > Right, but local variables must get saved too. That's what someone was
>  > saying about how difficult it is to save the stack under Unix.

Generally you just move the stack to wherever you've set up your persistent
memory, but you have to make sure that things like pointers don't break.
Which is an impressive problem you'll have if you have checkpointing
that is disjoint over applications.
 
> Well, in the simple situation, one calls (dumplisp) from the top level
> environment and the environment isn't multithreaded.  I guess what
> you're saying would hold in a multithreaded environment if one were to
> try to checkpoint while other threads are running.  I guess it hasn't
> come up for CMUCL because it doesn't have threads.  What does ACL do
> in this situation?

Threadedness shouldn't be an issue here, th solution above will work
as well with 50,000 threads as with 1.

> I think you have a very good point here - basically, if programs are
> saving all sorts of data just by relying on object persistence, then
> how does one go about cleaning his workspace?

One organizes his workspace so that it can be cleaned. 
Consider the functional nature of a unix directly. We can
'add', 'list', 'remove', 'link', 'timestamp'.

So we can think about using something even as simple as an alist.

((important-files (read-me . ??) (todo . ??)) (boring-files (dont-readme . ??)))

then we can write an (ls) function if we really want to and get this

(ls)
.
..
important-files
boring-files

(ls important-files)
.
..
read-me
todo


So, I don't really see that we need to beat people around the head
to make them organize themselves in one particular way, its pretty
trivial to do.

> Correct me if I'm wrong, but it the only way seems to be to not make
> object saving transparent - to put it under application control, in
> which case you loose alot of the beauty of a tranparent persistent
> store, which is to say that you have to go about figuring out exactly
> how to save your state.  Unless it's merely a matter of saying which
> global variables one cares about.

Well, I would have thought that you'd flag things at creation time
as to how you wanted them to persist. Something like wrappers, for example;

(allocate-synchronous
  (make-instance 'big-important-thingy))

or

(allocate-i-dont-want-this-to-get-saved-because-im-scared-that-it-might-slow
-me-down-too-much-thank-you
  (make-instance 'completely-trivial-ray-tracer-that-i-dont-care-if-it-doesnt
-finish-as-long-as-it-does-it-quicky))

:)
 
> As you indicate, (defvar-persistent 'foo) would also have to take lots
> of additional arguments, such as application name, so that a
> housecleaning application could display all the persistent objects and
> allow the user to remove unwanted ones.

Again, this is easy prey to disjoint checkpointing, and horribly limiting.

Brian