POS

[Harvey J. Stein]

Paul Prescod writes:
 > Harvey J. Stein wrote:
 > > If I understand what's being proposed (which is admittedly a little
 > > difficult given that no one's formally submitted an API proposal), I
 > > don't think you'd have to do this.  I think, for example, you'd just
 > > have to make sure that some global symbol contains a list containing
 > > those little bits of data that you're refering to, and that all the
 > > rest of the data is created inside of (let ..) constructs so that it
 > > ultimately gets thrown away.
 > 
 > I understood that, but I don't understand what happens in the meantime.
 > Let's say I'm writing an email program that stays "up" for days at a
 > time. Presumably the local objects get stored to disk once in a while so
 > that if that machine crashes the mail program can resume itself at the
 > last "check point". After all, this app may NEVER close so presumably
 > information on the stack gets saved every once in a while. Obviously we
 > want this automatic check-pointing to occur fairly often.
 > 
 > Now imagine that I'm ray tracing. I'm using a lot of RAM over several
 > days. I do *not* want my local data to take up space. I do *not* want
 > them to use disk space even while the program is running. I do *not*
 > want to waste time saving these objects to disk every so often.

Why not?  Are you doing a ray tracing that takes several days?  If so,
do you want to have to start it again if the machine crashes?  What
about swapping to disk inactive memory because half of a large chunk
of your image has already been computed so as to free up ram for other
applications?  And if you checkpoint again after the ray tracing is
done, then that'd have the effect of freeing disk space.

 > Maybe I misunderstood the proposal, but I don't see how to reconcile
 > these two apps without giving the programme control over the
 > checkpointing process. In otherwords, a "commit" function.
 >  
 > > What you'd gain is that you don't have to write anything to explicitly
 > > write those little bits to disk, and you wouldn't have to write
 > > anything to explicitly read them - they'll always be in that global
 > > variable into which you put them.
 > 
 > Right, but local variables must get saved too. That's what someone was
 > saying about how difficult it is to save the stack under Unix.

Well, in the simple situation, one calls (dumplisp) from the top level
environment and the environment isn't multithreaded.  I guess what
you're saying would hold in a multithreaded environment if one were to
try to checkpoint while other threads are running.  I guess it hasn't
come up for CMUCL because it doesn't have threads.  What does ACL do
in this situation?

 > > If you want versioning of these little bits, you'd basically do the
 > > same as you'd do if you had a file system, but it'd be easier because
 > > you wouldn't have to try to munge version info into the file names.
 > 
 > You don't have to munge version info into an object based file system.
 > Nobody is advocating for a Unix-style file system. I have used object
 > file systems with version info. That feature does not require filename
 > munging.

Well...  It was just a bad example of one common approach that people take
on a standard filesystem for such a purpose.

 > > You could, for example, keep a global variable with an assoc list of
 > > (version-number little bits of data), or keep afew global hash tables
 > > around - little-bits-by-date, little-bits-by-version-number, etc.
 > 
 > My concern here is that I don't understand how these global variable
 > names are stanadardized so that the user can manage the disk. For
 > instance let's take Netscape's page cache. On Unix, Netscape is forced
 > to give those things an end-user-readable name like .Nestscape-cache .
 > But on LispOS it could be only named by the global variable that
 > contains it, which may be readable, like *cache-object* but could also
 > be based on some internal Netscape idiom. Surely we've all read code
 > where variable names are only meaningful in the context of the source
 > code.
 >  
 > Worse, the object could be named (vector-ref *internal-data* 23) .
 > 
 > I admit that there is NO REQUIREMENT on Unix that Netscape store its
 > cache with a name that is human readable (the cache itself has a
 > meaningful name but the files within do not, whereas Microsoft's browser
 > uses meaningful cache-file names too). The separation of internal memory
 > state (which can only be managed by the program) and disk-space (which
 > must ultimately be managed by BOTH the user and the program) encourages
 > vendors to use meaningful names. Since they have to name it something
 > externally anyways, and since a bad name is likely to induce user
 > confusion, they will tend to choose a meaningful name externally no
 > matter what the internal name. The fact that those files are "declared"
 > externally also means that they can be manipulated by other programs.
 > 
 > So my contention is that in the end most objects that are persistent
 > will be treated specially and "named" explicitly. The effort of giving
 > them external names is approximately the same effort that would have
 > been required to explicitly save them. Instead of saying 
 > 
 > (make-object-manager-entry *Netscape-Cache* name: "Netscape-Cache"
 > version: ... date: ...)
 > 
 > You will say:
 > 
 > (store-object *Netscape-Cache* ...)

I think you have a very good point here - basically, if programs are
saving all sorts of data just by relying on object persistence, then
how does one go about cleaning his workspace?

Correct me if I'm wrong, but it the only way seems to be to not make
object saving transparent - to put it under application control, in
which case you loose alot of the beauty of a tranparent persistent
store, which is to say that you have to go about figuring out exactly
how to save your state.  Unless it's merely a matter of saying which
global variables one cares about.

Maybe just say (defvar-persistent 'foo) instead of (defvar 'foo), and
then let checkpointing save everything that's pointed to?  But if you
don't save and restore everything, then I'd think you'd run into all
sorts of pointer adjustment problems.  But, maybe it's not so bad if
one's saving everything pointed to by a set of top level objects.

As you indicate, (defvar-persistent 'foo) would also have to take lots
of additional arguments, such as application name, so that a
housecleaning application could display all the persistent objects and
allow the user to remove unwanted ones.

-- 
Harvey J. Stein
Berger Financial Research
abel@netvision.net.il