OOFS / POS

[Chris Bitmead uid(x22068)]

I'll just make one long posting about all the issues that have come up
WRT the OOFS / POS issues. I have quoted several posters below without
citation. I hope this doesn't offend...

Brian has suggested a scheme for persistency which is even more
transparent than what I had in mind. IMHO, this sounds too useful, and
too "clean" to ignore. If there are drawbacks to Brian's approach I
think we should look first to solving them rather than throwing our
hands up in the air and saying it's too hard.

One objection seems to be that objects will be "lost", and we won't be
able to find where they are to recover disk space. I'm afraid I don't
buy this argument at all.

If the way you program is to store references everywhere without any
sense, then you're going to get bitten by the garbage collector
telling you, you are out of RAM, long before disk space becomes an
issue.

I can't see that defining things without cleaning them up is any more
of a problem than those people who create hundreds of files in their
home UNIX account. People who do that should look at what they've
created and clean up what they don't need anymore.

Applications that use objects should be storing them in their own
environment or local variables. When the application terminates the
reference is gone and the object disappears. Only if the application
makes the concious decision to attach some objects to a known
persistent object will those objects become persistent beyond the
termination of the program. The application will make the *decision*
to do this when it is appropriate.

More comments below.

>It is possible (but rare) that the system could lose power when it is in
>an uncertain state where one object has sent a message and another has
>not received it. Personally, I am not willing to take a performance hit
>in order to get around this relatively rare problem. I don't even want
>to think about what garbage collection of a 7GB drive looks like.

There are proven commercial databases (Gemstone for example), that
implement GC across the entire database.

The reason it isn't a problem is that the database is clever about
what it does with objects that don't last for long. I guess you could
say it was a kind of generational scheme. But basically, its been done
before and it's working already out there in the real world.

Trying to deal with a persistent store that _doesn't_ have some GC
scheme is a major headache. It's a headache in Smalltalk, and it would
be a headache x 10 in lisp which encourages the production of garbage
even more so than Smalltalk.

>#1. If something is important enough that it should survive a reboot,
>give it a NAME. An OOFS (instead of a Transparent Persistent Store)
>enforces this. A "transparent store" that requires everything to have
>user-level names isn't that transparent anymore. You must still do a
>bunch of the work you were hoping to avoid: naming things and organizing
>pointers.

I don't want to name every cons cell that I want to be
persistent. What do suggest I name them? 

>#2. If you want to shut down and store EVERYTHING (which is a good idea)
>then that can be done by the operating system with no special filesystem
>or application support. But between shutdowns you don't need to store
>everything so you can actually save some disk space (equivalent to the
>amount of RAM you have).

Storing everything in one big dump is only going to be feasible if you
know in advance you're going to pull the plug.

>#3. If a particular execution thread wants to shut itself down and store
>all of its data it can do so in a standard OOFS by saving its
>continuation. Those who have a good reason for wanting apps that do not
>explicitly name their data can use this mechanism (I think that this is
>rare).

This could amount to hundreds of megabytes. This is not a good way to do it.


>I think you have a very good point here - basically, if programs are
>saving all sorts of data just by relying on object persistence, then
>how does one go about cleaning his workspace?

The same way as you clean your home directory on UNIX.

>What's so difficult about my save-my-state function? It should work
>fine on the Unix file system or any other operating system as long as
>there is an external representation of continuations. But that's a
>language implementation issue, not a file system issue.

And someone else....

>Chris Bitmead has been arguing for a transparent object store. I have
>not. I have been arguing that MOST of an application does not need to
>know about the issue. Only the small part that deals with persistence
>neesd to know about it.
>
>Mostly I've been arguing against Chris's slightly skewed world view
>that he seems to think it's impossible to keep a clean separation of
>portable and non-portable code when you have a persistent store.

I'm writing this telco network management application. I have a
pointer to an object which is the root of a tree of objects that
represents the entire telecommunications network for an entire
country.

I have this algorithm. It traverses the network. Sometimes it
traverses a lot of the network, sometimes only parts of it. The
algorithm makes random changes to the network to optimise it. The
algorithm does not want to care about disk vs memory. It just
traverses data structures, making calculations and changing things
here and there.

Now all those people who think you can just write some save-state and
load-state function, how is going to know what has changed without
run-time support? How is the (perhaps 1000000 lines of) algorithm
going to traverse data structures at will without having to have lots
of explicit loading and saving unless there is run-time support?

Maybe you know how, but I can't see it myself.

>Most of the apps I write have different persistence characteristics.
>They do a lot of work with data that should not be saved to disk and
>then return a little bit of data that SHOULD BE. So you're making your
>apps easier to write and making mine either harder (I must explicitly
>tag hundred of objects "not persistent") or slower.

Then store references to non-persistent things in local variables or a
local environment. When your program terminates they will be gone. How
does this make your work hard?

>Even if you have a POS based program, I'd think you'd still need to
>code up all these contortions so as to be able to import and export
>data from and to other systems.

That would be just a conversion function. It wouldn't have anything to
do with the way you write applications.