Persistence, Logging, Versioning

Fare Rideau
Mon, 5 May 1997 19:43:33 +0200 (MET DST)

>: Martin Cracauer

> If we are going to do an OS that doesn't loose files, we can either
> use a Log-structured filesystem (much work) or we can write all or
> parts synchonously, hurting performance (badly, given today's
> hardware, very badly).
Well, if we write our software cleanly,
we might begin with one, and replace by the other.
Or we might take Texas, as does the RScheme project?

> Whether this filesystem runs under in a Unix kernel or some Lisp
> system is going to make the slighest difference in this respect.
Well, yes it does. Under Unix, we could reuse an existing library,
but this would restrict the way we can organize memory management.
Of course, we could use Unix as just a slow hardware abstraction,
and write our Lisp code right away...

> What makes a difference between Unix and Lisp semantics is that we are
> free to add semantics for the filesystem API.

[Semantic issues with Unix]
Problem there: our implementation on top of POSIX can only
approximate this, and cannot enforce consistency,
as applications running outside of Lisp can mess things up.

> What we really want is a VM/filesystems/networking thing that provides
> a useful API, don't we?
> It is my opinion that the FreeBSD kernel has so much *implementation*
> done right, especially in the memory and networking area, that we
> should try to use the implementation with a different API.
I guess this means I'll be studying the *BSD kernels closely.
What scares me is that there are so many of them
(well, the two main seem to be FreeBSD and OpenBSD).

> The right thing for performance and savness is clearly a
> Log-structured filesystem and the FreeBSD folks are going to make
> their implementation usable.
Ahem. Log-structured, great. *File*system, scary.
We want a *persistent object store*, not a filesystem.
Of course, if they did their job right,
lots could be saved from their work.
We also want hooks for distributed GC/sync/replication/versioning.
I doubt their Logged FS can be used as is...

[CVS better than LispM versioning]
Well, the great disadvantage of CVS is its being *file*-based.
What will it become when we have no more a tree of files,
but a graph of objects?

> We should define a good API and people are free to reimplement lower
> level things as they like.

> The most work will be independent of the kernel anyway.
There will still be lots of kernel-dependent work:
implementing efficient GC and synchronization between
distributed persistent stores will be non-trivial.
   Of course, the advantage of LISP over C is that a major modification
in low-level details won't affect the high-level code in any way, whereas
C is so low-level that it any low-level choice becomes a legacy burden.

> I'd say that versioning like the LispM had should not be part of
> our APIs, but a general system that can be layered on top of simple
> copying/renaming or to more complex things like RCS and CVS (the
> kernel then could drive the things).
Mml. I agree, but this also means we must be ready to rewrite
significantly any part of the system that will be affected
by the eventual API.

== Fare' -- -- Franc,ois-Rene' Rideau -- DDa(.ng-Vu~ Ba^n ==
Join the TUNES project for a computing system based on computing freedom !
                TUNES is a Useful, Not Expedient System
URL: ""