Persistence, Logging, Versioning

[Fare Rideau]

>: Martin Cracauer

> If we are going to do an OS that doesn't loose files, we can either
> use a Log-structured filesystem (much work) or we can write all or
> parts synchonously, hurting performance (badly, given today's
> hardware, very badly).
>
Well, if we write our software cleanly,
we might begin with one, and replace by the other.
Or we might take Texas, as does the RScheme project?

> Whether this filesystem runs under in a Unix kernel or some Lisp
> system is going to make the slighest difference in this respect.
>
Well, yes it does. Under Unix, we could reuse an existing library,
but this would restrict the way we can organize memory management.
Of course, we could use Unix as just a slow hardware abstraction,
and write our Lisp code right away...

> What makes a difference between Unix and Lisp semantics is that we are
> free to add semantics for the filesystem API.
Yup.

[Semantic issues with Unix]
Problem there: our implementation on top of POSIX can only
approximate this, and cannot enforce consistency,
as applications running outside of Lisp can mess things up.

> What we really want is a VM/filesystems/networking thing that provides
> a useful API, don't we?
>
> It is my opinion that the FreeBSD kernel has so much *implementation*
> done right, especially in the memory and networking area, that we
> should try to use the implementation with a different API.
>
I guess this means I'll be studying the *BSD kernels closely.
What scares me is that there are so many of them
(well, the two main seem to be FreeBSD and OpenBSD).

> The right thing for performance and savness is clearly a
> Log-structured filesystem and the FreeBSD folks are going to make
> their implementation usable.
Ahem. Log-structured, great. *File*system, scary.
We want a *persistent object store*, not a filesystem.
Of course, if they did their job right,
lots could be saved from their work.
We also want hooks for distributed GC/sync/replication/versioning.
I doubt their Logged FS can be used as is...

[CVS better than LispM versioning]
Well, the great disadvantage of CVS is its being *file*-based.
What will it become when we have no more a tree of files,
but a graph of objects?

> We should define a good API and people are free to reimplement lower
> level things as they like.
>
Right.

> The most work will be independent of the kernel anyway.
>
There will still be lots of kernel-dependent work:
implementing efficient GC and synchronization between
distributed persistent stores will be non-trivial.
   Of course, the advantage of LISP over C is that a major modification
in low-level details won't affect the high-level code in any way, whereas
C is so low-level that it any low-level choice becomes a legacy burden.

> I'd say that versioning like the LispM had should not be part of
> our APIs, but a general system that can be layered on top of simple
> copying/renaming or to more complex things like RCS and CVS (the
> kernel then could drive the things).
>
Mml. I agree, but this also means we must be ready to rewrite
significantly any part of the system that will be affected
by the eventual API.

== Fare' -- rideau@ens.fr -- Franc,ois-Rene' Rideau -- DDa(.ng-Vu~ Ba^n ==
Join the TUNES project for a computing system based on computing freedom !
                TUNES is a Useful, Not Expedient System
URL: "http://www.eleves.ens.fr:8080/home/rideau/Tunes/"