The feel of a LispM/List of running machines

[Martin Cracauer]

I think we got a little carried away here.

>    Well if you restrict yourself to the LispOS world, then nothing on the
>    Unix side of things should break. They only break when you do things
>    the Unix way.
>
>Wrong.  Sometimes Unix machines just mess up for no frigging reason at
>all.  For instance SDF (the Linux system I gateway my mail through)
>recently rebooted, then fsck cleared its own i-node.  Major catastrophe.

I am sorry, folks, but this is cleary Linux-braindeadness, not generic
Unix.

All Unix systems write file system data (that it, the bytes a file
consists of) asynchronously, but Linux also writes metadata (that is,
the bytes a directory entry or inode contains) async. That's why you
can loose files you didn't even open in write-mode when the system
crashed.

So, if you don't like this, use FreeBSD. If you want async metadata,
there's a mount option on FreeBSD to do so for each individual
filesystem.

Now back to LispOS ;-)

If we are going to do an OS that doesn't loose files, we can either
use a Log-structured filesystem (much work) or we can write all or
parts synchonously, hurting performance (badly, given today's
hardware, very badly).

Whether this filesystem runs under in a Unix kernel or some Lisp
system is going to make the slighest difference in this respect.

What makes a difference between Unix and Lisp semantics is that we are
free to add semantics for the filesystem API.

We can:
- tell one single file to be synchronous, without changing all other
  on the same filesystem
- we can tell a directory that is should be open synchronous so that
  it can't loose file in the same dir. Without constraining the rest
  of the filesystem's behaviour.
- we can add a function to ask a file where it lives (NFS, NetBIOS,
  local files, whatever) and then decide how to open it.
- add a 'append-only' opening mode (what Linux does, because this
  doesn't require a change to the API, just to the filesystem itself
  and modifying the single program 'chmod')
- add a firo filesystem that keep you the latest N bytes
- add version control
- add as many flags as we want by keeping files as sets of two 'real'
  files and store a list it the 'flags' part.

All of these are sematic issues and Unix doesn't do it because all the
application use the limited Unix API and if they didn't they wouldn't
be portable anymore.

But as well as we can provide such an extended API from a Lisp
"kernel", we can add all these features to FreeBSD or Linux and
provide calls to control them, so that our "userlevel" Lisp world can
use them.

Of course, if we used the Unix kernel, it is wise to store those flags
that have meanings for the kernel as well as bit parameters in the
inode and use the property list only for values that affect Lisp
only. But adding a simple property list parser to the Unix kernel is
not a big deal. If we encpsulate all these in an API instead of
manipulating the property list from Lisp directly, we can even keep
running code going while changing the implementation from bit-coded
Unix kernel semantic -> Unix kernel with property list support -> Lisp
implemented filesystem -> Lisp implemented kernel.

>The -only- way to get away from unix lossage is to not run unix.

Yes, but what is Unix? I don't want `make` and `sh`, of course. But
the FreeBSD *kernel* is just a memory system (a pretty good one), a
set of filesystems (most suffering from braindead API, but we need
those as well to read other people's disks) and TCP/IP networking (is
said to be the best TCP/IP implementation ever).

What we really want is a VM/filesystems/networking thing that provides
a useful API, don't we? 

It is my opinion that the FreeBSD kernel has so much *implementation*
done right, especially in the memory and networking area, that we
should try to use the implementation with a different API.

The right thing for performance and savness is clearly a
Log-structured filesystem and the FreeBSD folks are going to make
their implementation usable. If they succeed, we should not limit
ourselve from using it. It is a lot of work and most of the work isn't
limited to the API.

>WHY does unix still have no version
>numbers?  

Well, when I use CVS, I think it is clearly superiour than what the
LispM offered. I don't only want versioning, I want multiuser
capabilities, I want branching, merging, distribution over several
hosts of different OSes and some things more CVS offers.

>But I'm going along with this write-lispos-on-top-of-linux-first idea
>because as Fare Rideau (I think?) recently said, even the kludgiest lisp
>system is better than a C system.  At least it will give us something
>that mostly insulates us, relatively quick.  I just don't want us to
>lose sight of the eventual goal of 100% lisp.

We should define a good API and people are free to reimplement lower
level things as they like. The most work will be independent of the
kernel anyway.

I think it would be pretty easy to implement a disless pure LispOS
kernel. Take a driver for a simple ISA network card with some simple
TCP/IP stack, interface to the MMU to give you useful memeory areas
(CMUCL already works quite near that level). You'll have to get some
filesystem semantics, maybe NFS is the easiest, or even ftpt could do
it. Then you start sending X11 packets to some other machine and you
could run all the things other LispOS people implemented (including a
CLIM clone). The TCP/IP stack would be most of the work, but it seem
to attrach quite some people.

But the center of our work for now should be to concentrate on useful
APIs and *not* define interfaces for functionality that isn't
up-to-date and/or limits us in using *good* things running on other
systems.

I.e. I'd say that versioning like the LispM had should not be part of
our APIs, but a general system that can be layered on top of simple
copying/renaming or to more complex things like RCS and CVS (the
kernel then could drive the things). As long as we don't have a
proposal, don't define an API that is used by normal applications, but
implement the needed functionaly using implementation-dependent
calls. Before versioning enters our logical pathname implementation,
it should be clear how to get the most of it.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@wavehh.hanse.de>
http://cracauer.cons.org
Fax +49 40 522 85 36