The LispOS Project: a position paper, part 2

[Dwight Hughes]

| From: Peter.VanEynde <s950045@uia.ua.ac.be>
| 
| On Tue, 3 Jun 1997, Dwight Hughes wrote:
| 
| > Some general comments:
| > 
| > The use of a Mach-type kernel would certainly be better for
| > our purposes than using a monolithic kernel based Un*x --
| > that much less to wade thru. What is the status of MkLinux?
| Sorry? Why should a Mach-type kernel be better?
| You can use kernel modules to add functionality to the "monolithic" Linux
| kernel. You can even use kerneld to load them as they are needed, like I
| do for the printer, floppy, parallel port, minix fs, msdos fs,etc.
drivers
| at the moment.
| 
| It is my humble opinion that the Linux source is as modular as the Mach
| kernel, it's just the built kernel that is "monolithic". It's all in the
| eye of the beholder...

I am probably showing my ignorance here, but here goes. I see the 
"microkernel" as more hackable/modifiable/extensible, relative to the
rest of the OS, than the "monolithic" kernel. It would appear to
be easier to add new primitive operations with better control and
understanding of their interactions with previous primitives (needed
to support the rest of the system) due to the factoring of the
code into the absolute essential mechanisms of the microkernel.

 [ snip ]
| Problem: there is no native code generator for the PowerPC in CMUCL. But
| there is for the HP-PA, the Sparc  and the Alpha (all now supported by
| Linux). 

Hmmmm.... I've been concentrating on x86 and ignoring all the rest.
Well, T doesn't have a PPC *or* an x86 native code generator.

| > As for the goal of "Lisp down to the metal" -- that will be
|                       ^^^^^^^^^^^^^^^^^^^^^^
| I've been reading (In the little free time I still have left) the paper
on
| the original Lisp machine. There you can see that the machine was
| developed to be easy to program in Lisp... The PC isn't made to be easy
to
| program, it's made to be cheap. Just think of the dozens of registers in
a
| normal VGA card that are (officially) write-only... The interlocks
between
| the registers, the timing that is sometimes critical... IMHO "down to the
| metal" will only work if 
| a the metal is designed to be used by a LispOS.
| b the metal is a minimal kernel, I think Linux will fit the bill. Like I
| said before: just think that Linux is an abstraction layer.

I would, at least from time to time, like to get down to the metal with
some
lower-level Lisp (or a Lisp designed with that ability in mind).

| I would like option a, but nobody would buy it :-(.

I wonder what could be done these days with a handful of Altera or Xilinx
FPGAs - I've never really known enough about the LispM designs to be able
to make an estimate. Might be worth a bit of thought -- perhaps a
"freeIvory"
(well, "cheapIvory", hardware is never free) design on PCI bus cards?

| ...
| > 1) the tight integration of GC and virtual memory (both
| > hardware and software aspects), giving the ability to allow
| > each to control the other tightly and take maximum advantage
| > of resources (for example, being able to constrain GC to
| > only deal with in-memory pages whenever desired);
| 
| I was thinking of this. It _can_ be done with a minimal kernel module and
| mmap. But I've got philosophical problems: if you want to collect the
| garbage in memory, you need to know what memory is referenced by the
| swapped out memory. And how can you know that without swapping it in, or
| keeping an expensive "pointers from outer space" list? But I know only a
| little about GC, so can someone enlighten me?

One of the LispM guys here (sorry, I can't remember who exactly right now)
first explained a bit about this -- the details I'm still trying to figure
out, but the payoff is very high efficiency usage of resources (and thus
considerably better performance). One point was that the GC would progress
as far as it could to all objects that it could reach *within* memory
before
starting to swap - swapping only if absolutely necessary in effect; what
it can resolve it does - if not, postpone. You could, for example, have
the GC follow pointers within memory until reaching one that faults, then
add the pointer to a "to do" list to tend to later. There would be
limitations
of course - programs/processes that spew a lot of small short to medium
life
objects would seem to benefit the most from this approach. You would
probably
not want to run this way *all* the time (or at least be able to adjust the
GC's swapping-phobia level).

| > 2) being able to dynamically create/destroy multiple 
| > scheduling policies definable by the applications
| > themselves, not predefined in the kernel. These are somewhat
| > outside the range of adaptability of any "standard" OSs,
| > probably for several more generations of these OSs. I
| > know you don't disagree with any of this -- you just see it
| > as a lower priority item than I do (or at least lower on your
| > list of interests).
| Already mostly done :-).

This has been attempted and/or actually done in several experimental
OS but can it be done (or efficiently faked) within Un*x type OSs?
Most of these OS papers start, "We created this (name here) OS because
Un*x lacked the flexibility to (see 2 above)...." We can live without
it for now, but I want it someday!
 
| There is a "RT-Linux" project that runs above the normal kernel. It would

| be trivial to add extensions, or to add a call-interface...

Doesn't RT-Linux drastically limit or disable "normal" Linux capabilities
like networking and such? I think the limitations are due to the hard-
realtime constraints it is operating with though, not the essential 
idea.

| > My primary -reservations- about jumping on the Lisp-to-the-metal
| > wagon are 1) all the broken devices we will have to deal with
| > (I very much want to make maximum use of others' suffering
| > to create good device drivers, especially video drivers) and
| Sigh. Very true.
| 
| > 2) having to develop a good GUI interface for LispOS, along with
| > everything else, and create programs using it. Sure, we could
| > create a pure server LispOS (and make Kelly very happy), but that
| > is definitely not on my interest list (I would want the LispOS
| > to have the capability, but not *just* that).
| 
| I would just like people to help the free-clim project and add ports...
| I've been _running_ the first release, so it sort-of-works. It just needs

| programmers. Even bad ones can help... just look at me :-).

I've been studying it when I get a few moments, still have a ways to
go though.

| > Your point that our time would be better spent *not* recreating
| > the Un*x software library is well taken.
| As I said before, we don't need a revolution, we need slow infiltration.
| If LispOS is easier to program in, runs on all conventional OS, and can
| communicate with the other programs, then people will flock to it...

A slow infiltration may be all we can manage anyway, even if we're both
good
*and* lucky. This is one whale of a project, seen as a whole.
 
| Groetjes, Peter

-- Dwight