Kernel LISP - how low down can it go?

[Fare Rideau]

>: Dave Hudson

> Well, there's been discussion about the idea of using a kernel Lisp as a
> base for building a higher level system, but I can see two ways of doing
> this.  The first says that it's really just a core and everything else is
> library code that is built out of that core.  The second says that we
> really have something like the LIL language and that our higher level
> system compiles down to the implementation language.
>
In a reflective language, the difference between the two is blurred:
you can seamlessly mix code written in the lower Lisp with code
automatically generated for it, either statically or dynamically.
All you need is make sure that the overall code is well-founded,
which is a typical bootstrap problem, that can be solved with
the usual techniques.


>>: Dwight Hughes
>
>> While getting a full 32 bits within Lisp is a bit difficult, if your tag
>> for integers is to make the lowest order bit 0 (or make the two lowest
>> order bits 0) this covers the majority of uses you will have for
>> addressing or bitwise logic in a 32 bit word aligned architecture.
>
> Unfortunately of course the x86 isn't a word aligned architecture so
> things like exception handlers will need to be able to get at the full 32
> bits (this is one reason why I picked this as an example in fact).
>
See my previous message about safely accessing the full 32-bit,
even in a tagged architecture.
A more important problem with interrupting code that shares GC space
with other threads, is to recover the GC invariants broken by
interrupted code.
See in the Tunes LLL subproject page for some tricks I have devised
for allowing recovery of GC invariants after interrupts:

http://www.eleves.ens.fr:8080/home/rideau/Tunes/LLL/LLL.html


>> Well, the true Kernel Lisp would probably have to support explicit typing
>> and you would probably have to use it to get efficiency at the lowest
>> level.
Well, at the lowest level (above macro-assembly),
we'd probably have some kind of parenthesized BCPL/FORTH.
No need for complex typing here; one type, "register size", is enough.
Might be trickier on architectures with heterogeneous register sizes,
but these are rare these days. Or the very heterogeneity might be used
as a typing hint (e.g. data/address registers; FPU registers; etc).

> The idea of having the tags and dynamic typing is something that attracted
> me to the idea of using Lisp in the first place.
>
Tags are only a default straightforward implementation for interpreters.
A compiler can remove most tags statically inside the code,
and check/restore them at the dynamic interface between code chunks.

> All I was wondering was if anyone had a handle
> on how much performance loss I might expect to see
> from using them within an OS environment.
>
Optimizing compilers for Scheme/CL can successfully compete with C code.
Interpreters for Scheme/CL, well, have no C interpreter to compete with,
and compete well with tools like Perl.

> I'm sure that there are lots of examples of relative performance
> between say C and Lisp at an application level,
> but I've found that OS performance considerations can often be
> somewhat different.
>
Well, all low-level code should be compiled,
so I fail to see where Lisp would intrinsically lose in an OS setting.

> My other query is that if I wanted to be able to support
> dynamic typing all the way down (and not use any explicit static
> typing if I can avoid it), how much of a runtime system would I need that
> couldn't be coded in Lisp?
>
[Note that static typing doesn't mean explicit typing -- see ML].
Depends on what you call "Lisp". I'm convinced that
the whole system can be done in Lisp,
in the same way as all of Linux is done in C,
despite its having (well-encapsulated) spots of inline-assembly.

>>,> [Against assembly in runtime code]
Well, that's all the point about a suitably low-level lisp:
to do everything in Lisp, rather than in assembly, C, or CL.
This doesn't preclude the use of well-encapsulated spots
of assembly or meta-assembly. Again, see Linux.

> I'm starting to wonder if a new dialect of Lisp might not end up being
> necessary for the low level stuff anyway?
What's great about Lisp is that it can reflectively extend
into dialects of itself, so the same environment is at the same time
the old Lisp and the new one. Of course, the idea is of extending
a simpler, lighter Lisp into CL rather than the other way round.

> [...] (if anyone's read the plan I put on my web site
> for a project called Constellation they'll see this), [...]
Care to give a URL?

>> Until we develop the tools and techniques we need, if we can create the
>> needed structures in C without propagating "C-ness" into the LispOS
>> environment, then we should do so. This will also help us define
>> what the minimum runtime should be. We will need assembly for some
>> things, but this should be ruthlessly minimized (I want to run the
>> LispOS on PowerPC and Alpha boxes too, without having to completely redo
>> everything). (No the C part would not be any less opaque than the
>> assembly code to the LispOS, but it would be a hell of a lot easier
>> to change and to port.)
>
> I think one thing to be careful about is saying that C code is wonderfully
> portable and that we would have to be aware of this if we were to invent
> our own C-like Lisp for low level stuff.  My own experience is that whilst
> a lot of C at an application level can be very portable, all of that
> portability goes out of the window as soon as we start to manipulate the
> hardware.
>
Yeah. If Dwight wants LispOS on a PPC or AXP, well,
I guess he wants a compiler, too, and not an interpreter.
So porting LispOS means porting the compiler,
and this means we have assembly access anyway!
Of course, it could be possible to *steal* the compiler back-end
from say, GCC, but that's another independent story.

== Fare' -- rideau@ens.fr -- Franc,ois-Rene' Rideau -- DDa(.ng-Vu~ Ba^n ==
Join the TUNES project for a computing system based on computing freedom !
                TUNES is a Useful, Not Expedient System
URL: "http://www.eleves.ens.fr:8080/home/rideau/Tunes/"