What is a ``reflective'' system?

[Jordan Henderson]

Reggie S. Perry writes:
> 
> >"Jordan" == Jordan Henderson <jordan@Starbase.NeoSoft.COM> writes:
> 
> > I think that perhaps the goal of full reflectivity is not best.  If
> > we could define a LispVM separately, perhaps with several
> > implementations (C, JIT, interpreted Lisp for prototyping) then the
> > rest of the system could be fully reflective assembling/compiling to
> > this.  It would require that a LispVM backend be written for CMU-CL
> > eventually.  I can see objections already.  We have people who want
> > to use CMU-CL with native compilation for everything, without regard
> > to whether this is a good idea or not.  I think these people have
> > "come in late from the picture show" and didn't hear that we were
> > going to have a LispVM.
> 
> I am not going to talk about the VM, but I think that there is
> something people are overlooking in order to find the holy grail of
> portability. 
> 
> Every operating system I have used so far has had some small amount of
> assembler to talk directly to the hardware. It would therefore seem
> logical to me to clearly define the set of assembly routines we would
> need that would allow us to implement the rest of the system in some
> lisp dialect with extentions. There seem to be too many low level
> things that we really need that would best be addressed in assembly
> language. If we keep it small, we should be OK. Maybe define a sort of
> HAL.
> 

Well, I think I have already suggested elsewhere, in that last post that
you quoted from in fact, a High-level Assembler.  My suggestion was to
add an assembler to Lisp.  It should be a straightforward extension.  All
you really need are loader directives (to define data and text start
addresses) and a full set of assembler routines which place machine
code in the text segments and data definition routines which place 
initialized data in the data segments.  Of course, you need a full set 
of symbols for register constants and the like.  LAP (Lisp Assembly 
Program) was just such an extension that was available years ago under
Lisp 1.5 (I believe) and continued to be available years later on other
Lisps (I recall such a facility on a DEC-10 Lisp I used almost 20 years
ago).  I don't recall the syntax at all, and I traded away my Lisp 1.5
manual some years ago (why did I >do< that?).  

I couldn't imagine a higher level assembler than an assembler married to
a powerful macro system as is available in CL. 

Do any "modern" Lisps have such a facility?  If yes, how do they interface
the Assembly to the Lisp?  Is anyone here more familiar with this facility
than I am?  

It seems that integrating the Assembler into your Lisp environment would 
be all you would need to make a complete `reflective' system.  If the 
assembler generated LispVM machine code, then the entire system would
be portable after the LispVM were ported (of course, device drivers 
would have to be written, but the Virtual Memory and GC would be part
of the LispVM).

As to implementing the small set of assembly necessary to support the system
up front.  I think that's putting the cart before the horse.  I think we
should work at a fairly high level first and decide where to draw the 
lines on what is in assembly last.

I'm starting to think that the best approach is to use the Flux toolkit 
(which, to a large extent defines the small parts of assembly necessary to
support an OS for us) to bring up CMU-CL directly.  People working with
CMU-CL in this environment (and in others) can be working on prototyping
the VM and the LAP.  Once these are done, combine them into the bootstrap
environment we have under Flux and build up a completely reflective 
system, with or without a VM (undecided yet, either the VM could come
first, or much later as we first do a 386 implementation).  Doing the
386 implementation first has definite advantages in that we already
have a CMU-CL that generates 386, but then if we later rehost on the VM
then we have to throw away the 386 Lisp Assembler in favor of the VM Lisp
Assembler.

Any comments?

> I think that someone said this before. While stack machines simplify
> some things, every one that I have seen so far has mapped badly onto
> the hardware leading to a slow implementation. Since all of the major
> places we are planning to implement are register machines, we should
> avoid the stack machine route.

That was me that said that.  We can avoid the stack machine route and
still have a VM.  See http://www.lucent.com/inferno for details on Dis.

I recall that Spitbol4 used a Spitbol VM (there was a name for it, but I
can't remember) years ago that was register based and easy to implement
on any machine.  These "instructions" were implemented typically as
assembly macros on the target machine (with perhaps a bit of translation,
I can't recall).  Anyway, Spitbol4 managed to be a very fast
Snobol implementation, faster than natively compiled implementations,
(although I believe there were some trivial difference between Spitbol and 
Snobol that made Spitbol much faster but only sacrificed something that 
was very rarely used in Snobol and could be easily implemented otherwise).  
One explanation for Spitbol4 speed was focus on efficiency at a high
level (choice of algorithms and data structures), while most other 
implementations were just assembly hacks micro-optimized before high
level considerations.

I believe that a register based VM, compiled directly to machine code 
on the target processor is a very effective way to go.  It would be fast 
to implement and would provide fine performance.

> 
> 
> -Reggie
> 
> -------------------
> Reginald S. Perry                      e-mail: perry@zso.dec.com   
> Digital Equipment Corporation
> Performance Manager Group	               
> http://www.UNIX.digital.com/unix/sysman/perf_mgr/pmweb/index.html
> 
> My opinions are barely my own. Clearly Digital wants nothing to do with them
> 
>