Some comments on LispOS/LispVM/SmalltalkVM/misc

[Dwight Hughes]

Fare Rideau has a number of interesting questions and ideas concerning a LispVM -- some of which
served to reroute some of my thinking. (By the way, the TUNES project is fascinating -- putting these
concepts under the LispOS would be awesome, assuming it all reaches expectations.)

I come to this project from considerable experience with Smalltalk and from studying and following the
development of the Squeak Smalltalk project (which was mentioned by Michael Korns in one of his 
messages (the Disney Imagineering group)). So I have a bit of a bias toward a good VM. 

However, I believe the low level of the Smalltalk VM as it was first presented as a defacto standard has
been the source of many of the practical problems that have followed the language system to this day. Any
simplistic low level VM is going to give us problems in performance, access to underlying OSs, extension to 
encompass new functionality, and in general inertia when we see we *really* need to change it -- Java is 
basically stuck with that pitiful excuse for a VM from now on, with minor backward compatible updates.

If we want to design a VM to be implemented in hardware, then the low-level of the JavaVM and SmalltalkVM
are exactly what one would want (in defense of the Smalltalk team, I think that *was* the idea originally).
If we want to run fast on standard hardware, we have to design a high-level VM. Remember the example
of APL - straight interpreted APL was hardly slower than fully compiled APL since the overhead of 
translation was trivial compared to the actual processing time spent performing the operations.

Any VM we might create should use high-level constructs - ASTs  are a good start. For a nice reference 
point check out <http://www.ics.uci.edu/~juice/intro.html> for an AST based VM for the Oberon 
language/system -- note that there is no actual VM running - the VM is just a portable interface specification
between the front end and back end code generation. They distribute the whole Oberon System in this VM
code (previously just for the Mac and 680x0 or PPC - available for x86 now?) -- when a module is loaded from disk the
VM code is translated to native machine code in memory (not JIT - all of it on loading). They claim the compact
representation of the VM code actually allows them to load the VM code, translate to native, and begin execution 
faster than they would be able to just load the same generated native code from the hard drive. 

However, for now I think that we should concentrate on the creation of the "true" LispOS -- or DynamicOS
(probably a better commercial name) -- using the Utah Flux OS toolkit to do the grungy bits - probably with
an assortment of Linux drivers added to the mix. As the design develops and insight is gained we can then
define something of the sort used in Juice (Oberon is not a dynamic language after all), and we can pay attention to
support of other languages then also.

As for running Java bytecodes on and with the LispVM - YUCK!, and yet again YUCK! -- however, getting
a separate JavaVM to run within the *true* LispOS environment shouldn't be any big deal. Why screw up 
everybody and everything to force support of Java bytecodes directly? Windows and MacOS machines 
don't force their whole OS to run on Java bytecodes just to run a few applets. Try to run the Sun 
HotJava 1.0 browser and you will see some of the performance hit any significant app in Java takes.
I think they might even be using a JIT VM with it now (not sure about that). 

If you want to compile Java in the LispOS environment however -- I would have nothing against compiling Java 
to the eventual LispVM/DynamicVM or extending it a bit to help with that - but we don't need no stinking Java bytecodes
<g> in "our" LispVM. This could be a real plus actually -- if Java compiled to the LispVM roars compared
to Java compiled to the standard Java bytecodes and JavaVM, that would be just dandy! And it would be
a real and general incentive to move the LispVM out into the real world.

I do not think we need to create *anything* with that been there, done that flavor. There are too many good
ideas and concepts that have been crushed/dropped in the rush to Unix or MacOS or Windows due to these
OSs built-in hostility to everything that doesn't operate exactly like themselves. We can take the best ideas
and models to make a synthesis the likes of which has only been hinted at in the LispM. I must include Common
Lisp in the been there, done that status -- but we have to start somewhere. Can anyone give a bit of a synopsis
of EuLisp -- in particular, how it improves on Common Lisp?