Project: Some aid for low level programmer

[Francois-Rene Rideau]

On Fri, May 24, 2002 at 03:21:14PM +0200, Peter Wiehe wrote:
> no give a lttle framework of the concept you had in mind.
The concept I have in mind is some framework to dump low-level code,
and build functionality up from there.

If it's written in a declarative way in LISP, the same framework
can be used indifferently (through proper macros) to produce code
at compile-time, at runtime, etc. Also, with proper declarative style,
we can start with have simple strategies to produce bad code fast,
and define more elaborate strategies as we evolve into sophistication.

The kind of functionality I envision is at first a basic runtime system,
but the ability to control the emitted code can then be used to do
many things we can't do with current systems.
Things like GC, persistence, logging, debugging, trapping exceptional
behaviour, enforcing security, etc., can all be done by instrumenting
the code being generated at the right level of virtual machine.

By building virtual machines up from code generation, we gain access
to a continuum of virtual machines, and can choose
the right one for each job, all the while keeping our "feet on earth",
whereas the approach of starting from an arbitrary virtual machine
will likely force us to make some early decisions disconnected from reality.
This doesn't mean that intermediate representations are bad,
but rather that by remaining always grounded, we can keep the system
self-hosting all the while when developing new, more elaborate,
intermediate representations.

Let's be a little more concrete.
At first, we start from a runtime assembler,
that has a first-class representation of code being emitted,
including processor mode and linking information
(which can be either information about an external object file
or information about the current live image).
You have things like
	(leal eax (ptr ebx 5))
emit code in the current assembler context.

Then, you may add a layer of simple macros,
to extend the instruction set into something more manageable
(i.e. add support for arbitrary immediate constants on the ARM).
More elaborate macros, that might require more contextual information
about the code being emitted, can extend the instruction set considerably,
into more and more elaborate virtual assembler.
This is the approach used by lightning, and not unlike the CMUCL VOP.

Over this runtime assembler (or collection of assemblers, for each processor),
we can build a straightforward code generator for some high-level language,
by mapping semantic constructs directly into suitable macro instructions.
This way, we can hopefully achieve code that runs at acceptable speed
(i.e. under optimized code * 4) in all cases, for a simple
not so high-level language that we bootstrap (not so high-level:
although control structures are high-level, basic types might be lacking
in features and safety checks), and something somewhat slower when
we have all checks enabled.

>From then on, we can start doing more elaborate things:
do analyzes to select simpler VOPs and remove unnecessary checks.

Note that with a declarative framework, a relatively small change in glue code
should allow one to go from compile-time analyzes and code production
to runtime analyzes and code production; from a simple virtual machine
definition that dumps an interpreter for the virtual machine to a compiler
that rewrites the high-level instructions into virtual machine instructions
to be further compiled. "mixins" some layer of code generation can add
the code instrumentation that give us "for free" the same debugging
independently from the code being emitted as a virtual machine or as native
code, at compile-time or at run-time. etc.

So the whole idea is to have a high-level framework for dumping low-level code
and build compilers for high-level code in a declarative way over it.

I admit I'll have to look at MLRISC more in detail.
Since it's ML, it's likely not declarative enough for automatic
use of the thing in widely different settings,
although by using suitable higher-order functions,
I suppose you can always adapt somehow.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[  TUNES project for a Free Reflective Computing System  | http://tunes.org  ]
Philosophy is questions that may never be answered.
Religion is answers that may never be questioned.