LLL weekly [far32]

[Francois-Rene Rideau]

 I'd like to regularly post a summary of pros, cons, reqs, etc,
about a standard LLL (or what I began to call a LLL, but I'm sure
you'll soon find a better word/acronym).
 This is the first version of the summary.
 
LLL principle
~~~~~~~~~~~~~
 There is a (family of (slightly hardware dependent ?) ?) standard
coding format for objects, INCLUDING EXECUTABLE CODE in such a way
as trivial programs can be instantly interpreted on any machine,
while more complicated programs are compiled without the
parsing/front end step which is known to be the most limitating,
and with intermediate code optimisation done and info ready for
ultimate back end compiling.

Pros
~~~~
1) Greatly shortens compile time
2) allows quick, easy and efficient portable code&data (=object)
exchange inside a heterogeneous distributed system or between 
remote systems.
3) greatly reduces future compiler design: a new language will
only need a new (or adaptated) front end; tools may be provided
for parsing (an OO lex/yacc equivalent). The back-end part of
compilers will be language independent, only hardware/implementation
dependent.
4) may extend current existing HLLs by including features a OO,
genericity, recursive call to HLL from LLL, etc.
5) may be compact _efficient_ (_standard_) interpreted code for HL
tasks, sometimes quicker than hand-written assembly, while much more
portable (don't you know most adventure games use their own
interpreter ?)
6) Is simple enough to be used as back-end by anyone willing to
compile his own expressions (see spreadsheets, function graph
drawers, etc)

Cons
~~~~
1) Is an important human-time investment (but aren't we Moosers
here to work ? :-)
2) Requires our respecting programming rules (but anyway, it's
inevitable)
3) Needs new compilers (but anyway, a new OS always requires new
compilers, to comply new object code format) (so why not profit
to build also a new HLL ? :-) for its features to be available to
programmers/users (again, I make no clear difference).
4) Compiler editing companies might not appreciate competition
(see Microsoft)
5) if it works, ANSI will have to produce again new papers (but
that also is inevitable, with MOOSE, isn't it ? Think about ANSI
MOOSE ! :-)

Reqs
~~~~
1) We must find a "good" preferably extensible format. A quick
linear conversion with a small memory occupancy is to be considered,
to provide at the same time external portability and internal
efficiency (in/external relative to each host). If we forget
something important, we'll have to produce un-upward-compatible
formats and thus forma converter that will occupy system time and 
space.
2) The standard must provide a quick, efficient, unspacy way of
coding external links: an object will almost always need external
objects to run/evaluate, that shouldn't/couldn't/oughtn't be
included in the object itself; however, when loaded/saved, the
object needs the link not to be forgotten, so as to be properly
restored next time. My opinion is have tables/virtual dictionaries
at module level (that'll be my definition for what Dennis called
modules)
3) As the format includes not only LL executable code, but also
data, etc, it's a full language. The main difference from usual
languages then is the fact that it is a binary format as opposed
to text format. But as text is a particular kind of binary, a
point of view would be that HLL is an LLL subset (as well as
machine language) (actually, if my LLL is extended to a full
object description standard, any object will be coded through it,
so why not HLL source fies in particular ?); thus, there is no
more compilation, but only simplification ("optimisation") and
evaluation, from subsets of objects to others.

Comments and Compile info
~~~~~~~~~~~~~~~~~~~~~~~~~
 If you want your LLL code to be efficient not only interpreted, but
also as intermediate code for compiling, you must add info (taken
from HLL compile) that will help optimization (optimizing for LLL
interpretation often is in conflict with LLL optimizing for further
xxx CPU assembly). More generally, we must include in our object info
that is not used while computing the object, but while compiling it,
while scheduling its use, while displaying it or interfacing it to
any external reading-only program.


First crude sketch
~~~~~~~~~~~~~~~~~~

** Modules **
A module will include
- a "Magic number" or equivalent, followed by info about coding format:
endianness, integer length, etc.
- a external module identifiers (names, version ?), followed by
id. of included objects.
- a list (only number ?) of public subobjects.
- a list of relations between the module's public object,
including their type and/or content.
- a pointer to info/comments about the objects. Info/comments have
their own format, different from module format. Module & info CAN
be packed in a single object/file, but don't have to. Info includes
optimization info, source file correspondance, showing/printing/
drawing/displaying format info, object equivalent representation
in different environment, etc.
- LLL include no builtin instruction, only calls to imported
instructions; however, standard instruction libraries are provided.

386 memory implementation
~~~~~~~~~~~~~~~~~~~~~~~~~
** Interpreted LLL **
 Objects are translated into a stack of pointers to LLL routines in
a LLL segment (devices can be added to the LLL through virtual memory).
Execution is done in privilege level 3, and the "code" is put into a
read-only stack (whatever, a level 3 stack is recommended not to interfere
with interrupts). LLL interpreter just pops data and near rets to next
instruction. Escape instructions can change mode into far ret and/or direct
assembly execution, etc. Another stack is ready for use in assembly mode,
and a ds:si, es:di LLL stacks are used for data/address push/pop.
 What d'ya think of it ?

** Compiled LLL **
 It's just 32 bit protected mode machine code. Standard programs run in
lowest privilege level, but lower level programs may run in PL2, 1, or 0.
Pointer/data have different stacks. Programs not trusted will run in PL3,
and won't be able to access external modules but through system calls to
modified paging a/o segmentation.


Policy propositions
~~~~~~~~~~~~~~~~~~~
1) (Fare')
First have a LLL interpreter in C/C++; then build C/C++ and/or
NewHLL compiler to LLL (and why not NewHLL <-> C/C++ compilers,
understanding extensions of C/C++ through added info inside
comments ?)
2) (?)
Forget all this, and "just" adapt an existing compiler (GCC ?
but I thought Michael agreed against plain C/C++ ?)





Feel free (bound ?) to reply/contribute and/or to correct my poor
english and my mistypes. Reply for small contributions/corrections,
group reply for large controverse, but please make me know your
opinion.
 If you feel someone is better suited than me to maintain this issue,
just tell him and me; if he agrees, I'm ready to let him this task.

   ,
Fare


P.S.:
I'm preparing some NewHLL examples for next week.