Tunes

[spc@gate.net]

On some network somewhere in cyberspace, Nathan Hawkins transmitted:
> 
> Organization of Low Level code in layers:
> 
> 1. very processor specific code: on x86, all kinds of stupidity.
> 2. LLL core: words which can be represented in a relatively small number of
>    instructions, and can always be compiled inline.
> 3. memory management and hardware stuff: page management, interrupts
> 4. LLL environment: heap management, "user" variables, etc.
> 5. LLL modules: standard modules?
> 
> Note: these layers are just useful ideas to keep in one's head. It wouldn't
> be reasonable to expect everything to fit the above oversimplification.
> 
> VERY LOW LEVEL CONVENTIONS:
> 
> I'm currently using the following register conventions:
> 
> ebx is used for TOS, and eax, ecx, and edx are scratch.
>   Possibly eax should be used for TOS? ebx is used in most Forth, but I
>   can't remember why, and if it's important. eax would be better for various
>   stupid x86 reasons. Gcc saves ebx, so maybe that was a factor...
> 
  EBX is used probably because in several Forth implimentations on the 8088
(80186/80286) used BX for TOS, since once could use BX to reference memory
(on the 8086/8088/80186/80286 only four registers could be used as an index,
BX, SI, DI and BP, with BP using SS: as a default segment).

  On the 386 or higher, any of the following can be used to reference
memory: EAX, EBX, ECX, EDX, ESI, EDI, EBP and ESP (with EBP and ESP using
SS: as a default segment).  If you write your "Forth" in C, avoid using EAX
for TOS, as most (if not all) C functions don't bother saving EAX.

  You might also think of using EAX, EBX, ECX, EDX, ESI, EDI and EBP as TOS. 
Then SWAP, ROT, DUP, etc become fairly cheap and if you generate native
code, those words can be optimized out (by keeping track of which register
is "on top".  JaxForth for the Amiga does this I think).

> esi always points to the beginning of the current word.
>   This allows PIC code within a word, if used correctly. Other uses will
>   probably occur to me.
> 
> edi points to "user" variables.
>   CURRENT, CONTEXT, sort of things. This is for things that aren't
>   sharable between threads, and which need relatively frequent access.
>   There shouldn't be very many of these variables.
> 
> esp and ebp are the stack and r-stack respectively.
>   This is helpful if we want to use segments, but otherwise, not strictly
>   necessary that we use ebp. edi would do as well. I chose to do it that way
>   for easier C interfacing.
> 
> 
> MEMORY MANAGEMENT
> 
> Level 1: Page allocation
>   I favour a free list for this. Bitmaps are annoying.
> 
> Level 2: Address space allocation
>   This could be done more than one way, but I like page-based.
> 
  Is this for the 386, or for the LLL in general?  Not all CPUs support
paged memory (say, the 68000 (no MMU) or the 80286 (can address the same
amount of memory as the 68000, but in segments)).  Also, some CPUs (most
notably recent RISC chips) allow page sizes from 512bytes to 16K.

> Level 3: Address range allocation
>   There should be a way to manage groups of addresses, independently of
>   whether or not actual memory is mapped to them. Also, a good level to
>   do guard pages, for automatically growing stacks and heaps.
> 
> Level 4: Heaps and Stacks
> 
> Heaps:
> 
> I need to have some kind of heap management, and I'm leaning toward using
> something more like malloc than the Forth setup. I want to be able to
> FORGET individual words, as well as whole vocabularies. Normal Forth can't
> really do either, which I think is stupid.
> 
  Who says you have to use Forth?  A Forth like language could be used,
taking the best features of Forth, and dumping the bad stuff (I don't like
the way Forth handles vocabularies myself).

  You might also want to approach heaps in a manner like Exec (the kernel on
the Amiga) handles memory.  The two basic allocation functions in Exec are
AllocMem() and Allocate().

  AllocMem() returns memory from one of several system heaps (since the
68000 doesn't support virtual memory, Exec gets around non-contiguous memory
regions by treating each continious memory block as a heap.  This isn't that
much of an issue with virtual memory, but because of the implimentation an
interesting feature comes out), depending upon circumstance.

  Allocate() returns memory from a given heap.  So what one can do is grab a
large chunck of memory from AllocMem(), set it up (actually, initialze a
structure at the beginning of the block returned) and then call Allocate()
with it to allocate smaller chucks, without fragmenting the system heaps.

  And really, AllocMem() is Allocate(somesystemheap); internally.

> I also need to get some suggestions about supporting GC for such a heap, and
> whether it would be better to GC only whole modules, instead of individual
> words. Also, I have some problems with the idea of integer/pointer
> distinction. This could be quite inconvenient. Less than 32 bit pointers would
> be a nuisence: either it would limit address space, or it would make accessing
> characters awkward. Both are very difficult to accept.
> 
  Again, are you talking about in general, or on teh 386?  It's best to
treat pointers as pointers, integers as integers and never the twain shall
meet.  This is one reason I avoid GNU products as much as possible (their
code shall only run on 32 bit systems, where *=int=long.  Bah!).

> Also, a fairly important question: can threads own resources at all levels,
> or would they only own higher level memory objects? There are tradeoffs
> either way. (Not that high-level programs would need to know the difference,
> but there's still the question.)
> 
  Traditionally, a "process" owns resources.  A "thread" ("task" in
Amigaspeak) doesn't.  A "thread" is a resource.  Along with "devices",
"files", "memory".  A "thread" is a unit of execution, and in a loose sense,
"owns" the CPU, which is a resource, and some memory (out of which it is
executing), which is a resource. 

  In other words, make the abstraction fit whatever you're trying to do in a
consistent and elegant way.

> LANGUAGE DETAILS
> 
> Also, I do like the vocabulary concept, but it is very weak. I'd like to
> expand it, so that vocabularies are more like directories, with a search
> path, current vocabulary and root being specific to each thread. (Done with
> the "user" variables.) This shouldn't be too hard to do, with a "slight"
> modification to the parser.
> 
> Example:
> 
> ~ would be the name of the root vocabulary word, specific to each thread.
>   A pointer to it would be kept in a "user" variable.
> 
> ~foo~bar would be a full reference to some word bar in the foo vocabulary.
> 
> If the current vocabulary is ~foo, "bar" by itself would find ~foo~bar.
> Standard pathname resolution as found with Unix and Dos directories would
> apply, including .., but not . for Forth reasons.

  Hint.  Don't use Forth (or: take the good, throw away the bad)

  Also, what's wrong with using '/'?  I know you dislike the ' ' as a
separator (I think we discussed this in private).  Why avoid it?  Is it
because it encroaches upon the filesystem?  Is this such a bad thing? (hint
hint)

> This is essentially a tree
> structure, which is good for modularity, and hopefully good for the HLL.
> I'm open to suggestions for a better separator than ~.
> 
  And the Unix heiarchial filesystem is (hint hint)?

> An alternative would be a design I've been told about, involving nestable
> word definitions, like this:
> 
> : foo : bar ; ;
> 
> where foo:bar or some related syntax would be used. I think this would be
> too complicated, although I as I recall, Pascal allows something like this for
> functions. I never used it.
> 
  Pascal used it, but the only function that could call the nested functions
was the one it was nested in (in your example above, foo could only call
bar, since bar was local to foo).

  I'm doing some work to my own Forth like language to address some of these
issues, but work on it is going slowly (I have other things to keep me
occupied alas).

> Also, on the x86, address spaces can be done either through segments or
> through page directories. I like page directories much better, but they can
> be wasteful of memory.
> 
  The 386 allows both at the same time, with segmentation being processed
first, to produce a 32 bit address, then paging, taking a 32 bit "logical"
address and converting it to a 32 bit physical address.

  -spc (Just another low level guy)