11.3 distributed memory management: Resource allocation

[Mike Prince]

On Tue, 6 Dec 1994, Chris Harris wrote:
> On Tue, 6 Dec 1994, Mike Prince wrote:

> > An agent is a thread of execution, which has it's own local stacks

> How do you plan to represent agents  Would they simply be an offset into 
> a toolbox function, along with stacks, or would they store actual 
> platform-independent code?  How would this differ for native toolboxes?

An agent is a slice of processor time, plus some data for storing that 
state when not in use (plus some of it's own local data).  An agent has 
no code and only "resides" in a toolbox while it's executing that 
toolboxes code.  A second later it might be in another toolbox, or 
another system across town.

The actual implementation is a chain of objects (representing the 
execution queue) of indexes of the agents local data.  So... Each agent 
has one index pointing to the objects that are local to it, plus an 
object holding the virtual processor state.

> > The OS storage primitive is a segment of memory.  The handle contains 
> > fields which link that segment to its parents, children, etc (more about 
> > this later).  A segment can be a toolbox, tool (either virtual machine 
> > code or binary), stack, agent_state (for storing thread information), 
> > jump_list ( for redirecting exectution within a toolbox), or a link_list 
> > (for redirecting execution outside a toolbox).
> 
> This seems too structured to me.  Unless these limits are enforced only 
> be an easily-changable module, we're going to end up finding its too 
> limiting one of these days.  What if we implimented this, and then 
> decided that we needed some sort of storage for a parallel-processing 
> agent?  How could we extend this?
> 
> I'd definitely like to hear more about parents and children.  At least 
> for the original MOOSE project, object-oriented was one of the key goals, 
> and I'm not hearing much about it here.  I know I'd like to see a 
> "truely" (subjective word here) OO OS, rather than simply an object 
> programming model, like NextStep....

I'm trying to adapt this thing so it will work efficiently in a 
distributed environment.  It may look complex at first, but I've pretty 
much listed ALL the parts we'd need.  By saying OO from the beginning, 
but not talking exactly how you'd do it, it may sound simpler/easier, but 
the proof's in the puddin'.  Please give me an alternate implementation.  
I'd love to be proved wrong/shown a better way!

> Here's another thought I just got.  Instead of having one 
> VCPU/stack/whatever low-level language, let's allow lots and lots of 
> them.  Each tool, or whatever they're called, is then written for the 
> most appropriate low-level language, weather it be stack-based, VCPU, 
> list based, whatever, and an ID number would tell it which one to use.

One of the goals is to take low-level code and shuffle it around a 
homogeneous processor pool.  If each of your code fragments needs a 
different compiler, then your overhead is going to get out of control.  
Remember how Taos does it, RISC like virtual processor, small core 
services, fast, etc.

> You present some interesting arguments, but I would be inclined to allow 
> more multiple types of data, so that only the most appropriate tool needs 
> to be used.  I guess I need to hear more about how you plan to merge the 
> stack model and objects....

> One last question for you....  Can toolboxes/stacks/etc be spread across 
> multiple machines, only partly loaded into memory, etc.?  It sounds like 
> your toolbox model doesn't fare to well if you wanted to have two 
> connected agents executing in parallel on two seperate machines.

If two agents want to be working on the exact same data, both reading 
and writing to it, then you can't break the problem up onto more than one 
machine.  But by the same token, even on multiple CPU's you'd have a 
beauty of a headache trying to synchronizing the access to that same memory.

But step back a little, most problems can be broken into sub-problems and 
farmed out.  Or problems involve reading data, then writing new data 
elsewhere (the results).  These can be solved with agents.  You can have 
a toolbox which can replicate onto another machine, taking half it's 
agents, or onto 50 machines, etc.  Each toolbox will have either a flag 
or a duplication function built into it (we still need to decide which).  
It's the same kind of problem of me needing a chunk of code (toolbox) on 
my machine, but my neighbor has it and is using it.  We can also dup it.

I had a similar discussion with Fare earlier about the same thing 
involving a database.  Instead of having one big one, it's broken into 20 
small ones, then you can better utilize an array of processors.

It's all in the programming.

Mike