coherent states (continuing old thread)

[David Manifold]

I forgot this message in my postponed folder ... I just found it and
finished it up: 

On Sun, 2 Aug 1998, Jecel Assumpcao Jr wrote:

> > > [how can state be decentralized (distributed)?]
> > (Allowing access is done by providing a function).
> 
> Here you are using sharing in the sense of resource protection.
> I agree that a design can make this linked to distribution
> (this is one of the really new things in Merlin) but it is
> normally a separate issue.

Ok, it is a separate issue.  I didn't exactly answer your question. 
Resource protection is accomplished by NOT providing a function to access
some resource.  Distribution is accomplished by providing generic
functions:  The local-access and remote-access functions have the same
syntax. 

> > So what I'm saying is
> > that once the system works standalone, it should be easy (almost fully
> > automatic) to extend to distributed systems, because a single system is
> > "distributed" already.
> 
> This is true in theory. But you must remember that when
> quantative differences become great enough they become
> qualitative differences. You can't replace instant
> communication (shared memory) with a low bandwidth
> alternative (modems) and expect the same design to keep
> on working. 

The same high-level semantics are adapted with separate specifications to
the two different low-level media of shared memory and modems.  The
latency changes with each medium, but that's why we explicitly state all
relationships of relative speed and latency, and then allow the system to
adapt itself to the optimum configuration.

> > So let me translate your question into the equivalent
> > in local terms:
> > 
> > Would objects A and B, who are both sharing object C, both have a full
> > copy of object C?  Would A and B each get half of C?  This last option
> > isn't practical unless you decide beforehand what A and B's roles are going
> > to be.  A and B might each get a copy of some common part of C plus some other
> > part of C that is not shared.  In any case, if A changes its part of C then
> > when do these changes propagate to any other objects that may be looking at C?
> > What if A and B simultaneously try to do conflicting things to C?
> > 
> > I can't answer these questions without more specific detail, since there
> > are so many different possibilities.
> 
> Ok. What I am asking is: will the system choose one of
> these possibilities? Will there be some toolkit to
> support them all? Knowing Tunes, I am sure that the
> answer is the latter. So I am interested in what building
> blocks make up this toolkit. I think this must be detailed
> before the first line of Tunes code can be written.

1) how much of object C goes to A and B?

You see objects A, B, and C as separate sets of constraints.  TUNES sees
one big list of constraints, made up of the union of A, B, and C.  If the
union is inconsistent, you must change one of the objects before you can
proceed.  Between the explicit specification and a concrete
implementation, you see "possibilities".  TUNES sees an incomplete
specification.  Possibilities must be eliminated in order to completely
resolve the constraints.  So TUNES needs to know:  What are the
relative priorities A and B get?  The system is likely to choose based on
system-wide constraints such as "Applications the user is watching get
first pick of resources".  How often are A and B going to access C?  If C
is heavily interfaced, it gets reimplemented optimized for speed of
access.  If C is loosely interfaced, it can be optimized for space.
Ideally, you don't have to answer these questions every time they come up.
You answer once, and that answer becomes a system-wide default.  Then the
settings should be easy to get back to in case you want to change them
again.

how do changes to "object C" propagate?

The functions that modify object C are annotated with functions to notify
A and B, possibly in different ways or at different speeds.

I haven't studied parallelism too much, but I have a book about it, and I
know it describes mechanisms for programming threads that share data.
We'll implement these in a high-level abstract way so that they are
distributable across the system and integratable into user programs (so
nobody has to reimplement anything).

> > [you meant:]
> > > > > You mean like  (car (quote (cdr car cond x))) or (car '(cdr car cond
> > > > > x))?
> > 
> > What exactly does this change in the order of evaluation?  Let me
> > guess...if (cdr car cond x) evalutes to '(a b c d), the above expression
> > will evaluate to (car '(a b c d)).  Right?  It won't evaluate to 'cdr,
> > will it?
> 
> The above expression will return the symbol 'cdr. If you eliminated
> the quote then you would get 'a (though (cdr car cond x) isn't a
> valid Lisp expression, so it couldn't really evaluate to '(a b c d)).

This mechanism isn't nearly as powerul as we need.  We need to be able to
say something like, pause evaluation depending on some condition, and then
resume on some other condition (manipulating the evaluation context from
within the function that is being evaluated).  The (quote ..) is just a
special case that is known by the evaluator.  Programs may need to adjust
the evaluator, or specify their own complete evaluator.

> > > [various models of coherent state, including WWW and NNTP]
> > 
> > Just think of the internet as one giant hard drive.  When you
> > install tunes, you get a window onto it.
> 
> Good, solid start. I am interested in the case where your window
> into the "global string" is read/write and you make a modification.

There has to be a tree of constraints defining what to do.  The action
gets passed along from one context to the next as expressions are
simplified and then eventually stops when the modification is made.

Now, I still don't know what coherent state means.  But, if it means what
I think, probably TUNES won't have it.  If you drop a rock into a pool,
the effects ripple outwards.  Waves don't hit all sides of the pool at the
same time.  In the same way, changes don't propagate across the Internet
at the same speed, or even throughout the local system at the same speed. 
I don't think we can expect them to.  At best, we can approximate a
coherent state (by the persistent image on disk)... it could be forced, at
expense of some time.. which would probably only be useful as a kind of
'sync' to prepare for shutdown.  (Syncing projects across the Internet
could be done for major version releases.)

> When I started working on Merlin, some 15 years ago, I had to
> choose between three exciting programming styles: OO, Logic and
> Functional Programming. I looked around me and saw great OO
> systems that had actually been built and were being used, but no
> real logic or functional systems (this hasn't changed a bit,
> as far as I know). So I abandoned the more attractive alternatives
> (you don't have to worry about emulating a single coherent state
> if you can exclude the notion of time, for example) and went with
> objects. Though Merlin doesn't exist right now, it did in the
> past (three systems were build, but never sold) and I can say that
> objects haven't let me down. I've learned that nothing really
> scales very well - the large insects of horror movies could never
> exist, for example (strutural strength scales with are (square
> of length) while weight scales with mass (cube of length)). I'd
> hate to have Tunes built around ideas that are wonderful in small
> experiments but which can't be built up into an NT replacement.

* We use objects with functional programming instead of message passing. 
* Logic programming is implemented using objects and functions, and is
used to meta-program objects and functions. 
* Current systems don't scale because they're built in layers.  In TUNES,
everything has full source so at any time the system can eliminate any
layers.  This allows easy absorption of new modules, decentralization of
control, and intelligent optimizations that make use of all the
meta-information available.  You can scale the system by adding additional
abstraction levels at exactly the necessary points, including at the low
level or at the high level.
* Linux is already turning out to be a great NT replacement.  TUNES is
not an operating system, it's an interactive modeling system, that happens
to have a built-in OS because it's reflective.

David Manifold <dem@pacificrim.net>