Reflection

[David Jeske]

On Sun, Jan 03, 1999 at 10:49:39AM +0100, Anders Petersson wrote:
> [snip: linking in all information]
> >Was that more concrete?
> 
> Yes, thanks.  The question is whether this is your personal view or
> the "official" version, and if your wish to actually do this in
> Tunes - you said "One example of how an existing system ...".

I'm glad. This was an example of features which make a system 'more
reflective'. The idea of tunes is to extend this idea to it's logical
conclusion.

> >My understanding is that the intent of Tunes is to allow people to
> >specify lower-level information as much as they want, but to never
> >require that lower-level optimization information to run the
> >functional code. Cases which deal with low-level issues like hardware
> >access do not fall into this category because the low-level issues are
> >part of the required functional description.
> 
> Lovely, another - maybe even blurrier - version. This is just getting
> better. (irony)

Are you asking for clarification of what I said above? Here it is:

Here is an example. If I write a program which has an aggregate
data-structure. (i.e. like a C struct). In tunes, the system would be
deciding what the physical layout of that data-structure is. It might
just pick it arbitrarily, or there might be an optimizer component,
which watches access to the data-structure and rearranges for optimum
cache performance after you use it for a while. 

You can see this kind of technology today, Sun has a compiler which
performs similar operations for C/C++ code. You compile the code in a
'profile form', then you run it for a while and the profile code
records access patterns. Then you run it again and it rearranges the
code and data to avoid cache misses. Most of what it does is align
code loops, and arrange structure members.

However, with Sun's compiler, the developer has to do it once, and the
code is going to get optimized for the cases that he runs. In Tunes,
this kind of optimization can be performed at any time, because the
information to regenerate machine instructions is always available. In
this way, the system will be able to perform target specific
optimization based on the exact environment the code runs in.

In my paragraph above, I say that people should be able to specify
low-level information as much as they want. By this, I mean that a
developer/user, should be able to "be" this optimization agent I talk
about above. A developer should be able to say "screw you and your
silly optimizer, I know best", and they should be able to specify a
physical layout of the structure in memory. 

However, I also say that "...never require that low-level optimization
information to run". This means that we could throw out the
developer's hints which say what the structure layout should be, and
the program should still be functional. In fact, we might choose to
trust our in-system optimizer more than developers.

Finally, I say that:

   "Cases which deal with low-level issues like hardware
    access do not fall into this category because the low-level issues
    are part of the required functional description."

Consider a driver written in C. It's common to use a 'struct' to
explain the layout of a bank of memory mapped registers on a hardware
device. In this case, the struct is not just layed out that way for
convinence, it must be that way, because it explains the layout of a
hardware implementation. This relates to the above, because if I, as a
developer, explain that a particular piece of hardware has a
particular memory mapped register orginization, then that information
about the 'layout of the aggregate type in memory' is not optimization
inforation. It's a functional description of the hardware. We can't
change it with some optimization engine, because it deals with real
hardware which isn't going to change. 

> >>> As reflection changes the way the system is structured, without any
> >>> help from the user, reflection could confuse for the user by
> >>> changing the way *the user* sees the system.
> >
> >Certainly. I'd like to rephrase what you said. The more indirection
> >which is possible in a system, the more confusing it is going to
> >be. Directness is much easier to understand. However, directness is
> >also much harder to maintain. I personally think the challenge of
> >presenting a system like Tunes is to find a way to represent a
> >reflective system in a direct way.
> 
> That's not quite what I meant. What if a user learns how his system look,
> but abra kadabra, reflection does its job. Where is now that report? "In a
> better place" the reflection algorithm would say, but we all know how
> stupid algorithms are. Remember that newbies have a hard time just
> understanding an ordinary directory structure.

No, you misunderstand. The idea is for us to keep enough information
around about all active objects such that we can provide the 'view of
the system' that any given object needs, even if it's not the same
view as some other object (or the user) needs. We would not be using
this capability to pull the sheets over the user's eyes. 

I'll try to provide another concrete example. Keep in mind that I'm
making analogies with existing systems to try and bridge the gap, but
the direct implementation of these ideas in a new system is slightly
different.

Under linux, you can use dosemu or wine to run programs which are
foreign to the Linux environment. However, programs which run inside
one of these environments are mostly 'trapped' in that environment. In
a proper reflective system, there is enough information available
about each component, no matter where it lives, to provide a mapping
into another 'foreign view'. For example, dosemu allows you to use
proper 'linux serial ports' with a dos app by emulating the register
interface that a dos app sees. It does not matter what kind of serial
port you are letting the dos app use, it does not have to be some
16450/16550 serial chip, but no matter what, the dos app goes through
the 16550 register interface. Core to the idea of reflection is that
we should be able to do this for all items in all 'views'. If we have
a spell checker, we should be able to easily export that spell checker
into the 'dosemu' environment and the 'wine' environment, and it
shouldn't matter where it runs. If the user chooses to make 'wine' his
primary view, that's fine. We don't have a problem with that. There is
no 'root level' of the system, instead everything is related to
everything else in a horizontal fashion.

To bring the "report" you talk about above into the picture. We're not
talking about the idea of moving the physical location of the
report. Instead, we're talking about the idea of having the system
actually _understand_ the form of the report. Imagine that we wrote
the report in Word, and saved it in word ".doc" format. The system
should know about .doc format, and what the document contains. We
might have a ".doc to .txt" converter. If I ask the system to search
all the reports I've ever written, it will have the smarts to: 

  1- know that the stuff is in .doc format
  2- convert it into .txt to extract text-search type information
  3- transparently build an index of this type of information if I do
     searches often. Including the ability to automatically update
     the index anytime I change a document, transparently, with no 
     changes to word processor software, or filesystem software.
     no custom hacks.
  4- be able to provide me with the search results, and show me the
     documents.

-- 
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + jeske@chat.net