Intent

[btanksley@hifn.com]

From: Jason Marshall [mailto:jmarsh@serv.net]

>Recently, btanksley@hifn.com wrote:

>> And who is this "everyone else" with whom you wish to associate
>> yourself and disassociate Brent?

>Some days I wonder.

Me too.  You know, that would have been a lot cooler as a quote if I hadn't
spent most of the rest of my reply sounding like I had a terrible headache
and blamed you personally.  I didn't have a headache, so I don't have that
excuse; thank you for tolerating my uncouth abruptness.  I apologise, and
will attempt to do better.

>> >I feel the need to reiterate your comments on intent.  This
>> >has been the foundation on which I have been trying to build a
>> >language of my own. 'Metadata' sounds more flowery, but down
>> >here in the trenches, 'intent', gets nods of comprehension much
>> >earlier in the conversation, in my limited experience.

>> Just because you personally
>> don't understand the theory and practice of combinators doesn't mean
>> that combinators don't reveal intent.

>What I 'know' is that other than Fare, I'm not picking up on any
>meaningful
>discussion of metadata going on in this list, which is the main reason
>I don't participate more in the discussions.  I can see plenty of
>evidence
>of incidental or inferential metadata, but I can get that out of any
>language, no matter how advanced, or how obtuse.  Talk of 
>whiz-bang ways of implementing yet-another-language-with-some-
>induction-logic is, while interesting, old hat, and does nothing for
>readability before I've had my morning coffee, which is exactly when 
>I'm going to botch the code.

Grin.  I certainly understand that.

>No one talks of formalized metadata, which is what I am after, and what
>I thought/think Fare is after.  Why?

Because you don't talk about it?  I mean, if it's interesting, talk about
it!  Don't dis my language because I don't talk about what you're interested
in.

Okay, the topic is a lot clearer to me.  You want metadata, and you want it
embedded in the language.  IMO, it would be best as an optional thing, but I
don't mind it being required.  You don't actually object to concatenative
languages; you simply want a language with metadata provided.  (You should
have _said_ that, instead of just stating that concatenative languages were
inferior.)

Okay, try this on for size: a concatenative language with two types of
annotation.  The first, in the form ( x -- y ), appears at the beginning of
the word definition and is a type description (x and y may be any number of
type names, seperated by spaces).  The last item in x is the type of the top
of the stack (as with traditional Forth stack comments).  Types are handled
entirely statically, and are fully polymorphic (including return values).

The second annotation, in the form ( x ), appears anytime within the
definition of a word, and is a stack naming (x may be any number of one-word
names).  Names for stack elements may be traced at compile time, and must be
consistent within a given definition.

example:
 3 2 ( adder addend ) swap ( addend adder ) + ( sum )
or
 ( number[myType] number[myType] -- number[myType] ) 
    ( adder addend ) swap ( addend adder ) + ( sum )
or
 3 2 ( adder addend ) swap ( adder ) + ( sum )

The last one of these demonstrates that the stack names don't have to name
everything on the stack.

>> The way for a programmer to reveal intent in a concatenative language
>> is to
>> choose appropriate factorings (and function names) for his problem.
>> The way
>> to reveal intent in your language is to choose appropriate parameter
>> names
>> for everyone who's ever going to use your function in their problems.

>This does zero to address unintentional misordering of function
>parameters on the part of client code.

Correct.  As parameter naming does zero to address people rewriting
functions which already exist, but are embedded inside poorly factored code.

>> Of the two of us, who's more likely to reveal the most intent?

>By sheer volume of citations of intent?  Your users. You wrote the code
>once, and now everyone else is having to live with your mistakes (and
>you are human, so don't imagine you haven't made any, because you have).

You missed my point.  Take two people, you and me, writing code in our
seperate languages: yours with named parameters and mine being
concatenative.  Which OF THOSE TWO PEOPLE will reveal more intent by means
of their language's mechanisms which are designed for that purpose?

 - The concatenative language is designed for quick, easy factoring and
refactoring, allowing the author to create many functions which work
together to solve his problem, and leave many component functions which may
be reused.
 - The parameterized language is designed to allow the programmer to name
his parameters, and communicate those names to later reusers.

Both languages have ways of communicating internal intent to the external
world, as well as structuring to preserve internal intent.  Do you really
think that one of these MUST be more revealing than the other?

>> >Not only is the code more readable by a human

[some editing]
>> you simply aren't familiar with concatenative languages.

>I am familliar with the classes of mistakes programmers in team an API
>settings make, in real development environments.  I am 
>familiar with the problems
>that crop up when one wants to modify an interface to their code, and
>there arises an argument number and type collision with an existing 
>part of the interface that can not be mitigated in any more useful 
>fashion than by sorting the arguments in some arbitrarily different 
>order so as to make them unique, or by inferring/overspecifying 
>parameters, or by giving up and trying something else.

>I understand that Kyle has grasped this, and that you see it as a
>non-issue.

No, I don't see it as a non-issue -- I simply see many other issues to deal
with, which add up to readability.

Note that Forth traditionally has no type checking at all (although my
suggestion for type checking, above, is based on a Forth type checker), and
even large projects in Forth don't run into that particular problem very
much -- almost certainly because it's so painful to use large numbers of
parameters in Forth.

>> >If you for instance declare
>> >that you intend to visit every item in a list or container, and
>> >perform
>> >an action on it, this is easier for the optimizer to sort 
>> >out than if
>> >you
>> >write 50 different minor variations on an iterator idiom by hand,
>> >some
>> >with unintentional out of bounds bugs that you didn't bump into
>> >(yet).
>> >And it's easier for the compiler to parrallelize it on the target
>> >hardware if the facilities are available (MPP, vector math, etc).

>> This is utterly true.  And completely and totally irrelevant.

>Not in the least.  People want flexibility in a language in case
>the environment isn't subtle enough for their tastes.  Then they turn
>around and complain because everyone comes up with a 'clever' synonym
>for the code they would have written, sometimes correctly, sometimes
>totally wrong, but either way, totally greek to their peers.  Formalism
>takes the edge off such concerns over abuse of power.  Here is your
>rope sir, but you have to justify any requests to tie it into a knot.

The odd thing is that I think I understand what you're saying (both times
you've said it), and I'm just about certain I agree entirely with it, but I
_still_ don't see what on earth it has to do with our discussion.  What does
this have to do with named parameters versus implicit parameter passing?

>> >to painful mechanisms of inter-language communication. All the
>> >code will exist within a single virtual machine, with a common
>> >calling convention where possible, dynamically generated
>> >wrappers where not.

>> So, in the final analysis, you have a virtual machine.   Our 
>> languages are not so different

>Being a virtual machine-based language provides little similarity.

Forgive me; I was thinking ahead of my typing.  My thought process was
analysing our runtime and compile time systems, and I was trying to figure
out how one would be superior to the other in optimization; beyond that, I
have no idea what I was thinking.  :-)

>> -- except that you have a huge translation layer, and I have a tiny
>>    one.

>I have a segmented translation layer, which is an important 
>distinction.

I have three low-level segments:

 - the virtual machine-->native processor layer (which is very thin; it's
essentially only a specification of which registers hold which stacks, and
what registers are preserved on task switches, and how I implement the
processor's native instruction set as though all of the instructions were
stack based -- for example, "ADD reg, reg" becomes "POP BX/ADD AX, BX" (or
something like that).

 - the quotation system (which defines a 'function structure' and provides
ways of accessing its fields and compiling natively the function it
indicates; the function structures contain properties such as profiling
data, frequency of calls, and so on; this layer is also written in processor
dependant code).

 - the main system (which contains, amongst other things, several different
optimizers and code generators, all parameterised based on values in the
layers below, as well as a profiler to drive the use of the optimizers).

>You have a tiny translation layer, or you're falling victim to the
>incomplete implementation fallacy?

Well, there are two systems like this that I know of which have been in
commercial use for five to ten years now.  Add a third, if you count Elate
(which uses a similar design but with a slightly different emphasis on the
underpinnings).

>How big is your standard library going to be?

If it includes everything TUNES wants, infinitely large.  However, I expect
it to fit on a floppy.

>What fraction of that is your minimal interpreter?

0.  Okay, seriously though, it'll be very small; perhaps we'll be able to
get it down to half a K of processor-specific executable code, plus another
512 bytes of parameters.  Add in the boot sector, and that's all the
nonportable code.

>How big is a mature implementation of your
>interpreter going to be?

Hard to say, because although the VM I discussed above contains an
interpreter, there's also languages above that; for example, there's one
which contains the same basic features of the VM language, but with the
addition of complex data structures and garbage collection.

The most basic interpreter, though, including all appropriate libraries and
data, will total about 256K -- at any rate, it WILL fit on a floppy disk,
because I Have Declared It ;-).

>How many things is it going to do internally,
>how many will be pushed into required standard functionality to perform
>the same task?

I don't know what you mean by "standard" there, but anything done by the VM
will be the Right Way to do things.  Any code which re-impliments a feature
of the VM is broken :-).

>Concerns over how big an 'unabridged' edition of the machine would be
>have some merit, but not as much as you're attributing to the issue.

Okay, here's the concern I'm expressing: contrary to your statement, I
believe that my optimiser, with or without metainformation, will produce
compact, efficient code.  I base this opinion on systems similar to mine,
such as Quartus Forth and 4THCOMP.

>There is no requirement that a compiler issue, or the 
>interpreter honor,
>any optional data.  If the optional data represents 80% of the
>dictionary,
>and 90% of the implementation complexity, who really cares?  All I've
>done
>is provide a formal vessel into which optional tools can place their
>data, in an attempt to avoid ad hoc formats.  What does that hurt?
>Nothing, according to the industry examples I can find.

Our discussion used to be about a language which _required_ parameter names
to be included in every call.  I don't know what you're discussing now, but
it's definitely not the same thing.  Could you provide an URL, or give a
description of what you're talking about?

>Different environments will contain a different subset of the content
>handlers for the language.  My deployment environments won't care about
>debug data.  My dev environment will.  My palm pilot doesn't care about
>MPP or high precision math or vector math, or advanced speed-for-space
>tradeoffs.  My workstation doesn't care about space-for-speed 
>tradeoffs,
>but when I'm downloading a new multiplayer game off the net, I'd much
>appreciate if it was usable the first time I ran it.  No VM I know of
>is giving me that.

Check out Elate, formerly known as Tao OS, at
http://www.tao.co.uk/2/tao/index.html.  Very cool.  Very similar to what Sam
and I are building, and does exactly what you say.

>-Jason

-Billy