On design processes

[Matthew Tuck]

Hans-Dieter Dreier wrote:

> It would really be good if you could manage to get a copy of SqlWindows 
> or Centura Builder. Apparently the benefits of editing in an outline
> tree are hard to communicate if you never tried it yourself. It's quite
> different; all those issues like open/closed form, syntax of
> declarations and the like simply vanish because the outline editor
> automatically takes care of that.

I gather they're commercial.  I think I can figure most of the dynamics
of tree editing.  I especially like they way they easily adapt to
extensible parse trees, whereas in textual views, you would need to
define a shorthand.

> Which ones do you mean? I haven't found any that describe the technical
> aspects of managing an open source project.

"Harvesting the Noosphere" talked a bit about the benevolent dictator,
etc., if that's not what you want, can you be more specific?
 
> > It could be useful to start writing some summaries of what has been
> > discussed?
 
> Good idea.

Would someone like to volunteer?  At the moment I've got so much I need
to do on this project, I can't keep taking on stuff faster than I do it.

> Sure. The slim binaries proposal may be something for the future. I
> also thought about starting with an interpreter. It can be turned into
> a compiler quite simply by replacing the "output" functions. Actually,
> it should be designed for easy exchange of the output part from the
> very beginning.

I was talking more about just writing our interpreter to operate on a
program tree, that way we don't need to convert to a low level.

>> I don't know that using a low-level syntax is a good idea.  What I
>> would do is write some stuff in high-level code and embed the
>> implementation in the interpreter for initial implementation.
> If you start with some low-level syntax, that will be much quicker than
> a high-level one. If the parser is done right, there will be no problem
> extending it. Example: Replace the initial symbol lookup function
> (which would only handle a single unstructured namespace) by a version
> that looks into the operand stack of the parser first, to determine the
> namespace to be used, as soon as functions and namespaces are supported 
> by the syntax.

I might be confused by what you mean by "low-level syntax", I was
thinking of a bit above assembly language, you're probably talking about
a primitive HLL.  This shouldn't be a problem.

> I'd like to use a bottom-up, LR(1) parser, table-driven, with some
> "hooks" that allow for easy treatment of special cases (such as control
> structures and parentheses). But the C++ code should be independent of
> the syntax used, except for those "hooks".

What would these hooks be used for?

> In the compiler, the behaviour which is needed for declaration
> modifiers like "private" could by put into objects, so that even these
> things could be "plugged in" with little changes to the compiler's
> innards.

I'm not sure what you're talking out here.  Example?

>> As a consequence if the next token is a "procedure", it says "while"
>> expected, whereas it really expected any statement.
> Actually, if you take that to an extreme, it would just point to the
> start of your module and say: "Not a valid program" (because this is
> the outermost production rule). It's not so easy to determine at which
> level to produce an error message. Bottom-up parsers generally are
> better on this.

No, it should really talk about the most specific language element (ie
makes sense to the programmer) that encompasses all the possible
situations.  For example, saying "while expected" is wrong, as a for
statement would have been equally OK, but if you say "statement
expected", there is nothing else it could expect.  But certainly, this
is only an issue with recursive descent.

> Parsing, yes. If it has to be done interactively (the user changes
> something in mid-program). But it would only be complicated if input
> were free-text, not if input is already done into an outline. Then the
> editor will always know what sort of item you are editing at some
> particular place.

Yes, that's true, but I think the problems are in modifying the
program.  Some modifications can occur in ways which an intelligent
editor might not allow.  Cut-and-paste allows all this quite easily as
it is not restrictive.  So we should try and determine what these ways
are, and make them easier.

For example, I have a statement A.  I want to put it inside an if.  But
if I just type "if", then I'm likely to get something like:

if cond then

end if
A

which is not what I want.  But an intelligent editor could make up for
this by a drag and drop facility.

> Just processing an input file to produce an outline for the editor is
> relatively easy because there is nothing interactive to it. In fact, we
> could use the compiler with a special syntax fitted for that purpose.
> 
> To me, it doesn't seem so complicated, although I have to admit that I
> haven't figured out everything yet. But I've been working with such a
> tool for a long time (SqlWindows, which I already mentioned). And, boy,
> what an opportunity to do something which is not just main stream, but
> better!

Here's a hint: how many of those editors are around?  Some, not many. 
It takes effort.

> You mean you could replace the output section of bison to generate a
> syntax table for a bottom-up-parser?

There isn't really an "output section" as such.  When a rule is
recognised, some C code is executed, which could be anything.

> > To produce code at runtime using bison we would need to be able to
> > interface with C.
> 
> Why that? It only makes sense to me if it generates C source, which I
> wouldn't like.

Yes it does, it has to especially since it has embedded C.  Since we're
talking about writing it in C++ anyway, I don't see the problem.  Once
bootstrapped, we would abandon it, but that's not a major problem since
it would not likely work in an intelligent editor environment anyway.

> In a truly integrated environment like that which I think of, there is
> no "back end", because part of the source code may need to be executed
> by the compiler and/or editor while development is in progress. Such a
> thing would not be possible if the "back end" could not host our
> development system in a sensible way, and be changed while running.

> Maybe output to JVM could be done as a separate run after development
> has been completed, but debugging or any other form of interaction with
> the source code while the program was running would be almost
> impossible. That would be a step backwards compared to current tools.

There's no problem with having a debug mode and a standalone mode. 
Although you could complain that the debug mode would be slower since it
uses uncompiler code, there are techniques to improve it, such as caches
of compiled methods.  The SELF project actually does this sort of thing
so they can get compiler method performance while still maintaining
flexibility (SELF is a delegation inheritance language, basically
meaning that methods can be updated rather than being constant).

-- 
     Matthew Tuck - Software Developer & All-Round Nice Guy
                              ***
       Check out the Ultra programming language project!
              http://www.box.net.au/~matty/ultra/