[stack] Digest Number 24

[Massimo Dentico]

Mark van Gulik on Concatenative mailing-list wrote:
> [.. discussing about OLAP ..]
> I myself have worked on a smaller scale project with an Objectivity database
> containing information for about 50 million customers (about 800 Gig, if I
> recall).  We did most of the development in *Smalltalk*.  No regrets -- it
> was a *lot* of fun, and a complete success.
> 
> Languages like Forth would be completely inappropriate for such constructs.
> The amount of semantic information in the models easily exceeds Forth's
> complexity limits (in my estimation).  *Parts* of the database could be
> dealt with by Forth, but its entirety is too intrinsically complex (e.g.,
> all the indexing structures, keyed by a new kind of spherical coordinate
> system and the spectra, must allow well-choreographed storage clustering on
> disk and tape without overcomplicating the high-level code requiring the
> access).

[Well, I'm not here to sell Forth but to highlight its qualities because
I'm interested about the design of a better concatenative language of the kind
that Bill Tanksley has illustrated previously. I'm convinced that it needs
improvement, in particular about its safety, thus my interest in a hypothetical
type system that fits well in the general framework of Forth: IMHO the approach
"the language is the type system", that you also seem to embrace with Avail,
looks very promising.]

You miss completely my point. Forth is a meta-tool, this is its point of
strength. You know better than me (you say that you have written one) that
Forth is extensible, at the syntactic/semantic level thanks to its
open interpreter/compiler, in whatever direction: toward low level (direct
access to hardware, embedding of assemblers) and toward high level (domain
or even application specific languages). Smalltalk, to a certain extent,
shares these characteristics of uniformity and extensibility.

Numerous Object Oriented extensions, with very different characteristics,
and a Garbage Collector are available in Forth, written in it. I have seen
also subsets of other programming languages embedded in it, like Lisp and
Prolog or a tagging language like HTML.

All that in a coherent, uniform framework, *without* leaving the language
(or better, the environment) and on-the-fly (load, use and then discard, the
extensions you need). Note that this is different in comparison to statically
compiled modules, it's a dynamic precess: every time you load a screen (a little
chunk of code) you interpret/compile it perhaps in a different context; this
opens the door to adaptive interpretation/compilation/execution which is specific
to actual context, so any sort of specialization/optimization which exploit
this dynamic information is potentially possible (depending of what make sense).
Note even that only most frequent executed parts of the program need to be
heavily optimized and these parts could vary during the execution itself or
between different executions.

Traditional batch compilers don't know much about the dynamic execution context
of the code, they compile alla and blindly apply every possible optimization
technique statically. This means long compilation times and calls for separate
module compilation. But separate compilation introduces complexity and constrains
the semantics of the language: every feature that is not statically known,
at compile time *and* at the module boundary, is discarded or restricted (for
example, separate compilation in Standard ML, which have a powerful type system,
is quite difficult and limits the language). Obviously, moving this to link-time
or load-time (of compiled code, a.k.a. DLL) complicates the process and don't
solve anything: the problem here is too much pre-compilation. Forth avoids this
issue entirely and Smalltalk too (I know that, for some implementation, is possible
to compile to C/C++ but usally this feature is used at the end of a development
cycle).

> Clearly these projects need the cooperation of many software developers.
> What kind of tools are available for Forth in this regard?  As far as I'm
> aware, Forth doesn't even have namespaces.  It also lacks (if memory serves)
> any ability to specify interfaces separately from implementations.  High
> complexity parallel development is simply not possible (at any price) in
> such an environment.
> 
> Your mileage may vary.

Experienced Forth programmers assert that their productivity is greater of
an order of magitude when they use Forth rather than other programming
languages. I think that, thanks to the characteristics discussed above, they
tend and succeed to avoid complexity, thus the necessity of big teams of
programmers and complex management.

But forthers are no more alone in their commitment toward simplicty.
"Do the simplest thing that could possibly work" [1] is also the motto of
Extreme Programming [2], a lightweight, humanistic discipline of software
development, grown in a part of the Smalltalk community:

--------------------------------------------------------------------------------
[...]

We find that we have to remind ourselves of this rule continually. Developers
like to develop, and most of us have years of experience in creating a "general
solution" to whatever we’re asked for. Real progress against the real problem
is maximized if we just work on what the problem really is.

[...]
--------------------------------------------------------------------------------


This is surprisingly similar to what Charles Moore said [3]:

--------------------------------------------------------------------------------
[...]

Don't Complexify

Simplify the problem you've got or rather don't complexify it. I've done it
myself, it's fun to do. You have a boring problem and hiding behind it is a much
more interesting problem. So you code the more interesting problem and the one
you've got is a subset of it and it falls out trivial. But of course you wrote
ten times as much code as you needed to solve the problem that you actually had.

[...]
--------------------------------------------------------------------------------


This seems to contradict what I have written in another e-mail, my concern with
generality, but it's not so: abstraction, taking out of context, is a process that
necessarily needs more concrete "objects" by which to abstract. Infact in Forth
and Smalltalk communities (and probably other, like Self community) this process
of abtraction is called factoring or refactoring and moves from more specific/
concrete pattern of code/data to more abstract one, in an utilitarian way because
only what is really general, in the actual context, is generalized. This contributes
to the compactness and then tractability of code (no redundances). I think that
generalization and specialization are not necessarily in contrast.


References

[1] eXtreme Programming web site
    - http://xprogramming.com/

[2] "Do the simplest thing that could possibly work"
    - http://www.xprogramming.com/Practices/PracSimplest.html

[3] "1x Forth"
    - http://www.ultratechnology.com/1xforth.htm


-- 
Massimo Dentico