POS, OOFS, CL v Scheme, etc.

[Marc Wachowitz]

hbaker@netcom.com (Henry G. Baker) wrote:
> > Neither threads nor reflective capabilities would imply non-conformance
> > (of the implementation) with the standard, as far as I can tell; just
> > make sure extensions live in a different package, or extend a standard
> > feature in an accepted way (e.g. a function which understands a larger
> > variety of arguments and does interesting things with non-standard types).
> 
> Surely you're dreaming if you think that adding threads to CL can be
> done in a 'conforming' way by simply adding another package.

Of course I would be dreaming if that was what I thought; furtunately,
I don't. My above comment was not at all about implementing or designing
a thread system for Common Lisp, which surely gives some "interesting"
questions. However, conformance to the standard does only mean that you
have to make your extensions available in a way which doesn't damage
the standard model, i.e. how to keep the language lawyers happy. They
don't demand anything about how you should protect users of extensions
from throwing their system into pieces due to lacking sychronization,
or what you should use as initial value in a new thread for variables
which have been dynamically bound at the time of thread creation. Such
stuff is mostly off limits for the standard. See the ANSI Common Lisp
specification, sections 1.5 (Conformance) and 1.6 (Language Extensions);
the following is from the latter:

"A program can have extensions, provided they do not alter the
 behaviour of conforming code and provided they are not explicitly
 prohibited by this standard."

> If you
> thought that adding immutable cons cells required going through the
> entire CL book and deciding for each function about what kinds of cons
> cells it would consume and emit, you have the same problem in spades
> with something like threads.

Certainly, though the situation is somewhat easier for _conformance_
(not necessarily implementation), since in the case of immutable data,
you'll probably want to get benefits even for programs which aren't
aware of this nice feature of your implementation, whereas as conforming
program, which doesn't know anything about threads, won't be hurt much
by the existence of threads _facilities_ in your system (it may be hurt
by another thread going wild, of course).

> For example, the LispM used 'stack groups' for its threads, and these
> turned out to be pretty 'heavy weight', due to the requirement to bind
> and rebind a bunch of shallow-bound 'special' variables.  I'm not sure
> that this is such a hot idea in 1997.  Therefore, since CL was based
> substantially on the LispM model, I'm not so sure that CL itself can
> survive without change into 1997 LispOS.

We probably agree about the dubious benefit of some features for advanced
environments. I guess even many of the people who wrote the standard would
agree about that, but still put those in for various non-purely-technical
reasons (preserving existing source code and implementations, keeping their
customers happy, lack of time to get something better together, political
games among implementors with different strategies for what's beyond the
current specification, and so on). The X3J13 issues in the HyperSpec hint
at some of those concerns.

For the specific case of dynamic binding, I think these could be made
sufficiently efficient, given the fact that the special variables in
ANSI Common Lisp are mostly needed in a context where a few additional
indirections won't hurt much, and modern code won't use much dynamic
binding for other cases (at least I hope so). Global variables which
merely happen to be special due to DEFVAR and friends, but are never
dynamically bound, shouldn't be a problem. Note that nothing forces an
implementation to allocate values as part of SYMBOL objects, as long
as the semantic association is available. Thus you could favour the
preferred cases, making e.g. the first dynamic binding of a variable
more expensive - even by rewriting compiled code which so far hoped it
would stay global, allocating the slot now on a special set of memory
pages which are allocated separately for each thread, and can easily
be remapped on thread switch, or simply accessed indirectly via one
pointer which is changed on thread swich. No, that's not elegant, but
it should work, and for high-performance CLOS you'd probably need a
few facilities for dynamic code rewriting anyway. (Sigh, don't get me
started to talk about the misfeatures of CLOS.)

-- Marc Wachowitz <mw@ipx2.rz.uni-mannheim.de>