LISP for Concurrent Systems

[Francois-Rene Rideau]

tfb@apocalypse.OCF.Berkeley.EDU (Thomas F. Burdick) writes on comp.lang.lisp:

> Francois-Rene Rideau <fare+NOSPAM@tunes.org> writes:
>
>> they also have good features that Lisp never had: [...]
>> support for building robust massively concurrent systems for Erlang.
>
> I certainly never had any experience with Lisp*, but I'd be surprised
> if it failed here

Maybe you mean *LISP, the language for the Connection Machine;
but AFAIK, it wasn't exactly a LISP,
just a language metaprogrammed from LISP.
So it was used to build massively concurrent systems,
but was very idiosyncratic with the CM, kind of SIMD,
and didn't survive to the demise of Thinking Machines.
I don't think it was designed for robustness, either.

Other LISPs with concurrency (threads) are kind of nice,
but not really built for massively concurrent systems:
the global garbage-collected shared-memory model of LISP
strongly limits the scalability such systems, and
their robustness when faced with fault-containment problems.
Also, no concurrent LISP extension correctly handles
problems of synchronization of interrupted threads with
application-defined invariants - see for instance the problems
that Joe Marshall had with interrupted transactions.
(LISP here is not worse than Java,
but JOCaml and Erlang can do much better).

Of course, it is possible to build distributed systems
atop of LISP (just like you can atop any language), but
then (just like you would have in any such setting), it means
that you have a fight a multi-level programming system 
that imposes arbitrary constraints on your programming style
and forces you to make arbitrary early low-level implementation
choice as to which problem will be treated at which level.
LISP's metaprogramming abilities can help you factor a lot,
so you will be better with LISP than with anything else,
but if you take this approach seriously, you will but end up
reimplementing Erlang or JOCaml in LISP, and the idiosynchrasies
of LISP systems (designed around a global namespace) will end up
getting in the way, whether you try to emulate them (expensive
runtime distributed coherence enforcement) or avoid them
(difficult to trace inconsistent behaviour when they are mistakenly used).

Erlang, on the other hand, is a generic tool for building robustp
concurrent systems. It does not depend on the idiosynchrasies of
any particular hardware architecture. Its model of internally pure
communicating processes easily scales to building large distributed systems,
while allowing for real-time behaviour (soft-real-time in practice;
but nothing in theory prevents a hard-real-time implementation if needed).
and provides a framework for fault-containment as well
as for asynchronous interruption handling, and more.
It is actually used in mission-critical telecom switch systems, 
in services using internet-standard protocols, etc.

I don't mean that it would be impossible to design a LISP so as to be
as good for concurrent programming as Erlang is. But it would require
deep changes to the language and its implementations, and the end-result
wouldn't quite be like Common Lisp. For instance, instead of global
namespaces and am implicit global object heap, it might have something
like first-class name-spaces and first-class object-spaces;
the behaviour of primitives of the language in face of concurrency
would have to be clarified -- maybe by introducing a linear type system.

Yours freely,

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[  TUNES project for a Free Reflective Computing System  | http://tunes.org  ]
How would the certainty of the near end of the world affect Ethics?