Modernizing Common Lisp: Recommended Extensions

Francois-Rene Rideau
Sat, 10 Feb 2001 23:07:07 +0100

Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

Rainer Joswig sent to me this interesting summary of directions in which
to extend Common LISP. Together with, e.g. Henry Baker's Critique of DIN
Kernel LISP, it gives good hints as to what are the challenges in designing
a modern programming language, independently from the evaluation paradigm...

[ François-René ÐVB Rideau | Reflection&Cybernethics | ]
[  TUNES project for a Free Reflective Computing System  |  ]
Individualists unite!

Content-Type: message/rfc822
Content-Disposition: inline

Received: from samaris (localhost) [] (root)
	by Samaris with esmtp (Exim 3.20 #1 (Debian))
	id 14J55D-0002Iq-00; Thu, 18 Jan 2001 03:45:51 +0100
Received: from []
	by localhost with IMAP (fetchmail-5.5.1)
	for fare@localhost (single-drop); Thu, 18 Jan 2001 03:45:51 +0100 (CET)
Received: from ( [])
	by (Postfix) with SMTP id E81E33E845
	for <fare@TUNES.ORG>; Wed, 17 Jan 2001 18:29:24 -0800 (PST)
Received: from by via INTERNET with SMTP id 15196; 18 Jan 2001 03:17:43+0100
Mime-Version: 1.0
X-Sender: joswig@
Message-Id: <p0433010db68c02266ad8@[]>
Date: Thu, 18 Jan 2001 03:16:32 +0100
From: Rainer Joswig <>
Subject: Modernizing Common Lisp: Recommended Extensions
Content-Type: text/plain; charset="us-ascii"

you might be interested in this old mail - if you don't have it:

Date: Wed, 7 Jul 1999 16:39:52 -0400
From: "John C. Mallery" <>
Subject: Modernizing Common Lisp: Recommended Extensions

	   Modernizing Common Lisp: Recommended Extensions

	  John C. Mallery, Howard E. Shrobe, Robert Laddaga,
	    Thomas F. Knight, Kalman Reti, Paul Robertson,
      Andrew J. Blumberg, Susan Felshin, and Patrick H. Winston

		  Artificial Intelligence Laboratory
		Massachusetts Institute of Technology

			     July 7, 1999

The document suggests enhancements to Common Lisp that modernize the
language in light of developments since that last ANSI standardization
round. These elements can be incorporated as changes to the:

	* Language

	* Suggested "yellow pages" that implementations or
	repositories may supply as libraries

	* Experimental components intended to explore areas not yet
	stable enough for inclusion within the language proper or the
	suggested libraries.

The presentation is as an outline of topics in order to keep the
document short and facilitate discussion. It is intended that separate
two-page outlines will be developed by proponents for each of the
substantial topics for more careful consideration. If the community
decides to move forward on a specific recommendation, we expect
proponents to develop a detailed proposal for critical review, a
working implementation, and ultimately, a standards specification.

The most important recommendation with the most leverage is the
recommendation to develop a PORTABLE BASE LANGUAGE FOR COMMON LISP.
This strategic move provides:

	* An opportunity to reengineer the core language;
	* A minimal base to port to new platforms;
	* Opportunities for new hardware and compiler efforts
	(base-to-platform & portable-to-base);
	* Increased sharing of higher-level code.

How do we identify the evolutionary trajectory for Common Lisp?  One
approach is to identify what features a modern programming language
requires by examining at other recent languages (eg, Java, Dylan) and
selecting features that merit refinement and generalization. Another
approach analyzes the deficits of contemporary programming languages
and environments and proposes new facilities to address the identified

These recommendations use both approaches but are tempered by
recognition of the constraints of previous standardization rounds and
the need to maintain continuity. Although some of the suggestions
below may seem beyond the conventional purview of a standards body, we
believe that language leadership is not possible without reference to
the kinds of facilities that a modern programming environment must
offer.  Feedback from independent efforts to develop libraries and
experimental components will provide a critical input into the
language evolution effort.

The entries below adhere to the following style for clarity of

	 1. Title: 		A telegraphic title.
	 2. Abstract: 		Three sentences describing the feature.
	 3. Category:		Extension, Library, Experimental
	 4. Impact:		Low, Medium, High
	 5. Examples: 		Are examples available in LISP or other languages?
	 6. Implementation:	Are existing implementations available?
	 7. Longevity: 		How long will this feature be relevant?
	 8. Difficulty: 	Easy, moderate, hard.
	 9. Benefits: 		What specific benefits does the feature provide?
	10. Approach:		How can this feature be incorporated?


A. Core Language Extensions

	* Package System: CL should provide relative package names and
	associated operators. This feature is important for smooth
	evolution of large Lisp systems. Language, Examples,
	Implementations, Easy, Low.  SF, JM, KR.

	* Declarations: Add the ability to declare returned values and
	their types for functions and methods. Provide a standard
	arglist function to return the arguments and values of a
	function. Language, Examples, Implementation, Easy, Low.  SF,
	JM, KR.

	* Modern Character Sets: CL should support Unicode, which is
	becoming the standard character set for multi-language
	applications. Extension, Examples, Implementations, Moderate.

	* Unit Awareness: Provide extensions to numeric operations
	that support conservation and coercion of units. Extension,
	low Impact, Easy. TK, JM

	* APL Array Operations: Provide a standard array package
	analogous to the one found in APL but building on top of
	current CL array substrate. Implementations should provide
	highly optimized specializations for functions. Library, Low
	Impact, Examples, Implementations, Moderate. TK, JM

B. Networking Components

	* Uniform Process Model: A standard API for working with
	processes (aka threads) should be developed and provided as a
	standard part of CL. It should include locks,
	store-conditional, and timers.  The design should allow
	applications to exploit multiple processor hardware.
	Extension, High Impact, Examples, Implementations, Moderate.

	* Uniform Foreign Function Model: A single, standard API
	should be provided by CL for calling foreign functions. The
	facility should provide some basic operators for marshalling
	and unmarshalling arguments that can be used by applications
	to support FF over other transports (eg HTTP) in either
	direction. Extension, High Impact, Examples, Implementations,
	Moderate. JM, HES

	* Standard Streams: CL should provide a standard API for
	streams in order to allow stream code to be portable. The grey
	stream proposal from CLIM is a good starting point, however,
	the actual standard should consider the requirements of
	networked applications, graphics display, and streamed
	video/audio. Streams should define generic buffer-in-place
	operations and provide a set of generic "internal" operations
	which enable, for example, encapsulation or metering during
	OPEN or other standard stream operations.  Extension, High
	Impact, Examples, Implementations, Moderate. JM, HES, KR

	* Networking Substrate: Incorporate a standard API for
	performing generic networking operations. Modern programming
	languages need to be network-aware. Java provides an example.
	A networking substrate assumes a thread model, streams, and
	locking mechanisms. Crucially, the networking model should
	provide a hierarchy of conditions relevant for networked
	applications. The networking API should be generic and capable
	of spanning multiple different networking standards, e.g.
	Internet, Chaos, AppleTalk, ATM. Networking implementations
	should keep track of the costs associated with sending and
	receiving data over specific links, so that data flow can be
	optimized by user code based on standardized metrics. Library,
	High Impact, examples, implementations, moderate. JM, HES

	* Uniform Resource Locators: The IETF URL standard should be
	incorporated as an extension to physical pathnames in order to
	allow lisp programs to refer to data stored remotely via
	multiple access protocols (eg FTP, HTTP). URL have received
	wide deployment in the World Wide Web and will remain relevant
	for the foreseeable future.  Incorporation involves making
	"URL:" a privilege host name that serves to invoke the URL
	parser. "URI:" and "URN:" should be reserved for use future
	use. A standard locator scheme should be provided to allow
	reference to specific versions of definitions (see Defsystem).
	Extension, High Impact, Examples, Implementations, Moderate.

	* Uniform Event Model: The Design should include support for
	externally generated events (OS generated). It should allow
	applications to exploit window systems I/O devices with high
	performance by mapping naturally onto their event model.  The
	design should extend the condition system in a way that is
	more targeted at non-exceptional circumstances (event driven
	programming).  Extension, High Impact, Examples,
	Implementations, Moderate. JM, PR, RL

	* CORBA: A standard CORBA API should be provided as a library
	for CL.  Library, Examples, Implementations, Moderate.
	* Java CLIM Binding: CLIM should provide a Java binding along
	with support within CLIM lisp so that applications can drive
	clients on conventional web browsers Presentations and
	graphics are most important here. Library, High Impact,
	Examples, Implementations, Moderate. JM, HES
C. Descriptive and Organizational Tools
	* Defsystem: CL should provide a two tiered system definition
	model.  There should be a base system definition language and
	data structure that can be automatically produced by Lisp
	environments, but be human readable and editable.  The base
	model should be definition centric, and use the persistence
	model (see below).  A separate tool layer should provide
	extended capabilities, including 1) automatically mapping the
	definition based dependencies to files if files are required;
	2) a patching and versioning facility (this facility should be
	URL aware so that resources can be drawn from multiple
	locations over the network); 3) a module system that clearly
	groups functionality supporting specific external interfaces;
	4) a functional dependency tracking facility (Apple Dylan)
	that allows coherent system versioning based on definition
	versioning and supports portable "tree-shaking". Library, High
	Impact, Examples, Implementations, Moderate.  JM, HES, PR, RL

	* Documentation Facilities: Enhance CL to support automatic
	generation of documentation for definitions. A series of
	generic functions (or a single one with parameterization)
	should write definition components to a stream. Components
	should include the definition name, argument list, summary
	documentation, full documentation, and returned values.  An
	assertion interface for making statements about definitions
	should be provided as a standard, but left to implementations
	to determine its implementation. A default module grouping
	scheme can use the assertion interface to support
	documentation units smaller than package internal or external
	symbols, or completely orthogonal to the package system.
	Library, Examples, Implementations, Easy, Moderate. SF, JM

D. Advanced Capabilities

	* CL Portable Base Language: Develop a base language to which
	all extant Common Lisp implementations can be targeted. This
	language should include locatives and basic OS operations like
	processes, file I/O, networking.  This base layer will
	increase portability of lisp applications and enable work to
	improve performance with specialized hardware or compilers.
	This layer should make efforts to support real-time and
	parallel dialects.  Experimental, Very High Impact, Examples,
	Implementations, Hard. KR, RL, JM
	* CL Virtual Machine: Develop a virtual machine capable of
	efficiently executing CL, multimethod dispatch, multiple
	inheritance, and GC.  Library, Examples, High Impact,
	Implementation, Hard. AJB, HES, JM

	* Persistence: CL should provide a persistence binding for
	CLOS.  The facility should provide transactions and caching
	suitable for low latency operation.  There should be a native
	object binding for all Lisp forms, so that they can be
	persistently stored. Library, High Impact, Examples,
	Implementation, Moderate. JM, HES, PR, RL

	* Mobility: Provide a framework for mobile LISP that allows
	computations to be stopped and moved to different machines.
	Include a layered, encapsulating security model that allows
	environments in which a computation occurs to assign
	selectable levels of trust. Use the type system to enforce the
	trust policy. Deploy a browser plug-in that supports
	client-side execution and exploits standard browser
	facilities.  Experimental, High Impact, Examples,
	Implementations, hard. PHW, JM, HES

	* Debugger API: Development of a low-level debugger API would
	allow developers to debug LISP applications remotely and via
	different, perhaps third part interfaces. The API should
	provide for stack movement, restarts, access of data from
	stack frames or the environment. Experimental, Moderate
	Impact, Examples, Implementations, Moderate. AJB, KR, JM

	* Java Integration: Compile Java into Lisp and execute.
	Library, High Impact, Examples, Implementations, Moderate. TK,

E. Experimental Facilities

	* Compiler Advice: Provide a general mechanism for
	communication between the compiler, the application, and the
	programmer. Clean up the declaration mechanism: 1) provide
	accurate scoping of declarations; 2; Separate declarations
	effecting semantics from those effecting efficiency. Provide
	an assertion facility orthogonal to the code base that allows
	the programmer to provide advice to the compiler and for that
	advice to be easily separated according to operating
	environments (eg Dylan Compiler, CMU Python Compiler).
	Experimental, Moderate Impact, Examples, Hard. AJB, JM, RL,
	* Profiling and Test Vectors: Provide portable operator for
	determining the amount of consing within functions and the
	speed with which they execute. These facilities should work in
	multiple process and multiple processor contexts. Facilities
	for asserting test vectors for definitions should be provided
	and higher level facilities should run the defined test
	vectors. Automatic fault localization could be explored. These
	facilities should provide the substrate for developing
	compilers able to exploit runtime feedback from the
	application.  Library/Experimental, High Impact, Examples,
	Implementations, Moderate/Hard. TK, JM, HES, KR
	* Adaptive Programming Model: An example of this sort of model
	is a multi-level categorization of functionality, such as the
	generic network system under Genera.  In such a system, high
	level functionality are characterized as Services, where
	services are implemented by alternative protocols, and
	similarly for protocols.  Thus a declarative method exists for
	specifying alternative implementations, along with
	implementation of glue code to allow the interoperation (read
	plug-and-play) of alternatives. The design should include
	support for at least three levels of service specification,
	and glue code to implement the specification layers, and
	utilization of the condition system to control switching
	between alternatives.  Experimental, Very High Impact,
	Examples, Implementations, Moderate.  PR, RL, JM, HES, AJB
	* Transactional Memory: Provide a facility for atomic and
	coherent computations in a multiple process environment. This
	may provide coherence for persistent or volatile storage.
	Experimental, High Impact, Examples, Hard. TK, KR, AJB, JM
	* Low Latency Garbage Collection: Networked applications in CL
	can suffer serious performance degradations when the main GC
	is tripped.  Ideas for low latency GC should be explored,
	possibly in conjunction with the portable base CL language.
	Experimental, Examples, Implementations, Hard. JM, KR
	* Parallel Dialect: Develop a dialect as close to ANSI CL as
	possible that is suitable for MIMD parallel processing.
	Iteration constructs should be deprecated and replaced with
	higher level abstractions that prevent the programmer from
	making commitments about the underlying computational model.
	Numeric operations should be vectorized.  Experimental, High
	Impact, Examples, Implementations, Hard. TK, JM, KR

Rainer Joswig, Hamburg, Germany