Modernizing Common Lisp: Recommended Extensions
Francois-Rene Rideau
fare@tunes.org
Sat, 10 Feb 2001 23:07:07 +0100
--gBBFr7Ir9EOA20Yy
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
Rainer Joswig sent to me this interesting summary of directions in which
to extend Common LISP. Together with, e.g. Henry Baker's Critique of DIN
Kernel LISP, it gives good hints as to what are the challenges in designing
a modern programming language, independently from the evaluation paradigm...
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[ TUNES project for a Free Reflective Computing System | http://tunes.org ]
Individualists unite!
--gBBFr7Ir9EOA20Yy
Content-Type: message/rfc822
Content-Disposition: inline
Received: from samaris (localhost) [127.0.0.1] (root)
by Samaris with esmtp (Exim 3.20 #1 (Debian))
id 14J55D-0002Iq-00; Thu, 18 Jan 2001 03:45:51 +0100
Delivered-To: fare@bespin.org
Received: from bespin.org [206.63.100.249]
by localhost with IMAP (fetchmail-5.5.1)
for fare@localhost (single-drop); Thu, 18 Jan 2001 03:45:51 +0100 (CET)
Received: from CORPORATE-WORLD.lisp.de (lispm.lisp.de [194.163.195.72])
by bespin.org (Postfix) with SMTP id E81E33E845
for <fare@TUNES.ORG>; Wed, 17 Jan 2001 18:29:24 -0800 (PST)
Received: from PBG3.lisp.de by CORPORATE-WORLD.lisp.de via INTERNET with SMTP id 15196; 18 Jan 2001 03:17:43+0100
Mime-Version: 1.0
X-Sender: joswig@194.163.195.72
Message-Id: <p0433010db68c02266ad8@[194.163.195.67]>
Date: Thu, 18 Jan 2001 03:16:32 +0100
To: fare@tunes.org
From: Rainer Joswig <joswig@corporate-world.lisp.de>
Subject: Modernizing Common Lisp: Recommended Extensions
Content-Type: text/plain; charset="us-ascii"
you might be interested in this old mail - if you don't have it:
Date: Wed, 7 Jul 1999 16:39:52 -0400
To: x3j13@ai.sri.com
From: "John C. Mallery" <jcma@ai.mit.edu>
Subject: Modernizing Common Lisp: Recommended Extensions
Modernizing Common Lisp: Recommended Extensions
John C. Mallery, Howard E. Shrobe, Robert Laddaga,
Thomas F. Knight, Kalman Reti, Paul Robertson,
Andrew J. Blumberg, Susan Felshin, and Patrick H. Winston
Artificial Intelligence Laboratory
Massachusetts Institute of Technology
July 7, 1999
The document suggests enhancements to Common Lisp that modernize the
language in light of developments since that last ANSI standardization
round. These elements can be incorporated as changes to the:
* Language
* Suggested "yellow pages" that implementations or
repositories may supply as libraries
* Experimental components intended to explore areas not yet
stable enough for inclusion within the language proper or the
suggested libraries.
The presentation is as an outline of topics in order to keep the
document short and facilitate discussion. It is intended that separate
two-page outlines will be developed by proponents for each of the
substantial topics for more careful consideration. If the community
decides to move forward on a specific recommendation, we expect
proponents to develop a detailed proposal for critical review, a
working implementation, and ultimately, a standards specification.
The most important recommendation with the most leverage is the
recommendation to develop a PORTABLE BASE LANGUAGE FOR COMMON LISP.
This strategic move provides:
* An opportunity to reengineer the core language;
* A minimal base to port to new platforms;
* Opportunities for new hardware and compiler efforts
(base-to-platform & portable-to-base);
* Increased sharing of higher-level code.
How do we identify the evolutionary trajectory for Common Lisp? One
approach is to identify what features a modern programming language
requires by examining at other recent languages (eg, Java, Dylan) and
selecting features that merit refinement and generalization. Another
approach analyzes the deficits of contemporary programming languages
and environments and proposes new facilities to address the identified
gaps.
These recommendations use both approaches but are tempered by
recognition of the constraints of previous standardization rounds and
the need to maintain continuity. Although some of the suggestions
below may seem beyond the conventional purview of a standards body, we
believe that language leadership is not possible without reference to
the kinds of facilities that a modern programming environment must
offer. Feedback from independent efforts to develop libraries and
experimental components will provide a critical input into the
language evolution effort.
The entries below adhere to the following style for clarity of
presentation:
1. Title: A telegraphic title.
2. Abstract: Three sentences describing the feature.
3. Category: Extension, Library, Experimental
4. Impact: Low, Medium, High
5. Examples: Are examples available in LISP or other languages?
6. Implementation: Are existing implementations available?
7. Longevity: How long will this feature be relevant?
8. Difficulty: Easy, moderate, hard.
9. Benefits: What specific benefits does the feature provide?
10. Approach: How can this feature be incorporated?
--------------------------------------------------------------------------------
A. Core Language Extensions
* Package System: CL should provide relative package names and
associated operators. This feature is important for smooth
evolution of large Lisp systems. Language, Examples,
Implementations, Easy, Low. SF, JM, KR.
* Declarations: Add the ability to declare returned values and
their types for functions and methods. Provide a standard
arglist function to return the arguments and values of a
function. Language, Examples, Implementation, Easy, Low. SF,
JM, KR.
* Modern Character Sets: CL should support Unicode, which is
becoming the standard character set for multi-language
applications. Extension, Examples, Implementations, Moderate.
JM, KR
* Unit Awareness: Provide extensions to numeric operations
that support conservation and coercion of units. Extension,
low Impact, Easy. TK, JM
* APL Array Operations: Provide a standard array package
analogous to the one found in APL but building on top of
current CL array substrate. Implementations should provide
highly optimized specializations for functions. Library, Low
Impact, Examples, Implementations, Moderate. TK, JM
B. Networking Components
* Uniform Process Model: A standard API for working with
processes (aka threads) should be developed and provided as a
standard part of CL. It should include locks,
store-conditional, and timers. The design should allow
applications to exploit multiple processor hardware.
Extension, High Impact, Examples, Implementations, Moderate.
JM, HES
* Uniform Foreign Function Model: A single, standard API
should be provided by CL for calling foreign functions. The
facility should provide some basic operators for marshalling
and unmarshalling arguments that can be used by applications
to support FF over other transports (eg HTTP) in either
direction. Extension, High Impact, Examples, Implementations,
Moderate. JM, HES
* Standard Streams: CL should provide a standard API for
streams in order to allow stream code to be portable. The grey
stream proposal from CLIM is a good starting point, however,
the actual standard should consider the requirements of
networked applications, graphics display, and streamed
video/audio. Streams should define generic buffer-in-place
operations and provide a set of generic "internal" operations
which enable, for example, encapsulation or metering during
OPEN or other standard stream operations. Extension, High
Impact, Examples, Implementations, Moderate. JM, HES, KR
* Networking Substrate: Incorporate a standard API for
performing generic networking operations. Modern programming
languages need to be network-aware. Java provides an example.
A networking substrate assumes a thread model, streams, and
locking mechanisms. Crucially, the networking model should
provide a hierarchy of conditions relevant for networked
applications. The networking API should be generic and capable
of spanning multiple different networking standards, e.g.
Internet, Chaos, AppleTalk, ATM. Networking implementations
should keep track of the costs associated with sending and
receiving data over specific links, so that data flow can be
optimized by user code based on standardized metrics. Library,
High Impact, examples, implementations, moderate. JM, HES
* Uniform Resource Locators: The IETF URL standard should be
incorporated as an extension to physical pathnames in order to
allow lisp programs to refer to data stored remotely via
multiple access protocols (eg FTP, HTTP). URL have received
wide deployment in the World Wide Web and will remain relevant
for the foreseeable future. Incorporation involves making
"URL:" a privilege host name that serves to invoke the URL
parser. "URI:" and "URN:" should be reserved for use future
use. A standard locator scheme should be provided to allow
reference to specific versions of definitions (see Defsystem).
Extension, High Impact, Examples, Implementations, Moderate.
JM, HES
* Uniform Event Model: The Design should include support for
externally generated events (OS generated). It should allow
applications to exploit window systems I/O devices with high
performance by mapping naturally onto their event model. The
design should extend the condition system in a way that is
more targeted at non-exceptional circumstances (event driven
programming). Extension, High Impact, Examples,
Implementations, Moderate. JM, PR, RL
* CORBA: A standard CORBA API should be provided as a library
for CL. Library, Examples, Implementations, Moderate.
* Java CLIM Binding: CLIM should provide a Java binding along
with support within CLIM lisp so that applications can drive
clients on conventional web browsers Presentations and
graphics are most important here. Library, High Impact,
Examples, Implementations, Moderate. JM, HES
C. Descriptive and Organizational Tools
* Defsystem: CL should provide a two tiered system definition
model. There should be a base system definition language and
data structure that can be automatically produced by Lisp
environments, but be human readable and editable. The base
model should be definition centric, and use the persistence
model (see below). A separate tool layer should provide
extended capabilities, including 1) automatically mapping the
definition based dependencies to files if files are required;
2) a patching and versioning facility (this facility should be
URL aware so that resources can be drawn from multiple
locations over the network); 3) a module system that clearly
groups functionality supporting specific external interfaces;
4) a functional dependency tracking facility (Apple Dylan)
that allows coherent system versioning based on definition
versioning and supports portable "tree-shaking". Library, High
Impact, Examples, Implementations, Moderate. JM, HES, PR, RL
* Documentation Facilities: Enhance CL to support automatic
generation of documentation for definitions. A series of
generic functions (or a single one with parameterization)
should write definition components to a stream. Components
should include the definition name, argument list, summary
documentation, full documentation, and returned values. An
assertion interface for making statements about definitions
should be provided as a standard, but left to implementations
to determine its implementation. A default module grouping
scheme can use the assertion interface to support
documentation units smaller than package internal or external
symbols, or completely orthogonal to the package system.
Library, Examples, Implementations, Easy, Moderate. SF, JM
D. Advanced Capabilities
* CL Portable Base Language: Develop a base language to which
all extant Common Lisp implementations can be targeted. This
language should include locatives and basic OS operations like
processes, file I/O, networking. This base layer will
increase portability of lisp applications and enable work to
improve performance with specialized hardware or compilers.
This layer should make efforts to support real-time and
parallel dialects. Experimental, Very High Impact, Examples,
Implementations, Hard. KR, RL, JM
* CL Virtual Machine: Develop a virtual machine capable of
efficiently executing CL, multimethod dispatch, multiple
inheritance, and GC. Library, Examples, High Impact,
Implementation, Hard. AJB, HES, JM
* Persistence: CL should provide a persistence binding for
CLOS. The facility should provide transactions and caching
suitable for low latency operation. There should be a native
object binding for all Lisp forms, so that they can be
persistently stored. Library, High Impact, Examples,
Implementation, Moderate. JM, HES, PR, RL
* Mobility: Provide a framework for mobile LISP that allows
computations to be stopped and moved to different machines.
Include a layered, encapsulating security model that allows
environments in which a computation occurs to assign
selectable levels of trust. Use the type system to enforce the
trust policy. Deploy a browser plug-in that supports
client-side execution and exploits standard browser
facilities. Experimental, High Impact, Examples,
Implementations, hard. PHW, JM, HES
* Debugger API: Development of a low-level debugger API would
allow developers to debug LISP applications remotely and via
different, perhaps third part interfaces. The API should
provide for stack movement, restarts, access of data from
stack frames or the environment. Experimental, Moderate
Impact, Examples, Implementations, Moderate. AJB, KR, JM
* Java Integration: Compile Java into Lisp and execute.
Library, High Impact, Examples, Implementations, Moderate. TK,
PHW, JM
E. Experimental Facilities
* Compiler Advice: Provide a general mechanism for
communication between the compiler, the application, and the
programmer. Clean up the declaration mechanism: 1) provide
accurate scoping of declarations; 2; Separate declarations
effecting semantics from those effecting efficiency. Provide
an assertion facility orthogonal to the code base that allows
the programmer to provide advice to the compiler and for that
advice to be easily separated according to operating
environments (eg Dylan Compiler, CMU Python Compiler).
Experimental, Moderate Impact, Examples, Hard. AJB, JM, RL,
HES, TK
* Profiling and Test Vectors: Provide portable operator for
determining the amount of consing within functions and the
speed with which they execute. These facilities should work in
multiple process and multiple processor contexts. Facilities
for asserting test vectors for definitions should be provided
and higher level facilities should run the defined test
vectors. Automatic fault localization could be explored. These
facilities should provide the substrate for developing
compilers able to exploit runtime feedback from the
application. Library/Experimental, High Impact, Examples,
Implementations, Moderate/Hard. TK, JM, HES, KR
* Adaptive Programming Model: An example of this sort of model
is a multi-level categorization of functionality, such as the
generic network system under Genera. In such a system, high
level functionality are characterized as Services, where
services are implemented by alternative protocols, and
similarly for protocols. Thus a declarative method exists for
specifying alternative implementations, along with
implementation of glue code to allow the interoperation (read
plug-and-play) of alternatives. The design should include
support for at least three levels of service specification,
and glue code to implement the specification layers, and
utilization of the condition system to control switching
between alternatives. Experimental, Very High Impact,
Examples, Implementations, Moderate. PR, RL, JM, HES, AJB
* Transactional Memory: Provide a facility for atomic and
coherent computations in a multiple process environment. This
may provide coherence for persistent or volatile storage.
Experimental, High Impact, Examples, Hard. TK, KR, AJB, JM
* Low Latency Garbage Collection: Networked applications in CL
can suffer serious performance degradations when the main GC
is tripped. Ideas for low latency GC should be explored,
possibly in conjunction with the portable base CL language.
Experimental, Examples, Implementations, Hard. JM, KR
* Parallel Dialect: Develop a dialect as close to ANSI CL as
possible that is suitable for MIMD parallel processing.
Iteration constructs should be deprecated and replaced with
higher level abstractions that prevent the programmer from
making commitments about the underlying computational model.
Numeric operations should be vectorized. Experimental, High
Impact, Examples, Implementations, Hard. TK, JM, KR
--------------------------------------------------------------------------------
--
Rainer Joswig, Hamburg, Germany
Email: mailto:joswig@corporate-world.lisp.de
Web: http://corporate-world.lisp.de/
--gBBFr7Ir9EOA20Yy--