OS design...

[Ray Dillinger]

On Tue, 20 Oct 1998, Francois-Rene Rideau wrote:


>> What I'm talking about here is how programs establish and access
>> services, including but not limited to those services provided
>> by a typical operating system.

>Do you mean a foreign function interface to existing systems/programs,
>or just a standard library for your system?
<snip>

>Sure. And the foundation is: the compiler. Not anything in the "kernel".
>Choose an existing compiler, or implement a new one; calling conventions
>for "system services" will follow automatically from that choice.

Here, I think, is our fundamental conflict of opinion.  You are
assuming that system services will and must be accessible as
procedure calls.  

In writing compilers though, I have learned that each language
embodies assumptions about the semantics of a procedure call. 
Different semantics mean that different underlying "low level
details" are needed for the most efficient support.

The most general procedure call in the LISP family is the Scheme
procedure call.  It is lexically scoped and can be captured by a
continuation; it needs a scope pointer as well as an activation
record pointer, and, if the compiler can prove that it can't be
captured in a continuation, and the hardware provides stack
support (the intel family of chips do), then it is most
efficiently implemented on a stack.  If however it *can* be
captured in a continuation, then the most efficient
implementation is to dynamically allocate it and garbage collect
it when there is no longer a continuation that has a live
pointer to it. 

The Common Lisp procedure call is lexically scoped and cannot be
captured in a full continuation (though it may be "unwound" by
the use of catch/throw, and with very little effort it is
possible to implement an almost-continuation that can be used
exactly once). Its most efficient implementation in machines
that have hardware stack support is on the stack, and it
requires a scope pointer in each invocation frame.

The Elisp or Emacs-Lisp procedure call is dynamically scoped and
cannot be captured in a continuation; its most efficient
implementation on machines with hardware stack support is on the
stack, without a scope pointer.

I am sick to death of building FFI's that map the semantics of
one language's procedure calls to the semantics of another's in
order to access services.  I think it is a mistake to make one
compiler more priveleged than another, or complicate the task
of implementors by forcing them to learn the internals of some
particular compiler in order to build effective compilers for
the system, and then force them to track changes in that other
compiler on penalty of having theirs break.  (this is where most
LISP FFI's that call C++ code are today). 

Also, I don't want to tie the system to my compiler's choices:
three years from now I may learn something new and cool and
change the way I implement procedure calls in my scheme
compiler. When that happens, there are a bunch of things I don't
want to break;  

	1) Compiled code produced by earlier versions of the
		compiler.  A commercial application's customers
		don't always have the source lying around to
		recompile, and they may have very costly
		recertifications they'd prefer to avoid if 
		they do.  

	2) Compiled code produced by Common Lisp and Elisp 
		compilers (not to mention C compilers) that 
		used the "interface" defined by the system 
		up until that time.

	3) Other compilers.  For instance I may discover that
		static testing to see whether an invocation
		frame can be captured in a continuation is 
		wasting more time than stack-allocating it 
		saves.  I would then start heap-allocating
		all invocation frames in Scheme programs.
		If the Common Lisp compiler on the system
		compiles to Scheme, then suddenly CL programs
		will start running slower -- because cutting that
		test didn't save *them* any time at all and 
		they *NEVER* need heap-allocation of invocation
		frames.  Common Lisp compilers have no such
		worries, so why should they pay a penalty for
		something that buys scheme programmers a little
		more speed?? Why indeed should the time spent on
		the test needed to eliminate the penalty from
		scheme code be built into their development loop
		in the first place?

Right now, on systems where services are defined in terms of C++
procedure calls (which is most of them) LISPS pay an overhead
for "glue code" in their FFI because C++ is statically typed and
doesn't use long numbers!  

If C++ were implemented by a system that compiles to LISP, then
C++ would pay either a killer compile-time overhead as the LISP
compiler laboriously and exhaustively eliminated the dynamic
types from the LISP code, duplicating effort already expended by
the C++ programmer, or a killer run-time overhead as the
compiled code generated and checked dynamic type tags it didn't
need.

You want to think about how silly the machine code would be if
we compiled FORTH procedure calls to Scheme procedure calls??

My point is, let's just not go there, that would be continuing
the kind of mistakes that current OS's make by priveling one
language (or at least one set of calling conventions) over all
others. Instead, let's have some standard method for accessing
services that is reasonably language-neutral and easy for a
compiler writer (compiling *whatever* language) to cope with. 

			Ray