Tunes

[Francois-Rene Rideau]

Dear M. Stallman,
   I thank you for having answered my previous mail,
and I appreciate your fine advice.
I understand you might be too busy to continue this discussion for too long,
(SIGSTOP and SIGHUP won't be ignored; SIGKILL can't be, but it isn't nice)
but there are some points I'd like to clear out.
(Note that I freely reordered some of your comments).


>        While GNU focused on Unix compatibility, so as to allow a seamless
>     transition from a corporate racket into a free software market,
>     Tunes focuses on providing cleaner abstractions
>     to free software developers, so as to reduce the overhead of
>
> I would say that that is the purpose of the HURD, also:
> to provide a clean set of features, not *only* Unix compatibility.
>
Up to now, I thought the HURD API would not make any major enhancement
to the unix model ("data through file descriptors, code through processes"),
so as to allow an easy POSIX layer.
Will you support full per-process scoping of system services, as in Plan9 ?
Will your system include a safe way for programs to communicate with
semantically meaningful fine-grained objects instead of raw byte streams ?
I guess I'll have to get the HURD documentations and read...
Sorry for my ignorance. I'll be reading more about HURD just today.


> You seem to envision that all programs that run on TUNES will be
> written in your new functional language.
>
Well, we envision our language to become the reference language
on the platform, much like C is the reference on Unix.
People would be free to use other languages,
which could be supported in multiple way:

* translation into our HLL. This could be fully automatical,
 or done in a manual/programmable way. In many cases,
 it could be detected that only a subset of the source language is used,
 that makes it possible to achieve translation
 through local code transformations instead of global ones.
 [I think your Guile project is meant to do the same.
 Maybe I can steal code from you, and perhaps give it back someday.]

* translation into the implementation-dependent calling convention
 used by the HLL, abiding by the GC requirements et al,
 in a way that cannot crash the system. Again, this can be used
 only if a strongly-typable subset of the language is used in
 the translated program; but runtime checks can be added to test that.

* execution in a protection box.
 This is the way Unix and such do for every program.
 We could very well do that for unsecure programs.
 This sure means a high overhead on memory usage,
 inter-object communication, etc.
    But there is no reason why this couldn't be done, too,
 with completely a programmable protection wrapper, that could do
 anything Plan9 allows and much more.
    Of course, "native" Tunes application shouldn't need the mechanism,
 but recursive Tunes sessions could be done this way,
 which could ease debugging, separated use of conflicting assumptions, etc.

As you see, even though it is not its primary goal,
Tunes shall eventually support recycling of C programs in many ways,
for we do believe too, that allowing smooth transition is needed to be useful.
However, we'll stress on the fact that programs
should rather be made secure and proven correct
than be kept emulated in paranoid boxes.


> No one will rewrite all his existing programs into a
> new functional language just to use TUNES.
>
> Likewise, Unix compatibility (including POSIX support) will remain
> absolutely essential; a system which lacks this compatibility will not
> be useful. It is true that you can provide that compatibility in a
> higher layer above a simpler cleaner interface.  That is, essentially,
> what the HURD does.
>
Yes. That's why people won't *have to* rewrite anything,
and we will eventually provide a full POSIX emulation box,
where the user can arbitrarily filter system calls,
to provide his own view of the "file system", etc.
   But the POSIX emulation to me is much like DOS emulation,
just that POSIX programs can be much more useful than DOS programs.
   We'll make using our language attractive,
and encourage and help people writing native applications.


> I certainly would not be
> interested in rewriting GNU Emacs into that language.
>
I fully understand, and you won't have to.
However, if ever the project is advanced enough,
I'd be interested in translating Emacs and/or other great GNU software
into native Tunes programs, with an elisp and C translators.
Hopefully, we can separate Emacs into a low-level layer than we can
map to our own system services,
and a high-level layer that will be strongly typable.


> You seem to have in mind developing both a language and a new
> operating system.  I think you are planning too many large projects.a
> If you try to do too many jobs, you are less likely to succeed in any
> of them.
>
Yes, that's why my effort has been towards finding people who would
help me with the OS part while I focus on the language part.


> So I suggest that you start by implementing a functional language
> compiler that can work on existing Unix-like systems.
> Given your ambitious goals for the language, that by itself
> could take a few years.
>
> If and when you're finished with that, then you could start
> designing a new kernel, if you still want to.
>
I personally decided that I would focus on the language thing.
However, I think that whenever the language and OS sides are independent,
two teams can develop them independently,
while whenever they closely interact,
it is a win to have the other being done at the same time.
   The major place where close OS interaction is required
is in implementing reliable orthogonal persistency.
POSIX makes it an unsecure mess, at best.


> My understanding is that functional languages
> generally don't give good performance; figuring out how to get good
> performance from a functional language could take a long time.
>
I'm not convinced that functional languages are any more difficult
to give good performance than other languages. What I'm convinced is
that people have invested much more time and money in C and FORTRAN.
Some LISP/Scheme/ML compilers like CMU CL, Bigloo, OCAML,
and surely many others, could achieve performance
not far from the best tuned C and FORTRAN compilers,
sometimes faster on some tests, and of course, without the programmer
having to worry about manual GC, so he can focus on good algorithms.


> To get good code quality and support many CPU types, I suggest
> implementing the functional language with a front end for GCC [...]
>
Thanks for the advice. I would dare thinkint about it without your telling me.
I'll give it a try, after the first prototype is out.
The main problem I see is that we want fine-grained dynamic compilation.
Could the GCC backend be used as some kind of a daemon,
or be integrated as a library ?
How strongly-typable is the C code inside ?

> I suggest implementing an existing functional language, if one exists
> that is suitable for what you have in mind, so that your compiler
> might appeal to the people who already like that language.
>
Well, once we have a compiler ready,
we could use a straightforward Scheme->TunesHLL frontend,
and get a Scheme compiler (or any other such frontend).
Thanks for the advice.


>        Dynamic partial evaluation and invalidation will be used to
>     provide excellent performance in a dynamic system,
>     and will encompass code generation, caching, etc.
> 
> Automatic inlining may not be a good idea.  It is inconvenient for
> debugging.  If the function foo has been inlined in many places then
> you cannot type "break foo" in the debugger and catch all calls to foo.
>
Mind that I said partial evaluation **and invalidation**.
Because we support object migration, we must be able
to invalidate any code that would be specialized against local assumptions.
As a side effect, it becomes possible to respecialize the code with
a breakpoint enabled (which needn't be implemented in such a naive way).
   Surely the partial evaluator will be a critical part of the system,
and proving it correct will be a priority,
so that we can be sure that partially evaluated code
has the same behavior as original code.


Well, many thanks again for sharing your wisdom with me,
and good luck in your free software activities,
for which I just can't thank you enough.

--    ,                                         ,           _ v    ~  ^  --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
--                                      '                   / .          --
Join the TUNES project for a computing system based on computing freedom !
                 TUNES is a Useful, Not Expedient System
WWW page at URL: "http://www.eleves.ens.fr:8080/home/rideau/Tunes/"