microkernels

[Francois-Rene Rideau]

On Mon, Sep 06, 1999 at 02:50:49PM -0400, shapj@us.ibm.com wrote:
> Actually, I was probably the cause for this round of the debate,
> so I'ld like to participate.
Thanks! You're welcome!

> [...] First, the comparisons begin with a questionable assumption:
> that the two systems should run comparable APIs. Typically,
> the POSIX API is used as the interface to measure.
> But let's test the assumption!
>
I think you're confusing two separate issues here.
I agree that it's wrong-headed to compare a POSIX implementation
with the implementation of something else;
but that's independent from the uK vs monolithic kernel issue.
It would be equally meaningless to directly compare performance
of a <foo> POSIX implementation to that of a <foo> EROS implementation,
for <foo> in "monolithic", "micro-kernel", or "kernel-less"!
What is meaningful is to consider the relative interest of
monolithic vs microkerneled vs kernelless implementations of <bar>,
for <bar> in "POSIX", "the EROS API", etc.

And my claim is that the very micro-kernel concept is wrong-headed,
for it focuses on low-level concerns, whereas interesting points
are user-level concerns. Unless you accept that what the user wants to see
is low-level interfaces to low-level black-box components. Now,
only a very few users care about low-level interfaces, and these users
won't care about black-box components if they have free software.

> Given that UNIX is likely the fastest POSIX implementation,
> If you went to a microkernel architecture you were looking
> for something other than performance.
> Perhaps you wanted better fault isolation.  Perhaps you wanted to be able to
> configure the system more easily.  In any of these cases, you decided that
> something was more important than the last bit of performance,
> and you hope to get whatever that may be by using a microkernel.

This reasoning _assumes_ that microkernel actually brings something.
I question that assumption, all the more when it's done _before_ even
considering the non-POSIX thing to implement. As for the given examples,
you can get fault-isolation in a monolithic kernel
using call-backs to user-level daemons for system services
(as has been done for Linux using NFS wrappers for FS code,
or gpm, or X, or bunches of other stuff);
or you can get prevent fault altogether by having a proper type-system
(as is done in SPIN, Fox, etc).
As for configuration, I fail to see how micro-kernels help.
Embedding a language in the OS (POP-n, FORTH, LISP, Modula-3, SML)
looks like a much more effective way to achieve maximal configurability;
C-written daemons with CORBA or similar bindings are a poor substitute
to a real concurrent programming language, as far as configurability goes.

> In either case, the performance comparison is moot, because you decided
> going in that performance wasn't the primary concern.
Well, performance IS a concern about the _low-level_ part of the design.
Once you accept that modularity is a _high-level_ concept,
and see that microkernel is a _low-level_ design,
you can only question the interest of the latter.

If performance is secondary to you, that's one more reason not to enter
into low-level considerations. Instead, to focus on the high-level
language/API you want to use, such as for instance Java/EJB, and see
that it has implementations, even souped-up interpreters
(not that I consider Java/EJB a perfect choice, but at least it's a choice).


> I used to use this as a metric for microkernels.  I don't anymore.
I wonder what makes something a "microkernel" or not, these days.
Perhaps another hint that the concept is perhaps not very relevant.

> If there is one, I think the difference between a microkernel
> and a monolithic kernel lies in simplifications of the main control flow
> and data flow paths, and the minimization of the number of abstractions
> that lie within the performance "core" of the operating system. Reductions
> in the number of abstractions and
> interdependencies increases implementation options.
I wonder if this distinction is very relevant when having
extensible systems (where the number of abstractions vary),
and/or systems based on a large language (where there are lots and lots
of available abstractions, but the kernel is nowhere to be seen).


>>> 1)  Micro kernels look good for hard realtime systems:
>>>     A) The source code is smaller and easier to prove and verify,
>>>        and predict.
>> Is it the micro-kernel approach that simplifies things,
>> or is it not the fact that you have smaller and simpler overall systems?
>
> Actually, Eric has a point here -- though I disagree about predictability.
> There is a lot of data that says that bugs rise faster than lines of code
> -- some would suggest exponentially faster.
> To the extent that it is possible to divide things
> into forcibly isolated components with well-defined interfaces,
> and to *enforce* the isolation,
> one achieves better separation of concerns and better testability.

What you're describing is *modularity*, and I sure do agree it helps.
But again, does the microkernel design help about it,
as compared to using a suitable module system in your favorite language?
Microkernels sound too much like a very klugy way
to emulate by hand a module system in C. Go get the real thing.

> One problem with libraries is that they don't really achieve this.
Not with C libraries indeed.
Because C offers no static checking for any modularity.
Now, microkernels are just pretending that a two-level system
with a library of C-programmed components glued together by
a second-grade shell language will do the thing...


> [Francois writes:]
>> Can you show me one single thing of interest other than [proprietary
>> software] in micro-kernels, that cannot be done in a safer and more
>> efficient way by using a suitable module system in whatever high-level
>> language is used to program the system?
>
> You seem to be confusing "binary" and "proprietary".
> Once again, I caution you not to get so caught up in your rhetoric
> that you lose track of what's substantiable.
No I'm specifically not confusing them.
A case in point is monolithic Linux, that allows for binary modules
to be inserted at runtime, yet won't work with proprietary modules,
since they are not portable accross changes in kernel version
(and even less in kernel architecture).
The concept that may escape you is to consider the meta-tools
that generate binaries as part of the overall system:
even though these tools may run on another host than the one
where the binary modules are inserted, they are conceptually
part of the overall system that is to be trusted.
That's why you may trust binary modules under Linux as much as other
parts of the system: they are produced by the very same development chain.
That's why you cannot ever trust third-party proprietary black-box code
as much, and want isolation from it, which microkernels do provide.

> There are many reasons to want to support binary integration.
Sure. But unless the binary modules are to be typed bit-by-bit by a human,
it is more enlightening to consider the whole system,
including the compilation chain, when assessing security.
And then, microkernels don't look so good anymore.

> The most obvious are enabling add-on modules that users can safely accept.
Again, I question the safety brought by microkernels.
When modules can access the whole memory through DMA,
hang out in interrupt code, or do nasty I/O,
it is dubious what MMU protection brings,
all the more since type-safety already brings more than MMU!

> There exists no language-based protection system today
> that has proven secure in the field.
Depends on what you mean by "language-based" and "proven secure".
I do consider all the current C APIs as C-language-based, for instance.
And it is known that a strong typing ensure stronger invariants
than an MMU can about illegal memory accesses.

> This is in part because language systems are inherently more complex
> than operating systems.
Here, I will say "MU".
You just can't separate language and operating system like that.
All operating systems are defined by their API in some language(s),
even if the language is 6502 assembly.

> A language-based module system is a fine thing to have,
> but solutions that restrict programmers to a single language
> empirically do not receive wide acceptance.
I've seen this argument many times, but I don't buy it.
Why not apply this reasoning to C and C++ based systems?
Or conversely to systems that support integration of multiple
high-level languages (POP-11)?

As for the acceptance of programming systems, I see much more
relevance to arguments such as commercial user support,
intellectual property barriers to creation of a market for services,
and consequential general convergence toward low-level interfaces
when anything high-level is "protected" by IP barriers
on software and hardware alike.
As a corollary, I can explain the success of C-based systems
(as opposed to, say, PL/1 systems, or assembly based system)
as freer-than-alternatives (free compilers)
somewhat-portable (independent from hardware IP)
low-level (independent from software IP) systems.
Add usual theorems about network effects, and you're done.
Granted, you may count IP barriers as just part of larger
technological evolution barriers to avoid;
but it's still a large part of it,
and a definitive part in that it doesn't even give a chance
for other criteria to emerge.

Ahem, I guess such arguments would rather go to cybernethics@tunes.org
than to tunes@tunes.org. Suffices it to say here that I question
the choice of using the non-modular C language for implementation
as being remotely a token of "language-independence" of the OS.

>> That's why C is not such a good systems programming language,
>> and why Modula-3, SML, and even Perl, are better choices.
>
> Plus ca change, plus ca reste la meme
>
> We certainly thought C was a poor choice when the EROS effort started.
> We'll shortly be converting to C because the level of abstraction in C++
> is too high to get a good, comprehensible kernel.
I prefer not to repeat here and now what I think of C++.
Let me just say that it has very little useful support for modularity.

> There are many (perhaps most) systems
> that can and should be written in high-level languages.
> A nucleus isn't one of them.  Also, can you explain how you propose
> to verify the correctness and
> security of the runtimes and compilers for any of these languages?
I don't know that C compilers are any simpler.
Also, consider that MMU-based protection relies on complex MMU circuits;
if you fear complexity in the compiler type-checker,
why not fear complexity in the chip design?
I do agree that overall complexity has to be reduced;
but you must consider the whole system,
not just myopically look at it tiny part by tiny part.
Formal correctness proofs look like to me a much more promising research path
for overall system security than is hardware protection.
And again, just because you used a high-level language doesn't mean
you can't or shouldn't peek at the produced binary code, if you so like.

> For a nucleus, I'ld rather have a non-optimizing compiler for a low level
> language for which I can verify the compiler.
Why just for the nucleus? What if what you actually need to trust
is an all-encompassing system, and/or an application?
And why even trust the non-optimizing compiler?
Why not look at the assembly output? why not write things in binary directly?
Why trust transistor layouts? Why not lay them out yourself?
Why trust the transistor specs? Why trust the laws of physics?
Did you reflect on trusting trust, lately?
(see Ken Thompson's classic ACM Turing Award speech.)

Security is an holistic concept; like everything,
it has some irreducible complexity; and like all complexity,
you cannot handle it without going at the meta-level; kernel or no kernel.
And the same methods you use to grow trust in kernel-based systems
can be used to grow trust in kernel-less systems:
modularizing things, splitting them in clearly separated meta-levels,
disclosing non-random material for peer review,
having an active and responsive community, etc.,
and of course, making (formal and experimental) tests on your programs.

> The cost argument, however, is empirically false.
> Separation into components leads to other simplifications in the code
> that cannot be realized in the absence of an encapsulation boundary.
> With KeyKOS, they frequently found a net
> *improvement* from breaking applications up.
This sure is an argument for *modularity*.
I again contend that microkernels are a poor way to achieve modularity.

>> Java, for all its warts, at least recalls us that security
>> is a high-level concept, not a low-level concept.
>
> Perhaps, but it also reminds us that *protection* is a low level concept.
> Java got this wrong, which is why Java has no meaningful security
> and never will. Even in the no-kernel approach,
> one must build on the right primitives.
>
Sure. Sometimes, isolation is the Right Thing(TM),
all the more in presence of Reflection
(you don't want information to leak through some borders),
and hardware protection mechanisms are clearly a way to enforce it.
However, domain-specific type systems that do information-tagging
of some sort may do the same trick, and more.
Which implementation technique is more adapted depends both
on the kind of isolation to express and on its grain.

I do agree that the most important point is to have the right primitives;
and that's perhaps what I'd like a no-kernel to be about:
focus the design attention on the semantics of primitives,
rather than on particular implementation mechanisms,
that should be hardware-dependent
(which is I think a positive contribution of exo/nano-kernel approaches,
independently from their kernelful aspect).

Best regards,

[ "Faré" | VN: Уng-Vû Bân | Join the TUNES project!   http://www.tunes.org/  ]
[ FR: François-René Rideau | TUNES is a Useful, Nevertheless Expedient System ]
[ Reflection&Cybernethics  | Project for  a Free Reflective  Computing System ]
Tradition is the matter of which civilization is made.
Anyone who rejects tradition per se should be left naked in a desert island.
Innovation is the matter with which civilization is built.
Anyone who rejects innovation per se should be left naked in a desert island.
		-- Faré