microkernels

[shapj@us.ibm.com]

--0__=K2tbEw39lUVRpflaDh1M7nu5mjHO1qF7AogfWfAITHB7RNuysB3kb3NJ
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline

I suggest the following changes in your new text.

Change:

     The prototypical Microkernels is Mach,
     originally developed at CMU,

To

     Perhaps the best known example of microkernel design
     is Mach, originally developed at CMU.

There were many microkernels prior to Mach, one of which is the predecessor to
Chorus.

You should completely drop:

     Microsoft Windows NT is said to originally
     have been a microkernel design, although
     one that with time and for performance reasons
     was overinflated into a big piece of bloatware
     that leaves monolithic kernels far behind
     in terms of size.

First, it simply isn't true.  Dave Cutler (the NT architect) has said over and
over again that NT was never a microkernel.  The label was applied incorrectly
by other people after the fact.

Second, the polemic stuff simply doesn't add any value.  Given good information
followed by *reasoned* explanations, people are quite good at drawing correct
conclusions.  Polemics are therefore interpreted by intelligent readers as a
place where the author actually didn't have any facts that supported their
opinion and so resorted to name-calling.  They conclude from this -- often
correctly -- that the author doesn't know what they are talking about and should
not be taken seriously.  In short, I think this kind of stuff damages your
credibility.


Your comments about academics are equally inappropriate and more than a little
offensive. I don't know of any academic who would laugh at a monolithic design
proposal these days. EROS is a case in point.  It's a major departure in design,
and people asked hard questions about that.  It is not a microkernel, but this
has never come up as a design issue one way or the other.  The kernel
architecture is structurally similar to a microkernel, and has some carefully
thought out layering.  So does Linux.  Nobody at the University of Pennsylvania
laughed at it, and several commercial efforts based on it are now underway.

If you have an example of an academic who has laughed at monolithic designs when
research on them has been proposed, make that known.  If you do NOT have several
concrete examples, then your text is an undeserved slur on a lot of people, and
it is a fundamentally dishonest thing to say.  You are certainly entitled to
that opinion, but a "glossary" is a place for statements of fact, not for
opinions.

I think it's worth noting that POSIX benchmarks are part of the problem, not
part of the solution.  The importance of these benchmarks became dominant with
the releast of the lmbench microbenchmarks, and has been promulgated by a number
of individuals in the OSDI and SOSP communities. One result is that most
microkernel architects have been forced to work on getting lmbench numbers as a
condition of publication rather than working on whatever made their system
interesting.  That is, the research has been compromised in a way that means we
don't really know what the current generation of microkernels can do, and we are
really unlikely to find out.

As an aside, an argument can be made that this sort of politics is essential to
the process of research.  An argument *has* been made that for this reason
science is primarily concerned not about fact but about orthodoxy, and is
therefore indistinguishable from religion.

My opinion is that such benchmarks are pragmatically important, but not
ultimately very helpful.  If your primary goal is to run UNIX, you will find
that UNIX implementations are surprisingly good at running UNIX and everything
else will tend to be less good at being UNIX.  If this is your primary goal, a
microkernel is inappropriate.  In fact, after a certain point, adjusting a
microkernel-based system to do well on POSIX benchmarks proves to be a good way
to make it bad at doing microkernel stuff -- the stuff for which you built it in
the first place.

If, on the other hand, modularity, security, or debuggability are important,
microkernels or systems like EROS that encourage dekernelized design may be a
good choice.  I have read about the problems that the HURD folks are having.
These problems reflect poor architectural choices in the structure of HURD
rather than any fundamental problem with microkernels.

Finally, I think your arguments about modularity are not really to the point.
That is, you are correct that disciplined programmers can achieve modularity in
large systems, and that this is not a compelling advantage of microkernel
systems from a theoretical point of view.  From a practical point of view, with
a lot of experience writing and modifying large systems, I can also say that the
devil is in the details, and that various Linux subsystems have "cheated" for
years before sufficiently modular interfaces became available and migration to
them slowly occurred.  It can be argued that this is good, but it should be
noted that the resulting evolution was more often accidental than intentional.
Also, your claims about the weak value of such modularity are most dependent on
arguments about drivers, which are not by their nature isolatable.  Anything
with access to physical DMA is not terribly isolatable in real systems.

The more critical concerns involve fault isolation *in applications*.  HURD is
not far enough along to really be able to exploit this, and without process
persistence it will be hard to achieve.  Driver fault isolation is largely an
illusion, because driver errors leave hardware in states that take whole
machines down.  This is true whether the driver is in user-mode code or
kernel-mode code.

>  Now, as far as polemics go, what is your
> take on microkernels? Maybe you have pointers
> to articles that support or contradict some
> of my opinions, and that I may link to?

I gave some in my netnews posting.  My basic response is that you either have
hard data supporting your position or you are simply shouting uninformed
opinion.  If you have the hard data, present it.  All of the commentary I have
seen on your web site is anecdotal.  Much of it has been examined in the
literature.  Each of the problems you identify existed in some particular system
or family of systems. Several present good arguments for why those individual
systems made some poor choices.  NONE of them, as far as I can tell, present a
fundamental argument for why microkernels are unsound.  All of the substantive
issues you raise have been addressed in subsequent designs.

For example, you complain about the shrinkage of microkernels into interrupt
dispatch + messaging.  A case in point, I think, would be Lava (the IBM
derivative of L4).  You present no data that this shrinkage is a bad idea, and
you completely ignore the fact that these systems have primitives that are a
factor of 100 to 1000 faster than Mach, which is the system you have the best
data for.  You also fail to note that the *reason* that UNIX on Mach failed
didn't have anything to do with the time spent in the UNIX emulator, and had a
*lot* to do with the time spent in the Mach primitives.  ALL of the hard data
has been collected makes this painfully clear.  Bryan Ford has done a good paper
on possible improvements in the Mach IPC implementation, and on the limits of
such advances within the basic Mach architecture.

Based on this, Lava ought to be very good at running Linux.  In actual
measurements, their published numbers show a degradation of 3% to 5% on typical
workloads relative to native Linux.  The Linux rehosting was a relatively
untuned quick and dirty port, and it was later determined within the Lava group
at IBM that the system benchmarked was suffering from a fairly serious bug in
the microkernel's memory mapping mechanism.  Current numbers are a good bit
better.

Now you may say that a 3% to 5% degradation is still a degradation.  I agree,
but remember that the primary purpose of Lava isn't running Linux.  Also,
remember that this is after an effort of only a small number of months, which
argues that the modularity structure actually works very well.


Personally, I don't think microkernels are better or worse.  I think they are
suited to a different set of problems.


Jonathan S. Shapiro, Ph. D.
IBM T.J. Watson Research Center
Email: shapj@us.ibm.com
Phone: +1 914 784 7085  (Tieline: 863)
Fax: +1 914 784 7595


Francois-Rene Rideau <fare@tunes.org> on 08/15/99 08:44:25 PM

Please respond to Francois-Rene Rideau <fare@tunes.org>

To:   Jonathan S Shapiro/Watson/IBM@IBMUS
cc:   tunes@tunes.org
Subject:  microkernels




--0__=K2tbEw39lUVRpflaDh1M7nu5mjHO1qF7AogfWfAITHB7RNuysB3kb3NJ
Content-type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-transfer-encoding: quoted-printable



Dear Prof Shapiro,
   thank you for your e-mail.

> Whatever the "truth" about microkernels may be,
> a "glossary" isn't the right place for the discussion.
Thanks for your most adequate remark.
It seems our glossary has grown
into something much too much polemical,
and should be reorganized into a knowledge-base
with raw facts being well delimited from associated opinions.
We haven't invested in the technology necessary
to develop such a knowledge base,
having waited for TUNES to appear and provide this technology for too l=
ong.
Do you have any suggestion?

> At the very least the entry should give a definition
> before it engages in polemics.
>
> How about adding a definition?
Ok. Since I wasn't fully satisfied with the entry in FOLDOC,
I wrote the one below. What about it?
Can you either correct it, or point me to an existing satisfying defini=
tion?

Now, as far as polemics go, what is your take on microkernels?
Maybe you have pointers to articles that support or contradict
some of my opinions, and that I may link to?

Appended to this message is the text I prepended to the current entry.

Best regards,

[ "Far=E9" | VN: =D0=A3ng-V=FB B=E2n | Join the TUNES project!   http:/=
/www.tunes.org/  ]
[ FR: Fran=E7ois-Ren=E9 Rideau | TUNES is a Useful, Nevertheless Expedi=
ent System ]
[ Reflection&Cybernethics  | Project for  a Free Reflective  Computing =
System ]
My opinions may have changed, but not the fact that I am right.


PS: I invite Tunes members to proof-read the current entry
and the below text as well. My questions are also to you, guys.
Be careful not to include Prof. Shapiro in replies that only concern Tu=
nes.

------>8------>8------>8------>8------>8------>8------>8------>8------>=
8------
microkernel (also abbreviated =B5K or uK) is
an approach to operating systems design
by which the functionality of the system
is moved out of the traditional "kernel",
into a set of "servers"
that communicate through a "minimal" kernel,
leaving as little as possible in "system space"
and as much as possible in "user space".
<LI>
microkernels were invented as a reaction to traditional
"monolithic" kernel design,
whereby all system functionality was put in a one static program
running in a special "system" mode of the processor.
The rationale was that it would bring modularity in the system architec=
ture,
which would entail a cleaner system, easier to debug or dynamically mod=
ify,
customizable to users' needs, and more performant.
<LI>
The prototypical Microkernels is Mach, originally developed at CMU,
and used in some free and some proprietary BSD Unix derivatives,
as well as in the heart of GNU HURD.
MICROS~1 Windows NT is said to originally have been a microkernel desig=
n,
although one that with time and for performance reasons
was overinflated into a big piece of bloatware
that leaves monolithic kernels far behind in terms of size.
Latest evolutions in microkernel design led to things like
"nano-kernel" L4, or "exokernel" Xok.
<LI>
At one time in the late 1980's and early 1990's,
microkernels were the craze in official academic and industrial OS desi=
gn,
and anyone not submitting to the dogma was regarded as ridiculous.
But microkernels failed to deliver their too many promises
in terms of either modularity (see Mach servers vs Linux modules)
cleanliness (see Mach horror), ease of debugging (see HURD problems),
ease of dynamic modification (also see HURD vs Linux),
customizability (see Linux vs Mach-based projects),
or performance (Linux vs MkLinux, NetBSD vs Lites).
This led some microkernel people to compromise
by having "single-servers" that have all the functionality,
and pushing them inside "micro"kernel-space (WindowsNT, hacked MkLinux)=
,
yielding a usual monolithic kernel under another name
and with a contorted design.
Other microkernel people instead took an even more radical view
of stripping the kernel from everything
but the most basic system-dependent interrupt handling
and messaging capabilities,
and having the rest of system functionality in libraries
of system or user code,
which again is not very different from monolithic systems like Linux
that have well-delimited architecture-specific parts separated
from the main body of portable code.
With the rise of Linux, and the possibility to benchmark
monolithic versus microkernel variants thereof,
as well as the possibility to compare kernel development
in various open monolithic and microkernel systems.
people were forced to acknowledge the practical superiority
of "monolithic" design according to all testable criteria.
Nowadays, microkernel is still the "official" way to design an OS,
although you wont be laughed at when you show your monolithic kernel an=
ymore.
But as far as we know,
no one in the academic world dared raise any theoretical criticism
of the very concept of microkernel.
Here is our take at it.
------>8------>8------>8------>8------>8------>8------>8------>8------>=
8------

=

--0__=K2tbEw39lUVRpflaDh1M7nu5mjHO1qF7AogfWfAITHB7RNuysB3kb3NJ--