Writing an OS for PC

[Francois-Rene Rideau]

>   Hi!  I don't know at this point if I'm interested
> in helping you work on your operating system.
Sure. I appreciate your being careful !

> This being said primarily due to the fact that I don't know
> what kinds of plans you have.
Have a look to the TUNES page "http://acacia.ens.fr:8080/home/rideau/Tunes/"
for more info.


> ============================================================================

>    Hello, and thanks for the reply.  Every OS designed has to be based off
> of something, I guess. (Or multiple somethings).
   And TUNES is based on quite some deep concepts. It is truely a
revolutionary OS design (at least as compared to commercial software and
concepts everyone seem crazy about).


>   1. Of course, true preemptive multitasking.
   Yes and No.
   Preemption is made necessary by the complete unsecurity that has deep
roots in traditional OS design. But when possible (most of the time),
cooperation yields *far* better performance. In TUNES, we will provide
the required security for cooperation, and of course will take advantage of
the enhanced performance of cooperation. See the TUNES LLL page about that:
"http://acacia.ens.fr:8080/home/rideau/Tunes/LLL/LLL.html#multithreading".
   Of course, this security is just impossible to achieve using C [well, not
completely impossible; but then, C would be only a very heavy burden, and would
lose *all* its advantages: existing libraries and compilers]. Which is why
TUNES refuses C as a standard system language.


>   2. Of course, true 32-bit programming.
   Well, sure the implementation on 32-bit computers like intel PCs should
be 32-bit. But TUNES intends to allow dynamic distribution on heterogeneous
platforms with different word size (e.g. 8, 16, 20, 32, 64 or 128 bits).
Thus 32 bit should not be hardwired anywhere in the system, be purely
implementation dependent, and should not interfere with dynamic code migration.


>   3. Refrain from HUGE amounts of memory required to run the OS, like some
>      other OS's that I know.  Keep in mind that this may conflict with the
>      goals below.  However, if it does, then the goals below become the
>      priority, because at least lots of functionality comes from the size.
   Well, again, features occupy memory. The problem with existing systems is
that they are coarse grained: you cannot isolate a useful feature from lots
and lots of useless but costly (hence harmful) ones.
   TUNES will provide fine grain at all levels: you can determine exactly
what modules you have. A stripped specialized system should fit the small
ROM of some embedded micro-controller, while a full featured one could
take advantage of thousands gigs of distributed shared memory.
   See the TUNES Features page:
"http://acacia.ens.fr:8080/home/rideau/Tunes/Review/Features.html".



>    Now for some of the things that I want that most other OS's don't have.
> 
>   1. VERY extensive error recovery and error handling.
>      a.  The OS itself NEVER crashes unless some PL0 task trashes me BEYOND
>          REPAIR.  (This implies an attempt to repair and recover).
   I fully agree. TUNES shall include support for extensive debugging.
Now, with existing software tools, writing usable error recovery code is
*very* difficult. The TUNES system shall have support for error-recovery
at language level. Exceptions, first-class continuations are good. But not
enough. Higher-order programming should allow programmable debug code
generation from annotated sources.


>      b.  All tasks, all functions are given the opportunity to recover/repair
>          their own errors instead of just being killed.
>      c.  Dumps, traces, etc. of where tasks came from, where they went, what
>          happened, how everything looks, etc. for problem resolution when
>          something does go badly.
   This is just not enough: bugs may be completely hidden at low-level.
Have you ever tried to assembly debug code generated for an OO language ?
Or the interpretation of byte-code or threaded code ? Debugging is needed
at source level. A language that requires debugging is lame. A language that
doesn't allow debugging is lamer even. TUNES proposes that programs be
annotatable with debugging information, as well as with full specifications and
proof of correctness.


>   2.  Reportability and accountability.  I want to know what my machine is
>       doing, why, when, how long, and who is causing it.
   This is administration. Again, TUNES proposes that this be solved with
dynamic active annotations: you can annotate a function f with some
code (e.g. annotate login with user access accounting, memory
allocation with quota checking, tree walking with action at node, etc).
Of course, for efficiency, functions can be closed to further active
annotation, so that it be optimized.


>   Note here, that the likelyhood of this OS being designed for "home use" is
> probably about 1%. (Less, probably).
   TUNES aim at world-wide utility: we want to provide some OS to kick out
traditional coarse-grained, unsecure, short-term design that poisons the
computing industry and forbids fair competition.
Even if TUNES does not become a world-wide OS, we hope the TUNES experience
can speed up the appearance of such an OS.




>   3.  Support for multiprocessing.  I'm including as much code as I can for
>       multiple processor support.
   TUNES strives to support dynamic code migration over heterogeneous
networks. multiprocessing is just a very particular case of this.


>   4.  Versatility.
> 
>   User programs, intermediate 3rd party products, administrator-written user
> or OS code, etc. can be very versatile, using the system in whatever
> ways they
> need or would like.  Good networking capability, excellent
> (as much as possible
> with the Intel hardware support) cross-memory services, etc.
   TUNES provides fine-grained modularity, and ends all arbitrary
distinction between OS, drivers, applications, third party code,
local configuration and user interaction. Code naturally differenciates
according to its objective properties, far better than any company or
even technical committe.
   TUNES will just provide higher-order fine-grained modules, that
can be linked together automatically or manually in any useful way
by the various people that manage or use the system.




>   Specifically, some of the things that I have in the plans or are working on
> implementing:
   Firstly, let's make it clear that all PC specificities should not modify
in any way the user-visible specifications of the system. Only the system
and compiler back-end implementors should /need/ know them.
   TUNES will be as portable as can be. Once portable code is running,
everyone is free to use the niftiest trick to make one's modules faster
and smaller on one's computer.



>   1.  Only two selectors in the GDT are static (also only 2 in the LDT).
>       The 2nd is the selector to the Common Memory Area in the GDT, the Local
>       Memory Area in the LDT (OS reserved virtual storage).
   Of course, as few selectors as possible should be static,
but more than two may be useful, be it just to allow use of code
in multiple priviledge rings, return to real or v86 mode, etc.


>   2.  OS memory can be fixed or pageable.  It can also be protected or
>       non-protected.  The selectors themselves are DPL3, allowing
>       user tasks to
>       read most of OS storage, but not modify it
>       (based on page table R/W bit).
>       Obviously, some things must reside in protected storage (U/S bit = S),
>       like network buffers, security package control blocks, etc.
   Sure, we shall take the most out of the hardware capabilities.
Now, I can't see why the programmer would want to interfere with such
low-level unportable things. We shall first limit all that as much as
possible. Later modules can implement their own abstractions and management
of these.


>   3.  An address space (or sets of segment/page tables) are not related to a
>       task (or TSS).  Multiple tasks can be run under an "address space".
   Tasks are some coarse-grained abstraction, that only introduce
overhead with no gain. Only emulation boxes and unsafe code need run in
isolated address space.


>   4.  Recovery/termination.  Any task can make an operating system call
>       at any
>       time to establish an "error recovery" routine to be called in the event
>       of an error.
   In TUNES, there is no such thing as "system call". Everything is specified
at high level, and code should be dynamically compiled/linked to achieve the
best performance (which centralized kernel call in no way allows).
   See e.g. the MIT Aegis exokernel project, to see how removing lame kernel
concept already speeds up 100 times system services.


>       This routine can correct/bypass the error, and request the
>       OS retry the origional routine, or continue the termination process.
>       Required in all OS routines, especially in global ones that can be done
>       under any TSS.  Also required in any tasks/address spaces that
>       would hang
>       the system in the event of a failure.
   Now, as for recovery, I'm sure dynamic active annotations, lazy/partial
evaluation (futures), exceptions and first-class continuations make things
easy enough, without requiring any "OS" support. TUNES does not differentiate
some "OS" from other software components. All software come into fine-grained
higher-order modules that dynamically link together. "Registering" a function
is just a very special case of dynamic linking.


>    5. Special environments for the double fault, NMI,
>       and stack fault handlers.
>       Make EVERY effort to protect these tasks code/control blocks so in the
>       event of a serious failure, the system doesn't hang.
>       These routines will
>       support dumping of the system for problem resolution.
>       (Dumping of all or
>       parts of the system are supported in all cases, initiated either by the
>       OS or a user task.  It's just that these three tasks are very
>       critical to
>       recovery of the OS, and at least problem resolution if recovery is
>       impossible).  The stack fault handler becomes "special" only in the
>       event
>       that a stack fault occurs on a PL0 stack.
   Of course we shall have code to report such things. Let's just create
a hook for it into high-level abstractions. At the first place, using only
safe code from safe compilers out of strongly typed or proven programs
should remove all such errors. If we restrict unproven untyped code to the
least possible, we know where the bug comes from, and bugs shall eventually
disappear.
   That's how TUNES see things: instead of building a huge number of paranoid
low-level checks, provide a high-level way (strong typing and/or correctness
proof) to ensure safety. It is infinitely more reliable. As for self-foundation
problem, of course there will have to be some code we trust; but again this
should eventually be as small a portion of code as possible, and its sources
shall be freely available so anyone can check it and correct it. The least
unreliable thing is what everybody checked but nobody could make fail.


>   If you're still interested after reading this book, let me know!
> I'd be glad
> to hear new ideas, implementation concepts, and even willing to help code!
   Sure I'm interested. Even if you decide not to join TUNES (which I will
be sad of), I hope you succeed. Good luck !
   And do have a look at the TUNES page (see URL below).

--    ,        	                                ,           _ v    ~  ^  --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
--                                      '                   / .          --
Join the TUNES project for a computing system based on computing freedom !
		   TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"