Don't hardwire OS features (was Re: Build OS features in So

[Francois-Rene Rideau]

> On  2 Dec 96 at 9:29, Francois-Rene Rideau wrote:
>
>> I named my previous reply to the vague "New OS idea" thread
>> "Build OS features in Software, not Hardware".
>>    But it is more general than that:
>> it is about not hardwiring any feature at any low-level place.
>> After all, it is the hardware+OS, not the mere hardware,
>> that constitutes what the user-programmer sees!
>> And this also applies to applications with hardwired interfaces, etc.
> 
> Yup. The precise placement of the line is arbitary, but it cannot be 
> above the line the programmer sees as the OS <-> my stuff interface, 
> unless the compiler is real smart and factorises complex semantics 
> out of the instructions it gets!
>

>>> I'd phrase it like so: MMUs check software for reliability the
>>> /emperical/ way; let it out to play, on the end of a rope; if it falls
>>> down a hole, we can haul it out.
>>
>> Yes, and that would be fine if tasks did not have to communicate
>> and rely on each other.
>> Only no task can be useful if it is thus isolated!
>> Utility of a task is proportional to the knowledge
>> it can exchange with the rest of the system;
>> hence an MMU is not very useful, because it only
>> manages gross approximations to security concerns,
>> that can be done better in software,
>> given proper compiler support
>> (of course, MMUs are fit to current unsafe compiler technology).
> 
> I love your capacity for damning remarks,

> usually including the word 'lame' ;-)
Yeah. I admit I don't know many familiar deprecatives.
Perhaps I should purchase this french-english dictionary of slang...

>>    With Tunes, you could add a system-enforced prerequisite
>> that input to the routine should be of the right format,
>> and it would ensure that only objects of the right format
>> are given to the routine
> 
> That'd be a precondition in Eiffel or the proposed ARGOT language.
>
Only Eiffel just CAN'T PROVE anything about the precondition.
It can only test it at runtime. And even then, this means that Eiffel
preconditions should be only computable statements.
Hence, not very expressive. All that is better than nothing,
but this doesn't remotely mean "good".

>> (which could involve full slow checking when no other proof is found).
> 
> It could involve satanic divination rituals if the implementor sees 
> fit!
>
Yeah! If the rituals yields any interesting result to you,
you should be free to implement it!
That's how reflectivity works:
the user-programmer can modify the system behavior wrt his own objects.

>>    In Tunes, some resources could have a prerequisite
>> that they cannot be locked by objects
>> not proved to release the lock in time.
> 
> Proof of this could result in having to prove that an algorithm will 
> terminate - that old chestnut!
>
The problem is when you try to prove something about an arbitrary
program chosen at random. We're talking about requiring
that programs be specifically designed not to crash the system...

> I'd be more inclined to research alternatives to standard locks.
> Not that I have yet, though, so don't get too excited.
>
There's NBS and suches. Doesn't remove the need for locks at times.

>>    In Tunes, the interrupted task would be basically be told
>> "now do your best to yield execution ASAP,
>> after putting the system in a rightful state".
>
> Yes, that's the kind of thing I'm aiming at with implicit yields. But
> there's a temporial and informational efficiency penalty!
>
Time penalty: not sure, because cooperation reduces the need
for blind context saving (and according page faults),
while saving the need to explicitly check the most frequent locks,
that will have been inlined into the cooperative convention.


>> Inserting yields shouldn't bloat the object code,
>> since it will be compressed out in disk-stored versions of object code,
>> and only inserted at execution time, when they can indeed be inlined!
>> Plus all the bad-case-checking made unneeded by cooperation
>> will help win back the space lost by those yields().
>
> I was worrying about the in-memory space! Larger machine code = more 
> swapping...
>
We also have
Not cooperative
<=> more context to save& more code to acquire frequent locks
<=> more swapping
So that a study must be done about it...


>>> Trap handlers in the OS don't take up much space, and I don't expect a
>>> runtime compiler to be a small piece of software.
>>>
>> Trap handlers in the OS do take place.
> 
> Yes, but not an incredible amount! They fit in QNX's 8k, along with 
> some other stuff, right :-)
>
What you are talking about is only trap redirectors.
If traps are not to be used, this is already too much;
if they are to be used, then you also need a real handler
outside of QNX, which will take place, etc.


>> Minimal Instruction Set Indeed!
>> As for bloat, I see no problem with size of speed-critical sections,
>> as long as it runs blazingly fast!
> 
> Yeah, but it's /all/ big, not just the good bits :-)
>
No, because of threading techniques.
e.g. 15-bit onpage "call" instruction to compress code...

> Unless we have pairs of CPUs - a MISC controller, with a set of CISC
> coprocessors to do stunning single-opcode FFTs, raytrace in hardware,
> that sort of thing!
>
I'd see it as the coprocessors as being stupid specialized hardware
rather than CISC CPUs: for instance, the FFT copro
would take the address of a matrix in RAM (or would I/O the coefficients),
a trigger I/O, and do its business as it likes.
Anyway, unless your processor is *SLOW*, the FFT will take so many
cycles that the few ones to set it up through I/O are a save
in comparison with the slowdown due to the logic for 8087-like
coprocessing.

>> For other sections where code density counts,
>> then compression should be achieved by threading methods,
>> and MISC computer allow *very* compact and efficient threaded code,
>> with calls that fit 15 bit!
> 
> That's interesting. What do you mean by that?
>
Because a call instruction is so short,
you can compress your code by calling well-factored routines,
instead of inlining them. The stack helps factoring a lot
(as compared to register machines); all in all,
for non-critical code, you can achieve great code compacity
by having lots of subroutine calls.
(compare that to 40 bits needed for a subroutine call on
intel architecture, 64 bits on most RISC architecture).


>>> Hmmm. I would beg to disagree; I interpret that as the RISC-style
>>> argument that we should have real simple hardware being driven by
>>> software. The software has a small set of operations from which it
>>> builds everything else up.
>>>
>> Exactly! Don't have bloated hardware that costs a lot,
>> can't adapt, and does things slowly,
> 
> Bloated hardware can do things faster... do you disagree that if 
> something can be done in software on a general purpose CPU, a circuit 
> can be made to do it much faster? Yes, it's hardwired and inflexible, 
> but if we're building a computer for multimedia applications we can 
> guess that it'll benefit from hardware MPEG decoding!
>
Specialized != bloated.
An PentiumPro is bloated when it tries to decode CISC into RISC,
schedule it at runtime between several pipelines,
include several levels of cache to do "as if"
it were a Von-Neumann architecture, etc.
An MPEG decoder is not bloated in so far as it
spends just enough chip resource to decode enough blocks in time.
Bloated == uselessly big and wasteful


>>    FFT should NOT be in any instruction set.
>> Instead, it should be in compiler libraries where its place belongs.
> 
> That's semantically identical to it being in an instruction set! Note 
> that qsort(), the quicksort, is in the C instruction set...
>
Nope. Being in a library extension to C is not the same as being in C.
The former means that you can strip it off the C implementation,
that you needn't special compiler support,
and that other features can similarly be added.
Making it an instruction means that it has a special compiler support,
that other extensions can't have.

>> Yes yes yes. Object code should be irreversibly produced for a particular
>> target, while a *high-level* language or maximal possible expressivity
>> should be used to represent user-level and/or portable objects.
> 
> Until we develop a truely fundamental theory of computation, 
> anyway...
>
Not sure what you mean.
If you mean that we don't currently have a nice
theory for what "expressiveness" is that makes the Tunes HLL
better than a Universal Turing Machine,
that's what I'm heading toward...


>>> Hmmm. Modern compilers (such as gcc) can write better assembler than
>>> most humans - except for such chestnuts as C's semantics being too
>>> lightweight; there's no rotate instruction in C, and the compiler
>>> cannot isolate pieces of code that are equivelant to rotates well
>>> enough to use hardware ROT instructions.
>>
>> This is no problem. See how linux/include/asm*/*.h does it. Neat!
> 
> Yeah, but that's not part of the C standardl writing "portable" ANSI 
> C, we can't assume anything like that, and have to write rotation in 
> terms of << and>> and | and all that stuff.
>
Two things: 1) GCC can also compile <<|>> statements to ROT.
2) even if it couldn't without the help of a asm/*.h macro, so what?
People do not program C because ANSI C is a nice language
that should do everything;
C is chosen but because it is actually available on actual platforms.
There's no problem with providing features
through architecture-dependent files.
C is architecture-dependent anyway (pointer sizes, etc).
Moreover you can still have generic versions as in
linux/include/asm-generic/*.h


>>> A language like Eiffel or maybe Ada is well suited for high and low
>>> level coding.
>>
>> Eiffel and Ada *suck*. Use functional languages instead:
>> CLOS, OCAML, Clean, etc.
> 
> Another of those cutting comments :-) My plan for ARGOT (the 
> language) bytecode contains some functional features. I won't be 
> studying much functional theory until I get into University, though. 
> Unsurprisingly, I want to take a degree in computer science :-)
> My admissions interview for Cambridge is on Thursday - wish me luck!
>
Good luck!
Do take courses in functional language theory and practice.


>>>>   This means the end of the need to explicitly generate/parse,
>>>>check/blindly-trust and convert/ignore file formats,
>>>>which eats most CPU time,
>>> 
>>> But there need to be agreed formats!
>>>
>> Yes, but agreement should be found within the system
>> without the casual user having to know about it!
> 
> Yup; I don't see how this can be done without the programmer's 
> input, though - as in completely automatically!
>
Complete automaticity means that *one* programmer did it *once*,
and the user doesn't have to worry about it thereof.
Of course the computer won't invent the formats,
the conversion routine, and the call to it!


>>> How do we transfer a copy of A's picture to B?
>>>
>> *We* don't. The computer system does it silently thanks
>> to its RGB->YUV tactic.
>
> The royal we :-)
>
> The programmer must have told the system about some kind of way of
> folding an image to a data structure consisting of only standard
> objects, which can be reconstructed elsewhere, though - independently
> of the internal representation of images at either end.
>
Sure. But that's the same as above:
once it's done for the first time,
no one needs to care about it anymore,
unless specifically tweaking it for optimization.

>>> If A and B used the same internal representation of pictures, then we
>>> could transparently transfer that array of integers. But they don't. 
>>>
>>> Converting Yu'v' to RGB will induce a loss of information, for a 
>>> start!
>>>
>> Only if the converted version is meant as a replacement for the RGB version!
>> On a Good System (TM), the YUV version will only be a cache for displaying
>> the original RGB object on YUV systems. The system knows that
>> the original object where the information is taken from is the RGB thingy.
> 
> But this is assuming that the RGB system understands some concept of
> Yu'v', and the Yu'v' system understands RGB. And they must both have
> a concept of the UVW space used by the truely wierd computer in the
> corner. 
>
My point is that you can apply irreversible lossy transforms to a RGB
picture for display on a YUV display, but that this will be
no information loss problem, because you will have remembered the
original RGB value for use when extracting information lost
by the YUV conversion.
   The RGB system needn't understand YUV, etc.
   What is needed, is that the system knows
* where the information comes from,
* which operations are reversible or not.
   Therefrom, knowing that the YUV conversion was lossy,
it won't recompute RGB back from YUV when asked for RGB,
but have kept RGB as the original object.
When asked about UVW, it will know that the direct
conversion from RGB is better than indirect conversion through YUV,
hence do in consequence.


> Things like "array" are 'atomic' for transmission, in that they 
> explain themselves, as do things like integers.
>
Yes. I wouldn't use the term "atomic", though.
Rather "terminal", "context-free", "self-sufficient",
or "easily reifiable", or perhaps only "pure".

> I'd be inclined to say "Right, Yu'v' with 32:16:16 bit precision is 
> enough to specify an image for the human eye. We can transmit an 
> image as three arrays, one for each component, and we can tell the 
> system it's safe to apply lossy waveform compression to these arrays 
> in transit.
>
> We might want to allow variable precision on those Yu'v' channels. In 
> fact, why not. Let the precision be controllable by the programmer, 
> defaulting to 32:16:16.
>
I see no reason to define overkill image formats
just for the sake of uniformity. If you need the original image,
take it, instead of wasting bits in a 32:16:16 Yu'v' stuff.

>>> Complex a,b;
>>> a.process_command_string("Please add to thyself the complex number " +
>>> b.render());
>> 
>> Now you also know why /bin/*sh, m4, TCL, some C/C++ packages
>> and all other such stuff suck:
>> they use dirty backdoor evaluation as a hack to compensate
>> their lack of recursive or reflective expressiveness.
> 
> Another wonderful nice sentence :-) I'm not flaming you for these, 
> BTW, don't be offended... I just admire your directness!
>
I'm not offended. I take it as a compliment.

>>> How do you intend to generalise the definition of interfaces like
>>> that?
>>>
>> Do you want a formal definition for an interface?
> 
> If you think it'd help me!
>
Well, now that we have formalized what an interface is,
writing a program without specifying the exact interface
is just a matter of
having
   ...
   interface(foo)
   ...
   interface(bar)
   ...
The interface macro will expand into an epsilon of type Interface_T
that the system will reflectively fill into whichever interface
is appropriate to the user
(WIMP interface, text windows, tty, voice, smell, etc).

>> You can add criteria on the above objects
>> for the computability/acceptability/complexity/whatever
>> of displays and interfaces.
> 
> Right! I take this to mean, you /do/ explain how the object should be 
> represented/interacted with when you are designing/modifying it :-)
>
I'm not sure what you mean.
What I mean is that the formal definition for an interface that I gave
could be refined with other criteria to judge to quality of the interface
(e.g. Kolmogorov complexity,
amount of information gatherable per single view,
vs amount of information to feed the system needed
to choose the "right" view, etc).

>> Surely, any knowledge in the computer must have be put inside by humans!
>> This is not what I pretend to change. No one can.
>> What I'd want to change is the fact that currently specific interfaces
>> are hardwired into object code,
>> whereas I want generic interfaces to be defined apart from it,
>> by a system of annotations to code and tactics to dynamically
>> generate interfaces.
> 
> What I take this to be getting at is not that you're aiming for
> "self interacting objects", but more for a philosophical thing in
> programming style that we keep the interface seperate from the
> "gubbins" (innards).
>
I achieve one by the other.
Thanks to reflective programming,
I can postpone interface and implemenntation stuff
outside of formal object definition.
When I have enough information, I can

> Ie, don't even think about designing the higher
> level interfaces until you've made the object work from the most
> basic and fastest interface of "a.b(x,y,z)" or whatever your
> favourite syntax is!
>
I'd rather say "don't need think about it",
as the system can "think it over" for you.
Surely, you should be able to do it if you want.

>>> How will you implement this interface generation thingy?
>> By reflective epsilon-calculus:
> Oh boy.
> [jargon snip]
> Um... can you recommend a net-available introduction to 
> epsilon-calculus?????
>
Unhappily not currently:
I only recently invented the concept for use in reflective programming.
Perhaps the concept has been previously invented by someone else
under some other name; but I don't know it. If you find something,
please do tell me! [I've been told about some kind of mu-calculus...]
   The name epsilon comes from the Hilbert epsilon symbol,
as used in formal logic, where
epsilon x.P which represents a object x fulfilling the property P,
if there exists any (if there doesn't, either the statement
is considered invalid, or an arbitrary object might be returned).
Basically, epsilons are to existential quantifiers
what lambdas are to universal quantifiers
(also see Curry-Howard isomorphism).
   Only epsilons are NOT computable in the general case,
so that no one before me seems to have had this ridiculous idea
of using some epsilon-calculus in computer programs!
But after having tried hard to formalize reflective programming,
I found that epsilon-calculus was exactly
what reflective programming was about:
externally and dynamically defining tactics
to fill those holes in computable foundation.
Much more general than late-binding
(one of the two principles of traditional "OO", besides encapsulation).
   All this might look difficult to understand for
people without a background in formal logic and its relation
to computational semantics. Please excuse me.
Writing a well proven clean and understandable article about that
could be a job for a masters thesis...
   Any taker?

>     Finally, the fact that different languages are given to
>> "programmers" and "end-users", each being crippled in many ways,
>> constitutes a barrier between them.
>
> Uhuh. Not that we want to bring back the days of BASIC as the OS CLI, 
> like on the old BBC micros - although that sure merged users and 
> programmers, the implementation was naff!
>
Today, FORTH systems, HP RPL-based hand calculators, Mathematica,
some LISP systems (Emacs?) and other such software,
do provide a unified computing environment,
much better than old BASIC-based systems of the early 80's.
Only they don't ambition to be generic enough
so as encompass the whole of computing in a unified environment:
FORTH is for low-level programming;
RPL for hand-calculators only;
Mathematica is directed toward doing math,
and is a purely commercial product;
Emacs ain't efficient.
Other LISP systems might be nearer to ,
but there's no effort to build 

> Right. You'll be comforted to know that I designed the ARGON
> paradigm, then invented a security layer for it, rather than vice
> versa :-)
>
Ahem. And did you choose scoping as the basic protection paradigm,
as in Tunes?

> --
> Governments are merely protection rackets with good images.
Some are, or tend to be. Happily they are not completely so.
Or else I would leave my country, or revolt against it!

== Fare' -- rideau@ens.fr -- Franc,ois-Rene' Rideau -- DDa(.ng-Vu~ Ba^n ==
Join the TUNES project for a computing system based on computing freedom !
                TUNES is a Useful, Not Expedient System
URL: "http://www.eleves.ens.fr:8080/home/rideau/Tunes/"