MOOSE Project -- new OS ?

[Francois-Rene Rideau]

> bit, but because I'm on an 8086, I have to say 
> 	TEST BYTE PTR etc
> 	OR   BYTE PTR etc 		;rather than:-
> 
> 	BTS  BYTE PTR. 
To me, this means a hardware abstraction layer must provide abstraction for
all those kind of bit operations, where C (for instance) only provides ands,
ors, xors, and shifts. Being able to do this directly on registers also means
this hardware abstraction layer should still allow hardware access when needed
(much like _AX statements in turbo pascal).
But as I'd like to write portable code, I want the capability to have
multiple versions of the same procedure, for different processors, and a
version without hardware dependency, that the compiler will have to optimize
itself. When possible, I'd also like to give more or less info to the
optimizer. It doesn't always need exact register allocation directives (on
RISC computers, reg. alloc. is done much better by computers than by humans)
GCC allows such things, but in a clumsy way. 

> [...]
> Now, What you are saying is that what counts is the semantics,
> not the syntax.
> Ultimately, that is correct, but my concern is that if the syntax is not
> appropriate to the task, the semantics never get off the ground.
I also said that the syntax should be designed to make the semantics easily
accessible. I never meant to say that syntax is not important at all.
Perhaps I've not been clear about that.


> I have seen
> too much commercial software that is complete crap because (at least in my
> opinion) the whole design philosophy and implementation is poor.
> [...]
> that are difficult to trace. Great ideas. Poor implementation. Or, to put
> it another way, great semantics, poor syntax? 
I've never ever seen any commercial software with good semantics, and syntax
was even more horrible.


> In other words, I think that there is a minimum level of syntactic
> "correctness" (for want of a better word) without which your Airbus 400s
> plunge into the ground (if they even get off it in the first place)! The
> semantics work with this, rather than leading it.
Ok. I agree.


> [about 3 levels of abstraction]
ok, there is the machine, the software, the human.
But as you said in your previous message, but the software
is infinite in diversity, and sub-levels.

> Here is really the nub. Unfortunately, one could sit and speculate forever
> about an ideal language (on level 1) but in real life I feel that you have
> to steer a middle course between the Scylla of making things too general
> (and sinking beneath the waves because everything is too wishy washy to
> actually work in real life), and the Charybdis of being too specific (with
> the consequence that you are rapidly overtaken by technology, and, even worse,
> you have to continually modify your code, indeed your whole approach, with
> the consequence that you go under in a welter of backward compatible 
> amendments).
I think that with the "have multiple versions of the code" technique, we
can have both the advantages of specific/optimized code and generic/portable
code.


> [...]
> So, in answer to your >>> I think that semantics and syntax are equally
> important, and in a workable system must be considered as working together.
> I hope I'm not being too simplistic!
Ok ok. They must be working together. I do not deny it.
But these last months have been working with a purely syntactic system,
which had no designed semantics, and where features had been added and added
without a general plan, ending with a syntactic monster (if you read french,
I'll send you my masters report). It's just unusable.
So what I say is you shouldn't add syntactic features on the fly without
studying the semantical effects, and that things that appear syntactically
simple are unusable semantical monsters (like this B stuff I've been working
with). Not to say that what YOU wrote is such (I do not know it very well,
and it did not look that way to me).

> [...]

> I think that in designing any system, half the art is in asking the right
> questions. So I have a few for you!!
I think it's 90% of the art, as the right formulation of the question leads
to the answer 80% of the time.

> Q1. Do you accept the general idea of having 3 levels, as defined above?
> If not, is this because you a. feel that my "drawing the line" at level 1
> is too rigorous (I feel you have to draw the line somewhere), or b. Some
> other reason?
Well, yes and no: to me, the level 0 (hardware dependency) is very limited,
and expands very slowly. Level 2 (human interface) we have little control on
it, and it's not the algorithmically interesting part of a program; let it
be automatically generated when possible. Level 1 (the algorithmic part of
the software) is infinite, and contains an infinite partially ordered only
hierarchy of sublevels.


> Q2. Do you have a "minimum component list" in terms of what you would like
> a CS to do? Do you perceive the CS as having "components", and how do you
> conceive of them interacting (I'm trying to keep this as general as possible).
As for a minimum component list, I see none. Well if you want a minimal system
for computing, then I'd had enough to allow lambda-calculus and end up with
Scheme (i.e. skimmed Lisp). If I wanted a portable system, I'd first abstract
level 0 completely. If I wanted  secure system, I'd first add a type-checking
system.
But I do not see a "minimal component" list independent from the task to
program. Again,  program has its own minimal component list, from which it
builds (say, by disabling function definition inside the current one).
What I see is the infinite hierarchy of level 1 modules, each using lower
ones. I also see some of these modules recursively defined, with multiple
sub-objects tightly interacting with one another.

> Q3. Have you thought about legible constructs for _parallel_ architectures?
Yes. first thing: do not use global variables when possible, so that
distribution be as simple as possible (without some global thread being
ever accessed). Then a generic FOR construct with a variable taking all values
in a set, and a recursive FOR construct to use recursively defined structures,
will provide distribution. Annotating objects by their expected/measured size
(in memory/time) will allow automatic scalable distribution. For optimal
thread migration, all thread information should be minimal; a portable
low-level language should also be used to allow distribution on heterogeneous
architectures; but architecture-dependent versions could nevertheless be
used to allow optimal performance on common architectures.


> (**)
> By the way, I have some slightly unorthodox ideas about assembler. Many 
> people seem to have completely the wrong concept of assembler, and this
> is often fostered by the totally incompetent assembly "environments" that
> are foisted on us by incompetent designers of assemblers (which at best
> work most of the time, but make IBM/370 JCL seem almost pleasant)! And
> assembly language programmers often seem to use totally bizarre constructs
> either for the hell of it or just to save a miniscule amount of time or
> space! A lot of the blame must also be taken by the chaps who design the
> instruction sets. CISC should never have come about, and would not have
> had _any_ chip designers had a modicum of insight into how to _really_
> program in assembler! (This may sound arrogant, but I do not lay claim
That's why RISC chips were designed (see the ARMxxx series of commonly
available RISC chips since 1985).


> to any magnificent insights into assembler, dammit I'm still on the
> learning curve with no end in sight). Also, a lot of people seem to
> delight in going down to the assembler level and then immediately trying
> to climb back up again by creating really disgusting "high-level-type"
> constructs with "if then else" syntaxes etc. Yeeech. Assembler should
> be the easiest language to program in. Perhaps we can discuss a few ways
> in which this could be implemented (if you accept the "level 1" idea).
If you're designing an new chip, that sounds ok. But if you're writing a
computing system on existing architectures, then I think it's too late to
correct the asm.


> Me neither. I think that I'm too stupid to use OOLs. Or maybe the chaps who
> design them are too stupid to advertise them in a way that the common idiot
> can understand! My supply of metoclopramide ran out long before I got into
> C++. ((Metoclopramide is an anti-nausea agent)).
C++ is definitely not an OO language. It's a big bunch of crap, built over C
to make it look-like OO. To see real OO, look at Smalltalk, SELF, and BETA,
or Eiffel, or Sather (but BETA is seem better).




> >What we need is a single language that can extend itself naturally, specialize
> >to any field of computing.
> 
> Agreed, absolutely. If you cannot fit this into the framework of my ideas 
> above, _please_ tell me your ideas in this regard.
I think your ideas are ok.


> >> The solution as I see it is to keep things 
> >> absolutely simple (a la RISC), and rigorously exclude any instructions/ 
> >> constructs that impair lucid interpretation on any level.
> >Yes, and no (according to the point of view).
> >Yes, basic constructors should be simple, so the semantics is easy to
> >understand.
> >But no, everything shouldn't be done directly from those simple construtors.
> 
> Agreed. Abstract all you want on level 2, _and_ design level 2 to make
> abstraction easy. Ideas?
That's where the OO design is helpful (what I call OO is not what C++ or
commercial stuff present as OO): that is pure, completely parametrizable
generic abstraction, with multiple numbers of more specific versions that
go down and down to the hardware. The user will just ask for the abstract
construct, and the compiler (either computer or human) will choose the best
fit specialization for performance.


> > Every program must be defined in its most generic form,
> >and the user will specialize to the form that allows best performance.
> 
> Wellll.. Often performance is defined on the lowest level (level 0). You
> may _have_ to leap down to the lower levels to get best performance, but 
> all the dirty work should be done there, and be transparent to the top
> level user who doesn't want to know about it. BUT if you have knowledge
> of all levels, it should be _easy_ to get into the lower levels, with
> helpful debuggers etc as a standard part of every CS, even down to arcane
> details of timing etc. I think that obscenities where you _cannot_ even
> find out what some components do (eg certain registers on the pentium)
> should never be acceptable!!
I agree. But all portions of code do not need the same attention and
optimization; I really think the "multiple version" stuff should be
used. Automatic version translation should also be used, so that the user
justs enter the one version he needs now (using the proper abstraction),
and a generic version is generated.


> >Reuse *is* important. Today, reuse above language constructs is 0%, because
> 
> Hear, hear. That's one of the reasons why I'm all for simplicity, and 
> at least the basic defining of what is on which level. Also, it is then
> easier to define an intelligent "apprentice" program that will help you
> on several levels, and ultimately even understand _exactly_ (is this 
> possible??) what imported code does! At some level, all code should be
> lucid and explicit, and straightforward.
That's also why I insist on having code specification in some version
of code, so that we know what the program does externally, without knowing
its internals. That's the only way of reusing code you don't have time to
understand.


> >But also note that for reuse to be possible, security is a must, thus
> >type checking and more than that, program specification. If you can't be
> 
> Yep. Agreed. Details of implementation are another matter. Ideas?


> >> 6. So I see any programming attempt as part of a continuum. There may be 
> >That's exactly what I just said. We constantly need abstracting some part
> >of the code. There is no fixed limit of three or four or any finite number
> >of layers in abstraction (which is why ISO standards try to do with their
> >bullshit OSI layer model). Each program is well defined in its own proper
> >layer of abstraction; and layers are not in a total ordering, but a partial
> >one: if A needs B and C, D needs B and E, there's no reason to put A and D in
> >the same layer, or one above the other; they're just defined in different
> >non-disjoint contexts.
> 
> Hmm. Q4. How would you relate this to my ideas above? The way I see it is
> that ultimately everything devolves to level 0. Level 1 I see as a necessary
> evil, and what you do on level "2" (and how you subdivide/extend it)
> is your business.
Perhaps I did not get what exactly you called levels 1 and 2.


> >> memory and / or output ports. Unfortunately, we often seem to be driven 
> >> by the hardware designers  [...]
> >> THIS. THIS IS OUR STANDARD. WE WILL DEFINE OUR SOFTWARE AS SUCH AND SUCH. 
> >That's even worse than OSI: just one big level of abstraction, with an
> >arbitrary choice of functions.
> 
> I hope I have more clearly explained what I mean above (!?)


> >>If I go into a restaurant and order a steak, I ...
> >You too see that program specification is needed. I too have known it for
> >a long time, but now, I understand it and why it is needed (and not only a
> >toy). And timing specification can be as useful as effect specification,
> >as our computers will never ever execute programs instantaneously. But as
> >for the *exactly*, it is impossible. *enough* is quite enough.
> Absolutely. I would like to have timing considerations built into level 1.
> Maybe "absolute maximum time", "absolute minimum time" and "mean time"?!
> But keep the specs. as simple as possible. Life is complex enough as it is!


> >> Level 0: lowest level. May need considerable effort / ingenuity to ...
> >Yes, but this level should also be implementation-dependent, and reserved
> >for use by those who *really* want it, in time-critical portions of their
> >programs (i.e. ultra-fast graphics or signal processing, OS code, compiler
> >back-end, etc). Normal users shouldn't *have to* use it. And a code
> >portion could have several equivalents in several different low-level contexts.
> 
> Yes, as above, but accessible if required. Also, where you anticipate (in
> designing level 1) that several options are/will be needed [got your crystal
> ball handy?] make this explicit!!
It should be accessible to users *but* only in a way such that it can be
proved (automatically or manually) that their program is correct and won't
take over system security. This can be done if each component is
annotated by an exact specification.



> >> Level 1: The "assembler" level. But cf my example above about drawing a ...
> >Assembler is still in the implementation-dependent part to me.
> >Note that I do not want to forbid low-level parts; just to allow not to use
> >them, and to provide portable equivalents (thus a time-critical routine may
> >have several versions to run optimally on different hardware, whereas a
> >non-time-critical routine will only have a slow high-level version).
> In my scenario, the machine-dependent part is all on level 0. Level 1
> is "low level" but abstract and machine-independent. The "assembler" is
> a symbolic "assembler" that is idealised to make it user-friendly. (Lots
> of ideas here). You absolutely CANNOT use explicit level 1 code in level
> 2. Such an idea (inline code etc) is an abomination!!
To me, you can, but only if a specification and portable version already
exist (though these can be automatically generated from your inline code).


> >> Level2: The "top" level. Perhaps something along the lines of the 
> >> incredibly simple language I gave you a flavour of above.
> >Your language is too low-level to me. Again, I love low-levelness (or I
> >wouldn't be writing an OS !), but only when it is needed. Sometimes (most
> >of the time, once the OS is written, if it really is well written), a high
> >level of abstraction is needed.
> "Abstract" as much as you want! As explained. I'm really eager to hear
> you ideas in this regard.
I already told everything: you have absolutely high level abstractions,
annotated by lower-level versions, so that to be actually run, an object
must be ultimately build on level 0 (though as intermediates, there can
be arbitrary code, including interpreters, inference machines, and any kind
of inefficient but generic things, that could be further replaced by more
efficient things, *if you need it*).


> >I really need someone ready to dirty his hands with the low-level stuff in
> >my OS. Uhappily, whereas I have many ideas, I *know* that this stuff is the
> >non-essential part of the OS. By non-essential, I mean that if I am to convince
> >people helping me in my project and later using my OS, I must have a *working*
> >version, however slow, whereas low-levelness will bring me speed and
> >efficiency but nothing that would make my OS better than it would be if
> >running on a faster computer (that would be available for a higher price, or
> >just by waiting long enough). Thus, I deeply need help from someone who will
> >cope with the low-level details of the system. Will you be that man ?
> >I cannot promise any success, just tenacity.
> 
> I would be very keen to give it a bash. I can also be pretty tenacious.
> What time-frame are you looking at, and do you have any (no matter how
> tentative) specs on the "OS/CS"? What are your constraints? Note that I
> rather fancy top-down development (even here) so I would try and give you
> a skeleton working system ASAP.
Ok. I'll mail you as soon as I've gathered the team.