MOOSE Project -- new OS ?

[Johan Van Schalkwyk]

Dear Francois_Rene

>See ML for a typed functional language (CAML-light on ftp.inria.fr is a
>free imlementation that runs on macs, pcs and unices).
Thanks

>Well, I mean no offense, but syntax is not what's the most important to me.
>To me, semantics are quite more important, and user-level macros or parsers
>should allow any type of syntax one likes.

Maybe I've been a bit _too_ specific, but my problem is (as noted) that I
would attach less importance to syntax if I had found just _one_ language 
whose syntax appealed to me! I prefer to try and see a synthesis between the
message and its form (!?). Also, I have found (have you not?) that often when
programming, you say "dammit what I really want to do is test and set the
bit, but because I'm on an 8086, I have to say 
	TEST BYTE PTR etc
	OR   BYTE PTR etc 		;rather than:-

	BTS  BYTE PTR. 
or eg. on a Z80 (really going back a bit) you might want to simply exchange
two registers, but have to do this via the stack because the architecture 
simply does not permit you to do so directly. The meaning is the same, but
the form is different. Such examples are trivial (and make no difference to
the final result unless timing is critical) but unfortunately when we are 
working in real life, especially with very complex programs, it may be
critically important to 
	a. have readily comprehensible code (and a small difference can make
		an enormous one - cf. the early space probe, I think it was
		one of the earlier Mariners, where a comma was left out, 
		(in a DO loop) resulting in a minor course correction
		instruction causing the several hundred million dollar
		spacecraft to do a sharp left and never be heard of again)
	b. Define right from the start not only what you want to say, but
		how you want to say it. 

Let me give you an example. Let us say you are contemplating designing a
complex virtual reality system, and one component is obviously the display
of realistic visual information. There are several components:-
1. inputs which determine (ultimately) what should next be displayed
2. processing
3. display of the information.

Now, a simple nuts-and-bolts approach might be to have a perfectly generalised
and very fast system that displays an eg 2k x 2k pixel image in 32k colours.
Let's now look at this ideal system in terms of idealised levels of processing.
On the top level, you might have the user sitting in his abstract world
and interacting by speaking/pointing to a part of the picture, or whatever.
The next level down might be a complex high-level processing determining
"what to do next etc". Then we have a lower level of coding (perhaps a sort
of assembler level, where the nuts and bolts are assembled(!)), and finally
we have the hardware implementation. 

Now, What you are saying is that what counts is the semantics, not the syntax.
Ultimately, that is correct, but my concern is that if the syntax is not
appropriate to the task, the semantics never get off the ground. I have seen
too much commercial software that is complete crap because (at least in my
opinion) the whole design philosophy and implementation is poor. I'm sure
the chaps had great ideas and put an enormous amount of effort into the
product, but somehow they screwed up along the way, and the end result is
slow, clumsy and unattractive, and, worst of all, has intermittent errors
that are difficult to trace. Great ideas. Poor implementation. Or, to put
it another way, great semantics, poor syntax? 

In other words, I think that there is a minimum level of syntactic
"correctness" (for want of a better word) without which your Airbus 400s
plunge into the ground (if they even get off it in the first place)! The
semantics work with this, rather than leading it.

I also think that the buck has to stop somewhere. Let us say (in the above
VR example) that you want to draw a line. On the top level, you might be
an artist drawing an image, or an engineer drawing a machine component, 
or whatever. Lower down there might be the equivalent Draw instruction, 
and lower still there is the implementation that brings this about on the
actual image. This is really what I meant by having three levels - you can
have a lot more, but to me, three is the minimum, and in my plan you should
see everything on level "2" as high level (you can stack as many levels on
top of it as you want, in my idealised viewpoint, they are all much the
same), and level "0" as the machine-dependent level (ideally all done in
hardware, especially if speed is a consideration). Level "1" is the key,
and it is here that I would try and succeed where C (for example) fails -
I would make it in many ways resemble conventional assembly languages (BUT,
as I will explain later(**), not perhaps your perception of assembler), but
with the capacity for quite dramatic information hiding where appropriate.

Here is really the nub. Unfortunately, one could sit and speculate forever
about an ideal language (on level 1) but in real life I feel that you have
to steer a middle course between the Scylla of making things too general
(and sinking beneath the waves because everything is too wishy washy to
actually work in real life), and the Charybdis of being too specific (with
the consequence that you are rapidly overtaken by technology, and, even worse,
you have to continually modify your code, indeed your whole approach, with
the consequence that you go under in a welter of backward compatible 
amendments).

I really believe that the decisions you make about (that filthy word) syntax
on this level colour your whole approach to implementation, and ultimately
affect the quality of the result. A painter may have great ideas, but if
his technique and tools are poor, it is highly likely that his art will be
awful. Contrariwise, if his tools are good and his ideas are rubbish, he
will definitely produce rubbish. To me, the syntax and the semantics go
hand-in-hand, and you cannot neglect the fine structure if you want the
whole picture to turn out right!

So, in answer to your >>> I think that semantics and syntax are equally
important, and in a workable system must be considered as working together.
I hope I'm not being too simplistic!

I've mentioned how I consider the buck to stop on level 1, but have given
no answers to the question of how to steer the middle course alluded to
above. I have a few ideas about this, although I probably haven't considered
things in enough detail. I would like to illustrate a concept that I could
perhaps call "impedance matching". The basic idea is that if you want to
communicate with a human being using sounds, you would employ a frequency
range between a few hundred Hz and about 15-20 kHz. If you wanted to 
communicate with a bat, you might have to work up to 50kHz or more. But
if you want to communicate with a human and want him to _enjoy the experience_
you would have to modify your signal considerably in terms of amplitude,
frequency distribution etc etc etc.

Similarly, if we look at input / processing / output in a very general
fashion, can we not apply similar ideas, ie. make both the i/o and the
processing relevant to communication with real human beings (or, in
computer <-> computer communication, real computers)? 

Returning to my VR example, one can (with justification, I think) ask the
question "What are appropriate level 1 constructs that will allow us to
near-optimally visualise and implement a VR system, as a sub-component 
of all possible communication with humans using visual means"? You'll notice
that I'm keeping this entirely general.




Much recent progress in computers centres around eg. voice-activated control,
- can we not look at the input constructs on level 1, and "match the
impedance" to what is required? (I'm not making specific recommendations,
note, just asking a general question)! For example, if careful analysis
of the requirements of such a system suggests that a primitive instruction
"get a phoneme" is a good idea, then have it as a simple construct on level
one. The buck stops there. There may or may not be multiple layers of
processing on level 0, but this should be transparent to the level 1
user, and also to users on the higher "level 2". 




We have considered i/o. Now let's look at the actual processing. I see
level 1 as being completely general in this regard, with no committment
to (in your example) list / array-based processing. But the same criteria
that I applied above (ie impedance matching) should be applied to the 
choice of "processing" constructs available on level 1.

Likewise for the implementation of a language on level 2. Unfortunately, in
trying to be general, I became very specific. I was merely giving an
example of syntax in my language, but, if you think about it, this does
not preclude the use of e.g. arrays or whatever. I made no mention of
handling of memory or whatever, I was just giving an example of a convenient
syntax that I find fairly explicit, and, well, pretty general. (I have a
whole lot of ideas about memory handling that will probably make you tear
your hair out)! 



I think that in designing any system, half the art is in asking the right
questions. So I have a few for you!!

Q1. Do you accept the general idea of having 3 levels, as defined above?
If not, is this because you a. feel that my "drawing the line" at level 1
is too rigorous (I feel you have to draw the line somewhere), or b. Some
other reason?

Q2. Do you have a "minimum component list" in terms of what you would like
a CS to do? Do you perceive the CS as having "components", and how do you
conceive of them interacting (I'm trying to keep this as general as possible).

Q3. Have you thought about legible constructs for _parallel_ architectures?



(**)
By the way, I have some slightly unorthodox ideas about assembler. Many 
people seem to have completely the wrong concept of assembler, and this
is often fostered by the totally incompetent assembly "environments" that
are foisted on us by incompetent designers of assemblers (which at best
work most of the time, but make IBM/370 JCL seem almost pleasant)! And
assembly language programmers often seem to use totally bizarre constructs
either for the hell of it or just to save a miniscule amount of time or
space! A lot of the blame must also be taken by the chaps who design the
instruction sets. CISC should never have come about, and would not have
had _any_ chip designers had a modicum of insight into how to _really_
program in assembler! (This may sound arrogant, but I do not lay claim
to any magnificent insights into assembler, dammit I'm still on the
learning curve with no end in sight). Also, a lot of people seem to
delight in going down to the assembler level and then immediately trying
to climb back up again by creating really disgusting "high-level-type"
constructs with "if then else" syntaxes etc. Yeeech. Assembler should
be the easiest language to program in. Perhaps we can discuss a few ways
in which this could be implemented (if you accept the "level 1" idea).


>
>That's prolog-like definition of a name according to what is input or output.
>I like it. But perhaps the BETA way of doing it (having an x subobject, where
>any object setting calls an object-defined function) is better. I dunno.

Me neither. I think that I'm too stupid to use OOLs. Or maybe the chaps who
design them are too stupid to advertise them in a way that the common idiot
can understand! My supply of metoclopramide ran out long before I got into
C++. ((Metoclopramide is an anti-nausea agent)).

>
>
>But I think you're overstressing the syntax. I myself lost several years
>for the same reason. What Iearnt those last years when I could see the world
>through internet and CS courses, is that semantics (i.e. the meaning, the
>contained thing) is more important than syntax (i.e. the aspect, the
>container); this I've always known, but it somehow became clear and obvious
>to me. Of course, badly designed syntax won't allow easy
>description of the semantics, but I'm sure that semantics should be designed
>first, and that syntax should be designed and modified to conform to it.
>Please think about it. I'm talking from experience I'd like to share.
>I lost much too much time thinking about the perfect natural-language like
>or simple-constructor based syntax.

See the above.

>
>What we need is a single language that can extend itself naturally, specialize
>to any field of computing.

Agreed, absolutely. If you cannot fit this into the framework of my ideas 
above, _please_ tell me your ideas in this regard.


>> The solution as I see it is to keep things 
>> absolutely simple (a la RISC), and rigorously exclude any instructions/ 
>> constructs that impair lucid interpretation on any level.
>Yes, and no (according to the point of view).
>Yes, basic constructors should be simple, so the semantics is easy to
>understand.
>But no, everything shouldn't be done directly from those simple construtors.

Agreed. Abstract all you want on level 2, _and_ design level 2 to make
abstraction easy. Ideas?


> Every program must be defined in its most generic form,
>and the user will specialize to the form that allows best performance.

Wellll.. Often performance is defined on the lowest level (level 0). You
may _have_ to leap down to the lower levels to get best performance, but 
all the dirty work should be done there, and be transparent to the top
level user who doesn't want to know about it. BUT if you have knowledge
of all levels, it should be _easy_ to get into the lower levels, with
helpful debuggers etc as a standard part of every CS, even down to arcane
details of timing etc. I think that obscenities where you _cannot_ even
find out what some components do (eg certain registers on the pentium)
should never be acceptable!!

>
>Reuse *is* important. Today, reuse above language constructs is 0%, because
>there is no secure form of genericity in commonly used languages.
>A language like ML (see CAML-light again), allows genericity; but having
>complete genericity in ML leads to verbose programs (which is why actual
>genericity of existing ML programs is a compromise), because ML understands
>no kind of implicit or partial information passing (which is called
>Object-oriented techniques). I'm disgusted when I see that 99.9% at least
>of code being written is just rewriting old stuff for use in a new program,
>because no reuse thing exists.

Hear, hear. That's one of the reasons why I'm all for simplicity, and 
at least the basic defining of what is on which level. Also, it is then
easier to define an intelligent "apprentice" program that will help you
on several levels, and ultimately even understand _exactly_ (is this 
possible??) what imported code does! At some level, all code should be
lucid and explicit, and straightforward.


>But also note that for reuse to be possible, security is a must, thus
>type checking and more than that, program specification. If you can't be

Yep. Agreed. Details of implementation are another matter. Ideas?


>> 6. So I see any programming attempt as part of a continuum. There may be 
>That's exactly what I just said. We constantly need abstracting some part
>of the code. There is no fixed limit of three or four or any finite number
>of layers in abstraction (which is why ISO standards try to do with their
>bullshit OSI layer model). Each program is well defined in its own proper
>layer of abstraction; and layers are not in a total ordering, but a partial
>one: if A needs B and C, D needs B and E, there's no reason to put A and D in
>the same layer, or one above the other; they're just defined in different
>non-disjoint contexts.

Hmm. Q4. How would you relate this to my ideas above? The way I see it is
that ultimately everything devolves to level 0. Level 1 I see as a necessary
evil, and what you do on level "2" (and how you subdivide/extend it)
is your business.


>> memory and / or output ports. Unfortunately, we often seem to be driven 
>> by the hardware designers  [...]
>> THIS. THIS IS OUR STANDARD. WE WILL DEFINE OUR SOFTWARE AS SUCH AND SUCH. 
>That's even worse than OSI: just one big level of abstraction, with an
>arbitrary choice of functions.

I hope I have more clearly explained what I mean above (!?)



>>If I go into a restaurant and order a steak, I ...
>You too see that program specification is needed. I too have known it for
>a long time, but now, I understand it and why it is needed (and not only a
>toy). And timing specification can be as useful as effect specification,
>as our computers will never ever execute programs instantaneously. But as
>for the *exactly*, it is impossible. *enough* is quite enough.
Absolutely. I would like to have timing considerations built into level 1.
Maybe "absolute maximum time", "absolute minimum time" and "mean time"?!
But keep the specs. as simple as possible. Life is complex enough as it is!



>> Level 0: lowest level. May need considerable effort / ingenuity to ...
>Yes, but this level should also be implementation-dependent, and reserved
>for use by those who *really* want it, in time-critical portions of their
>programs (i.e. ultra-fast graphics or signal processing, OS code, compiler
>back-end, etc). Normal users shouldn't *have to* use it. And a code
>portion could have several equivalents in several different low-level contexts.

Yes, as above, but accessible if required. Also, where you anticipate (in
designing level 1) that several options are/will be needed [got your crystal
ball handy?] make this explicit!!


>
>> Level 1: The "assembler" level. But cf my example above about drawing a ...
>Assembler is still in the implementation-dependent part to me.
>Note that I do not want to forbid low-level parts; just to allow not to use
>them, and to provide portable equivalents (thus a time-critical routine may
>have several versions to run optimally on different hardware, whereas a
>non-time-critical routine will only have a slow high-level version).
In my scenario, the machine-dependent part is all on level 0. Level 1
is "low level" but abstract and machine-independent. The "assembler" is
a symbolic "assembler" that is idealised to make it user-friendly. (Lots
of ideas here). You absolutely CANNOT use explicit level 1 code in level
2. Such an idea (inline code etc) is an abomination!!


>> Level2: The "top" level. Perhaps something along the lines of the 
>> incredibly simple language I gave you a flavour of above.
>Your language is too low-level to me. Again, I love low-levelness (or I
>wouldn't be writing an OS !), but only when it is needed. Sometimes (most
>of the time, once the OS is written, if it really is well written), a high
>level of abstraction is needed.
"Abstract" as much as you want! As explained. I'm really eager to hear
you ideas in this regard.



>I really need someone ready to dirty his hands with the low-level stuff in
>my OS. Uhappily, whereas I have many ideas, I *know* that this stuff is the
>non-essential part of the OS. By non-essential, I mean that if I am to convince
>people helping me in my project and later using my OS, I must have a *working*
>version, however slow, whereas low-levelness will bring me speed and
>efficiency but nothing that would make my OS better than it would be if
>running on a faster computer (that would be available for a higher price, or
>just by waiting long enough). Thus, I deeply need help from someone who will
>cope with the low-level details of the system. Will you be that man ?
>I cannot promise any success, just tenacity.

I would be very keen to give it a bash. I can also be pretty tenacious.
What time-frame are you looking at, and do you have any (no matter how
tentative) specs on the "OS/CS"? What are your constraints? Note that I
rather fancy top-down development (even here) so I would try and give you
a skeleton working system ASAP.


>You really recall me my previous self two years ago: full of good ideas, but
>without an overview of the why it should be done in such perfectionistic
>way, and enthousiast about small things when I didn't see the big thing yet.
>Please do not see any offense in my words. I don't know if I'm older than you
>(born 28 nov 1973), but I *think* (may be wrong !) I've made more errors than
>you in the past, and thus have gained a little experience about how things
>are and should be.

No offence whatsoever!

I've made quite a few errors of my own. One may have been being born in 1960.
I'm glad that you consider me a trifle naive and enthusiastic. I like to
preserve this image!! I probably am.

Bye for now

JVS.


>(BTW, can you talk from your site to mine ? do you have IRC connection ?)

Not to the best of my knowledge. JVS.