From rainer@physik3.gwdg.de Fri, 5 May 1995 11:30:18 +0200
Date: Fri, 5 May 1995 11:30:18 +0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: Version Numbers
Faré, you wrote:
> 1) to express extensions and restrictions of object specifications.
> (i.e. object a++ is an extension to object a)
> 2) to check for dependencies between objects (i.e. are objects a and b
> actually different versions of a same package; is there some object c
> that supercedes both of them at lower cost than having them both ?)
> [...]
> * each object in a package points to a list of objects (possibly axioms of the
> system, possibly objects from the same package, possibly external objects),
> that define exactly the objects' semantics (its specifications),
> its dependencies (other modules it needs to run), its recent history (i.e.
> how it was obtained from a previous version and/or user interaction).
This seems to ambitious to me for now. We want to build a system that
enables us to work on this high a level, but not immediately so. I believe
the system should be layered.
The lower layer handles identification (numbering, naming) of versions
only and doesn't deal with the meaning of ids other than lineage. The id
is kept small and won't blow up the size of medium-sized objects (e.g
methods). This is what we need immediately.
The next layer(s) handles modules and the semantic relationships you
mentioned. That layer should be implemented on top of HLL (as a module!? ;-).
For the time being we'll emulate that layer by thinking.
> * there is a user-controlled programmable inference system to track
> the above dependencies between objects.
> [...]
> What are good ways to (semi-automatically) group objects into packages
> of closely-related objects that do not interact as much at all
> with external objects ? What information must we ask the programmer, must
> we compute when compiling, must we gather while running, must we forget
> when freezing, etc ?
This reference deals with that topic:
"Sifting out the Gold"
> * there is a ("intelligently") locally cached global (yes, really) database
> of known dependencies, that serves as a basis to the above inference system.
> [...]
> This database is really a world-wide knowledge database.
That is *way* beyond Tunes 1.0.
> * there is some global unique addressing scheme used by the above database.
This is essential for all (a lot) of Tunes.
Rainer
From rideau@clipper.ens.fr Fri, 5 May 95 12:12:36 MET DST
Date: Fri, 5 May 95 12:12:36 MET DST
From: Francois-Rene Rideau rideau@clipper.ens.fr
Subject: Version Numbers
> Faré, you wrote:
Wow, I received this accent right !
> This seems to ambitious to me for now. We want to build a system that
> enables us to work on this high a level, but not immediately so. I believe
> the system should be layered.
Of course. I was only talking about the final target. First implementation
steps should be merely allocating space for such information, as well as
security signatures/proofs. All this information won't be actually used by
preliminary TUNES boot code. That's step 1.
> The lower layer handles identification (numbering, naming) of versions
> only and doesn't deal with the meaning of ids other than lineage. The id
> is kept small and won't blow up the size of medium-sized objects (e.g
> methods). This is what we need immediately.
Well, even that is step 2.
> The next layer(s) handles modules and the semantic relationships you
> mentioned. That layer should be implemented on top of HLL (as a module!?
> ;-).
Exactly !
> For the time being we'll emulate that layer by thinking.
Or rather by not thinking at all, and trusting blindly.
This hopefully will be true only for boot modules.
As soon as possible, we should built safer module-managing modules...
>> * there is a ("intelligently") locally cached global (yes, really) database
>> of known dependencies, that serves as a basis to the above inference
>> system.
>> [...]
>> This database is really a world-wide knowledge database.
> That is *way* beyond Tunes 1.0.
Sure, but remember, TUNES is a long-term project. This is the general
direction. Of course the beginnings are much more modest, but we must
keep this main goal in mind.
>> * there is some global unique addressing scheme used by the above database.
> This is essential for all (a lot) of Tunes.
Indeed !
> Rainer
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From jecel@lsi.usp.br Fri, 5 May 1995 22:21:54 -0300
Date: Fri, 5 May 1995 22:21:54 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: Version Numbers
On Thu, 4 May 1995 11:40:43 +0200 rainer@physik3.gwdg.de (Rainer Blome) wrote:
> Again, if we had what we wanted (i.e. an OO system), it'd be trivial:
>
> o Every versioned object (preferrably down to single methods) maintains
> an (RCS-like) version-number of itself.
Ok. This is pretty easy in Self 4.0 - each object can have several
annotations ( not the same thing as Faré means by the term ) and the
version could be one of them.
> o This number changes (like in RCS) when the owner (locker) of that
> object decides to take a snapshot of the current state (i.e. the versions
> of the components).
I am not sure who this owner is in the style of version management that
we are talking about here. Would it be a user or the object which
"contains" the object being snapshotted ( there is a neat word ;-)
> o Branching and creating sets of versions with permutations of
> orthogonal features (versions/snapshots of components) is equivalent to
> cloning and modifying. RCS already allows arbitrary names for
> snapshots, but they should be in addition to numbers.
Cloning and modifying is certainly a way to have several "current versions"
at the same time. Sharing complicates this as the rest of the system
could not try out the new version - it would still point to the original.
> o Since this scheme is object-based, it is just as file-based and
> hierarchichal as the objects are, which is what you'd expect.
Objects aren't a hierarchy like Tunes is. They form a graph with
shared nodes. That is why it is not clear to me how all of this
would work.
> Multi-user access will surely complicate things, but that seems inevitable
> to me.
*That* is the one problem that doesn't worry me. I am pretty satified
with the simple multi-user semantics I have adopted for Merlin ( I think
I have already described this here, but can do it again if there is
interest ).
-- Jecel
From jecel@lsi.usp.br Fri, 5 May 1995 22:21:58 -0300
Date: Fri, 5 May 1995 22:21:58 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: Version Numbers
On Fri, 5 May 95 1:19:58 MET DST rideau@clipper.ens.fr (Francois-Rene Rideau) wrote:
> Yes, yes, yes. But still, there are implementation problems about that:
> the version control system should allow:
> 1) to express extensions and restrictions of object specifications.
> (i.e. object a++ is an extension to object a)
I think this is already handled by inheritance or delegation.
> 2) to check for dependencies between objects (i.e. are objects a and b
> actually different versions of a same package; is there some object c
> that supercedes both of them at lower cost than having them both ?)
That is the problem I was talking about. I have a certain version of
the Linux kernel, a certain version of gcc and lib gcc, a certain
version of XFree, and so on. How do these things depend on each other?
What is the global version of the system?
I understood that you were talking about intrapackage object dependencies
above, but I think the two kinds of dependencies are similar and might
be handled in the same way.
> 3) to have objects as small as possible, i.e. not having to move the whole
> version and hierarchy information if every single object
> 4) to have efficient way to store and/or compute all the above information.
I don't see this as a big problem. It has to be integrated into the
persistent store system, though.
> * objects are distributed in packages.
I agree - if you need to copy an object to a disk to send to
another person, there should be some unit whose objects are
copied while the others are not. Stuffing "the whole world" into
a floppy is not very practical.
> * each object in a package points to a list of objects (possibly axioms of the
> system, possibly objects from the same package, possibly external objects),
> that define exactly the objects' semantics (its specifications),
> its dependencies (other modules it needs to run), its recent history (i.e.
> how it was obtained from a previous version and/or user interaction).
Yes. I am not too clear on the specifications part yet, though.
> * there is a user-controlled programmable inference system to track
> the above dependencies between objects.
That would help with system-wide garbage collection too. But "inference
system" makes me think of complex rule-based programming. I would
rather something as simple as possible.
> * there is a ("intelligently") locally cached global (yes, really) database
> of known dependencies, that serves as a basis to the above inference system.
> * there is some global unique addressing scheme used by the above database.
This is a hard problem. In Merlin, I am using a three part global ID:
32 bits with the TCP/IP address ( or other uniq ID ) of the machine
on which the "package" was created ( it might have moved, but its ID
doesn't change ), 32 bits as a timestamp of when the package was
created and 32 bits indicating the object within the package. Actually
I divide these last 32 bits into an 8 bit version number and a 24
bit object number ( not address - this indexes an "entry table" ).
Most objects don't have a global ID, or this would be too slow to
use.
I might dump this in favor of something like Apertos uses.
> This database is really a world-wide knowledge database.
> To achieve any efficiency, it must be (dynamically ?) clustered.
> And this is yet another problem:
> What are good ways to (semi-automatically) group objects into packages
> of closely-related objects that do not interact as much at all
> with external objects ? What information must we ask the programmer, must
> we compute when compiling, must we gather while running, must we forget
> when freezing, etc ?
> Actually, this is *the* problem that arises when we want efficient
> distributed computing...
In Merlin, the objects are groups manually into packages. For a look
at how to group objects dynamically look at Jim Noble's www pages.
I don't remember the URL, but you can find it by starting at the
Self home page and looking at "other projects using Self".
cheers,
-- Jecel
From jecel@lsi.usp.br Fri, 5 May 1995 22:21:59 -0300
Date: Fri, 5 May 1995 22:21:59 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: Version Numbers
On Fri, 5 May 1995 11:30:18 +0200 rainer@physik3.gwdg.de (Rainer Blome) wrote:
> > [...]
> > What are good ways to (semi-automatically) group objects into packages
> > of closely-related objects that do not interact as much at all
> > with external objects ? What information must we ask the programmer, must
> > we compute when compiling, must we gather while running, must we forget
> > when freezing, etc ?
>
> This reference deals with that topic:
> "Sifting out the Gold"
Sorry to disagree, but the "grouping" in this papers is very different
from what we are talking about here ( I think :-). Ole Agesen was just
trying to recreate at the Self level the class-like information that
exists in the VM and compiler ( called maps ). He wouldn't need to
do this if Self's reflection was less limited.
-- Jecel
From rainer@physik3.gwdg.de Sat, 6 May 1995 19:49:35 +0200
Date: Sat, 6 May 1995 19:49:35 +0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: Version Numbers
Jecel, you wrote that the "grouping" in Ole's and David's paper was
different from what we are talking about here, that they were just trying
to recreate maps (classes) at the Self level. Do you really think they'd
write a paper about a task that trivial? Indeed, they use that information
to allow conclusions about groups of objects, but generating it is only the
first step of five.
You omitted the central sentence from your quote of my message:
> * there is a user-controlled programmable inference system to track
> the above dependencies between objects.
By "above dependencies" I referred to what Faré said in his message:
> * each object in a package points to a list of objects (possibly axioms of the
> system, possibly objects from the same package, possibly external objects),
> that define the object's [...] dependencies (other modules it needs to run)
In another message you said: [...] there should be some unit whose objects
> are copied while the others are not. Stuffing "the whole world" into a
> floppy is not very practical.
Exactly. In the mentioned paper, an implementation of a system is
described that uses an inference process to extract from the Self world all
the objects needed to perform a specific task (respond to a message send).
(Actually, they did a little more than that: the objects were even stripped
of unneeded slots.) In the abstract they say: "The extracted application
runs in a tenth (of) the space of the original environment."
Although they used their system on application startup messages (like
"benchmark quicksort start", it may as well be used on lower levels. What
counts is that the resulting set of objects is independent of the rest of
the system (with respect to that particular message) and may therefore be
viewed as a module.
Finding dependencies or recording them or automatically creating/bounding
modules was not the authors' main concern. But that information may easily
(I believe) be collected during the marking phases (this might even be a
way to do garbage collection?). When modules (packages) are introduced, the
extractor will sometimes cross boundaries between modules when following a
potential send. When not extracting an application but merely deducing
dependencies, that'd be the time to have the system record an
inter-module-dependency.
The central step dealing with dependencies in their system is called type
inference. The corresponding reference is (must be s.th. like):
Type Inference
of Self: Analysis of Objects with Dynamic and Multiple Inheritance
I have not bothered reading that so far coz I thought it'd deal with
run-time type inference only. Maybe I'll get to read it now.
Rainer
From jecel@lsi.usp.br Sat, 6 May 1995 22:19:58 -0300
Date: Sat, 6 May 1995 22:19:58 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: grouping (was: Version Numbers)
On Sat, 6 May 1995 19:49:35 +0200 Rainer Blome wrote:
> Jecel, you wrote that the "grouping" in Ole's and David's paper was
> different from what we are talking about here, that they were just trying
> to recreate maps (classes) at the Self level. Do you really think they'd
> write a paper about a task that trivial? Indeed, they use that information
> to allow conclusions about groups of objects, but generating it is only the
> first step of five.
As you say, grouping is trivial and is not worth a whole paper. In fact,
it only gets a paragraph or two. It is not even necessary for their
real task ( which is application extraction ), but the other steps
would be much slower with out it. It is much faster to work with
the "priorityQueue group" than with {obj19, obj37, obj41, .... } in
the type inference step.
Grouping objects with a similar structure ( the same map ) is different
from grouping objects that are used together ( which the paper also
describes: the fourth step ). I should have been more specific - it
is not that I think that the paper is not relevant to this discussion,
but just that the use of the word "grouping" is not the same.
> You omitted the central sentence from your quote of my message:
> > * there is a user-controlled programmable inference system to track
> > the above dependencies between objects.
Sorry. I answered four messages at the same time and must have
messed things up a little.
> By "above dependencies" I referred to what Fari said in his message:
> > * each object in a package points to a list of objects (possibly axioms of the
> > system, possibly objects from the same package, possibly external objects),
> > that define the object's [...] dependencies (other modules it needs to run)
I agree that there must be such a system ( I think that I complained about
"inference" as implying AI, but Ole uses the word too ).
> In another message you said: [...] there should be some unit whose objects
> > are copied while the others are not. Stuffing "the whole world" into a
> > floppy is not very practical.
>
> Exactly. In the mentioned paper, an implementation of a system is
> described that uses an inference process to extract from the Self world all
> the objects needed to perform a specific task (respond to a message send).
> (Actually, they did a little more than that: the objects were even stripped
> of unneeded slots.) In the abstract they say: "The extracted application
> runs in a tenth (of) the space of the original environment."
The problem with their system is that it extracts *everything* needed
to run the application, even things like the process scheduler. Many
of these objects will already be available to the person I want to
send the floppy to, so I would like some module system that would
limit even further the objects that have to be copied.
> Although they used their system on application startup messages (like
> "benchmark quicksort start", it may as well be used on lower levels. What
> counts is that the resulting set of objects is independent of the rest of
> the system (with respect to that particular message) and may therefore be
> viewed as a module.
Some objects will be common to several different extractions, so it
would be best to make them into a module of their own ( see above for
an example ).
> Finding dependencies or recording them or automatically creating/bounding
> modules was not the authors' main concern. But that information may easily
> (I believe) be collected during the marking phases (this might even be a
> way to do garbage collection?). When modules (packages) are introduced, the
> extractor will sometimes cross boundaries between modules when following a
> potential send. When not extracting an application but merely deducing
> dependencies, that'd be the time to have the system record an
> inter-module-dependency.
Self 3.0 and 4.0 do have modules. The extraction software ignores
the modules, but it would be very neat to do exactly what you
suggest. Currently the Self modules are managed entirely by hand.
> The central step dealing with dependencies in their system is called type
> inference. The corresponding reference is (must be s.th. like):
> Type Inference
> of Self: Analysis of Objects with Dynamic and Multiple Inheritance
> I have not bothered reading that so far coz I thought it'd deal with
> run-time type inference only. Maybe I'll get to read it now.
This is a great paper - I recomend it. It is not about run-time type
inference at all ( there are tons of Self papers about that - the
compiler does run-time type inference ). This paper talks about a
seperate application which can list all of the possible results of
any message send, or all of the objects that might be found in a
given slot. This is a great way to check that your application
won't cause any "message not understood" errors before shipping it,
much in the same way Lint was used with C programs in the old days.
You don't have to give up this advantage of static type checking to
use Self :-)
BTW, Ole has sent me some samples of the output of his extraction
programs and I agree that it is a vast improvement over what the
Smalltalk people do today to deliver applications. If the VM could
also be reduced, it might even make it possible to use Self for
embedded applications!
-- jecel
P.S.: about version numbers, David MENTRE (David.Mentre@enst-bretagne.fr)
has written me about CVS, a RCS extension that deals with whole
applications rather than individual files. I plan to look at it as
soon as possible ( it can be found in sunsite and mirrors in the
Linux/develop/version_contr directory )
From jecel@lsi.usp.br Wed, 3 May 1995 18:40:15 -0300
Date: Wed, 3 May 1995 18:40:15 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: Version Numbers
Does anyone know a good scheme for version numbers for a
complex piece of software such as Tunes?
The current xx.yy.zz.vv system is interesting, but has most of the
same problems as the schemes I will mention below.
At one time I tried simply using the release data as the version
name. That gave a relative ordering to the versions, but most
development is not very steady, so it is hard to tell how much
two versions differ using this system. You might just do a minor
patch in two years, or a major overhaul in three months.
A single number, like the Adobe Postcript interpreters have, has
the same problems as using the dates.
The Revision Control System ( RCS ) that comes with Linux uses
a two level system: 1.3, 1.4, 1.5, etc... The first number is
incremented manually to indicate a major release. To allow
several people to work on the project at the same time, you can
create derived versions. If you derive a version from 1.4, for
example, you get 1.4.1.1. The next change you make to a file
will result in 1.4.1.2 and so on. This allows you to build an
arbitrary version tree ( actually a graph, as different branches
can be manually merged to create a new version derived from both ).
The major problem I see with RCS is that is works with versions of
the individual files. What about the whole system? How should it
work in a hierarchical project like Tunes? If you update the
Interfaces project but keep the HLL project as is, for example,
should the system version number be incremented? Should each
subproject have its own version number?
In my 286 OS ( back in 1987 ) I adopted the system decribed in
"Managing the Evolution of Smalltalk-80 Systems" by Steve Putz
in pages 273-286 of the book "Smalltalk-80: Bits of History,
Words of Advice" edited by Glenn Krasner ( Addison-Wesley 1983
ISBN 0-201-11669-3 ). That system started out as informal practices
using email and evolved into a tool based system. The main
elements were:
bug report - a message detailing a bug found in a given version
of the system. It might also be a suggestion for
improvement ( in my use of the system, at least )
change - a message describing changes made to one or more source
files ( which are listed ) to fix one or more bugs as
described by the listed bug reports
version - a message that talks about the release of a new version
of the system that is derived from a given version by
adding the listed changes
Browsers allowed you to create bug reports, changes, versions ( each
with a unique ID given by the system ), anonymous test versions, to
look at these elements, to find relevant bugs for a version and so on.
When creating a new version, the system might warn you of conflicts
in several changes. The solution was to create a new change manually
that resolved the conflicts and to use it instead of the original
changes in the new version.
Changes that were not incorporated in a version were known as "goodies".
The system allowed you to try them out ( one at a time or in combination )
by making a test version.
This was a very neat system, but it also had a "flat" view of the
software, where a change to any part has the same effect on the
version number.
Isn't there any way to combine version numbers with the subproject
numbering scheme in Tunes to be able to say things like "a system
with old garbage collector and new GUI"? I wouldn't mind if the
resulting code turned out to be very large - we might build some
browser to filter it.
So, we have:
Date Number RCS Smalltalk What I want
----------------------------------------------------------------------
imposes order yes yes yes(1) yes yes(1)
quantity of change no no yes yes yes
tree of versions no no yes yes(2) yes
file based no no yes no(3) no
hierachical no no no no yes
----------------------------------------------------------------------
(1) only within a single branch
(2) allows anonymous test versions, not a real tree
(3) changes are file based, versions are not
How about it folks? Is it even worth wasting time worrying about this?
regards,
-- Jecel Mattos de Assumpcao Jr
Laboratorio de Sistemas Integraveis - University of Sao Paulo - Brazil
jecel@lsi.usp.br http://www.lsi.usp.br/~jecel/jecel.html
From rainer@physik3.gwdg.de Thu, 4 May 1995 11:40:43 +0200
Date: Thu, 4 May 1995 11:40:43 +0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: Version Numbers
> ... worth wasting time worrying about this?
Sure!
Again, if we had what we wanted (i.e. an OO system), it'd be trivial:
o Every versioned object (preferrably down to single methods) maintains
an (RCS-like) version-number of itself.
o This number changes (like in RCS) when the owner (locker) of that
object decides to take a snapshot of the current state (i.e. the versions
of the components).
o Branching and creating sets of versions with permutations of
orthogonal features (versions/snapshots of components) is equivalent to
cloning and modifying. RCS already allows arbitrary names for
snapshots, but they should be in addition to numbers.
o Since this scheme is object-based, it is just as file-based and
hierarchichal as the objects are, which is what you'd expect.
Multi-user access will surely complicate things, but that seems inevitable
to me.
Rainer
From rideau@clipper.ens.fr Fri, 5 May 95 1:19:58 MET DST
Date: Fri, 5 May 95 1:19:58 MET DST
From: Francois-Rene Rideau rideau@clipper.ens.fr
Subject: Version Numbers
>> ... worth wasting time worrying about this?
> Sure!
> Again, if we had what we wanted (i.e. an OO system), it'd be trivial:
Yes, yes, yes. But still, there are implementation problems about that:
the version control system should allow:
1) to express extensions and restrictions of object specifications.
(i.e. object a++ is an extension to object a)
2) to check for dependencies between objects (i.e. are objects a and b
actually different versions of a same package; is there some object c
that supercedes both of them at lower cost than having them both ?)
3) to have objects as small as possible, i.e. not having to move the whole
version and hierarchy information if every single object
4) to have efficient way to store and/or compute all the above information.
> Multi-user access will surely complicate things, but that seems inevitable
> to me.
Well, "as soon as" there is a version graph, and not just a version line,
multi-user access becomes easy. That's what it was meant for, isn't it ?
I agree version control should be implemented deep into the module system.
Here is how I see things:
* objects are distributed in packages.
* each object in a package points to a list of objects (possibly axioms of the
system, possibly objects from the same package, possibly external objects),
that define exactly the objects' semantics (its specifications),
its dependencies (other modules it needs to run), its recent history (i.e.
how it was obtained from a previous version and/or user interaction).
* there is a user-controlled programmable inference system to track
the above dependencies between objects.
* there is a ("intelligently") locally cached global (yes, really) database
of known dependencies, that serves as a basis to the above inference system.
* there is some global unique addressing scheme used by the above database.
This database is really a world-wide knowledge database.
To achieve any efficiency, it must be (dynamically ?) clustered.
And this is yet another problem:
What are good ways to (semi-automatically) group objects into packages
of closely-related objects that do not interact as much at all
with external objects ? What information must we ask the programmer, must
we compute when compiling, must we gather while running, must we forget
when freezing, etc ?
Actually, this is *the* problem that arises when we want efficient
distributed computing...
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From jecel@lsi.usp.br Tue, 9 May 1995 20:39:11 -0300
Date: Tue, 9 May 1995 20:39:11 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: grouping (was: Version Numbers)
Rainer,
> The (application-) extractor, the module system and the type inferencers
> (inferrers? ;-) (static and run-time) seem to have a lot in common. I
> guess one of the reasons why they didn't exploit that (yet?) are the
> limited reflective capabilities of Self.
Exactly. Programming at the VM level in C++ and programming at the
Self level are not mixable, so they must duplicate code sometimes. Note
that the current implementations of the Self compilers depend on
runtime type feedback and no longer do the sophisticated type
analysis described in so many of the Self papers. So the Self level
type inferencer could have shared code with the Self 2.0 compiler, but
would have to do type inference itself from Self 3.0 on.
> A lot of the information (all those constraints, esp.) would not have to be
> computed every time from scratch if Self featured reifications of notions
> like types (which are sets of clone families), send sites, arguments and
> (return) values. Reification of every(?) concept that programmers use
> should be a requirement for Tunes.
You can reify some concept statically or on demand. Examples of the
latter in Self and Smalltalk implementations are contexts ( or activations )
and messages. Most modern implementations use a simple stack for
efficiency, but will create the "traditional" context objects on the fly
whenever the programmer tries to use them. Message objects are also not
created for every message sends, but only for the very rare cases
where the message object itself is accessed.
If you reify something on demand, it doesn't make sense to keep
the results around for a possible reuse as you want to save time
and memory.
> Merlin is supposed to have a reflective architechture. What did you plan
> to include?
I am writting a paper called "Bootstrapping the Object Oriented Operating
System Merlin: Just Add Reflection" that I plan to send to the Meta95
workshop on reflection of ECOOP'95. I will make this paper available
in the net as soon as it is finished ( which must be before the 15th ).
It is important to contrast my goals with those of the Self group. They
allow the implementation to "cheat" on the programming model only as
long as it is impossible to get caught. They reject the common tail
recursion elimination technique, for example, because it is possible
to see that some stack frames are missing. They allow you to use maps
to save space and time in the VM, as long as it is impossible for the
Self programmer to find out that this has been done. This is what is
limiting their use of reflection and makes them duplicate code to
group objects. The advantage, of course, is that they can radically
change the implementation ( which they did ) without the danger of
anything breaking.
My goal, on the other hand, was that there should be no "black boxes".
I did not want to have one group of programmers that worked in Self
while another used C++ with an entirely different development
environment. I wanted the whole system to be defined in terms of
itself, so I allow reflection even though I try to keep it under
control in order not limit the evolution of the system. I also
allow the implementation to "bend" the programming model with things
like tail recursion elimination, though I try to limit this too.
The most important objects in my reflection model are:
map/reflector - this object "knows" all about the format and other
operations of one ( or more ) "base level" Self
objects. It knows which metaobjects are used to
manage that object's memory, which metaobjects
can send messages from/to it, etc...
context/message/future - this should be three different objects, but
have been lumped together to reduce the strain on
the garbage collector. As a context, it knows about
the executing method's program counter, its stack,
interrupt flags, execution mode, etc. As a message
it knows the receiver, the selector and the arguments.
As a future it knows whether the answer has been
returned yet and has a list of pending messages.
CPU - this object isolates the hardware and allows switching between
normal level and meta level execution. It knows how to do DMA,
how to setup interrupt handlers, how to flush caches, how to
initialize system state, etc. It is a lot like the Alpha PALcode.
I know this is too brief to be clear, but it is enough to get an idea
of how message sending works. This is the most crucial part, as in
Self we have "message sending at the bottom". Note that this is a
case where things are reified on demand - most of the time the
compiler will optimize the sequence below into nothing!
Sending the message:
1. switch to reflector/map ( actually, the mailer object )
2. if receiver is future
block and call scheduler
3. else
make future/message/context object
4. if receiver in other session
trap to "kernel mode"
5. else
put future/message/context in receiver's queue
6. switch back to calling context
Receiving the message:
7. object returns from previous message: move next future/message/context
to ready queue
8. start dispatcher object ( inside the receiver's reflector/map )
9. if meta-message
patch context and switch to the right metaobject
10. else
lookup the message selector
11. start slot "reflector" apply code
Anyone who has taken a look at the Moostrap language will see where most
of this came from. Note that steps 1, 4 and 6 involve the CPU object, as
do steps 9 and 11. All of the other steps should be replacable with
user supplied code in the final, fully reflective version. This, of
course, might be *very* dangerous. That is why "inter session" messages
go through a system trap - any damage you do can't affect the other
users in the system.
Slots, in my system, as pseudo objects that have map/reflectors and
can be sent messages ( like the "apply" message in step 11 ). They
might be reified into full objects on demand.
I hope it doesn't seem as confusing to you as it does to me ;-)
-- Jecel
From rainer@physik3.gwdg.de Tue, 9 May 1995 16:25:24 +0200
Date: Tue, 9 May 1995 16:25:24 +0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: grouping (was: Version Numbers)
Jecel, you wrote
> Some objects will be common to several different extractions, so it
> would be best to make them into a module of their own ...
You're right, of course. I just didn't bother going into more details.
> > Type Inference of Self: ...
> > I have not bothered reading that ... Maybe I'll get to read it now.
> This is a great paper - I recomend it. ...
Got it. Yes, a most useful tool that is.
The (application-) extractor, the module system and the type inferencers
(inferrers? ;-) (static and run-time) seem to have a lot in common. I
guess one of the reasons why they didn't exploit that (yet?) are the
limited reflective capabilities of Self.
A lot of the information (all those constraints, esp.) would not have to be
computed every time from scratch if Self featured reifications of notions
like types (which are sets of clone families), send sites, arguments and
(return) values. Reification of every(?) concept that programmers use
should be a requirement for Tunes.
Merlin is supposed to have a reflective architechture. What did you plan
to include?
Rainer
From chharris@u.washington.edu Sat, 13 May 1995 16:02:22 -0700 (PDT)
Date: Sat, 13 May 1995 16:02:22 -0700 (PDT)
From: 'Chris Harris' B. Harris chharris@u.washington.edu
Subject: No, folks, I haven't died.
Hello Tunsers,
Just wanted to drop a little note to suggest that I still do exist,
am living and breatheing. School has been rather very busy lately, and
my body has decided, to my annoyance, that it really can't get five and a
half hours of sleep each night and still be awake/happy. With all this,
computers, unfortunately, seem to have landed themselves on the back
burner of my life. So I haven't quit the project, and will be back ASAP
(hopefully before summer...). Just feeling a bit overwehlmed....
As for the project here itself, does anyone else think it would be
helpful to have some sort of vision around here? Isn't there at least
some semi-logical order of what needs to be decided when, so that we
could "get started"? I know the project is going forward, but certainly
not very fast. I know part of this is due to lack of manpower, but
couldn't some sort of layout make better use of our availible resources?
Have a good one!
-Chris
--------------------------------------------------------------------------
"Be careful lest in casting out your devil you cast out the best thing
that's in you." --Nietchie %% Me = Chris Harris (chharris@u.washington.edu)
Check out my web page: http://weber.u.washington.edu/chharris/
From fare@Sit-Hyps Sat, 13 May 1995 11:54:18 +0200 (MET DST)
Date: Sat, 13 May 1995 11:54:18 +0200 (MET DST)
From: Francois-Rene Rideau fare@Sit-Hyps
Subject: CANDO
Dear Tunespeople,
here is some announce I collected from comp.lang.forth.
I invited the author to join us, with the only objectioon that
specification in a formal system was much more secure, accurate and
doable, that is much more useful, than descriptions in english...
------>8------>8------>8------>8------>8------>8------>8------>8------>8------
From: mfiring@primenet.com (Martel Firing)
Newsgroups: comp.lang.forth
Subject: Old Dog's New Tricks
Date: 13 May 1995 21:34:18 GMT
Organization: Primenet Services for the Internet (602)395-1010
NNTP-Posting-Host: usr3.primenet.com
X-Newsreader: TIN [version 1.2 PL2]
CANDO - A Software Architecture for the Next Century
by
Martel Firing
This paper describes my ideas for a software architecture based on
observations, revelations and frustrations over 16 years of programming.
It began as an attempt to analyze why software development was so
difficult, unpredictable, and bug prone. My conclusion was that the
programmer was required to do too many things that the computer should
do, and the computer simply couldn't do the things only the programmer
could do. And in the end, I concluded that programmers need more
freedom, not a more rigid set of rules to create better software. And
the software itself needs to thrive on diversity, change and error
instead of choking up and quitting. So here are my ideas.
CANDO is a software structure designed to work in both large and
small systems, distributed and centralized hardware, and importantly,
able to adapt to changing requirements on-the-fly.
The guiding principal behind CANDO (which means Can Do!) is that
software should be built from competing independent modules advertising
their needs and capabilities. The purpose of the CANDO system is to
simply match needs with capabilities without regard for what these
actually are. The CANDO core does not, itself, generate any needs nor
have any other capabilities.
What's Lacking Now
Before discussing the conceptual details of CANDO, let's examine
some of the characteristics and failings of current software languages
and programming fashion.
1. The most striking observation about software is that it is designed
on a totalitarian style model, the 5 year Gosplan. Everything has to
be thought out in advance and every detail must be correct or the
whole edifice may crumble, and usually does at one time or another.
The massive software manuals for C++, Windows, etc. are proof of
this. In an attempt to enforce correctness, good design practice,
and standards, the "requirements" have gotten completely out of
hand.
2. A second feature of most software is that it is nearly impossible to
determine in advance how difficult it will be to develop and when
(or at what cost) it will be finished.
3. Most software is not designed to be re-usable, and if so designed, I
suspect that little of it is actually re-used in practice.
4. Many software projects become so large and complex that no single
person completely understands the whole thing. Documentation
becomes as large and daunting a task as writing and testing the
code.
5. And finally, all software has bugs. Most of the work in software
development involves testing, identification, and fixing of bugs.
Solutions have been proposed to solve some of these problems. The
most fashionable solution is OOPS, Object Oriented Programming Systems.
The advocates of the OOPS approach argue that their languages: a)
produce re-usable modules, b) allow better program design and team work,
c) reduce bugs, and d) produce more reliable/verifiable results.
OOPS does, undoubtedly, have some of the advantages claimed for it
and appears to be an improvement over prior art. The purpose of this
paper, however, is not to knock OOPS. But rather, to suggest that OOPS,
and languages that preceded it suffer from a singular, fundamental flaw
that will make them fail in the end.
The flaw behind all of the computer languages is that some
programmer must ultimately anticipate every conceivable requirement and
provide a "means" to handle it. And only one means at a time can be
used. Given the inevitable complexity of software, implementing this
"means" requires a great deal of knowledge about the overall system,
even in an OOPS environment. And if the "means" fails, the system can
fail, and usually does.
The flaw, then, is that the programmer has to know too much, has to
adhere to a rigid discipline, and has to produce code that always works.
These real life requirements are only realized consistently in our
dreams.
CANDO: The Free Enterprise Model
The CANDO system is based on what might be described as a
"free-enterprise" model. The core system matches requests for
information or services with advertised capabilities. Its job is simple
so it can be very fast. "Worker" modules post the capabilities ads and
make requests. The features of this model are:
1. A core system whose only purpose is to rapidly match "needs" with
"capabilities." The needs and capabilities of worker modules are
advertised on the core system's "bulletin board."
2. Each worker module posts an advertisement telling what it CAN DO,
i.e., what it supplies. When called upon to do a job, it posts a
"want ad," asking for something another module may be able to
supply.
3. The worker modules don't have to know anything about one-another, or
even if they exist. If a "want ad" goes un-answered, then it is the
programmer's or the user's job to either give it what it needs, or
produce a module that will do that automatically. The system itself
tells the programmer what modules are needed, and what each module
has to supply.
4. Consuming modules can judge whether or not their requests have been
satisfactorily met and will not supply the requested product until
and unless they first get what they need.
5. Modules can compete for business. A scoring system allows
successful, popular modules to remain close to the system and
migrates others to more distant points or eventually even to
oblivion. (Module usage could become a means for programmers to
receive royalties based on the utility of their work.)
The advantages of the CANDO architecture are:
1. It is by design adaptable to changing system requirements.
2. Modules are completely independent of one-another. Any language can
be used to implement any module so long as the basic communication
protocols and encapsulation are observed.
3. Worker modules can reside and simultaneously execute on multiple
processors.
4. The system should be bug tolerant. No system will ever be
completely bug-free.
5. CANDO lets each programmer be an artist, enforcing minimal
constraints on his productivity and creativity, yet gives him access
to all prior work accessible to the system.
There are several requirements for CANDO to work. First, data must
be described and encapsulated in a simple way that allows modules to
describe it, send it and receive it in a predictable manner. I propose
that all data be represented as strings with only the address, length
and type transmitted from one module to another. This needs to be very
simple and highly generalized.
Secondly, a means has to be devised for modules to tell one another
what they do or what they need. This is the most difficult problem.
I've worked on this for some time looking for a "computerish" solution
of formal words and phrases that could be used. My conclusion, at this
point, is that English would work best, because people (even
programmers?) understand it.
The programmer's job is to work out the matches and let the system
turn these matches into tokens that the system can then efficiently
manipulate. The system manages the facts while the programmers manage
the ambiguities.
An overall view of the CANDO system, then is like the input-output
matrix of a free-enterprise economic model. Suppliers are shown in rows
and consumers in columns. Most suppliers are also consumers and vice
versa. At the intersections of rows and columns you have potential
transactions, with the effects of these transactions rippling through
the structure as requests for services produce requirements for services
from others. When the system naturally generates requests for services
or information that don't already have a supplier, then the programmer
can create a new worker, and the process continues.
A Little More Philosophy
CANDO is modularized on the basis of an observation. If a computer
is represented as shown in the diagram, below, all of the capability
flows upward from the most primitive hardware functions. All
information flows downward from the real world toward the primitive
hardware level. This is information needed to perform a task. The
information desired by the user also flows "downward" to the hardware
and out to the CRT or the Printer. See below.
Real World Information: Payroll, Spreadsheet,
Text, Formula, Program.
--------------------------------------
all information - no capability
INFORMATION FLOWS DOWNWARD
+---------------------------+
| A program resides here, |
| feeding capability |
| up and information down. |
+---------------------------+
CAPABILITY FLOWS UP
all capability - no information
--------------------------------------
Hardware Capability: PUSH, POP, MOV, XCHG, IN, OUT etc.
CANDO worker modules ask for information when needed and advertise
capabilities. These modules can be employed at any level right down to
the assembly level. Lower level worker modules acquire information from
higher levels and capabilities from lower levels. However, although we
can talk about levels (of primitiveness), there is no actual layering.
The worker module that moves a cursor on the screen, for example, can
supply this service at the most primitive level, where it is most
efficient. There is no need to burrow down through multiple layers of a
structure to do a primitive function.
The significance of all this is that the programmer is typically
required to keep track of too many of these transactions and
capabilities in the context of an application. The software and
hardware are better suited to this task. What a computer language needs
is more "self-awareness." The language should tell the programmer what
is missing, based on what it has been told to do. The programmer should
not have to guess. The computer can't, by itself, decide what
capabilities are needed to accomplish a task, but it can do a good job
of keeping track of the capabilities that are available and letting the
programmer make that decision. Modern compiled languages do a miserable
job of this if they do it at all.
Language Structure
The CANDO worker modules each have a common structure that is part
of the language. The inputs (NEEDS) of one module are supplied by the
services (CAN DO) of another. A typical module looks like the
following:
WORKER MODULE
+-------------+
| I CAN DO X | This is the advertised capability
|-------------|
| I NEED Y | These generate requests when the
| I NEED Z | worker's capability is requested.
|-------------+
| CODE | Code that actually performs the task.
+-------------+
The worker module will only provide its service after its own
requests have been satisfied. In the example, Y and Z (NEEDs) can be
needs for data or capability. Data and capability are interchangeable
in the sense that a module must call on a (CAN DO) capability of some
other module to get information.
Simplicity is enforced on these modules because the order of
fulfilling a modules NEEDs is indeterminable. So one need can't depend
upon another. Interdependence or order-dependence of NEEDs suggests
that another, simpler module is required.
Data is supplied as a string with an address, and a length. The
user can ask: "WHAT IS z" to get type information about data string "z"
from its supplier. Another question: "WHERE IS z" was anticipated, but
it is probably better for the supplier to be capable of supplying its
data no matter where it is (on disk, on the network, in memory, in the
user's head, etc.), or for some other worker module to do this job.
Additional dialogues in addition to "WHAT IS" will probably be
needed to make it all work, but the philosophy is to keep these to the
absolute minimum required.
The Programming Environment
Programming in CANDO occurs at multiple levels. The highest level
consists of defining a high level task worker that usually makes certain
requests. As the module's requests are executed and no other module
supplies them, the programmer is notified. He may then may search for
descriptions of services that match the request of his module. Various
tools can be envisioned to make this search easier. When a candidate is
found, the system can tokenize the match and the new module can then be
tested. The important difference here is that a) the programmer can
work in English, and b) the system generates requests for missing
capabilities or information.
Lower level programming consists of creating new worker modules
containing execution code. This can be done without any prior knowledge
of the application(s) in which these capabilities may eventually be
used. The programmer supplies English language descriptions of the
worker's capability (its advertisement) and of its needs (requests for
information or capability). He also generates code that uses the
information requested to provide the capability advertised. The
language provides a framework for inter-module dialogues, data
verification, and entry/exit points for code execution.
Note that the execution of the worker module's code is not
scheduled by the CANDO core module, because execution depends on
fulfillment of the worker's own requests. Also, modules may reside on
multiple processors. Thus, execution scheduling is tied to the local
processor's time-slicing mechanism independently from the core module.
In effect, each worker module has its own independent thread of
execution.
Speed & Efficiency
In many respects this CANDO concept is similar to Smalltalk, a
language noted for neither speed nor efficient use of hardware
resources. Undoubtedly some of C++'s popularity comes from the
perception that the speed of C can be combined with the advantages of
OOPS. But it seems to me that both are compromised, and at great expense
to the programmer.
I don't think CANDO will ever be a speed demon, but it need not be
a slow poke either. The core mechanism for matching capabilities and
needs can be tightly coded and fast. The code internal to various
worker modules can also be tightly coded and fast. Most of the
performance penalty in software comes from the innermost levels of the
software that perform highly repetitive tasks, while the outer layers
make infrequent requests for system resources. Thus, a software system
that has tightly coded (fast) inner layers with very high level (slow)
outer layers may not be appreciably slower than a system coded entirely
in a fast language.
In the future, the greatest gains in performance will come not from
tight code, but from the use of multiple processors. This is because
much of the commercially produced software, although coded in C, is
seldom tight or fast. A system designed for multi processor execution
will be faster than one that is not.
Efficiency in the use of hardware resources is inevitably declining
as programs get bigger and do more. Languages that encourage multiple
use of standard modules will produce more compact applications as
applications become larger.
I suspect that CANDO can be implemented as a very compact system
except for the English language module descriptions, which can be kept
on disk. I also believe that the speed of performance for most
applications will be as good as any other comparably high level software
and better than most when multiple processors become the norm.
What's the Real Difference?
Although borrowing ideas from OOPS, Smalltalk, FORTH and lots of
other languages, CANDO departs from the crowd in several important
respects:
1. It introduces the idea that program components should be allowed to
compete with each other to perform a task, with the system selecting
more useful and efficient modules.
2. It permits the programmer to create and use modules based on English
language descriptions. The system, not the programmer, keeps track
of the tokens.
3. The System keeps track of the modules, their interconnections and
their descriptions. The programmer is notified when a requirement
is missing, rather than having to always anticipate the requirement.
4. CANDO worker modules can be created for use in applications not
anticipated by the programmer and can call on capabilities not yet
available to the system.
5. The concept of dedicated application programs essentially
disappears. Instead, the system is endowed with capabilities.
These dynamically configure themselves to solve problems presented
to the system. As the system grows, it can solve more problems.
The programmer becomes involved when a) the system can't solve a
problem because something is missing, or b) the programmer thinks of
a better, faster, or more efficient way to implement a module.
Implementing CANDO
CANDO can be implemented on today's hardware taking the
multi-processing future into consideration. I believe that OS-2's
multi-threaded multi-tasking mechanism is a suitable environment for
this.
It would be possible to define and implement CANDO entirely as a
kind of "communications protocol" similar to Microsoft's object
embedding, linking and other data sharing protocols. The advantage of
this would be speed of implementation -- if acceptance could be somehow
endorsed by a power that be. The disadvantage is that much of the
fundamental benefit of CANDO would never be realized.
The implementation I would favor is a bottom up approach starting
at the assembler level. A CANDO assembler would support higher level
structures. The ultimate implementation of CANDO should be in CANDO
itself. This would completely validate the concept and create a
platform from which the system could grow indefinitely.
Programming with CANDO will be completely interactive. There will
be no edit/compile/link/test cycle.
The actual syntax of the CANDO language will be sparse, with very
few reserved words. The system, in effect, generates and keeps track of
its own reserved words (tokens) invisibly to the programmer. The main
language components will be the core module and the encapsulation
components used to build worker modules, to implement inter-module
dialogues and to validate exchanged data and capabilities. Programming
tools will evolve and become part of the structure.
As an incentive to the growth of the system, I would advocate some
means for contributors to the system to earn royalties for successful
CANDO modules. This could be an integral part of the CANDO structure.
Users could have access to the work of thousands of independent
programmers via networks, with CANDO sorting out which modules are
useful and which are less so for a given set of applications. Competing
modules would survive or die based on the law of the jungle. Producers
of successful modules would be rewarded with growing royalties.
Unsolved Problems
There are many unsolved problems in the CANDO idea. A few spring to
mind.
A) How to let modules compete. How to score and select the best
modules for an application instance.
B) How to make the system bug-tolerant. I believe that this should be
a goal of CANDO. Some means must be devised to determine which of
competing modules is providing the desired result.
C) How should the user initate a process and interact as the solution
is being composed? Worker modules should handle this task, I
believe, not CANDO itself. But the open-endedness of a capability
based system makes this a real challenge for creative spirits. In
any case, it can evolve.
D) How to enable participants to receive compensation for their
efforts.
E) How to make CANDO simple enough that it will accommodate growth and
evolution.
F) How to get started.
Conclusion
As you have probably figured out by now, CANDO is not so much a
language as it is a new way of computing, in which the computer (and its
networks) is a repository of capabilities that are called upon to
perform tasks not necessarily guessed in advance by the programmers. A
program is a (possibly temporary) arrangement whereby these capabilities
are knitted together to perform some useful task. The task or tasks,
however, may change quite dynamically from day to day. As software is
improved, the improvements automatically appear and are used. As
software falls into disuse, it goes away. The overall system, however,
grows ever stronger and more capable.
This kind of dynamic, self-adaptive software is not possible in a
rigid centrally planned model. It requires new thinking: the idea that
a machine can compete with another machine - and survive or die based on
merit. And, as in economics, some chaos is helpful if not always
pretty. The "machines" we're talking about, after all, are the
encapsulated cleverness, skill and ego of programmers. Why not let them
compete?
The Role of Forth
CANDO could be implemented in any language and the design
encourages "worker" modules to be implemented in any language the
programmer prefers.
The reasons that this is posted to the FORTH newsgroup are:
A) I like Forth and use it myself.
B) Forth encourages simplicity, and the implementation of CANDO
requires simplicity.
C) A language-transparent applications architecture would be good for
the future of Forth.
Undoubtedly, some will ask: "Why hasen't Firing implemented CANDO
himself?" The answer is simple. I haven't thought it out completely and
don't see how to make a living on it. But I might do it anyway if
someone else doesn't do it first.
---------------------------------------------------------------------
Martel Firing
mfiring@primenet.com
---------------------------------------------------------------------
------>8------>8------>8------>8------>8------>8------>8------>8------>8------
From jecel@lsi.usp.br Mon, 15 May 1995 19:54:14 +0300
Date: Mon, 15 May 1995 19:54:14 +0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: papers
Here is a neat reference to a language called
Agora. It is Self-like in many ways ( currently
implemented in ParcPlace Smalltalk ) but might
be closer to what is intended for Tunes:
ftp://progftp.vub.ac.be/agora
It was also featured in the December 1994 issue
of Sigplan Notices. It is interesting that a
friend of mine developed a language very much
like this and it was also called Agora. Talk
about coincidences...
I wrote a paper called "Bootstrapping the Object
Oriented Operating System Merlin: Just Add
Reflection" that I have submitted to Meta'95,
the workshop on metaobjects and reflection at
ECOOP'95. I placed it at:
http://www.lsi.usp.br/~jecel/jpaper7.ps.gz
for now. I don't have a FTP area that I can
put it into, so you have to get via a web
browser.
More when I have more...
-- Jecel
From mfiring@PrimeNet.Com Mon, 15 May 1995 14:34:22 -0700 (MST)
Date: Mon, 15 May 1995 14:34:22 -0700 (MST)
From: Martel Firing mfiring@PrimeNet.Com
Subject: Old Dog's New Tricks
Thank you Francois-Rene for your message a few days ago. I wanted to
visit your WWW site before responding.
On Sun, 14 May 1995, Francois-Rene Rideau wrote:
> I've been very interested in your post about CANDO, and I completely
> agree with your diagnostic as of what is lacking now in the computer software
> world, as well as with the general philosophy you propose to asnwer the
> problems.
> Actually, I already began my own project just for the very same
> reasons, and with mostly the same goals (some aspects being stressed more
> or less by me than by you according to the history of the project):
>
> Here is where I disagree slightly:
>
> > I propose
> > that all data be represented as strings with only the address, length
> > and type transmitted from one module to another. This needs to be very
> > simple and highly generalized.
> Ahem. I propose we use some higher-order type system, much like what the
> Coq proof system does, but made more usable, by adding reflectivity, access
> to low-level considerations, and multiple level of error-tolerance. A proof
> system is really the most reliable thing possible, plus it's clean, the
> theory is well known, and its so generic it will last for years (with perhaps
> adjustements in encoding). It's much more secure and accurate than English,
> it is very well-known theoretically, and has already been implemented very
> successfully, though not integrated into a real-work oriented platform yet,
> only dedicated research software.
Unfortunately, I'm not familiar with Coq proof systems. The idea behind
the generalized representation is simply to avoid the complications of
data representation such as numbers being: 8-bit integer, 16-bit integer,
single precision floating point, etc. etc. My approach would be to just
call them numbers and represent them in ASCII. I realize that this has a
performance penalty, but believe the tradeoffs are worthwhile. Other
data types should be similarly generalized as well.
>
> > Secondly, a means has to be devised for modules to tell one another
> > what they do or what they need. This is the most difficult problem.
> > I've worked on this for some time looking for a "computerish" solution
> > of formal words and phrases that could be used. My conclusion, at this
> > point, is that English would work best, because people (even
> > programmers?) understand it.
> With a formal proof system like Coq, you can precisely define what modules
> can do and what they need. When formal specifications were not entered yet,
> and/or for people to understand what a module is meant to do, english is also
> a good point (but it can't be used as the basis for computer security,
> whereas proofs can). This is why the reliable modules will have formal
> specifications. But a fundamental concept in TUNES is annotations: you can
> annotate a module object with english text as well as formal proofs, types
> in any type-system, display options, or just anything. Understanding of
> english would be a fine longer-term goal, but it is still very remote.
>
Your solution sounds optimal if it isn't too complex. My notion is more
general in this respect. The idea of English to specify lets programmers
decide what to match. The machine or language would not do this itself.
However, if it could be automated, that would be so much the better.
>
> > A) How to let modules compete. How to score and select the best
> > modules for an application instance.
> This is a point that the TUNES team already considered: TUNES is going
> to be a distributed, persistent system. Which means there is a persistent
> distributed database of modules. By the means of a distributed publishing
> system, people can make available world-wide their feedback about any
> module (as well as any real-life subject they would), in any computer
> or human language (including real-life measurements).
Sounds good to me!
>
> > Undoubtedly, some will ask: "Why hasen't Firing implemented CANDO
> > himself?" The answer is simple. I haven't thought it out completely and
> > don't see how to make a living on it. But I might do it anyway if
> > someone else doesn't do it first.
> What I'll ask is: Why not join our efforts ?
> We seem to have very similar ways to envision the world how it is, and
> how it should be.
>
After spending a little bit of time with your WWW pages, I did, indeed
find some similar ideas and spirit. It struck me, however, that you seem
to have taken on an immense task. I was also puzzled by what seemed to
be a "conspiratorial" view of the computer biz. Having been in the biz
myself for some years and being critical of the direction things have
taken, I still don't believe that any conspiacy is involved -- just the
exercise of power in a free enterprise environment. It is customer
choice (whether we think this choice is rational or not) that determines
who weilds power -- and this tends to change over time.
You have some good ideas and I'll follow your progress with great
interest -- perhaps even participate as it develops.
Thank you, Francois-Rene, for your response and for your kind invitation
to join in your project.
Best wishes, Martel
---------------------------------------------------------------------
Martel Firing
mfiring@primenet.com
---------------------------------------------------------------------
From jecel@lsi.usp.br Tue, 16 May 1995 01:06:54 -0300
Date: Tue, 16 May 1995 01:06:54 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: CANDO
This is a very interesting proposal. The NEEDS and CAN DO parts of a
worker are exactly like the IMPORT and EXPORT of a module, except
that there is an indirection.
This is like the message send compared to subroutine call - the first
has an extra indirection that allows it to refer to any of several
subroutines depending on the type of one ( or more ) arguments. The
second alternative is limited to refering always to exactly the same
subroutine.
In Firing's system the NEEDS part can refer to any of a number of
modules, not just one like the traditional IMPORT. Unlike OOP, all
of the possible modules would be interchangable and the choice would
be made by some scoring system.
The problem with allowing runtime "link" failures is that Smalltalk
already takes such a beating for making the "message not understood"
error possible.
The closest thing I know to CANDO is the Linda system, where tasks
comunicate based on the contents of tuples - they know nothing
about each other.
Comming up with an agreed upon list of services would be very, very
hard. It would also be fundamental for the proposal to work. Based
on how long it is taking for the Object Management Group to do
something like this for CORBA, I would not be very optimistic
about this.
-- Jecel
From rdr@legislate.com Tue, 16 May 95 07:16 GMT
Date: Tue, 16 May 95 07:16 GMT
From: Raul Miller rdr@legislate.com
Subject: CANDO
The problem with allowing runtime "link" failures is that Smalltalk
already takes such a beating for making the "message not understood"
error possible.
There's another possibility here, as exemplified by the various
packages which do symbolic math.
Comming up with an agreed upon list of services would be very, very
hard.
Sounds like symbolic math, again.
--
Raul D. Miller
From rainer@physik3.gwdg.de Tue, 16 May 95 12:23:47 0200
Date: Tue, 16 May 95 12:23:47 0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: Old Dog's New Tricks
The following is a message that I sent to the Tunes mailing list some time ago.
The topic then was parallel distributed multitasking, but the reference fits
even better in this context. The book is a very good place to start reading
about research in market-oriented systems.
Rainer
------- old message follows --------------------------------------------
for those who are interested in this kind of system, i recommend the
following book:
Bernardo A. Huberman [Editor]: The Ecology of Computation
North Holland, New York etc. 1988. 342 p.
(Studies in Computer Science and Artificial Intelligence 2)
ISBN 0-444-70375-6
huberman is (was?) with xerox parc (if i recall correctly) and a lot of
interesting topics (agents, heuristics, agoric (market/ecosystem-oriented)
systems, meme-pools (memes code ideas/knowledge and their use, anologous to
genes coding biological structures and their operation) and authors (lenat,
stefik) are assembled there.
rainer
------------------------------------------------------------------------------
Rainer Blome mailto:rainer@physik3.gwdg.de http://www.physik3.gwdg.de/~rainer
From rainer@physik3.gwdg.de Tue, 16 May 1995 16:04:10 +0200
Date: Tue, 16 May 1995 16:04:10 +0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: CANDO
You say that symbolic math packages address runtime errors and sth. like
unification of services. I know what REDUCE and those do, but I don't use
them and don't see what you mean. Could you be more specific?
Rainer
From fare@Sit-Hyps Tue, 16 May 1995 18:39:15 +0200 (MET DST)
Date: Tue, 16 May 1995 18:39:15 +0200 (MET DST)
From: Francois-Rene Rideau fare@Sit-Hyps
Subject: No, folks, I haven't died.
> Just wanted to drop a little note to suggest that I still do exist,
> am living and breatheing.
Glad to learn it ;-)
> As for the project here itself, does anyone else think it would be
> helpful to have some sort of vision around here?
Yes, many asked about publishing some kind of overview of the project.
I guess I'll have to do it (though I'd be glad if someone else would
head up this task).
> Isn't there at least
> some semi-logical order of what needs to be decided when, so that we
> could "get started"?
The project is somehow parallelized already. My opinion is each
subproject should begin to write exploratory code to settle things
and find the critical points as they refine their specifications.
Synchronization between projects would be done later and progressively
(partial lazy evaluation ;).
> I know the project is going forward, but certainly
> not very fast. I know part of this is due to lack of manpower, but
> couldn't some sort of layout make better use of our availible resources?
With the current charter, the main problem is there lacking maintainers.
I'm open to any suggestion to improve the organization as well as anything
in the project...
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From rideau@clipper.ens.fr Wed, 17 May 95 0:41:59 MET DST
Date: Wed, 17 May 95 0:41:59 MET DST
From: Francois-Rene Rideau rideau@clipper.ens.fr
Subject: work ahead...
Dear Joyous Tunespeople,
it seems indeed that we have some scheduling problem. We can't seem to
communicate well enough to have any real common work getting done.
Anyone has any methodology to propose (not the administrative details,
but just the real-world things) ? Currently, I've received little help
in setting up the ideas in the WWW pages, and received little feedback.
Sure we shall codevelop the HLL and real applications using it (unless
one can find a language satisfying the creteria, of weaken these while
still maintaining the system's coherency).
I've released not very proudly 0.0.0.14 (no patch over 0.0.0.13, just the
archive at ftp://frmap711.mathp7.jussieu.fr/pub/scratch/rideau/Tunes/)
that includes mainly more pointers added to WWW pages
(there is definitely a lot to read on the internet), with some changes to the
TUNES homepage, and a major reorganization of the few that exists for the LLL
sources.
Among the pointers added, there's some Aegis project in MIT about some
"ExoKernel" architecture that is very similar to my no-kernel concept
(and opposite to the micro-kernel fad) and yields services much more flexible
and faster (10 to 100 times) than a monolithic kernel (Ultrix) on same
machine.
I'm sure semi-automatic partial evaluation in the whole system
(instead than just manual partial-evaluation in the system library)
could yield far better results. This only confirms results by other
teams (e.g. Apertos) that did the same kind of experiments but not
with unix-like systems.
Actually, I've much more read than written these weeks.
My conclusion is that there is definitely a place for our project,
that tries to put together innovations most of which were already implemented
separately elsewhere, plus its own that weren't (active annotations), into a
coherent system, whereas previous implementation of these (persistence,
distribution, exokernel, partial evaluation, dynamic scheduling,
self-extensibility) were isolated and out of synch with other parts of the
system.
So there's a lot of work ahead, but it seems to be useful work.
Anyone can propose any kind of organization to actually do common work ?
P.S.: here are topics we could discuss:
* how manage object identity, consistency, and resource-share with
dynamic active annotations ?
* What syntax for the HLL ?
* Who is willing to participate in writing the GC (garbage collector) ?
* Who is willing to participate in the OTOP (over unix) implementation ?
* Who is willing to participate in the i386 (over raw PC hardware)
implementation ?
* Is there any problem with using some mostly ANS FORTH as a LLL ?
* What preliminary bytewise encoding to use for interchanging
objects over the net ?
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From jecel@lsi.usp.br Fri, 19 May 1995 01:39:25 -0300
Date: Fri, 19 May 1995 01:39:25 -0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: Agora Language
I noted that the Agora ftp URL I gave was included in
the language comparison page. I couldn't get Mosaic
or Netscape to see the files in that directory even
though I had no problems with a normal FTp program.
I think this is related to their ftp server being
a Mac.
The reason I mentioned this language ( which was
featured in the December 1994 Sigplan Notices - didn't
I say this already? ) was that it has a Self-like
syntax and object model, but is extendable via
"reifiers" that look a little like Faré's active
notations. I have tried twice to get the papers
I mentioned, but have had the ftp connection fail
midway, so I can't give more details for now. The
language is implemented in ParcPlace Smalltalk and
the sources are available at that site.
Have fun,
-- Jecel
From jecel@lsi.usp.br Fri, 19 May 1995 14:33:41 +0300
Date: Fri, 19 May 1995 14:33:41 +0300
From: Jecel Mattos de Assumpcao Jr. jecel@lsi.usp.br
Subject: languages URL
Here is a WWW pages that points to several
interesting pages about programming ( and
human ) languages:
http://www.cs.brown.edu:80/people/dbc/work/praogramming/
Brown University has an interesting course on Self.
-- Jecel
From chharris@u.washington.edu Sun, 21 May 1995 16:40:25 -0700 (PDT)
Date: Sun, 21 May 1995 16:40:25 -0700 (PDT)
From: 'Chris Harris' B. Harris chharris@u.washington.edu
Subject: Must our computers be this "posessed"?
When I got this new linux machine, I thought all those "weird computer
things" -- those unexplainable happenings that only happend to you, and
that no tech support guy can explain -- would go away. After all, moving
to a memory-protected, multi-user environment from a wimpy little Mac had
to improve something, right?
Well I'm afraid I've been a bit disappointed. This new machine is
starting to feel just as "posessed" as my old one. Granted, it has yet
to crash on me, but crashes are far from everything. Today, for example,
I went to print some rather important documents, using the same technique
I've been using since day one. But instead of finding my printer soon
humming away, ghostscript just sat there, not printing, not giving any
error messages, just idling. A bit of experimenting determined that none
of my ways of printing -- including the generic ASCII dump to the
parallel port -- were going to produce anything more than a blank stare.
I tried rebooting, checked the printer's power, all the connections (all
which were fine), and still nothing worked.
After enough playing around, I finally gave up. While I'll admit I have
less than ideal knowledge of linux, as far as serial ports and printing
go, today I just don't have time to learn all that. And I do have a way
(however wonderfully inconvenient it may be) to print the stuff at
school. But even if I'm not going to loose out here long-term, why does
this need to happen? Is there some unwritten rule that all
programs/OSs/subsystems must act "posessed"?
Well, okay, enough complaining. But the point all this goes to
illustrate is that TUNES, should it ever emerge into existance, _must_be
stable_! TUNES objects and subsystems should provide good error
messages, be able to better diognose which of their dependencies is at
fault, etc.. TUNES (as with technology) has a lot of potential. But if
we end up not being able to trust such basic things as printing, then
what _will_ we be able to trust?
-Chris
--------------------------------------------------------------------------
"Be careful lest in casting out your devil you cast out the best thing
that's in you." --Nietchie %% Me = Chris Harris (chharris@u.washington.edu)
Check out my web page: http://weber.u.washington.edu/chharris/
From rideau@clipper.ens.fr Tue, 23 May 95 1:26:08 MET DST
Date: Tue, 23 May 95 1:26:08 MET DST
From: Francois-Rene Rideau rideau@clipper.ens.fr
Subject: Must our computers be this "posessed"?
>[problems setting up Linux]
Linux took me quite a lot of time to setup. I did it on a necessity
basis, though, part by part, and could reuse my common unix
knowledge from sunos.
>[machine being "possessed"]
> Is there some unwritten rule that all
> programs/OSs/subsystems must act "posessed"?
Well, the problem is that all the free software we're using
is not supported by the hardware manufacturers, so there is
no way to be sure it will work on past or future chips. The
only way is "test", and test cannot be centralized by anyone,
because no one can afford buying lots of hardware to test
free software.
TUNES will have the same problems, at least as of running
on uncommon hardware. Of course, TUNES could also run seamlessly
"On Top Of POSIX", or "On Top Of Windows", just slowlier and less
reliably. It could also run great on some adapted hardware (e.g.
the Forth hardware from the FIRE project). Or we could hack the
format of Windows drivers to use them ? Or drivers from other
software in general.
> Well, okay, enough complaining. But the point all this goes to
> illustrate is that TUNES, should it ever emerge into existance, _must_be
> stable_!
I hope it is, and we should arrange so that it be. But as soon as
nonstandard software is involved (see all those drivers in Linux, plus
all the drivers in the XFree project, plus all the timings to gather),
there's definitely a problem.
> TUNES objects and subsystems should provide good error
> messages, be able to better diognose which of their dependencies is at
> fault, etc..
Yes, that we can do, I hope: good error reporting, so that even if it
doesn't work, people can understand why, and what part of the software to
replace/modify.
> TUNES (as with technology) has a lot of potential. But if
> we end up not being able to trust such basic things as printing, then
> what _will_ we be able to trust?
At first, TUNES won't do much, I fear, except perhaps by using external
posix commands. Writing a terminal or network interface for tunes should
be one of the first things to do, when TUNES works, so we can truely
cross-develop it without having to reboot the development environment,
until we eventually fully bootstrap it.
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From rdr@legislate.com Tue, 23 May 95 00:09 GMT
Date: Tue, 23 May 95 00:09 GMT
From: Raul Miller rdr@legislate.com
Subject: Must our computers be this "posessed"?
if you have to set up linux again, might I recommend the debian
system? debian.cps.cmich.edu:pub/debian
Raul
From fare@Samaris Tue, 30 May 1995 00:21:45 +0200 (MET DST)
Date: Tue, 30 May 1995 00:21:45 +0200 (MET DST)
From: Francois-Rene Rideau fare@Samaris
Subject: release 0.0.0.15
I just released version 0.0.0.15.
I've not published any patch, as I'm not sure anyone uses it.
Actually, I'm not even sure anyone looks at any tunes release :( :( :(
Tell me if I'm wrong...
Here is an un-htmlified exerpt from the Changelog:
Changes from version 0.0.0.14 to 0.0.0.15:
* Modifications in many WWW pages, particularly the home page.
* In the Review subproject:
still more pointers added to the OS and Language pages
wrote skeleton and draft of the CD Database example;
made small enhancements to the Features page.
* In the LLL subproject:
more code in the sources;
redacted some implementation ideas
(particularly, added something about multithreading).
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From fare@Samaris Mon, 29 May 1995 20:31:07 +0200 (MET DST)
Date: Mon, 29 May 1995 20:31:07 +0200 (MET DST)
From: Francois-Rene Rideau fare@Samaris
Subject: Writing an OS for PC (fwd)
Dear TUNESers,
David Logan <73114.3621@compuserve.com> posted a message in
comp.os.research about his pet OS project. I proposed he join us,
and here is his answer (my answer back in next tunes post)...
------>8------>8------>8------>8------>8------>8------>8------>8------>8------
>From 73114.3621@compuserve.com Sat May 27 19:31:17 1995
Received-Date: Sat, 27 May 95 19:31:17 +0200
Date: 27 May 95 13:29:47 EDT
From: David Logan <73114.3621@compuserve.com>
To:
Subject: Writing an OS for PC
Message-Id: <950527172947_73114.3621_EHJ161-2@CompuServe.COM>
-----BEGIN PGP SIGNED MESSAGE-----
Francois -
Hi! I don't know at this point if I'm interested in helping you work on your
operating system. This being said primarily due to the fact that I don't know
what kinds of plans you have. Allow me to provide the letter that I am
providing to people who believe they are interested in helping me. If it
sounds interesting to you, perhaps we can talk some more. Thanks!
==============================================================================
Hello, and thanks for the reply. Every OS designed has to be based off of
something, I guess. (Or multiple somethings). Let me see if I can sum up some
of what I'm trying to do.
1. Of course, true preemptive multitasking.
2. Of course, true 32-bit programming.
3. Refrain from HUGE amounts of memory required to run the OS, like some
other OS's that I know. Keep in mind that this may conflict with the
goals below. However, if it does, then the goals below become the
priority, because at least lots of functionality comes from the size.
Now for some of the things that I want that most other OS's don't have.
1. VERY extensive error recovery and error handling.
a. The OS itself NEVER crashes unless some PL0 task trashes me BEYOND
REPAIR. (This implies an attempt to repair and recover).
b. All tasks, all functions are given the opportunity to recover/repair
their own errors instead of just being killed.
c. Dumps, traces, etc. of where tasks came from, where they went, what
happened, how everything looks, etc. for problem resolution when
something does go badly.
One of the things that has pissed me off no end is that no PC OS even
comes close to having problem determiniation aids outside of a single-step
debugger. In this area, I'm working on using concepts that come from IBM's MVS
operating system. (You'll notice that several ideas come from there).
2. Reportability and accountability. I want to know what my machine is
doing, why, when, how long, and who is causing it.
Note here, that the likelyhood of this OS being designed for "home use" is
probably about 1%. (Less, probably). Again, the concepts are being based off
the mainframe OS. The idea came from my LAN admin (at work) telling me that
he couldn't report on anywhere near the types of things that I can (in the
mainframe world).
3. Support for multiprocessing. I'm including as much code as I can for
multiple processor support.
Obviously, for a while a lot of the MP code will just be anchored (i.e. just
a RET statement), but as much as I can include and support for future use I am
doing.
4. Versatility.
User programs, intermediate 3rd party products, administrator-written user
or OS code, etc. can be very versatile, using the system in whatever ways they
need or would like. Good networking capability, excellent (as much as possible
with the Intel hardware support) cross-memory services, etc.
5. There's more, but it has escaped me for the moment. If we talk again
based on this letter, I'm sure you'll hear some more ideas.
Keep in mind that I don't believe I am a mainframe bigot. I've been an
expert in both mainframe and PC's for years and years. However, also keep in
mind that IBM has been developing high performance, multitasking,
multiprocessing operating systems for years, and their ideas, concepts, and
implementations are quite good. If we can figure out how to implement those
same concepts (as in error handling, recording/tracking, task versatility,
etc.), it has the capability to provide businesses with a full featured,
reliable, usable, excellent OS. Especially as PC's evolve in power and
functionality.
Specifically, some of the things that I have in the plans or are working on
implementing:
1. Only two selectors in the GDT are static (also only 2 in the LDT).
The 2nd is the selector to the Common Memory Area in the GDT, the Local
Memory Area in the LDT (OS reserved virtual storage).
2. OS memory can be fixed or pageable. It can also be protected or
non-protected. The selectors themselves are DPL3, allowing user tasks to
read most of OS storage, but not modify it (based on page table R/W bit).
Obviously, some things must reside in protected storage (U/S bit = S),
like network buffers, security package control blocks, etc.
3. An address space (or sets of segment/page tables) are not related to a
task (or TSS). Multiple tasks can be run under an "address space".
4. Recovery/termination. Any task can make an operating system call at any
time to establish an "error recovery" routine to be called in the event
of an error. This routine can correct/bypass the error, and request the
OS retry the origional routine, or continue the termination process.
Required in all OS routines, especially in global ones that can be done
under any TSS. Also required in any tasks/address spaces that would hang
the system in the event of a failure.
5. Special environments for the double fault, NMI, and stack fault handlers.
Make EVERY effort to protect these tasks code/control blocks so in the
event of a serious failure, the system doesn't hang. These routines will
support dumping of the system for problem resolution. (Dumping of all or
parts of the system are supported in all cases, initiated either by the
OS or a user task. It's just that these three tasks are very critical to
recovery of the OS, and at least problem resolution if recovery is
impossible). The stack fault handler becomes "special" only in the event
that a stack fault occurs on a PL0 stack.
6. A million others, but these should give you a taste of what I'm reaching
for.
If you're still interested after reading this book, let me know! I'd be glad
to hear new ideas, implementation concepts, and even willing to help code!
David Logan,
Boulder, CO
-----BEGIN PGP SIGNATURE-----
Version: 2.6
iQB1AwUBL8dhnxylXvjtppttAQGrlQL+Nk+SKi/OnQG2fxdt4nhGUGPAbV7T6Pfd
dLS9AyvtqB7K3+U/Y36BqjhudW9gvCZpixdX4qD2YUW5ZlORXLt7wkeJGSms45vs
Xc1fhf5SSHs10rbaULQajksx+qkeHp/a
=WPnk
-----END PGP SIGNATURE-----
------>8------>8------>8------>8------>8------>8------>8------>8------>8------
(end of forwarded message)
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From fare@Samaris Tue, 30 May 1995 01:07:02 +0200 (MET DST)
Date: Tue, 30 May 1995 01:07:02 +0200 (MET DST)
From: Francois-Rene Rideau fare@Samaris
Subject: Writing an OS for PC
> Hi! I don't know at this point if I'm interested
> in helping you work on your operating system.
Sure. I appreciate your being careful !
> This being said primarily due to the fact that I don't know
> what kinds of plans you have.
Have a look to the TUNES page "http://acacia.ens.fr:8080/home/rideau/Tunes/"
for more info.
> ============================================================================
> Hello, and thanks for the reply. Every OS designed has to be based off
> of something, I guess. (Or multiple somethings).
And TUNES is based on quite some deep concepts. It is truely a
revolutionary OS design (at least as compared to commercial software and
concepts everyone seem crazy about).
> 1. Of course, true preemptive multitasking.
Yes and No.
Preemption is made necessary by the complete unsecurity that has deep
roots in traditional OS design. But when possible (most of the time),
cooperation yields *far* better performance. In TUNES, we will provide
the required security for cooperation, and of course will take advantage of
the enhanced performance of cooperation. See the TUNES LLL page about that:
"http://acacia.ens.fr:8080/home/rideau/Tunes/LLL/LLL.html#multithreading".
Of course, this security is just impossible to achieve using C [well, not
completely impossible; but then, C would be only a very heavy burden, and would
lose *all* its advantages: existing libraries and compilers]. Which is why
TUNES refuses C as a standard system language.
> 2. Of course, true 32-bit programming.
Well, sure the implementation on 32-bit computers like intel PCs should
be 32-bit. But TUNES intends to allow dynamic distribution on heterogeneous
platforms with different word size (e.g. 8, 16, 20, 32, 64 or 128 bits).
Thus 32 bit should not be hardwired anywhere in the system, be purely
implementation dependent, and should not interfere with dynamic code migration.
> 3. Refrain from HUGE amounts of memory required to run the OS, like some
> other OS's that I know. Keep in mind that this may conflict with the
> goals below. However, if it does, then the goals below become the
> priority, because at least lots of functionality comes from the size.
Well, again, features occupy memory. The problem with existing systems is
that they are coarse grained: you cannot isolate a useful feature from lots
and lots of useless but costly (hence harmful) ones.
TUNES will provide fine grain at all levels: you can determine exactly
what modules you have. A stripped specialized system should fit the small
ROM of some embedded micro-controller, while a full featured one could
take advantage of thousands gigs of distributed shared memory.
See the TUNES Features page:
"http://acacia.ens.fr:8080/home/rideau/Tunes/Review/Features.html".
> Now for some of the things that I want that most other OS's don't have.
>
> 1. VERY extensive error recovery and error handling.
> a. The OS itself NEVER crashes unless some PL0 task trashes me BEYOND
> REPAIR. (This implies an attempt to repair and recover).
I fully agree. TUNES shall include support for extensive debugging.
Now, with existing software tools, writing usable error recovery code is
*very* difficult. The TUNES system shall have support for error-recovery
at language level. Exceptions, first-class continuations are good. But not
enough. Higher-order programming should allow programmable debug code
generation from annotated sources.
> b. All tasks, all functions are given the opportunity to recover/repair
> their own errors instead of just being killed.
> c. Dumps, traces, etc. of where tasks came from, where they went, what
> happened, how everything looks, etc. for problem resolution when
> something does go badly.
This is just not enough: bugs may be completely hidden at low-level.
Have you ever tried to assembly debug code generated for an OO language ?
Or the interpretation of byte-code or threaded code ? Debugging is needed
at source level. A language that requires debugging is lame. A language that
doesn't allow debugging is lamer even. TUNES proposes that programs be
annotatable with debugging information, as well as with full specifications and
proof of correctness.
> 2. Reportability and accountability. I want to know what my machine is
> doing, why, when, how long, and who is causing it.
This is administration. Again, TUNES proposes that this be solved with
dynamic active annotations: you can annotate a function f with some
code (e.g. annotate login with user access accounting, memory
allocation with quota checking, tree walking with action at node, etc).
Of course, for efficiency, functions can be closed to further active
annotation, so that it be optimized.
> Note here, that the likelyhood of this OS being designed for "home use" is
> probably about 1%. (Less, probably).
TUNES aim at world-wide utility: we want to provide some OS to kick out
traditional coarse-grained, unsecure, short-term design that poisons the
computing industry and forbids fair competition.
Even if TUNES does not become a world-wide OS, we hope the TUNES experience
can speed up the appearance of such an OS.
> 3. Support for multiprocessing. I'm including as much code as I can for
> multiple processor support.
TUNES strives to support dynamic code migration over heterogeneous
networks. multiprocessing is just a very particular case of this.
> 4. Versatility.
>
> User programs, intermediate 3rd party products, administrator-written user
> or OS code, etc. can be very versatile, using the system in whatever
> ways they
> need or would like. Good networking capability, excellent
> (as much as possible
> with the Intel hardware support) cross-memory services, etc.
TUNES provides fine-grained modularity, and ends all arbitrary
distinction between OS, drivers, applications, third party code,
local configuration and user interaction. Code naturally differenciates
according to its objective properties, far better than any company or
even technical committe.
TUNES will just provide higher-order fine-grained modules, that
can be linked together automatically or manually in any useful way
by the various people that manage or use the system.
> Specifically, some of the things that I have in the plans or are working on
> implementing:
Firstly, let's make it clear that all PC specificities should not modify
in any way the user-visible specifications of the system. Only the system
and compiler back-end implementors should /need/ know them.
TUNES will be as portable as can be. Once portable code is running,
everyone is free to use the niftiest trick to make one's modules faster
and smaller on one's computer.
> 1. Only two selectors in the GDT are static (also only 2 in the LDT).
> The 2nd is the selector to the Common Memory Area in the GDT, the Local
> Memory Area in the LDT (OS reserved virtual storage).
Of course, as few selectors as possible should be static,
but more than two may be useful, be it just to allow use of code
in multiple priviledge rings, return to real or v86 mode, etc.
> 2. OS memory can be fixed or pageable. It can also be protected or
> non-protected. The selectors themselves are DPL3, allowing
> user tasks to
> read most of OS storage, but not modify it
> (based on page table R/W bit).
> Obviously, some things must reside in protected storage (U/S bit = S),
> like network buffers, security package control blocks, etc.
Sure, we shall take the most out of the hardware capabilities.
Now, I can't see why the programmer would want to interfere with such
low-level unportable things. We shall first limit all that as much as
possible. Later modules can implement their own abstractions and management
of these.
> 3. An address space (or sets of segment/page tables) are not related to a
> task (or TSS). Multiple tasks can be run under an "address space".
Tasks are some coarse-grained abstraction, that only introduce
overhead with no gain. Only emulation boxes and unsafe code need run in
isolated address space.
> 4. Recovery/termination. Any task can make an operating system call
> at any
> time to establish an "error recovery" routine to be called in the event
> of an error.
In TUNES, there is no such thing as "system call". Everything is specified
at high level, and code should be dynamically compiled/linked to achieve the
best performance (which centralized kernel call in no way allows).
See e.g. the MIT Aegis exokernel project, to see how removing lame kernel
concept already speeds up 100 times system services.
> This routine can correct/bypass the error, and request the
> OS retry the origional routine, or continue the termination process.
> Required in all OS routines, especially in global ones that can be done
> under any TSS. Also required in any tasks/address spaces that
> would hang
> the system in the event of a failure.
Now, as for recovery, I'm sure dynamic active annotations, lazy/partial
evaluation (futures), exceptions and first-class continuations make things
easy enough, without requiring any "OS" support. TUNES does not differentiate
some "OS" from other software components. All software come into fine-grained
higher-order modules that dynamically link together. "Registering" a function
is just a very special case of dynamic linking.
> 5. Special environments for the double fault, NMI,
> and stack fault handlers.
> Make EVERY effort to protect these tasks code/control blocks so in the
> event of a serious failure, the system doesn't hang.
> These routines will
> support dumping of the system for problem resolution.
> (Dumping of all or
> parts of the system are supported in all cases, initiated either by the
> OS or a user task. It's just that these three tasks are very
> critical to
> recovery of the OS, and at least problem resolution if recovery is
> impossible). The stack fault handler becomes "special" only in the
> event
> that a stack fault occurs on a PL0 stack.
Of course we shall have code to report such things. Let's just create
a hook for it into high-level abstractions. At the first place, using only
safe code from safe compilers out of strongly typed or proven programs
should remove all such errors. If we restrict unproven untyped code to the
least possible, we know where the bug comes from, and bugs shall eventually
disappear.
That's how TUNES see things: instead of building a huge number of paranoid
low-level checks, provide a high-level way (strong typing and/or correctness
proof) to ensure safety. It is infinitely more reliable. As for self-foundation
problem, of course there will have to be some code we trust; but again this
should eventually be as small a portion of code as possible, and its sources
shall be freely available so anyone can check it and correct it. The least
unreliable thing is what everybody checked but nobody could make fail.
> If you're still interested after reading this book, let me know!
> I'd be glad
> to hear new ideas, implementation concepts, and even willing to help code!
Sure I'm interested. Even if you decide not to join TUNES (which I will
be sad of), I hope you succeed. Good luck !
And do have a look at the TUNES page (see URL below).
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From 73114.3621@compuserve.com 30 May 95 10:05:10 EDT
Date: 30 May 95 10:05:10 EDT
From: David Logan 73114.3621@compuserve.com
Subject: Writing an OS for PC
-----BEGIN PGP SIGNED MESSAGE-----
Francois -
I tried to send this reply to the internet address I recieved from your last
reply. It was returned, and the text in it's entirety is here. I'm going to
send a 2nd reply to this address.
David Logan
==============================================================================
Sender: mailer-daemon@compuserve.com
Received: from localhost by arl-img-4.compuserve.com (8.6.10/5.950515)
id WAA20467; Mon, 29 May 1995 22:52:18 -0400
Date: Mon, 29 May 1995 22:52:18 -0400
From: Mail Delivery Subsystem
Subject: Returned mail: Host unknown (Name server: samaris.ens.fr: host not found)
Message-Id: <199505300252.WAA20467@arl-img-4.compuserve.com>
To: 73114.3621@compuserve.com
The original message was received at Mon, 29 May 1995 22:52:17 -0400
from mailgate@localhost
----- The following addresses had delivery problems -----
root@samaris.ens.fr (unrecoverable error)
----- Transcript of session follows -----
550 root@samaris.ens.fr... Host unknown (Name server: samaris.ens.fr: host not found)
----- Original message follows -----
Received: by arl-img-4.compuserve.com (8.6.10/5.950515)
id WAA20462; Mon, 29 May 1995 22:52:17 -0400
Date: 29 May 95 22:49:54 EDT
From: David Logan <73114.3621@compuserve.com>
To: System administrator
Subject: Writing an OS for PC
Message-ID: <950530024954_73114.3621_EHJ47-1@CompuServe.COM>
- -----BEGIN PGP SIGNED MESSAGE-----
Francois -
You have some interesting concepts, some of which I'd like to ask a few more
questions about. The first thing that comes to mind after reading your letter
is that it seem you are attempting to create a very strictly controlled
environment. Hmmm. I don't know about that. I've always found difficulty in
very controlled environments. As a programmer, I always, but always, find
something that I need to do where there is no way (or support). But anyway,
let's go on:
>> Preemption is made necessary by the complete unsecurity that has
>> deep roots in traditional OS design. But when possible (most of the
>> time), cooperation yields *far* better performance. In TUNES, we will
>> provide the required security for cooperation
What is "required security"? Without preemption, you have no way of keeping
a task from locking up the entire system. You also have no way of executing
work with any type of priority scheme, as far as I can tell, unless you have
some OS implementation ideas I haven't grasped. I do agree that cooperation
can improve performance. However, after dealing with Windows for so long, I am
really tired of "cooperation", where one task decides he's the only one that
gets to make the CP wait for god knows how long.
>> Of course, this security is just impossible to achieve using C [well,
>> not completely impossible; but then, C would be only a very heavy
>> burden, and would lose *all* its advantages: existing libraries and
>> compilers]. Which is why TUNES refuses C as a standard system
>> language.
I'm not an advocate or an opposite of C. I use any and all programming
languages. However, I don't really understand how anything is impossible in C,
nor how C code could be a heavy burden on anything, esp. with a good optimizer.
(Could be kinda bias, though, I prefer assembler to all languages). So what
language DO you plan on using? I can't think of anything better than C than
assembler.
>> Well, sure the implementation on 32-bit computers like intel PCs
>> should be 32-bit. But TUNES intends to allow dynamic distribution on
>> heterogeneous platforms with different word size (e.g. 8, 16, 20, 32,
>> 64 or 128 bits). Thus 32 bit should not be hardwired anywhere in the
>> system, be purely implementation dependent, and should not interfere
>> with dynamic code migration.
Obviously, you do plan on using a high-level language that supports porting.
Again, one of the things that I don't understand is how you are going to
prevent dependencies on your word size. From the user perspective, it's much
easier. From the OS point of view, not depending on the hardware would be
virtually impossible, wouldn't it?
>> TUNES will provide fine grain at all levels: you can determine
>> exactly what modules you have. A stripped specialized system should
>> fit the small ROM of some embedded micro-controller, while a full
>> featured one could take advantage of thousands gigs of distributed
>> shared memory.
You are aware that recapturing efficiencies in other areas are going to be
spent on this arena? One of the reasons that I am choosing to avoid a message
based system is due to the fact that zillions of messages, function calls upon
function calls, strict interfaces to millions of little functions consumes vast
amounts of resources. "Fine grain" sounds like it will take a significant
amount of resource to get from the 1st user call, all the way down and back
again, depending on what was requested of it. I guess "fine grain" is
relative, though, and perhaps I don't understand your breaking points.
>> Now, with existing software tools, writing usable error recovery code
>> is *very* difficult.
With the MVS operating system, error recovery code is quite easy to write and
implement. The OS calls the current error recovery procedure, and keeps
cascading through previously catalogued error recovery procedures until one
handles it to the point it wants the OS to "retry" the user's process. The
funnest part is actually figuring out what to do in various error conditions.
>> Higher-order programming should allow programmable debug code
>> generation from annotated sources.
Programmable debug code generation? What's that? Is this the same stuff
that's written into Borlands C and ASM programs when you request debug support?
This is great for user programs, but often impossible for system-oriented
programs/tasks.
>> Debugging is needed at source level. A language that requires debugging
>> is lame. A language that doesn't allow debugging is lamer even.
I don't disagree with the need for source-level debugging. It is valuable
beyond ... Other things are required as well, though. No matter how well
anything is written, things end up in endless loops, memory does get trashed,
etc. Source level debugging *only* is simply not enough for all problems. The
statement about a language that requires debugging is interesting. How in the
world are you ever going to prevent logic errors? Errors like moving a count
to a pointer, a pointer to a count, and be-bopping through an array? Such a
high-level language where the statement "DO IT." generates and runs an entire
program (exaggeration, of course, but not sarcasm)?
>> Firstly, let's make it clear that all PC specificities should not
>> modify in any way the user-visible specifications of the system. Only
>> the system and compiler back-end implementors should /need/ know
>> them. TUNES will be as portable as can be. Once portable code is
>> running, everyone is free to use the niftiest trick to make one's
>> modules faster and smaller on one's computer.
I like portability. Everyone likes portability. It keeps people from having
to write 15 identical programs in 15 different ways. However, I don't believe
that PC specificities (nice word) should be shut out. A good programmer who
needs the thoroughput can always find the best way to do that with the hardware
they are on, as long as the capability is there. The point, I guess, is that I
always want the capability to use environment-dependent code if I choose. I
may choose to write some small functions this way if it can improve my process
X%, with the rest of the product calling the small functions. In this way, I
can maximize my program, and minimize the rewrite requirements in the case of a
port.
>> Of course, as few selectors as possible should be static, but more
>> than two may be useful, be it just to allow use of code in multiple
>> priviledge rings, return to real or v86 mode, etc.
Depending on the efficiency requirements, I suppose. My method is to have
two static selectors. The first selector is a data pointer to itself (both GDT
and all LDTs). The second is the OS virtual memory pool, where everything else
in the world is anchored to, somehow, someway.
>> Sure, we shall take the most out of the hardware capabilities. Now,
>> I can't see why the programmer would want to interfere with such
>> low-level unportable things. We shall first limit all that as much as
>> possible. Later modules can implement their own abstractions and
>> management of these.
Basically to report. Report on tasks, users, memory, device characteristics
or status, etc. In these cases, if the memory aquired by the OS to maintain
this information is DPL 0, it requires a task to go DPL 0 to get it. This is
just one more task that can kill the OS. If you make as much as possible DPL
3, write-protected, you have the ability to minimize OS stability. You even
have the capability of running much of the OS itself DPL 3 to prevent it from
munching itself. It only goes DPL 0 when it must perform something requiring
the privilege.
>> Tasks are some coarse-grained abstraction, that only introduce
>> overhead with no gain. Only emulation boxes and unsafe code need run
>> in isolated address space.
Please don't take offense, but I think this statement is ridiculous. The
first thing that I'd say is: unsafe code = EVERYTHING. Emulation boxes, of
course. But the first thing that comes to mind is that every task has the
capability to use a 4GB address space. If you don't task, then ALL tasks can
only share 1 4GB. Back to the days when people didn't know what virtual
address spaces were, almost. Also, I believe there is lots of gain. It
utilizes the hardwares capability to execute other tasks when the current task
cannot execute, instead of having the OS software have to try to perform a
state switch. This is what DOS has done for years with TSR's.
>> In TUNES, there is no such thing as "system call".
This is what I think I'm getting, but still wondering about. It sound like
you are trying to create a product where user tasks and system tasks alike hold
hands, and dance nicely together around in a circle. Non-preemption, linking,
load-modules for functions, no system calls, no tasks, etc. I still don't
understand how you are going to prevent a programmer from bringing your system
to it's knees by: a) Not following standards, b) Forgetting a piece of logic,
c) Intent, etc. If there's nothing I like more about MVS (One of IBM's OS's),
is that it is virtually impossible to kill, no matter WHAT you do. That single
item is probably priority #1 by FAR in my design.
>> That's how TUNES see things: instead of building a huge number of
>> paranoid low-level checks, provide a high-level way (strong typing
>> and/or correctness proof) to ensure safety. It is infinitely more
>> reliable. As for self-foundation problem, of course there will have
>> to be some code we trust; but again this should eventually be as
>> small a portion of code as possible, and its sources shall be freely
>> available so anyone can check it and correct it. The least unreliable
>> thing is what everybody checked but nobody could make fail.
That's a great thing to hope to achieve. I forsee problems in the future,
though, about workload mix in this environment. Do you see it as an OS where
anyone can run anything, anytime that it's required? Background/foreground
jobs, online users, file servers, communications/network programs, programmer
development, management reporting, etc. etc. etc.?
>> And do have a look at the TUNES page (see URL below).
I'll have to do that. One reason is to find out what "annotated" means to
you guys. I'm having trouble relating that word to programs.
David Logan,
Boulder, CO
- -----BEGIN PGP SIGNATURE-----
Version: 2.6
iQB1AwUBL8qIJRylXvjtppttAQGSDAL9EnU9pn1L26AHPkri6omP0eTLpN74LK8a
5C+pXiWE088BNbvxmGkw0q2mtB1waf7xwOyXJELcWLJBMWwbd+qJBCVRWs+C+O1n
5sNmzRE2G4xQWd/gtV97w3veF2oqBMkZ
=3TBs
- -----END PGP SIGNATURE-----
<<>>
-----BEGIN PGP SIGNATURE-----
Version: 2.6
iQB1AwUBL8slhxylXvjtppttAQGqygL6A8dRcMNi6dXE+kUvXFh9O8Impf9YPCnt
j354V1aWAcDXMtVgNTl1SkoyrnFffVnRT4BXns9Zn9052lAQ9C5Al07s21gaH9L6
mL7U06FvB/jQDYgLPD3MgkFJc+IhxuXy
=q3j8
-----END PGP SIGNATURE-----
From rideau@clipper.ens.fr Tue, 30 May 95 21:34:25 MET DST
Date: Tue, 30 May 95 21:34:25 MET DST
From: Francois-Rene Rideau rideau@clipper.ens.fr
Subject: Writing an OS for PC
> You have some interesting concepts,
> some of which I'd like to ask a few more
> questions about.
Please do...
> The first thing that comes to mind after reading your letter
> is that it seem you are attempting to create a very strictly controlled
> environment. Hmmm. I don't know about that.
> I've always found difficulty in
> very controlled environments. As a programmer, I always, but always, find
> something that I need to do where there is no way (or support).
The only kind of control in traditional systems is paranoid forbiddance.
Sure it's stupid and uneffective, and everybody hates it. Now, that's
not what TUNES will do. The kind of control we have is higher-order
correction proof, that is, all that a human can logically say to prove
that a program is safe.
Strong typing is a particular case of that (see the ML category of
languages). TUNES will eventually allow people to prove that assembly
routines are correct, so that people need not be root to use assembly.
Also, higher-order program manipulation allows to optimize code through
safe programmable code transformations.
>> Preemption is made necessary by the complete unsecurity that has
>> deep roots in traditional OS design. But when possible (most of the
>> time), cooperation yields *far* better performance. In TUNES, we will
>> provide the required security for cooperation
>
> What is "required security"?
All binary code is required to abide by cooperation rules.
Compilers are required to produce such code.
By default, the system checks some PGP signature to ensure that
code was emitted from a valid source. Paranoid people can have an
empty list of valid sources, and will thus refuse to run any code
that was externally generated.
> Without preemption, you have no way of keeping
> a task from locking up the entire system.
Yes there is: be sure beforehand that the task will not lcok up the system.
Do not run unsafe tasks. That's like removing runtime-checks when you know
the checked integer won't overflow. You can, if you're careful.
> You also have no way of executing
> work with any type of priority scheme, as far as I can tell, unless you have
> some OS implementation ideas I haven't grasped.
See the multithreading entry in the TUNES LLL page:
"http://acacia.ens.fr:8080/home/rideau/Tunes/LLL/LLL.html#multithreading".
You can very well mix cooperation and timing.
> I do agree that cooperation
> can improve performance.
> However, after dealing with Windows for so long, I am
> really tired of "cooperation", where one task decides he's the only one that
> gets to make the CP wait for god knows how long.
Because Windows sucks, and windows development tools suck, because they
are C based. Sure, there is no way to achieve fair cooperation using existing
tools. This is why TUNES will get rid of these.
>> Of course, this security is just impossible to achieve using C [well,
>> not completely impossible; but then, C would be only a very heavy
>> burden, and would lose *all* its advantages: existing libraries and
>> compilers]. Which is why TUNES refuses C as a standard system
>> language.
>
> I'm not an advocate or an opposite of C. I use any and all programming
> languages. However, I don't really understand how anything is
> impossible in C,
> nor how C code could be a heavy burden on anything,
> esp. with a good optimizer.
Yes, and with a good optimizer for Turing machine, it is possible to
compile efficient code for it, too...
Seriously, nothing is completely impossible in any Turing-equivalent
language. But some languages make program development, maintenance,
modification, such a mess, that people should really use another language.
> (Could be kinda bias, though, I prefer assembler to all languages). So what
> language DO you plan on using? I can't think of anything better than C than
> assembler.
Do you know FORTH ? Lisp ? ML ? Haskell ? BETA ? SELF ? Perl ? Icon ?
Have a try !
>> Well, sure the implementation on 32-bit computers like intel PCs
>> should be 32-bit. But TUNES intends to allow dynamic distribution on
>> heterogeneous platforms with different word size (e.g. 8, 16, 20, 32,
>> 64 or 128 bits). Thus 32 bit should not be hardwired anywhere in the
>> system, be purely implementation dependent, and should not interfere
>> with dynamic code migration.
>
> Obviously, you do plan on using a high-level language
> that supports porting.
> Again, one of the things that I don't understand is how you are going to
> prevent dependencies on your word size. From the user perspective,
> it's much
> easier. From the OS point of view, not depending on the hardware would be
> virtually impossible, wouldn't it?
No, it wouldn't. Why should humans be able to do it, and not machines ?
Determining the right size for a word is just a matter of knowing the
range of values some variable will represent. If people statically
know it, a machine can know it statically too. If people don't know it
statically, the machine can dynamically adjust the size much more reliably
than any human could do.
>> TUNES will provide fine grain at all levels: you can determine
>> exactly what modules you have. A stripped specialized system should
>> fit the small ROM of some embedded micro-controller, while a full
>> featured one could take advantage of thousands gigs of distributed
>> shared memory.
>
> You are aware that recapturing efficiencies in other areas are going to be
> spent on this arena?
Uh ? I'm not sure I grasp what you mean...
> One of the reasons that I am choosing to avoid
> a message
> based system is due to the fact that zillions of messages,
> function calls upon
> function calls, strict interfaces to millions of little functions
> consumes vast
> amounts of resources. "Fine grain" sounds like it will take a significant
> amount of resource to get from the 1st user call, all the way down and back
> again, depending on what was requested of it. I guess "fine grain" is
> relative, though, and perhaps I don't understand your breaking points.
Message passing is just a model among others, that is isomorphic to
function call. TUNES will not priviledge any model, and let people use
whichever model suits their purpose the best. Now, as of the number of
messages being passed or functions being called, have you ever heard about
inlining function calls ? I'm sure you have, and often do it manually.
The system can very well do it dynamically, which is commonly called
"partial evaluation", and yields great results while simplifying system
design a lot: compiling, interpreting, evaluating, lazy evaluation and
futures are just special cases of partial evaluation.
No more function calls than needed will actually happen in TUNES,
which is not the case with "kernel" or worse "microkernel" design where
lots of time is wasted in stupid lame useless runtime dispatchers.
>> Now, with existing software tools, writing usable error recovery code
>> is *very* difficult.
>
> With the MVS operating system, error recovery code is quite easy
> to write and
> implement. The OS calls the current error recovery procedure, and keeps
> cascading through previously catalogued error recovery procedures until one
> handles it to the point it wants the OS to "retry" the user's process. The
> funnest part is actually figuring out what to do in various error conditions.
Yeah. That's called throwing exceptions in modern programming languages.
Now, the problem is when you add a new kind of exception, lots of the new
code becomes obsolete, because you can't do automatic higher-order
transformations to make old code aware of the new exception. Thus you have
to manually update old code, and maintenance is a mess.
>> Higher-order programming should allow programmable debug code
>> generation from annotated sources.
>
> Programmable debug code generation? What's that? Is this the same stuff
> that's written into Borlands C and ASM programs when you request debug
> support?
> This is great for user programs, but often impossible for system-oriented
> programs/tasks.
No, it's much more than that. It's dynamic code annotation. You should be
able to dynamically annotate the debuggable code, to add arbitrary
features to it. Existing systems only allow to do very restricted things
at debugging points, with no way to automatize things.
For example, in TUNES, profiling is a triffle: just annotate all functions
(automatically, not manually) with some automatically generated accounting
code. You can personalize this annotation in any way you want, have special
cases for special functions, etc.
Have you ever tried to debug threaded code with an existing asm level
debugger ? Or debug a program being interpreted from a source-debugger
for the interpreter ? If you ever tried, you may have an idea about what I
mean...
>> Debugging is needed at source level. A language that requires debugging
>> is lame. A language that doesn't allow debugging is lamer even.
>
> I don't disagree with the need for source-level debugging. It is valuable
> beyond ...
> Other things are required as well, though. No matter how well
> anything is written, things end up in endless loops, memory does get trashed,
> etc.
Endless loops yes, memory being wasted, yes, (well, in many case, one
can *prove* it won't be, but if generic enough human interaction is allowed,
yes there are problems). Memory being trashed, no: strong typing prevents it.
> Source level debugging *only* is simply not enough for all problems.
Depends on what "debugging" is. See above.
> The
> statement about a language that requires debugging is interesting.
> How in the
> world are you ever going to prevent logic errors?
By allowing not to run any program that doesn't come with a proof,
and limiting the compulsory set of admitted axioms to a very
limited amount of programs whose sources are freely available.
> Errors like moving a count
> to a pointer, a pointer to a count, and be-bopping through an array?
Strong typing already solves such things.
Safely optimizing a strongly typed program can yield programs as
good as manually doing it will, and much safer.
> Such a
> high-level language where the statement "DO IT." generates and runs an entire
> program (exaggeration, of course, but not sarcasm)?
The system can very well (and will) track down the meaning of such
statements. Program manipulation is not done at ascii level, but at the
level of some internal representation...
>> Firstly, let's make it clear that all PC specificities should not
>> modify in any way the user-visible specifications of the system. Only
>> the system and compiler back-end implementors should /need/ know
>> them. TUNES will be as portable as can be. Once portable code is
>> running, everyone is free to use the niftiest trick to make one's
>> modules faster and smaller on one's computer.
> [...]
> The point, I guess, is that I
> always want the capability to use environment-dependent code if I choose.
I never denied it: see about being free to use the niftiest tricks.
>> Of course, as few selectors as possible should be static, but more
>> than two may be useful, be it just to allow use of code in multiple
>> priviledge rings, return to real or v86 mode, etc.
>
> Depending on the efficiency requirements, I suppose. My method is to have
> two static selectors. The first selector is a data pointer to itself
> (both GDT
> and all LDTs).
> The second is the OS virtual memory pool, where everything else
> in the world is anchored to, somehow, someway.
Hey ! You already need selectors for executable code ! Why have them
more or less static than the above ??
>> Sure, we shall take the most out of the hardware capabilities. Now,
>> I can't see why the programmer would want to interfere with such
>> low-level unportable things. We shall first limit all that as much as
>> possible. Later modules can implement their own abstractions and
>> management of these.
>
> Basically to report.
Only the system implementers need such reports, while debugging.
Hopefully, clean design will reduce the need for debugging...
> Report on tasks, users, memory, device characteristics
> or status, etc. In these cases, if the memory aquired by the OS to maintain
> this information is DPL 0, it requires a task to go DPL 0 to get it. This is
> just one more task that can kill the OS. If you make as much as possible DPL
> 3, write-protected, you have the ability to minimize OS stability. You even
> have the capability of running much of the OS itself DPL 3 to prevent it from
> munching itself. It only goes DPL 0 when it must perform something requiring
> the privilege.
In my vision, security is maintained at a much higher level than the priviledge
mode. Low priviledge would be used only to enable paging write trapping, and
run emulators. Protection is a software concept: have programs that don't
crash and trash. Hardware can't help. All these hardware protections are the
stupidest things ever conceived. Should the effort put in there have been
used to produce simpler hardware and more advanced software, computers would
have been way faster and more reliable.
>> Tasks are some coarse-grained abstraction, that only introduce
>> overhead with no gain. Only emulation boxes and unsafe code need run
>> in isolated address space.
>
> Please don't take offense, but I think this statement is ridiculous. The
> first thing that I'd say is: unsafe code = EVERYTHING.
No. Have you ever seen true strongly typing languages ?
Have you ever seen reliable compilers ?
Do you know about formal programming methods ?
All these can produce programs that you can be sure won't crash,
even if they don't do what you think they will.
> Emulation boxes, of course.
> But the first thing that comes to mind is that every task has the
> capability to use a 4GB address space. If you don't task, then ALL tasks can
> only share 1 4GB. Back to the days when people didn't know what virtual
> address spaces were, almost.
Yes. Who needs hardware virtual memory ? Virtual memory can be done much
better in software. The money spent in doing hardware virtual memory is
pure waste. Having fast on-chip RAM instead of lame TLBs, with fast
read/write instructions would allow truely faster computers (see the MuP21)
at much lower cost. TUNES will take advantage of hardware capabilities when
there are, but won't require any to exist.
> Also, I believe there is lots of gain. It
> utilizes the hardwares capability to execute other tasks when the current
> task
> cannot execute, instead of having the OS software have to try to perform a
> state switch. This is what DOS has done for years with TSR's.
Why should hardware switch be better than software switch ?
The software knows much better what it needs to switch. Hardware designers
should provide tools to software designers, and not impose constraints on
them.
>> In TUNES, there is no such thing as "system call".
>
> This is what I think I'm getting, but still wondering about.
> It sound like
> you are trying to create a product where user tasks and system tasks
> alike hold
> hands, and dance nicely together around in a circle.
Exactly !
> Non-preemption, linking,
> load-modules for functions, no system calls, no tasks, etc.
> I still don't
> understand how you are going to prevent a programmer from bringing
> your system to it's knees by:
> a) Not following standards,
If you mean not using a safe compiler,
the administrator can and will forbid the system to run such programs.
To run some unsafe program, you need to use some emulation box (TUNES
can emulate itself trivially) or the consent of a superuser.
> b) Forgetting a piece of logic,
Uh ? Sure a bad programmer can always crash *his* program, and all the
parts of the system he was granted random write access to.
But this will always be the case whatever you do. Nobody can ever
prevent you for doing "/bin/rm -rf /".
> c) Intent, etc.
Intent ? what's that ?
> If there's nothing I like more about MVS (One of IBM's OS's),
> is that it is virtually impossible to kill, no matter WHAT you do.
> That single
> item is probably priority #1 by FAR in my design.
I don't know MVS, and thus don't get what you mean by "kill".
Whose killing are you talking about ?
> Do you see it as an OS where
> anyone can run anything, anytime that it's required?
> Background/foreground
> jobs, online users, file servers,
> communications/network programs, programmer
> development, management reporting, etc. etc. etc.?
Yes, anyone is free to do anything he likes with his account.
No one is bound to trust what someone else is doing, though.
If you don't trust Mr X's fileserver, don't use it. Nobody will
force you do it. But if you trust it (or trust some abstraction
over it), you can use it seamlessly like any other fileserver.
Also, you can close access to your objects, so that others can't
write it, can't annotate it, or can`t read it (Actually, just can't
*see* it).
>> And do have a look at the TUNES page (see URL below).
>
> I'll have to do that. One reason is to find out what "annotated" means to
> you guys. I'm having trouble relating that word to programs.
Yeah. Unhappily, this particular point ain't always very clear
in the TUNES pages yet.
Basically, annotating is a way of extending the meaning of an object
to a broader context. In TUNES, it is the one way to define properties of
objects, and it can be done dynamically.
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From bromage@cs.mu.OZ.AU Wed, 31 May 1995 10:51:53 +1000 (EST)
Date: Wed, 31 May 1995 10:51:53 +1000 (EST)
From: Andrew Bromage bromage@cs.mu.OZ.AU
Subject: Writing an OS for PC
G'day.
Francois-Rene Rideau writes:
> > (Could be kinda bias, though, I prefer assembler to all languages). So what
> > language DO you plan on using? I can't think of anything better than C than
> > assembler.
I recommend the tech report EE9240 from the Department of Electrical
and Computer Engineering, University of Newcastle, Australia. It's
called "The Case Against C". Required reading.
(Not sure where you can get it. Try ftp.newcastle.edu.au for starters.)
> Do you know FORTH ? Lisp ? ML ? Haskell ? BETA ? SELF ? Perl ? Icon ?
> Have a try !
Just a point here on efficience. Would anyone here suggest that
most of these languages could ever beat C for efficient code?
WARNING: My opinions follow.
Lisp is not bad, mostly because of all the work that has been put
into compiling Lisp. However Common Lisp (the current standard)
is large, to put it mildly. (Just take a look at CLtL2 and count
the built-in functions if you don't believe me. It's on the web
at http://www.cs.cmu.edu:8001/Web/Groups/AI/html.)
Forth is not very expressive, and is almost as bad as programming
in PostScript. (Just my opinion.)
Haskell is lazy, which constitutes a significant run-time overhead
in itself. However Haskell has the advantage that the syntax is
extremely elegant, and also that it is purely declarative. It is
also large, but for different reasons than Common Lisp.
ML is faster than Haskell, but the syntax is not very nice at all.
The problem with Perl is that it has a user base that supports it
for the wrong reasons. They like it because you can do even better
hacks than you can in TeX.
I've never programmed in BETA, SELF or Icon so I can't really comment.
WARNING: Gratuitous plug follows.
At the University of Melbourne, we are working on a new language
called Mercury. It is a logic language (like Prolog) except that
it is strongly typed (like ML and Goedel), strongly moded and
strongly deterministic. There are no non-logical constructions
(unlike Prolog) such as the cut, and I/O is completely logical
too. Now here's the good bit: Compiled code runs extremely close
to the speed of handwritten C. This is despite the fact that we
haven't written an optimiser for it yet. Once we've done some
optimisations (and they won't be basic block optimisations either;
they'll be large-scale cross-module remove-orders-of-complexity
optimisations) we expect handwritten Mercury to overtake handwritten
C in speed.
Moreover, the speed of development is fast since the compiler
detects most bugs in your code. (This is a good argument for
strict languages.) Anyone who has programmed in ML or Haskell
will know what I mean.
The moral of the story? High-level, purely declarative,
theoretically superior languages need not sacrifice efficiency.
Don't think that C is the only choice just because it generates
fast code, because while it did generate fast code for the PDP11,
that doesn't mean that it will generate fast code for the machines
of the future.
Cheers,
Andrew Bromage
---
PS: I neglected to mention that Mercury compiles to C. :-(
However, I did mention that Mercury compiles to code nearly as
fast as _handwritten_ C.
From tanksley@san_marcos.csusm.edu Tue, 30 May 1995 18:30:43 -0400
Date: Tue, 30 May 1995 18:30:43 -0400
From: Billy Tanksley tanksley@san_marcos.csusm.edu
Subject: Writing an OS for PC
Andrew Bromage wrote:
_> Do you know FORTH ? Lisp ? ML ? Haskell ? BETA ? SELF ? Perl ? Icon ?
_> Have a try !
_Just a point here on efficience. Would anyone here suggest that
_most of these languages could ever beat C for efficient code?
Yes. It's absolutely certain, given a little weasel-wording. :)
C is one of the worst possible languages to write efficient algorithms in,
due to its long compile-times (and many theoretical disadvantages which we
all know or we wouldn't be here etc). Thus, although compiled C may be
faster than interpreted Forth (which it often, but not always, is),
for any two independantly developed programs, one in Forth and one in C, the
Forth one will outperform the C one.
I'm saying that although C may generate the best possible instruction
sequence for any given logical operation (no at all true), it causes
programmers to generate far poorer logical operations.
I've done a little work in Icon, but none in any of the others (although I
looked at Haskell). I won't comment on them. I like the small size of
Forth too much to spend long messing with the others.
-Billy
P.S. Porting Forth to Geos is NOT easy.
P.P.S. I'll ignore that crack about Forth not being expressive ;).
From Erwann.Corvellec@irisa.fr Wed, 31 May 1995 10:22:21 +0200
Date: Wed, 31 May 1995 10:22:21 +0200
From: Erwann Corvellec Erwann.Corvellec@irisa.fr
Subject: Writing an OS for PC
Kikoo Tunespepole !
This is my first participation as I am discovering the Tunes project...
Here's the problem:
On May 30, 8:34pm, Francois-Rene Rideau wrote:
> Subject: Re: Writing an OS for PC
> > I do agree that cooperation
> > can improve performance.
> > However, after dealing with Windows for so long, I am
> > really tired of "cooperation", where one task decides he's the only one
that
> > gets to make the CP wait for god knows how long.
> Because Windows sucks, and windows development tools suck, because they
> are C based. Sure, there is no way to achieve fair cooperation using existing
> tools. This is why TUNES will get rid of these.
Don't you how boring it is to reorganize your code just to do:
for i:=0 to 100
let_the_others_work();
end
This is even more true in OO applications !
Do you imagine yourself saying during the development of Tunes:
"Well, the files managing team have finished their work, so the video
managing team can start..."
NONSENSE !
Cooperation between objects is important of course...
But it must be cooperation, in terms of communication, between parallel
tasks supervised (Is it english ? ;-) by the teams' manager (the OS...) !
If you choose cooperation multitasking then Tunes is already dead !
Or proove me the way you are going to implement such a concept in Tunes
applications.
I mean in an efficient way ! That will not suck as you say ! ;-)
--
Erwann Corvellec.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Student & coder on PC (ASM, C, C++) & fond of LINUX V1.3 ;-D
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
From rideau@clipper.ens.fr Wed, 31 May 95 12:20:48 MET DST
Date: Wed, 31 May 95 12:20:48 MET DST
From: Francois-Rene Rideau rideau@clipper.ens.fr
Subject: Multithreading (was: Writing an OS for PC)
>>[talking about how cooperative multitasking is more efficient,
>> but requires more security than existing OSes and development tools
>> can provide]
> Don't you how boring it is to reorganize your code just to do:
> for i:=0 to 100
> let_the_others_work();
> end
> This is even more true in OO applications !
>
> Do you imagine yourself saying during the development of Tunes:
> "Well, the files managing team have finished their work, so the video
> managing team can start..."
>
> NONSENSE !
You seem to believe that because code must yield() execution,
it should explicitly schedule, which is false: programs just need yield(),
and with proper binding, this causes the system to schedule a new thread if
the current thread did time-out.
Actually, high-level programs using a high-level TUNES compiler need not
even explicitly yield() execution either: the compiler can very well do it
automatically; the programmer could still show the compiler how to optimize
things.
Basically, there must be a yield() often enough so that non-real-time
threads are run with sufficiently good resolution. That is, the (computer
or human) compiler must track how long each block takes to execute, and
be sure that for every long enough block, there is a yield() inside.
Loops are easy to manage: either the inside code is large enough, and will
have had a yield() inserted, or the loop will have to be split into smaller
loops each having a yield() at the end (which can be done at run-time or
compile time). The trickiest is recursivity, where you must insert a yield()
at every function call (but still, you could unroll the function's definition).
And you're not even required to yield if you know the execution will be
quicker than the required resolution...
Now the cost of a yield() is not very high: until the timer or some
event decides that a new thread must be scheduled, yield() calls a function
that returns immediately. On a i486 PC, this is some twelve cycles, not more,
and to have the same user-level resolution as a 100 Hz timer, you are
required to insert them only every 10000 or 100000 cycles...
Of course that may be more costly on a 16MHz 386sx computer, but still,
it's much more efficient than preemptive multitasking.
> Cooperation between objects is important of course...
> But it must be cooperation, in terms of communication, between parallel
> tasks supervised (Is it english ? ;-) by the teams' manager (the OS...) !
Uh ? "Cooperative" multitasking actually just means that programs abide
by some rule about yielding execution. Programs need not know what happens
when they yield, but that they will be given execution back some time later.
I don't think there is any direct relation between "cooperative" multit*ing
and the kind of cooperation you are just talking about (I may be confused,
however).
> If you choose cooperation multitasking then Tunes is already dead !
> Or proove me the way you are going to implement such a concept in Tunes
> applications.
> I mean in an efficient way ! That will not suck as you say ! ;-)
Well, I still have to see any argument why cooperative multithreading
is impossible, unefficient, or not desirable. Sure it is impossible
*using existing Oses and Language tools*. But as TUNES is redesigning an OS
from scratch, it can do it.
-- , , _ v ~ ^ --
-- Fare -- rideau@clipper.ens.fr -- Francois-Rene Rideau -- +)ang-Vu Ban --
-- ' / . --
Join the TUNES project for a computing system based on computing freedom !
TUNES is a Useful, Not Expedient System
WWW page at URL: "http://acacia.ens.fr:8080/home/rideau/Tunes/"
From rainer@physik3.gwdg.de Wed, 31 May 1995 17:46:05 +0200
Date: Wed, 31 May 1995 17:46:05 +0200
From: Rainer Blome rainer@physik3.gwdg.de
Subject: release 0.0.0.15
yes, i do read the pages (most of them), they're fine (which my thesis
isn't, that's why i'm silent), and i'd like patches if they require no more
than say two minutes of work (for each of us). bandwitdth is valuable, we
shouldn't use it if we don't need it. this applies to my reading
bandwidth, too: i just read the patch instead of hunt for the new leaves in
the tree.
rainer