A revolutionary OS/Programming Idea

[Alaric B Snell]

John Newman wrote:

> "i suggest some serious reading on the tunes.org,
> especially the wiki...
> you will be either torn apart or ignored on this list
> with ideas like
> this."

Well, do you feel torn apart, ignored, both, or neither? ;-)

> I wasn't meaning to say that a novice could sit down
> and suddenly make an OS.  Just make it so that while
> navigating their program files they have the option of
> navigating deeper into the source code, just as one
> would in a normal file system. 

Yes, that's as much a statement about the open nature of a system that 
does everything in terms of 'source code' with on the fly compilation - 
they're *beautifully* tinkerable ;-)

On a similar vein, I'd like to be able, when viewing the GUI an 
application has presented, to do something like "shift + right click" on 
a button to get a menu of programming actions - such as opening up the 
source code for the action handler for the button and so on.

 > If the primitives were
> easy enough to learn in an our or two (with no syntax
> crap to get hung up on), then the data-structures
> would slowly be learned through daily computer use. 

Hmmm... maybe; but that would depend on people bothering to look inside 
their apps and explore, which many wouldn't. Then again, those that DO 
are the kinds of people who will want to learn anyway ;-)

> With a few small tutorials one could easily learn to
> integrate different aspects of applications
> together--or create applications by mixing them.  
> 
> For instance, make a file; in it, create a file that
> checks the weather forecast (maybe "cin >>
> http:www.weather.com); in it, also create an 'if'
> file; in the 'if' file, make a file that says "rain" &
> "within 24 hours from now" (however your language
> would express that); then, in the 'if' file, open the
> 'then' file (which was already created by the if); in
> the 'then' file, drag-n-drop your email client in and
> make a 'send' file that says "bring your umbrella" to
> all the people in the 'to' file in your 'send' file.

Yep. That level of programming is what some would call 'scripting' - 
just using basic flow control to architect interactions between other 
software components; lots of UNIX people do this, perhaps without 
labelling themselves as 'programmers' or 'real programmers', but the 
syntax for shell scripts is a bit wierd to say the least, which must be 
an obstacle to many.

> 
> Later, Aleric, you say:
> 
> "3) Like the code browser, except that even function
> bodies are 
> represented diagrammatically, with a GUI editor;
> however, this gets 
> tricky. I can type "if (foo) {x++;}" faster than I can
> right click, 
> select "IF", then go to the condition bit, right
> click, select 
> "variable 
> reference", type "foo" (or drag 'foo' from elsewhere
> onscreen), go to 
> the other bit of the if, right click, select
> "INCREMENT", right click 
> again, select "variable reference", and type or select
> 'x'."
> 
> This is true, but if I am a novice, that doesn't
> understand "if (foo) {x++;}" then I may build the
> functions out of the primitives I already understand. 
> If I understood the primitives, then I could build my
> own incrimenter, or foo, simply by drag-n-drop.  I
> could even build the 'if' function myself an use that
> file for that purpose, if I was oblivious to the
> standard library.

That indeed holds for a novice, but experimentation with structured 
editors (see below) led the non-novices to get annoyed with them and 
shift back to editing strings.

Now, it could well be that this was not a fair test - maybe nobody tried 
teaching somebody to program from scratch in a structured editor, then 
got them to attempt to use a textual syntax to see how they got on. I 
don't know. Anyone got a reference?

> Plus, if we are building with more than one liners,
> with as little as even, say, a couple hundred lines of
> code, with a few extra one liners, then the
> drag-n-drop file system becomes a heck of a lot faster
> and easier, since syntax in textual coding would scare
> me away from cutting and pasting things where I don't
> know if they will fit/compile or not.  It emphesizes
> the reuse of data-structures.

There's a kind of editor called a 'strucural editor' or something like 
that, which (as you describe) works directly upon an abstract syntax 
tree rather than a string of text, although they usually try to make it 
LOOK like a string of text. You might see:

    IF   foo
    THEN bar
    ELSE IF   baz
         THEN wibble
         ELSE wobble

...but if you mouse over the keywords (which I've drawn in caps, but 
would probably be a different colour as well / instead) then the whole 
IF/THEN/ELSE and subtrees are highlighted, and you can cut+paste the 
construct as a whole. Yet put the mosue over 'foo' and just the 'foo' is 
highlighted, and you can delete it to leave a bright red symbol that 
means 'SOMETHING OUGHT TO BE HERE AND YOUR PROGRAM WILL NOT RUN UNTIL 
YOU FIX IT!', and so on, pretty much as you describe...

*google*

Like this:

http://www.nuprl.org/manual.with.index/node39.html#3732

> Thanks!  The whole reason I am posting here is because
> I like what TUNES is all about.  And I find it to be a
> noble cause.  Once I am a professional programmer (an
> inexolerable end-point that is already in motion), I
> plan to contribute extensively to an open source OS. 
> It is one of the most noble open source projects I can
> think of.  One of the things that I think TUNES prides
> itself in is empowering the user with the full power
> of their computer.  Making the IDE and the OS one
> system is a step in that direction.  It will also make
> the OS itself more evolvable, comporting with this
> 'reflexive' thing I keep hearing so much about.  I
> know, this idea wont make a programmer out of a novice
> over night, but it brings down various barriers.

I agree.

Reflection is pretty easy at heart... Imagine that, in your case of 
treating source code like a filesystem, what can be done if a program 
reaches into its own source and reads and writes it.

Apart from science fiction about software attaining sentience on its own 
and taking over the world (!), this is useful because you can avoid 
laborious code writing by writing a program to generate the code for you.

And if you're a mathematician, imagine a 'differentiate' function that 
accepts a numeric function and returns a new numeric function that's the 
differential of the input function, right? In most languages you'd have 
to write this using a numerical approximation: call the input function 
at x, x - 0.000001, and x + 0.000001, and feed the three results into 
the magic formula to get an approximate result.

But in a fully reflective system, you can write your differentiation 
function to ask to see the source of the function it has been passed, 
and if you're allowed it and the source does indeed look like a pure 
numerical function, proceed to use algebra to find the exact 
differential, and then return a function implementing that. Of course, 
the system may refuse you the source code - either due to security 
restrictions or because the function you are investigating is actually 
implemented in raw assembly or something because it's a primitive, or 
the function may perform mutating operations within itself and be too 
difficult to differentiate by following rules, in which case you can 
still fall back to the numerical approximation.

Which is really neat, since a properly differentiated function can be 
evaluated faster than the numberical approximation in most (all?) cases.

> I mean, think about it.  The open source collaborative
> community would grow exponentially!  Maybe, instead of
> just laboriously building software, we should build
> tools that make building software less laborious and
> less complicated to learn.  In the end, there would be
> a lot less work to do for the founding programmers. 
> Create an environment that builds itself.

Perhaps the barriers to learning programming can be split into three groups:

1) The language itself
2) The mental tools behind it all (abstraction, debugging skills, logic, 
data structures, etc...)
3) Other concepts you have to deal with (POSIX, TCP/IP, file formats, 
APIs, and so on)

You're attacking (1), but I don't think (1) is as much a hurdle as the 
other two, although as you rightly point out it's the *first* hurdle you 
have to cross.

(2) really comes down to good education, or having access to some good 
books on it. The things in (2) are already designed to be as simple as 
they can, because the academic types who come up with them want to make 
them as neat as possible. Books like "Structure and Interpretation of 
Computer Programs" are good. Have you read this book, John? If not, READ 
IT NOW! It's lovely!

(3) is something that I want to attack. There's lots of stuff you have 
to interface to when writing nontrivial programs, and each of those 
things was written by somebody different, in a different situation, with 
different goals, and so on. They're messy. Most of them weren't actually 
designed for what they're being used for, so you have to learn tricks to 
bend them to your will.

Take TCP - it was designed to work just like a UNIX pipe or a serial 
link or something. It actually does a lot of work for you at each end to 
convert a packet switching network into an illusion of a serial cable; 
it uses a lot of buffer space to do this, sometimes, and it doesn't make 
the most efficient use of available bandwidth.

And then people use TCP as the basis for protocols like HTTP and SMTP 
and so on that take this 'serial cable' abstraction and... turn it into 
a message-passing system. You send a request down the connection and 
wait for a response. You use some technique to mark the boundaries of 
requests and responses, since TCP has gone to all that effort hiding the 
boundaries of the network packets from you, etc.

Yet if you decide to ditch TCP and write it with UDP, which gives direct 
access to the underlying network, then you have to deal with your 
packets being randomly lost in the network, arriving in random orders, 
being duplicated in the network, being slightly modified, being rejected 
for being too large for some intermediate node... not to mention that 
you need a sensible rate limiting algorithm, too.

So you either have to do a lot of work yourself, or you have to use 
something that does too much work for you and do a little bit of extra 
work to undo some of the work being done for you. Neither of them are 
very efficient ways of going about it.

The obvious answer is to design a better communications protocol for 
this kind of application, that sits on top of UDP - such as ONC RPC. 
Which has a whole new API, and hides some of the networking issues that 
you'd actually like to have a bit more control over, etc.

Also, when communicating with other bits of software, you almost always 
have to get down to the level of dealing with byte strings. After all, 
files and memory objects and TCP connections and UDP datagrams are all 
defined as strings of bytes. There are a few offerings for higher level 
data representation around - most languages have a language-specific 
serialisation system for their own object model, and there are things 
like the CORBA mechanisms, ONC XDR, ASN.1, XML, and so on. But they're 
not used enough - few things you need to interface with will use them.

So there's a sprawl of concepts to learn in order to interface with the 
world, too...

I'm trying to think of simple, flexible, data formats and APIs and 
whatnot that might truly deserve the title 'universal'. It's not hard, 
maybe not possible, but is an interesting challenge.

> Do you think object oriented logic programming is
> implementable under my paradigm of software development?

Sure, I think so.

ABS