Slow down!!

[reti@wilson.ai.mit.edu]

    Date: Sun, 27 Apr 1997 03:26 EDT
    From: Mike McDonald <mikemac@titian.engr.sgi.com>


      I think everyone is in too big a hurry here. This discussion only
    started, what, four days ago? Already people are talking about "binding
    votes" and resource commitments. I think we need to come up with WHAT
    we want to accomplish, WHY we want to accomplish that, HOW we think we
    can succeed at that long before we start voting. 

I completely agree, and couldn't have said it better.

						     As I see it, they are
    still three camps in this discussion, the "let's build a better VM"
    camp, the "I love Genera/Xerox1186 camp", and the "let's build a
    better web server in lisp" camp. Let's take then one at a time. (In no
    particular order.)

      First the VM camp, why do you want to invent another VM? 

To support uniquely Lisp features well (e.g. efficiently), viz. GC,
multiple-arg dispatch for CLOS, perhaps also to support additional
features (object persistence).  There was a lot of discussion, for
example, on why the Java virtual machine wasn't suitable for running
Lisp and what additions were necessary to make it so.  I dropped 
out of reading the discussions of enhancing the Java virtual machine,
but I gather the Java folks thought the additions were unnecessary
and/or negatively impacted the performance of Java.

As for the question of why have a VM at all (instead of compiling down
to native code), the answer is platform independence (or the ability
to run on multiple platforms).  Of course, the performance issue at
once raises its head, but you can have the VM specify an "escape to
native code" and a "return to VM code" operation so that you can
generate native code for performance-critical stuff.  This is what
enabled the PowerPC Macintosh stuff to gradually become more and more
native and less and less emulated.

							       How will it
    be better than the existing ones? How are you going to beat the
    existing ones? What advantage does your VM have that the other VMs
    don't have that will make it the choice of the commercial world? 

I don't think that this question is relevant; the commercial world chooses
applications or solutions to problems, not VMs.  If the Lisp environment
built on top of the VM provides either of those suitably better along
some dimension (see below), 								     
								     (Do
    you really think the JAVA camp is going to take anything you do
    seriously? You're trying to play their game and they already have the
    press, the venture capital, and the "mind share" of the industry.)

      Next, the LispM camp. What advantage will an integrated Lisp and OS
    bring to the commercial world? 

To answer that question, I have to describe first the area in which I
see a Lispm-like system as indispensable.

As I see it there are a lot of applications in the commercial world which
require fairly complicated sets of software, not complicated in the sense
of doing one fairly difficult thing, but complicated in the sense of having
to react to a complicated changing environment, or to complicated changing
goals (e.g. due to business or organizational changes in the company), or
to the need to integrate a set of existing, already complicated and poorly
understood legacy systems, or the need to handle huge amounts of data and
be available 24 hours 7 days a week, or all of the above at the same time
(the usual case).  

I believe that the Lispm environment enabled solving several of these types
of applications (worth millions a year to the companies involved) because
solving them in other environments was both too costly and too slow. All the
ones I personally worked on had had several previous failed attempts using
different platforms.

The lispm environment was ideal for addressing such applications for many
reasons, among them

  1) the ability to change the application while it was running (via the
     patch loading mechanism)
  2) the ability to ship objects around among machines (essentially using
     bin dump/load across the network)
  3) the ability to capture backtraces and object descriptions for errors
     that occurred (including argument and local names, and DESCRIBEs of
     the relevant objects) and package them up (via either email or
     disk files) for someone to look at later to fix the problem.
  4) the ability to trivially define network protocols tailored to the
     application at hand, as well as the ability to cheaply mimic either
     the client or server side of an existing application-specific
     protocol running on legacy hardware.
  5) the relatively large headroom in the system, i.e. large virtual address,
     no limits on numbers of objects, huge limits on sizes of objects, etc.
  6) the ability to tune the OS characteristics to the application, e.g.
     change the scheduler or I/O characteristics to suit the application
  7) the ability to inject application code at any level (e.g. into the
     middle of network packet handing)

The first 5 could be addressed in a Lisp environment layered on top of another
OS, but the latter two were also crucial to the success of at least two
of these systems I worked on.

Another consideration is in developing and debugging these systems; in the
Lispm environment there is no hard boundary between OS code and application
code. I frequently found application bugs by looking that the OS code,
viz by seeing what it expected and did with the incoming data.

Finally, supporting such code and training new people to support such code
could be a nightmare; however, the tools uniquely available in the Lispm
environment (trap on call, location monitoring, advice, backtrace gathering
code, etc.) all made it easy to figure out what was going on when you were
trying to understand a new pile of code.
				   
So, to answer the question (why would the commercial world choose this?),
because it could get more functionality for less money, or in some cases,
functionality at all (since the projects when tackled in less flexible
environments would be virtually undoable, no matter how much was spent).

				   How will your tightly integrated LispM
    environment interoperate with the existing disjointed world? 

Via the network.
								 What has
    change in the "real world" to make a LispM viable today? Or what are
    you going to do different this time to win? (Cheap hardware isn't the
    answer to your prayers. The other guys have that too.)

Provide more (and more reliable) functionality for fewer dollars spent.
And to be able to maintain and improve that functionality for reasonable
cost.

      Finally, the web server camp. What do you need a LispOS for inorder
    to make a better web server? What do the current OSes do that prevents
    you from building your better web server? How are you going to compete
    with Netscape, Microsoft, IBM, and every other Tom, Dick, and Harry
    that smell big bucks in the "Intranet" market?

      Personally, I'd like to have a PC LispM because of sentimental
    reasons and it fits my personal needs. But I don't for one second
    believe there's any chance of it being a commercial success. Of the
    three, I think the better web server has the most chance os commercial
    success. But I don't see why that can't be accomplished on top of the
    existing OSes. (The more, the merrier!) If it's the distributed OS
    portion, I'd suggest you look into the Beowolf project. Otherwise, use
    what the OSes have to offer when it meets your needs and then bypass
    it when it doesn't, like most commercial database companies do.

      Mike McDonald
      mikemac@engr.sgi.com

    http://cesdis.gsfc.nasa.gov/linux/beowulf/beowulf.html