files, printers, etc. [Re: The feel of a LispM/List of running machines]

[Christopher Stacy]

1. FILES AND NETWORKING

The Lisp Machine has an object-oriented "generic network system" that allows
it to operate well in a heterogenous networking environment (many different
kinds of hosts and networking protocols).

For example, let's talk about transparent file access.  
You can OPEN a file on any machine on the network, and it just works.
It does not matter what operating system the file was stored on, nor did you
have to know or worry about which network file protocols the target machine
supported.  You just use the syntax of the target file system.

This also directly benifits the application users, since they can refer
 to files in whatever normal manner that they already know.  
The file server host name is specified at the beginning of the 
file name, like these examples:

  "AI:STAN.K;LINES >"
  "<PDL.SRC>ZORK.MID.3"
  "Secrets:>udd>ManOfMystery>Deep>Dark>crpyto>nuke>plans"
  "Quabbin.SCRC.Symbolics.COM:>CStacy>hacks>demo.lisp.newest"
  "BARNEY:\FROGS\EYES.TXT"
  "prep:/usr/local/lib/etc/etc/etc/biteme.arf"

Here's a simplified explanation of how it works inside.  

When you call OPEN on a filename, the system begins by parsing the file name
into a PATHNAME object.  Different kinds of hosts had different classes of
pathname objects, with assorted various behaviours mixed in. 

If the programmer needed to manipulate the pathname, for example merging,
pathname functions can be used to take the pieces apart and put them back
together.  The pathname system "understands", provides robust support for,
and tries to do the right thing with all the different kinds of incompatible
syntaxes that the different server hosts use: case sensitivity, file types,
version numbers, devices, directories, component seperators, defaults etc.

(If you wanted to do a hairy directory operation involving wildcards, say,
copying only the latest versions of some Lisp source files from one directory
into part of some other directory.  Maybe one file server machine is UNIX
which does not have file types and versions, while the other machine is a
TOPS-20 or VMS machine that has those things.  Genera will try to figure 
it all out for you, even to the point of emulating the underlying
unimplemented host file directory operations.)

When the pathname was parsed, it looked up the named host in the host 
object database.  It needed to do this to figure out the class of the host,
and consequently what kind of pathname object to create.

Continuing along, OPEN asks for a particular abstract network service: 
file access ("FILE") to be invoked on the host object.  The system calls
some methods on the host object to figure out what networks the host is
participating in (Internet, DECnet, CHAOSnet, etc.), and what kind of
file access protocols it supports on what networks.  For example, for
an Internet host, the methods could use Domain Name Service to get
the WKS (well-known services) resource record bitvector of TCP services.

Now it knows something like: this is a host on the Internet at 128.86.0.099,
and it supports FTP over TCP.  There might be more than one way to get
file access to the host: maybe it also supports TFTP or NFS, or maybe it
also has an address on some other kind of network like DECNET.

The system matches up what protocols the host supports with the set of
protocols enabled on the Lisp Machine, and picks the best one according 
to some configurable preferences.  This results in an invoked-network-service
IO control object, which in turn invokes the appropriate network stack,
and an appropriate file stream with your bits on the other end is handed
back up to OPEN.

Besides the familiar OPEN function, some other kinds of abstract services
that a programmer can call for include TIME, and REMOTE-TERMINAL (telnet),
and SHOW-USERS (finger).  In each case, the network and protocol are
determined dynamically and automatically, and the programmer normally
just writes the application using generic sorts of function calls.

Everything about all of this is object oriented and extensible.
It's very easy to implement new network protocols and support 
for new kinds of hosts and file systems.

In addition to the usual TCP (and RPC) based file access protocols, 
the Lisp Machine also has its own proprietary protocol called NFILE.
Think of it as sort of like NFS, except there's no mounting, configuring,
pretending that file names all look like UNIX, getting hung up, locking
problems, or anything else.  It just works perfectly, all the time.


2. DATA STORAGE
   
The Lisp Machine has three basic ways of storing data.

First, there are its own native file systems.  The one intended for use by
applications wanting a conventional file system is called LMFS.  Is had long
case-preserving file names with seperate type and version (generation)
fields, also arbitrary user-definable property lists on files, hierarchical
directory structure, ACLs, guaranteed un-delete, and various date and flag
fields (dont-delete, backed-up, etc.), and wildcard operations.  
It includes a nice backup/restore system, and a de-fragmenter.
It's an ultra-reliable place to keep your bits, 
and can cope with broken disk hardware.

The FEP (front-end processor) file system is a more low-level file system, 
used to contain things such as LMFS partitions, swapping partitions, 
certain front-end bootstraps and debugger overlay modules, and saved bootable
Lisp world images.  FEPFS also has hierarchical file names and some stuff.
If you don't have a LMFS on your machine, but the network is hosed or your
only file server is down, in a pinch you might save your files on the FEPFS.

Second, there is a distributed, object-oriented database called the Namespace.
This is mostly an used for managing system and program configuration parameters.
It's not really very flexible, it's not fast, it's hairy to use, and had bugs.
Think of it as a little bit like DNS.   Mostly not used, except by the system.

Last, there is Statice, which is a persistant Lisp object system with good
performance, intended to be used by both the system and applications.  
One could imagine a development environment based not mostly around project
source files, but rather around objects stored in Statice.  
However, Statice was a late addition to Genera, and this sort of thing was
never done -- programs were just hacked as text using ZMACS, 
the Lisp Machine's fancy version of Emacs.

3. PRINTERS

Printers were represented by objects in the Namespace, having names and
configuration properties such as their spooler location and serial port
number and so on.  A user accesses a printer by just typing its name,
like on any kind of system.  For example, in a command interface the
programmer would write (CLIM:ACCEPT 'NET:PRINTER), which would activate a
completing reader or menu-list of printer names to select from.
This would return a printer object, upon which the obvious 
operations could be invoked.

4. Even though it far surpasses the capabilities of today's operating
   systems, this is all ancient history.  It shouldn't be hard to think
   about models beyond this stuff.  (Well, at least I can say that
   it wasn't hard to think beyond this 15 or more years ago; nowadays most
   people may have been taught to only consider more limited horizons.
   They definitely have far lower expectations of their operating software.)

Chris