JOS variant "familiarization" version available (almost)

[John Morrison]

This is a multi-part message in MIME format.
--------------3CACEF4D84B8B0D3B86D7608
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi All,

I have just uploaded JJOS-98-10-09.tar.gz to ftp.spin.de:/incoming
(ftp.jos.org wouldn't let me write).  It is a big, 3MB download, and it
uncompresses to just over 10MB.  I am assuming Markus will put it on the
ftp.jos.org FTP server so everybody can get it (ftp.spin.de, conversely,
won't let me read).

There is a doc/README.html file which pretty much explains everything
(and I will attach it), but a few words first:

(1) This is a "familiarization" release -- it doesn't do a whole lot,
although it does enough that someone could start writing and/or
integrating a JVM with it.  It is intended to elicit feedback.

(2) There are two build targets: a self-hosted Linux "emulation"
version, and a cross-development (again, x86-Linux-hosted) version
targeted at a bare, i386+ PC-compatible.  (Actually there's already an
erratum in the README file, now that I look it over -- the nativetarget
can do everything the nativehost can do.)

(3) In order to prevent my at-work inbox not from getting swamped,
please DO NOT REPLY TO/CC ME, but instead send replies to the usual
jos-kernel mailing list (of which I am a member).  This will also enable
other mailing list members to answer questions about embedded
development, too (I don't have time to do the day-job, work on JJOS, and
answer 50 emails a day).

(4) I expect to field lots of complaints about: the current license or
lack thereof -- I have been too busy to figure out which NPL/PSI/BSD
license to use yet, so I've put a ruthlessly brief copyright in the
source files ("FASCIST!  Running-Dog Capitalist Pig!  Code-NAZI!"); the
lack of functionality ("That's IT?  That's ALL?  Where's the REST of
it?"); the decision to use C++ (I've already gotten a good deal of
that); the decision to write code that does not conform to the
in-process "jos-kernel" interface standard (I keep saying "it's NOT A
KERNEL!"); the decision to use a different JVM than the Politically
Correct ones (gotten some of that, too); etc.  You get the idea.

There are enough decisions here such that it is virtually impossible to
make each and every one of you personally happy with everything I have
done and am doing.  Please recognize that fact, and please make some
effort to spare me from gratuitous flamage, and avoid giving truth to
the old saying "No good deed goes unpunished."  If what I'm doing
doesn't work for you, please just silently ignore it and hope it just
goes away.  If, on the other hand, you're keen to help, please reply to
the jos-kernel list.  Believe me when I tell you there's no shortage of
things to do.

(Hey, did we make it under the 1-year anniversary wire?)

Very Truly Yours,

-jm

-- 
==== John Morrison ==== jm@mak.com == http://www.mak.com/welcome.html
==== MaK Technologies Inc., 185 Alewife Brook Parkway, Cambridge, MA
02138
==== vox:617-876-8085 x115
==== fax:617-876-9208
--------------3CACEF4D84B8B0D3B86D7608
Content-Type: text/html; charset=us-ascii; name="README.html"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="README.html"
Content-Base: "file:///D|/JJOS/README.html"







JJOS


VERSION 0.00


last updated 17 September, 1998 by John
Morrison


Copyright, 1998 by John Morrison,
All rights reserved.


THIS JOS VARIANT'S GOAL

We are trying to build a portable JOS
(hereafter referred to as "JJOS" so as not to be confused with other JOS
development efforts).  JJOS' overarching design goal is to do as much
as possible of this in Java, and as little as possible in native code. 
The first target hardware platform is an i386+ PC-compatible.
OVERVIEW OF DESIRED OPERATION

We anticipate that JJOS will boot thus:
a boot loader will load: a native-code runtime binary (hereafter referred
to as "NCR"); a ZIP file of Java bytecode OS code; and a command-line argument
string  -- all off of a local device (hard disk, CD-ROM, or network
card)

the boot loader will jump to the NCR entry point

the NCR, which contains the JVM, will do as little as possible to prepare
the hardware to execute the JVM

the NCR will invoke the JVM on the Java entry point indicated by the command-line
arguments

the Java code will finish probing and initializing the hardware, and finish
booting JJOS

We anticipate that the paging daemon thread, device driver threads, and
as much of the GC as we can manage, will all be written in Java.  
Virtual memory page faults will result in suspension of the faulting thread,
and invocation of the paging thread, which will page in the required memory
and re-invoke the faulting thread.
SYSTEM REQUIREMENTS

Currently, we can build the JJOS NCR two ways:
 for UNIX in an "emulation" mode, so that one can develop most things
(with the exception of hardware-specific drivers) rapidly and in the comfort
of UNIX with the aid of a debugger

for the target i386+ PC-compatible system.

For developing the JJOS NCR for the host (hereafter called "nativehost"),
only the UNIX-based host is required.

Development of the JJOS NCR for the x86 PC-compatible (hereafter called
"target") is not self-hosted (yet).    For the bare
i386+ PC JJOS NCR (hereafter called "nativetarget"), one must use 2 PCs,
each connected to a common Ethernet.  The development host PC upon
which JJOS binaries are built (hereafter called "host") must be a Linux,
x86 PC.  The target must be an i386+ PC.

I am using a P5 Gateway for the host, and a P5 Baby AT for the target.
REQUIRED TOOLCHAIN FOR NATIVETARGET

As of this writing, the following tools are required on the host in order
to build nativetarget:
Linux -- I am using Red Hat 5.1, Linux kernel 2.0.34

g++ (really egcs, I guess) -- I am using version "egcs-2.90.27 980315"

The NASM assembler -- the distribution
uses version 0.97

The Etherboot network boot
package -- the distribution uses version 4.0


INSTALLATION


How the Sources Are Organized:

There's no fancy autoconfiguration process (is this a bug or a feature?). 
Just unzip/untar the sources.  You will see several directories in
the root directory.
bytecodecommon -- bytecode common to host and target

bytecodehost -- common bytecode + host-specific bytecode

bytecodetarget -- common bytecode + target-specific bytecode

doc -- documentation (such as it is)

nativecommon -- C++ sources common to both nativehost and nativetarget

nativehost -- C++ code specific to nativehost

nativetarget -- C++ and assembly code specific to the target

tools -- any tools required to build some part of JJOS (The developers
of both NASM and Etherboot have graciously consented to redistribution
of their respective packages along with JJOS subject to their original
licenses.)


Other Nativetarget-Specific Tools:

In order to build the nativehost, you do not need any special tools. 
However, to build the nativetarget, you will.  All such tools (currently
NASM and Etherboot) should be found in the tools directory.

Please read the NASM and Etherboot manuals for installation.  In
short, first, you'll have to build NASM and Etherboot (including our customizations).  
Second, you'll have to convince the host to service BOOTP and TFTP requests
from the target (read the Etherboot docs).  Third, you'll have written
a bootable (by the target) Etherboot floppy, selecting the appropriate
Ethernet card driver for the target.
 
BUILD PROCESS


For nativehost:

Just type "make" in the nativehost directory.  The makefile will also
run the emulation.  (Ain't Life grand?)
 
For nativetarget:

This is unavoidably more complicated.  First, you've got to build
and prepare the NCR/ramdisk image for downloading each time you change
the NCR sources.
Type "make" in the nativetarget directory.

The makefile will build an ELF image of the NCR.

The makefile will then "objcopy" the ELF format NCR into a flat, "binary"
image suitable for downloading.

It will then call a customized mknbi (MaKe NetBootable Image) to stitch
together a header block, the NCR image, the command-line arguments, and
the ramdisk file.

It will copy the resulting netbootable image to the TFTP directory.

Assuming the bootable Etherboot floppy is sitting in the target's floppy
drive, just hit the reset button.
The target machine will boot off the Etherboot floppy, and then broadcast
a BOOTP request ("who am I? what should I boot?") which will be caught
by the host (the first time I do this, it seems to take several tries to
get the host to respond -- I'm not sure why).

The host will respond, giving the target its IP address and the name of
the netbootable image we just made and copied to the TFTP directory.

The target, in reply, will TFTP the netbootable image into RAM and jump
to it.

It takes about 30 seconds end-to-end, from typing "make" on the host to
execution start on the target.  (In fact, I often hit the reset button
on the target before I make, to parallelize the process.)  This is
WAY, WAY faster than the traditional, reboot-intensive, single-hosted kernel
development process, and so it's well worth the added hassle.  
It's also WAY WAY lower risk (you're not likely to trash your development
host).
HOW THE SOURCES ARE ORGANIZED

Wherever possible, we used an object-oriented approach.  nativecommon
sources, for the most part, are C++ base classes which are subclassed for
either host or target.  We figure this is an extensible approach for
portability.  Currently, virtual functions are used, but this is a
bad idea for performance reasons.  So, as soon as performance becomes
a bigger problem than functionality (or whenever we get enough volunteers),
this will be revisited (e.g., using nativecommon sources as templates instead).
 
LIMITATIONS ON C++ NCR SOURCE CODE

There are currently quite a few limitations to be observed when writing
NCR C++ code:
C++ exception handling is not supported (I have no plans to remove this
limitation).

No C++ global or static objects which have constructors can be used --
the constructors will fail to be called prior to jumping to the NCR's entry
point (I have no plans to remove this limitation).

The current NCR is not even anywhere nearly close to POSIX-compliant (I'll
do what I have to do and no more).

As of this writing, templates have not been tested (I will test this ASAP).


STATUS


Nativetarget

As of this writing, the nativetarget can:
boot via Etherboot

go to flat 32-bit protected mode

printf to the console

figure out how much extended memory there is

use C++ virtual functions

(after a certain point when the heap is initialized) new C++ memory objects

field interrupts and vector to C++ static methods of the base machine class

"open" up read-only stream objects to files contained in the uncompressed
ramdisk zip file


Nativehost

The nativehost emulation can do everything the nativetarget can do, 
plus:
find its command-line arguments and pass them to the JVM stub.


INCREMENTAL DEVELOPMENT PLAN

We are attempting to achieve a self-hosted development system as soon as
possible.   We are using a rapid-prototyping, breadth-first approach
to do this.  Consequently, we are being ruthlessly minimalist about
what gets written.  Capabilities that are not absolutely essential
(e.g. virtual memory, JNI, JIT, "persistent" object systems) will not be
implemented until they become unavoidable.  Inferior but expedient
approaches will be used wherever possible (e.g., virtual functions on base
C++ NCR classes, reference-counting GC instead of copying GC).

As of this writing, here is the incremental medium-to-long-term development
plan in rough chronological order:
non-virtual memory, reference counted GC, running simple "hello world"

adoption/incorporation of balance of GNU classpath

implementation of VGA JJOS (requires AWT)

boot off hard disk (requires non-Etherboot loader, disk driver, file system)

self-hosted development (requires Java-language Java compiler)

think about compliance with any evolving specifications

more drivers for more hardware

mark-compact GC (for avoiding disk I/O for performance and battery life)

virtual memory

better GC (e.g., copying generational GC)

perhaps, native threading

perhaps, a Lisp Machine like "save world" / "boot world" capability


SHORT-TERM "TODO" LIST


put in less paranoid, less fascist copyright

find the command-line arguments for the nativetarget

release the NCR

get makefiles and build process for ramdisk zip files in bytecode* dirs

incorporate _Quinn's JVM

run his "hello world" program


PRIMARY AUTHORS/CREDITS

If you would like to assist the following developers, please see the JOS
homepage.
_Quinn (is writing a C++ JVM)

 Matt Tiswell (has volunteered
to write Java device detection and drivers)

John Morrison  (is writing the C++
NCR, has assembled the tool chain, and has written this document)

 


--------------3CACEF4D84B8B0D3B86D7608--