let's talk

[Emmanuel Marty]

Hi Sam :)

> > Do you mind if we use tunes-lll in the meantime?
> 
> If you're going to do this, how does one sign up?  I too would like to be on
> the XCOM 98 Consortium.  :-)

Just go to http://www.tunes.org/collaboration.html#Mail and subscribe
to tunes-lll (which stands for low level language, just FYI). There might
be some discussion you don't care about there, but that's where xcom
is currently discussed until maybe David Manifold creates a new list..

And you are more than welcome to be part of the "XCOM 98 Consortium" ;),
as you've brought lots of effort into the format.

> BTW: My name is Samuel Anthony Falvo II, so the "L" for a middle name was
> incorrect.  But that's just a cosmetic "bug"; the functionality is still
> there.  In case you forget, just remember I'm one of those annoying,
> migrating Italians.  ;-D

Oops. My apologies; I really thought I saw your name spelled this
way somewhere. Recorded :)

> Also, is Andreas Beck going to be joining us in this venture?  I saw he had
> a few words of input to the XCOM Specificationt too.  Not sure if he'll be
> interested or not, seeing as how he's involved with GGI and all.

Andreas gave his valuable opinion on several issues even though he didn't
directly work on the format; if we spawn a new list I'm sure he'll
subscribe to it and criticize us :)

> > Tom, Sam: I want to change two things in XCOM. 1) the architecture
> > description string is useless. It's not used by anything important
> > right now, so I'd like to discuss an useful way. 2) the
> 
> Thank you for bringing this up so soon -- XLink is basically a loader right
> now (See?  I *AM* making progress!), due to lack of time.  I should have it
> reading in fixup records and coalescing sections by the end of next week (I
> hope) or the week after.  I will definately need to know how to handle
> INTERFACE sections by that time.  If not, that's OK -- I can always go back
> and change it (and pulling my hair out in the process!  :-) ).

I will try to work it out this weekend. If not I will make the changes
to the code if you want to keep your hair..

> I plan on extracting the first two passes of the linker for the purposes of
> writing a 'XRun' command, which allows one to load and run an XCOM
> executable (usage is similar to su under Linux; "xrun ls.xcom" for
> instance).
> 
> *WARNING*
> 
> DO NOT run xlink when trying to link 16MB XCOM components.  More likely than
> not, you will run out of memory!  :-)  xlink works by loading all program
> sections into RAM, then loading relocation records, coalescing them in RAM
> (no realloc() or memory copying, so it's fast), and then writing the final
> result from the coalesced sections.

That's great! This is much needed. Loading, fixup and running is how I
envisioned it anyway, so this is ok. It is only for the purpose of testing
components in a friendly development system.

> > interface section, I would like to find a way to make it so that
> > the linker can do fixup inside it, and still not have to know
> > the format used by the OS it is destined to. That should
> > be relatively simple, as only Clementine has interfaces right
> > now, and I can break them as much as I like.
> 
> Why not just use relocation records?  Does an XCOM_SECTYPE_INTERFACE section
> need symbol exports?  I must be missing something important about INTERFACE
> sections.  :-)

Not symbol exports but it makes references to the string table for 
compactness (it could have its own table, but since some strings like
symbol mappings are common that would be space consuming for no reason);
these references need to be translated at link time, when a new table
is made out several objects. 

I'm thinking we should devise an operating environment-independent way
for the linker to fix those up, rather than individual support for
every possible interface.

> > XCOM doesn't support position-independent code right now but
> > it will sometime soon.
> 
> Could have fooled me.  The fixups alone make code position independent upon
> loading.  As far as run-time relocation (ie., having floating chunks of code
> during run-time), I personally don't see the utility in that.  We might want
> to consider making some portions of XCOM "Optional Extensions," just as the
> TCP and IP protocols have optional extensions.

I mean position-independent code the way .so and dll files mean it, and
what motorola means by pc-relative code. The interest is that the same
page of code can be executed in different processes at different virtual
addresses, accessing process-specific data (ie. global variables of
the PIC code are actually different for every process, so that it doesn't
need explicit reentrancy support by only using local vars).

This is indeed the only case where it is useful. I'm not too much
for extensions, rather for designing it in a way that isn't a bother
for the runtime loader (it's fine to transfer complexity at
compilation time). I will not let xcom loaders become complicated,
as simplicity is one of the main goals.

> > * SGML'ing the documentation
> 
> I'd like to add "Better Documentation" into this as well.  The existing
> documentation does not yet address the issues which I've raised with you in
> the past, with the exception of the formal meaning of xc_int64 datatype, nor
> the issues which will be discussed in the future on the XCOM development
> list. Perhaps we should wait on this...
> 
> We should also re-evaluate the include file xcom.h too -- it seems to have
> Clementine-specific definitions present, if only just a few.  :-)

That's true. If we redesign the INTERFACE sections accordingly, all
the clementine-specific code and definitions will disappear. For keeping
the doc in sync this is also true and my fault.. I'm afraid I don't
remember all issues. If you want to make a little summary.. :)

> > * Writing an XCOM loader as an usermode program for POSIX OS'es.
> 
> This is what lead to the development of XLink, actually,  I wanted to try
> and make a loader, but I realized that I needed a linker first.  And since
> they both share a good portion fo the code, I figured working on the linker
> first would assist in the writing of the loader.
> 
> If someone else wants to take up the task of the loader, that's fine by me
> -- less code I have to write!  :-)

Well you seem to have well progressed with that so it'd be silly to
dump it for other work now :) Maybe someone will care about maintaining
it when you want to move to other tasks though ?

> > * Writing an XCOM loader for the Linux kernel, as a 'misc
> >   loader' module? That shouldn't be too hard, and should
> >   nicely insert as a module with no need to patch the
> >   base kernel.. This would also probably require designing a
> >   small 'INTERFACE' section for it.
> 
> An interesting idea.  I wasn't aware such an interface even existed in Linux.

Yeah, there is.. You can load an executable and spawn any interpreter
you like; there is a small loader for java applets that spawns appletviewer
as the interpreter for example.

For a very simple test we could make one that calls XRun for instance.. I
was more thinking of linking actual linux programs as xcom and having
a loader for them in the kernel though, so that we can validate the
format some more.

--
Emmanuel