Persistence: a proposal

[Scott L. Burson]

   From: Patrick Logan <patrickl@servio.gemstone.com>
   Date: Thu, 22 May 97 14:31 PDT

   >>>>> "Scott" == Scott L Burson <gyro@zeta-soft.com> writes:

       Scott> ...although small commits are relatively expensive in
       Scott> ZetaBase, it makes committing tens of megabytes of stuff at
       Scott> a time quite cheap, compared to many of the designs I have
       Scott> seen.

   Are you talking about tens of MB of delta or tens of MB of objects,
   where the actual delta is less than tens of MB?

The former (that is, it's the former case where I think ZetaBase will do
well).  The commit time is affected by a variety of factors, but it's not very
sensitive to the size of the delta.  This, I'm suggesting, is a blessing when
the delta is large and a curse when it's small.

   What is the expected transaction rate?

If you just wrote a loop incrementing a counter in an object and committing,
and you weren't doing anything else, you might get somewhere in the range of 1
to 5 TPS (this is a guesstimate; I have no actual measurements).

But that would really be an odd use of ZetaBase.  Commit times in more
complicated cases are affected by a variety of factors.

One thing I would point out is that most of the functionality of ZetaBase is
distributed.  There is a centralized server, but its functions are those of
coordination rather than data handling.  Processes running on different
machines, modifying different segments which are stored on different
filesystems, will very rarely slow each other down.  (They can be reading
segments in common without penalty as well.)

   Multiple concurrent sessions, i.e. transactions?

Locking is on a segment basis; multiple writers to a segment are not
supported.  It's conceivable that someday I could do merging like you do in
Gemstone, but there's nothing like that in there now.

   Gemstone has fast commits by using binary sequential transaction logs
   (optionally on raw partitions) and very well encoded representations
   of the deltas to be committed. The work of actually getting the bits
   back to the repository is a background task and happens potentially
   long after the transaction has been committed (except "full check
   points" will write everything back to the repository). Shared object
   caches and efficent per-session object tables make these changes
   available to multiple sessions without having to make round-trips to
   the repository all the time.

Sounds good.

You know, ZetaBase has had maybe 8 months of development while Gemstone has
had 15 years.  Not only is Gemstone clearly far more sophisticated, but in
fact ZetaBase hasn't even been used for anything ever.  The danger definitely
exists that it harbors some fatal design or performance flaw, although I've
tried to anticipate all of those I could think of.  What I'm saying is, in any
head-to-head comparison, ZetaBase isn't going to sound very good.

I know you're not necessarily speaking critically -- you're just trying to
find out what it is.  But I wanted to point this out.

Clearly, if you were to make Gemstone freely available for use in LispOS, we
should use it :-)

   I would be interested to find out if it could be tuned to your
   characteristics, if you could provide more information.

A lot of the information doesn't exist.  The thing hasn't been exercised more
than minimally.  Like I said, I ran out of money :-(

-- Scott