[gclist] The old container problem

[Nick Barnes]

Digging through my old mail....

At 1998-09-22 15:55:00 UT, David Chase writes:
> >At 1998-09-21 16:26:51 UT, David Gadbois writes:
> >> 2. Allocate the containers in a special "pre-natal" area that is
> >>    younger than the youngest normal generation.  Collect this area by
> >>    doing a straight non-generational GC at fairly infrequent
> >>    intervals.  The downsides to this approach are that it requires
> >>    what amounts to an extra, special-purpose collector and that it
> >>    complicates already fragile storage system invariants.
> 
> At 01:32 PM 9/22/98 +0100, Nick Barnes wrote:
> >Something like this is a good technique if your collector is
> >sufficiently flexible to make it possible.  You shouldn't have to have
> >an extra collector; in particular note that collection policy (when
> >and how often a given part of the heap is collected), promotion policy
> >(when and whether parts of the heap are promoted to older
> >generations), and maintaining graph invariants are theoretically
> >independent of each other, and a good GC will offer the client total
> >control over the first two, while having a number of approaches to the
> >last.
> 
> >This isn't rocket science; I suggest that all GC implementors consider
> >how best to achieve this sort of flexibility.  Hard-wiring collection
> >and promotion policies is as unnecessary and inflexible as hard-wiring
> >the number of generations or the maximum heap size.
> 
> My knee-jerk reaction to this is "oh dear, more knobs".

My answer to this is that by proper modularity one can (nay, should,
nay, must) turn N^M into NM.  Make each policy module guarantee the
appropriate well-defined invariants for the memory under its
management.  These can be tested independently of any other policy
modules, and if they all work independently then they will all work
together.  If you check them at runtime then you can tell exactly when
and where you went wrong before it blows up in your face.

Of course, this can not work if implemented in a huge hack of C or C++
with one big namespace, a zillion global variables, no consistency
checking, and no careful design.  Given that most garbage collectors
fit this description, it is hardly surprising that they are very hard
to modify in such a way.

> I've also observed that, given knobs, the best you can
> hope for is that users will ignore them, because if
> they don't, odds are good that they'll be set incorrectly.

Only add knobs which you know will be used, but always write software
in such a way that you can add knobs.  This is the essence of
requirements-based development.

> What, by-the-way, are the performance gains obtained by
> this sort of tailored garbage collection?

In certain applications, factors of ten or more.

> Sorry to be such a curmudgeon, but I've been burnt by
> this before.  I realize that I am taking a bit of a
> defeatist attitude towards bugs (i.e., I assume that
> they exist, and therefore take defensive measures like
> reducing the size of the bug search space for testing,

No, no, this is entirely sane of you.  Read "Software Testing" by
Glenford Myers.

Nick Barnes