[gclist] Re: Articles expiring too fast!

[Mark Hittinger by way of hbaker@netcom.com Henry Baker]

[hbaker: I thought that people on this list might get a kick out of
people with _real_ GC problems!]

Mark Hittinger@freebsd.netcom.com (Netcom/Dallas) writes:

hockings@netcom.com (Mark Hockings.) writes:
>Mark Hittinger (bugs@freebsd.netcom.com) wrote:
 [5 gigs of usenet news/day; 12 or 18 gigs of disk =>]
>: 6.  12/5 = ?  18/5 = ?
>er, how about having someone do the unthinkable?
>Plan ahead!  This is just like the yahoos who build a new bridge or widen a
>road. by the time they finish,  it needs to be widened again. (always hated
>that)
>Since newsfeeds are going to keep getting bigger, it would behoove Netcom to
>plan ahead a bit. If they think they are going to need 20 gig of space, get
>40. (actually, if the newsfeeds are 5 gig per day now, then I would double
>that (minimum) multiply by one weeks worth, then double THAT total.
>(meaning they should go to 140 gig of drive space)


Well now

because of the design of our current netnews infrastructure we'd need to
get two disks for every one because of mirroring.  This technique allows
a drive to go bad without having the server role over.

In order to keep all the servers in sync we'd need to purchase the same 
amount of space for each server.  So you want 40g spools hmmm then I need
to go ask for 20x2x36 gig or 1440 gig.  The street price on this will be
around a half million dollars.  Not a problem I've got that kind of clout! :-)

Then I need to arrange the staff for the project and some downtime for
each server.  

Of course, expire is now going to run a lot longer since it is going to
traverse 40gig of unix directories instead of 12gig.  I'll need to implement
the 64 bit hash history database because the 32 bit hash will roll over and
die once we go above a 20g spool.  I'll probably need to add some additional
servers to spread the load out a bit more since expire will essentially
be running constantly.

And all of this expense so we can have access to last weeks binaries which
were reposts from the prior week...prior week....week...week....week....

As a slight reality check here the newer servers that we have put into service
have 18g spools.  We do plan to upgrade the older 12g servers to 18g, and once
that is done we can increase expire times across the servers by 33%.  This
will impact server load and retrieval speed - but we are working on some 
software changes to nnrpd that will enhance its performance.

Regards :-)

Mark Hittinger
Netcom/Dallas
bugs@freebsd.netcom.com