[gclist] Re: Articles expiring too fast!

[Mark Hittinger by way of hbaker@netcom.com Henry Baker]

[hbaker:  More details from Hittinger on a _real_ GC problem...]

jonzonk writes:
>In article <5cds77$1tg@dfw-ixnews1.ix.netcom.com>,
>Mark, could you clarify for me how the expires are currently working.
>Is there one expire running on the "master" news server or does each
>server have its own expire based on the size of the server.
>IOW, does the server that us shellers are looking at already have the
>extra 33% or is it something that we can look forward to.
>(and at the risk of sounding like a broken record ... where *is* expire.ctl)

The servers are all time synchronized with 'ntp'.  The expires are all set
to run at the same time across all 20 of the current INN servers.  The other
11 or so nfs mounted reader servers don't need to run expire.  

Expire runs at 1 AM Pacific time and 1 PM Pacific time on all the IX news 
servers, the goal is to expire the same articles on all the servers at 
roughly the same time.  Expire generally takes about three hours to run.
All the servers use the same expire.ctl in order to achieve synchonization
of the spools.  Jonzonk the 33% increase is something which is in the cards
for a later date.

The expire process executes the following steps with approximate times:

1.  Process the ~260mb history file to find articles to delete  ~  40 mins
2.  Remove the target articles from the overview database       ~  60 mins
3.  Remove the target articles from the news spool              ~  60 mins
4.  Final cleanup pass, remove cancelled articles from overview ~  20 mins

I've not exported the true /usr/local/news directory to shell, so you can't
see the true expire.ctl file.  We are running the "poison" expire program
now and I'm tweaking the expire times anyway.  When I'm finished tuning it
I'll put a world readable copy on /ssa/expire.ctl.

Regards,

Mark Hittinger
Netcom/Dallas
bugs@freebsd.netcom.com