Re: pg_clog questions

Tom Lane wrote:
> "Benjamin Krajmalnik" <kraj@illumen.com> writes:
> >>> While checking our server, I noticed quite a few files in the pg_clog
> >>> directory.
> >>> Is there a maintenance task which can be run to purge the files which
> >>> are no longer needed?
>
> >> VACUUM.  If they're not disappearing, you aren't running an adequate
> >> vacuum regime --- missing out some databases, perhaps?
>
> > I have autovacuum turned on with the default settings.
>
> Hm.  I believe that autovac only does database-wide vacuums when it
> thinks they're necessary to prevent transaction wraparound failures.
> Which would mean that it'd let pg_clog grow to something on the order
> of half a gig before any truncation would happen.  That's probably
> insufficiently aggressive :-(
>
> Alvaro, Matthew, any thoughts about improving that?

Hum, IIRC there is a test somewhere to check pg_database.datvacuumxid
and issue a database-wide vacuum if it gets too old, but the test uses
a-little-less-than-2-billion to fire :-(

This is the code:

        /*
         * We look for the database that most urgently needs a database-wide
         * vacuum.  We decide that a database-wide vacuum is needed 100000
         * transactions sooner than vacuum.c's vac_truncate_clog() would
         * decide to start giving warnings.  If any such db is found, we
         * ignore all other dbs.
         *
         * Unlike vacuum.c, we also look at vacuumxid.  This is so that
         * pg_clog can be kept trimmed to a reasonable size.
         */
        freeze_age = (int32) (nextXid - tmp->frozenxid);
        vacuum_age = (int32) (nextXid - tmp->vacuumxid);
        tmp->age = Max(freeze_age, vacuum_age);

        this_whole_db = (tmp->age >
                         (int32) ((MaxTransactionId >> 3) * 3 - 100000));

Maybe it was foolish to use such a large constant in the vacuumxid case
:-(

> It strikes me that Alvaro's work-in-progress on maintaining per-table
> xmin info would allow truncation of clog without actually insisting on
> a database-wide VACUUM, but that's not going to be any help for
> existing releases.

Yeah.  The patch is almost ready BTW and I was about to post it on
Saturday but I got distracted.  I have a problem with sinval messages
:-(  I'll post it later today.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support