Обсуждение: vacuum freeze performance, wraparound issues

Hi all,

I have a few questions related to recovering from a near-miss with transactionid wraparound.

I'm currently running a vacuum freeze in single user mode on our largest database (about 36 TB).  It's been running for
about10 days (since the database shut itself down to avoid xid wraparound). One cpu has been basically running at 100%
thewhole time (except during short periods of write activity when it drops briefly to around 30%). Any idea how to
guesshow long this might take? Is there anything I can adjust to speed the vacuum freeze up? It seems to be CPU
limited--cananyone tell me what it would be doing that is so CPU intensive? 

Runs postgres 9.1.9 on FreeBSD 9.1 with ZFS, database was built --with-segsize=10, storage is a pool of 20 2-disk
mirrors
Maintenance_work_mem is set to 10GB; there is about 48 GB of memory in the machine.


The size of our database may be unusual for postgres, but honestly, it has performed quite well for us over the years.
Ourtrouble comes when every once in a while we get a perfect storm that causes the autovacuum to fall behind. We are
generallyvery careful to try to keep this from happening, since all it takes is one query at the wrong time to take a
largedatabase offline for days (or weeks). This is definitely our biggest cause of postgres outages, taking one of our
databasesoffline about once a year, it seems. With the speed postgres is capable of, and the ever-falling prices of
storagemaking larger, faster databases possible, has the possibility of changing the transaction id to a 64-bit (or
even128-bit!) value been considered?  


Thanks!
Natalie

Natalie Wenz <nataliewenz@ebureau.com> writes:
> ... With the speed postgres is capable of, and the ever-falling prices
> of storage making larger, faster databases possible, has the possibility
> of changing the transaction id to a 64-bit (or even 128-bit!) value been
> considered?

Not terribly seriously --- the penalties from making row headers 8 bytes
bigger have always seemed to outweigh the advantages.  (128 bits is right
out; we don't even have 128-bit LSNs.)

We'd probably take a patch to make 64-bit XIDs available as a compile-time
option, if someone wanted to do the legwork to write and test it.  But
let me ask you this: if such an option existed, would you be willing to
dump and reload your database to take advantage of it?  The conversion
costs of changing row header format seem like they'd discourage exactly
those people whom such a feature could help.

            regards, tom lane

Yes, absolutely! We would dump and reload all of our large databases in a heartbeat if there were an option for a
64-bitxid!  

Another quick question: When this database took itself offline to avoid transaction id wraparound, we opted for a
vacuumfreeze in single-user mode. Was that the right choice? In other words, what is the fastest way to get a database
backonline when this occurs? Maybe a plain vacuum would have been better?  

Thanks!
Natalie


On Aug 12, 2013, at 11:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Natalie Wenz <nataliewenz@ebureau.com> writes:
>> ... With the speed postgres is capable of, and the ever-falling prices
>> of storage making larger, faster databases possible, has the possibility
>> of changing the transaction id to a 64-bit (or even 128-bit!) value been
>> considered?
>
> Not terribly seriously --- the penalties from making row headers 8 bytes
> bigger have always seemed to outweigh the advantages.  (128 bits is right
> out; we don't even have 128-bit LSNs.)
>
> We'd probably take a patch to make 64-bit XIDs available as a compile-time
> option, if someone wanted to do the legwork to write and test it.  But
> let me ask you this: if such an option existed, would you be willing to
> dump and reload your database to take advantage of it?  The conversion
> costs of changing row header format seem like they'd discourage exactly
> those people whom such a feature could help.
>
>             regards, tom lane
>
>
> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin