Re: ERROR: MultiXactId XXXXX has not been created yet -- apparent wraparound

On 08/15/2016 07:40 AM, Ioana Danes wrote:

Hello Adrian,

On Mon, Aug 15, 2016 at 10:00 AM, Adrian Klaver
<adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:

    On 08/15/2016 06:48 AM, Ioana Danes wrote:

        Hello All,

        I am running postgres 9.4.8 on centos 7 and few days ago I
        started to
        get this error MultiXactId 32766 has not been created yet --
        apparent
        wraparound in 2 instances.

        1. On one database server during the nightly task that does the
        "vacuum
        analyze". I upgraded to postgres 9.4.9 as there is a reference
        to a bug
        fixed with a reference to this error but I am still getting the same
        error when I vacuum analyze. This database was created using
        pg_dump and
        pg_restore (not pg_upgrade) from the previous installation which was
        still postgres 9.4.6 but on SLES instead of CentOS. I only have one
        postgres version installed on the server.

        Autovacuum is turned on and I also have a task that runs vacuum
        analyze
        every night.

        postgresql94-9.4.9-1PGDG.rhel7.x86_64
        postgresql94-plperl-9.4.9-1PGDG.rhel7.x86_64
        postgresql94-libs-9.4.9-1PGDG.rhel7.x86_64
        postgresql94-server-9.4.9-1PGDG.rhel7.x86_64
        postgresql94-contrib-9.4.9-1PGDG.rhel7.x86_64

        psql (9.4.9)
        =# vacuum analyze;
        ERROR:  MultiXactId 32766 has not been created yet -- apparent
        wraparound

        This server only runs since June 2016.

        2. The second instance of this error is on a new database server
        during
        pg_restore on few CREATE INDEX, PRIMARY KEYS and FOREIGN KEYS
        statements
        (postgres 9.4.8 on centos 7). In this case I dropped and
        recreated the
        database and the second try succeeded without errors.

        postgresql-14.csv:2016-08-14 05:33:59.753
        CST,"postgres","test",19555,"[local]",57b055d4.4c63,11,"CREATE
        INDEX",2016-08-14 05:28:20 CST,1/27230,5381,ERROR,XX000,"MultiXactId
        1667854355 has not been created yet -- apparent
        wraparound",,,,,,"CREATE
        INDEX...;


        Please let me know if I should provide more info.


    Are these the same VM's as in your previous error(Corrupted Data) post?

    Is the first error you mention on the db3 server from the previous
    error(Corrupted Data)?


They are not the same servers but they have similar setup. Fortunately
they are qa and development.

Should have asked previously, are they on the same host machine?

Basically, something is corrupting data, just trying to narrow it down to hardware or software. Looking to figure out if the problem(s) follow the physical machine or the software the Postgres is running on.

On the same servers (qa) I also started to get some corruption messages
on pg_dump.

2016-08-15 00:04:17.238
CST,"postgres","abrazo",7600,"[local]",57b15a62.1db0,6,"COPY",2016-08-15
00:00:02 CST,13/6895726,0,ERROR,XX000,"compressed data is
corrupt",,,,,,"COPY ... TO stdout;",,,"pg_dump"

Also yesterday I dropped the table that had the problem in the previous
post and restored it from db1 (the good database) and all seemed good
but few hours after I fixed that table, db4 that is a the PITR slave of
db3 started in production with a checksum error...

All the problems started to appear when we switched to CentOS with kvm.

On the same machines or new/different machines?

I have instances on postgres 9.4 in production and qa on SLES with XEN
since few years ago and never had any kind of issues, and that would be
around 70 database servers. We also run on postgres since 12 years ago
without major problems...


Thank you very much for your help,
ioana



        Thank you in advance,
        Ioana








    --
    Adrian Klaver
    adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

--
Adrian Klaver
adrian.klaver@aklaver.com