Re: pg_upgrade < 9.3 -> >=9.3 misses a step around multixacts

On Fri, Jun 27, 2014 at 05:05:21PM -0400, Alvaro Herrera wrote:
> Bruce Momjian wrote:
>
> > Assuming this is true, here is a query we can put in the next 9.3 minor
> > release notes to tell users if they need to remove the '0000' file:
> >
> >     WITH list(file) AS
> >     (
> >             SELECT * FROM pg_ls_dir('pg_multixact/offsets')
> >     )
> >     SELECT  EXISTS (SELECT * FROM list WHERE file = '0000') AND
> >             NOT EXISTS (SELECT * FROM list WHERE file = '0001') AND
> >             NOT EXISTS (SELECT * FROM list WHERE file = 'FFFF') AND
> >             EXISTS (SELECT * FROM list WHERE file != '0000')
> >             AS file_removal_needed;
> >
> > Do they need to remove the members/0000 file too?
>
> No, the members file shouldn't be removed; if they have used multixacts
> at all since upgrading, the members/0000 file contains the members of
> the first bunch of multis.  Removing it would remove valid data.  (Also
> I still think that removing the members/0000 file that initdb creates is
> a mistake.)

We never remove offset or members files with user data;  we are only
discussing the removal of empty files created by initdb.  Can you
explain why we would remove the offsets file but retain the members file
created by initdb when we are _not_ preserving files from the old
cluster?  We are certainly setting the offset and member value from the
old cluster:

        exec_prog(UTILITY_LOG_FILE, NULL, true,
                  "\"%s/pg_resetxlog\" -m %u,%u \"%s\"",
                  new_cluster.bindir,
                  old_cluster.controldata.chkpnt_nxtmulti + 1,
                  old_cluster.controldata.chkpnt_nxtmulti,
                  new_cluster.pgdata);

How are we not creating a gap in members files by setting the members
value?  Why will that not cause problems?

> I think the query above is correct.  One possible quibble is a FFFF
> segment that isn't allocated in full.  I don't think this is a problem,
> because although the 0000 file would survive deletion, it would be
> overwritten with zeroes as soon as the counter wraps around.  Since 0000
> is seen as "in the future" as soon as file 8000 starts being used, it
> shouldn't be a problem there.

Understood.   Here is an updated query with a better column label --- I
think this is ready for the 9.3.X release notes, along with some text:

    WITH list(file) AS
    (
        SELECT * FROM pg_ls_dir('pg_multixact/offsets')
    )
    SELECT     EXISTS (SELECT * FROM list WHERE file = '0000') AND
        NOT EXISTS (SELECT * FROM list WHERE file = '0001') AND
        NOT EXISTS (SELECT * FROM list WHERE file = 'FFFF') AND
        EXISTS (SELECT * FROM list WHERE file != '0000')
        AS file_0000_removal_required;

> Another thing probably worth doing is have pg_upgrade set
> pg_database.datminmxid to the minimum of pg_class.relminmxid on each
> database.  Right now these values are all initialized to 1, which is a
> problem.  We have set_frozenxids() which appears to be related, although
> I don't readily understand what that is doing.

I will address this in a separate email.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +