Re: Changing the autovacuum launcher scheduling; oldest table firstalgorithm

On 02/28/2018 12:04 PM, Masahiko Sawada wrote:
> Hi,
>
> I've created the new thread for the changing AV launcher scheduling.
> The problem of AV launcher scheduling is described on [1] but I
> summarize it here.
>
> If there is even one database that is at risk of wraparound, currently
> AV launcher selects the database having the oldest datfrozenxid until
> all tables in that database have been vacuumed. This leads that tables
> on other database can bloat and not be frozen because other database
> are not selected by AV launcher. It makes things worse if the database
> has a large table that takes a long time to be vacuumed.
>
> Attached patch changes the AV launcher scheduling algorithm based on
> the proposed idea by Robert so that it selects a database first that
> has the oldest table when the database is at risk of wraparound. For
> detail of the algorithm please refer to [2].
>
> In this algorithm, the running AV workers advertise the next
> datfrozenxid on the shared memory that we will have. That way, AV
> launcher can select a database that has the oldest table in the
> database cluster. However, this algorithm doesn't support the case
> where the age of next datfrozenxid gets greater than
> autovacum_vacuum_max_age. This case can happen if users set
> autovacvuum_vacuum_max_age to small value and vacuuming the whole
> database takes a long time. But since it's not a common case and it
> doesn't degrade than current behavior even if this happened, I think
> it's not a big problem.
>
> Feedback is very welcome.
>
> [1] https://www.postgresql.org/message-id/0A3221C70F24FB45833433255569204D1F8A4DC6%40G01JPEXMBYT05
> [2]
https://www.postgresql.org/message-id/CA%2BTgmobT3m%3D%2BdU5HF3VGVqiZ2O%2Bv6P5wN1Gj%2BPrq%2Bhj7dAm9AQ%40mail.gmail.com
>
> Regards,
>
> --
> Masahiko Sawada
> NIPPON TELEGRAPH AND TELEPHONE CORPORATION
> NTT Open Source Software Center

Hello!
I`ve tried to compile your patch on Fedora24 with gcc 6.3.1 20161221:

gcc -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -g3 -O0 -I../../../src/include  
-D_GNU_SOURCE   -c -o autovacuum.o autovacuum.c
In file included from ../../../src/include/postgres.h:46:0,
                  from autovacuum.c:62:
autovacuum.c: In function ‘get_next_xid_bounds’:
autovacuum.c:1979:9: warning: implicit declaration of function 
‘TransactionIdIsVaild’ [-Wimplicit-function-declaration]
   Assert(TransactionIdIsVaild(frozenxid));
          ^
../../../src/include/c.h:713:7: note: in definition of macro ‘Trap’
    if (condition) \
        ^~~~~~~~~
autovacuum.c:1979:2: note: in expansion of macro ‘Assert’
   Assert(TransactionIdIsVaild(frozenxid));
   ^~~~~~
autovacuum.c:1980:28: error: ‘minmutli’ undeclared (first use in this 
function)
   Assert(MultiXactIdIsValid(minmutli));
                             ^
../../../src/include/c.h:713:7: note: in definition of macro ‘Trap’
    if (condition) \
        ^~~~~~~~~
autovacuum.c:1980:2: note: in expansion of macro ‘Assert’
   Assert(MultiXactIdIsValid(minmutli));
   ^~~~~~
autovacuum.c:1980:9: note: in expansion of macro ‘MultiXactIdIsValid’
   Assert(MultiXactIdIsValid(minmutli));
          ^~~~~~~~~~~~~~~~~~
autovacuum.c:1980:28: note: each undeclared identifier is reported only 
once for each function it appears in
   Assert(MultiXactIdIsValid(minmutli));
                             ^
../../../src/include/c.h:713:7: note: in definition of macro ‘Trap’
    if (condition) \
        ^~~~~~~~~
autovacuum.c:1980:2: note: in expansion of macro ‘Assert’
   Assert(MultiXactIdIsValid(minmutli));
   ^~~~~~
autovacuum.c:1980:9: note: in expansion of macro ‘MultiXactIdIsValid’
   Assert(MultiXactIdIsValid(minmutli));
          ^~~~~~~~~~~~~~~~~~
<builtin>: recipe for target 'autovacuum.o' failed
make[3]: *** [autovacuum.o] Error 1

-- 
Grigory Smolkin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company