Re: [HACKERS] Block level parallel vacuum

On Fri, Dec 20, 2019 at 12:13 PM Masahiko Sawada
 wrote:
>
> I've attached the updated version patch that incorporated the all
> review comments I go so far.
>

I have further edited the first two patches posted by you.  The
changes include (a) changed tests to reset the guc, (b) removing some
stuff which is not required in this version, (c) moving some variables
around to make them in better order, (d) changed comments and few
other cosmetic things and (e) commit messages for first two patches.

I think the first two patches attached in this email are in good shape
and we can commit those unless you or someone has more comments on
them, the main parallel vacuum patch can still be improved by some
more test/polish/review.  I am planning to push the first two patches
next week after another pass.  The first two patches are explained in
brief as below:

1. v4-0001-Delete-empty-pages-in-each-pass-during-GIST-VACUUM:  It
allows us to delete empty pages in each pass during GIST VACUUM.
Earlier, we use to postpone deleting empty pages till the second stage
of vacuum to amortize the cost of scanning internal pages.  However,
that can sometimes (say vacuum is canceled or errored between first
and second stage) delay the pages to be recycled.  Another thing is
that to facilitate deleting empty pages in the second stage, we need
to share the information of internal and empty pages between different
stages of vacuum.  It will be quite tricky to share this information
via DSM which is required for the main parallel vacuum patch.  Also,
it will bring the logic to reclaim deleted pages closer to nbtree
where we delete empty pages in each pass.  Overall, the advantages of
deleting empty pages in each pass outweigh the advantages of
postponing the same.  This patch is discussed in detail in a separate
thread [1].

2. v39-0001-Introduce-IndexAM-fields-for-parallel-vacuum.patch:
Introduce new fields amusemaintenanceworkmem and
amparallelvacuumoptions in IndexAmRoutine for parallel vacuum.  The
amusemaintenanceworkmem tells whether a particular IndexAM uses
maintenance_work_mem or not.  This will help in controlling the memory
used by individual workers as otherwise, each worker can consume
memory equal to maintenance_work_mem.  This has been discussed in
detail in a separate thread as well [2]. The amparallelvacuumoptions
tell whether a particular IndexAM participates in a parallel vacuum
and if so in which phase (bulkdelete, vacuumcleanup) of vacuum.


[1] - https://www.postgresql.org/message-id/CAA4eK1LGr%2BMN0xHZpJ2dfS8QNQ1a_aROKowZB%2BMPNep8FVtwAA%40mail.gmail.com
[2] - https://www.postgresql.org/message-id/CAA4eK1LmcD5aPogzwim5Nn58Ki+74a6Edghx4Wd8hAskvHaq5A@mail.gmail.com

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com