Hackers,
Finally, here is the patch to implement multiple concurrent workers in
autovacuum. What this patch does is augment the shmem autovacuum struct
so that it can support having several workers running at any time; and
the launcher process schedules workers to be started on the existing
databases at regular intervals.
The other important change is that a worker rechecks whether a table
needs vacuum, before actually doing it. This is to support the
situation in which a previous worker is still running. This algorithm
is very simple-minded at present, so hot table starvation is a problem
not attacked in this patch.
Something that hasn't changed is that the algorithm for picking a
database to start a worker on, is purely based on pgstat info on last
autovacuum start.
The launcher keeps a doubly linked list of databases, along with a "time
of the next worker". This list is sorted by that time; autovacuum picks
the nearest time and sleeps until that time. When it awakes it launches
a worker, updates the database list to reflect the new time, and goes to
sleep again.
The database list is recalculated when a database is started on a
database that the launcher didn't expect. This can happen because the
doubly linked list only tracks databases with stat entries; but if you
create a new database, the database-picking algorithm will choose a
database that wasn't on the list.
There are several open issues still. For example, memory management of
the database list. Currently the launcher just leaks some memory. This
of course needs to be fixed. One thing I noticed is that Dllist uses
malloc for allocations, claiming that front-end code does not support
palloc. However, I didn't find any front-end code using Dllist. I
think this is just a leftover, and maybe this could be changed and that
code cleaned up a little.
No docs are changed yet, and there are some things utterly untested
(like the handling of Xid wraparound, dropping workers during CREATE and
DROP DATABASE, etc). Also there are some FIXMEs in the code. Fixing
most of them is mostly a SMOP, but I wanted to throw the patch out for
other to comment, review, improve, test, complain, ignore, or whatever
they feel like doing.
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support