pgsql: aio: Adjust I/O worker pool automatically.

aio: Adjust I/O worker pool automatically.

The size of the I/O worker pool used to implement io_method=worker was
previously controlled by the io_workers setting, defaulting to 3.  It
was hard to know how to tune it effectively.  That is replaced with:

  io_min_workers=2
  io_max_workers=8 (up to 32)
  io_worker_idle_timeout=60s
  io_worker_launch_interval=100ms

The pool is automatically sized within the configured range according to
recent variation in demand.  It grows when existing workers detect that
latency might be introduced by queuing, and shrinks when the
highest-numbered worker is idle for too long.  Work was already
concentrated into low-numbered workers in anticipation of this logic.

The logic for waking extra workers now also tries to measure and reduce
the number of spurious wakeups, though they are not entirely eliminated.

Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2Bm4xV0LMoH2c%3DoRAdEXuCnh%2BtGBTWa7uFeFMGgTLAw%2BQ%40mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/d1c01b79d4ae90e52bf9db9c05c9de17b7313e85

Modified Files
--------------
doc/src/sgml/config.sgml                        |  69 ++-
src/backend/postmaster/postmaster.c             | 175 +++++--
src/backend/storage/aio/method_worker.c         | 636 ++++++++++++++++++++----
src/backend/utils/activity/wait_event_names.txt |   1 +
src/backend/utils/misc/guc_parameters.dat       |  34 +-
src/backend/utils/misc/postgresql.conf.sample   |   6 +-
src/include/storage/io_worker.h                 |  11 +-
src/include/storage/lwlocklist.h                |   1 +
src/include/storage/pmsignal.h                  |   1 +
src/test/modules/test_aio/t/002_io_workers.pl   |  15 +-
src/tools/pgindent/typedefs.list                |   1 +
11 files changed, 802 insertions(+), 148 deletions(-)