pgsql: Support asynchronous execution of MergeAppend subplans.

Поиск
Список
Период
Сортировка
Искать

pgsql: Support asynchronous execution of MergeAppend subplans.

От:
Etsuro Fujita <efujita@postgresql.org>
Дата:
Support asynchronous execution of MergeAppend subplans.

Commit 27e1f1456 added asynchronous execution for Append nodes, letting
multiple async-capable children -- in practice postgres_fdw ForeignScans
-- run concurrently instead of one at a time, but MergeAppend was not
covered.

This commit extends asynchronous execution to MergeAppend.  On the first
call, requests for all valid async subplans are fired off up front, the
synchronous subplans are pulled while those requests are in flight, and
the async results are then collected before the merge heap is built.
After that, the merge cannot emit another row until the next tuple from
one specific subplan (the current heap top) is known, so at most one
request is outstanding during steady-state execution, and we always wait
indefinitely for it rather than polling as Append does.

Some other notable changes to the core and postgres_fdw code:

* create_merge_append_plan now marks children async-capable.  Unlike a
  plain Append, which disables async whenever its output must be
  ordered, a MergeAppend always has pathkeys and still preserves
  ordering by merging the per-child streams, so having pathkeys is not
  disqualifying here; a child is eligible only if it is ordered well
  enough on its own that no Sort node was inserted above it, as Sort is
  not async-capable.  The new GUC enable_async_merge_append (default on)
  controls this.

* The event-loop body of ExecAppendAsyncEventWait is factored out into a
  new ExecAsyncEventWait in execAsync.c and shared by both node types,
  with the caller responsible for creating and freeing the WaitEventSet.
  A new MergeAppendReady wait event reports waits in the new event loop.

* Since an FDW's ForeignAsyncConfigureWait callback can no longer assume
  the requestor is an Append, add ExecAsyncGetRequestorEventSet to fetch
  the requestor's WaitEventSet without knowing its node type, and use it
  in postgres_fdw.

* postgresForeignAsyncConfigureWait also needs adjusting: the
  optimization of skipping event configuration, rather than processing
  an in-progress request made by a different requestor, is valid only
  when our requestor is an Append; a MergeAppend cannot make progress
  unless the given request produces a result, so in that case we process
  the pending request and configure the event.

Full disclosure: I (Etsuro) reworked Alexander Pyhalov's original patch
with Claude Opus 4.8.

Author: Alexander Pyhalov 
Author: Etsuro Fujita 
Reviewed-by: Matheus Alcantara 
Reviewed-by: Alexander Korotkov 
Reviewed-by: Alena Rybakina 
Reviewed-by: Álvaro Herrera 
Reviewed-by: solai v 
Reviewed-by: Alexander Pyhalov 
Reviewed-by: Etsuro Fujita 
Discussion: https://postgr.es/m/59be194c5a409fb9fc9f2031581b8a44%40postgrespro.ru

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/11591f273e6a26fb55728b410538fa2627b19b28

Modified Files
--------------
contrib/postgres_fdw/expected/postgres_fdw.out  | 442 +++++++++++++++++++++++-
contrib/postgres_fdw/postgres_fdw.c             |  60 ++--
contrib/postgres_fdw/sql/postgres_fdw.sql       | 119 ++++++-
doc/src/sgml/config.sgml                        |  14 +
doc/src/sgml/fdwhandler.sgml                    |  24 +-
doc/src/sgml/postgres-fdw.sgml                  |  10 +-
src/backend/executor/README                     |  15 +-
src/backend/executor/execAsync.c                | 127 +++++++
src/backend/executor/nodeAppend.c               |  94 +----
src/backend/executor/nodeMergeAppend.c          | 301 +++++++++++++++-
src/backend/optimizer/path/costsize.c           |   1 +
src/backend/optimizer/plan/createplan.c         |  26 ++
src/backend/utils/activity/wait_event_names.txt |   1 +
src/backend/utils/misc/guc_parameters.dat       |   7 +
src/backend/utils/misc/postgresql.conf.sample   |   1 +
src/include/executor/execAsync.h                |   8 +
src/include/executor/nodeMergeAppend.h          |   1 +
src/include/nodes/execnodes.h                   |   6 +
src/include/optimizer/cost.h                    |   1 +
src/test/regress/expected/sysviews.out          |   3 +-
20 files changed, 1115 insertions(+), 146 deletions(-)

FAQ