Обсуждение: pgsql: tableam: Add tuple_{insert, delete, update, lock} and use.

Поиск
Список
Период
Сортировка

pgsql: tableam: Add tuple_{insert, delete, update, lock} and use.

От
Andres Freund
Дата:
tableam: Add tuple_{insert, delete, update, lock} and use.

This adds new, required, table AM callbacks for insert/delete/update
and lock_tuple. To be able to reasonably use those, the EvalPlanQual
mechanism had to be adapted, moving more logic into the AM.

Previously both delete/update/lock call-sites and the EPQ mechanism had
to have awareness of the specific tuple format to be able to fetch the
latest version of a tuple. Obviously that needs to be abstracted
away. To do so, move the logic that find the latest row version into
the AM. lock_tuple has a new flag argument,
TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
version, rather than the current one.  It'd have been possible to do
so via a separate callback as well, but finding the last version
usually also necessitates locking the newest version, making it
sensible to combine the two. This replaces the previous use of
EvalPlanQualFetch().  Additionally HeapTupleUpdated, which previously
signaled either a concurrent update or delete, is now split into two,
to avoid callers needing AM specific knowledge to differentiate.

The move of finding the latest row version into tuple_lock means that
encountering a row concurrently moved into another partition will now
raise an error about "tuple to be locked" rather than "tuple to be
updated/deleted" - which is accurate, as that always happens when
locking rows. While possible slightly less helpful for users, it seems
like an acceptable trade-off.

As part of this commit HTSU_Result has been renamed to TM_Result, and
its members been expanded to differentiated between updating and
deleting. HeapUpdateFailureData has been renamed to TM_FailureData.

The interface to speculative insertion is changed so nodeModifyTable.c
does not have to set the speculative token itself anymore. Instead
there's a version of tuple_insert, tuple_insert_speculative, that
performs the speculative insertion (without requiring a flag to signal
that fact), and the speculative insertion is either made permanent
with table_complete_speculative(succeeded = true) or aborted with
succeeded = false).

Note that multi_insert is not yet routed through tableam, nor is
COPY. Changing multi_insert requires changes to copy.c that are large
enough to better be done separately.

Similarly, although simpler, CREATE TABLE AS and CREATE MATERIALIZED
VIEW are also only going to be adjusted in a later commit.

Author: Andres Freund and Haribabu Kommi
Discussion:
    https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
    https://postgr.es/m/20190313003903.nwvrxi7rw3ywhdel@alap3.anarazel.de
    https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/5db6df0c0117ff2a4e0cd87594d2db408cd5022f

Modified Files
--------------
contrib/pgrowlocks/pgrowlocks.c                    |   6 +-
src/backend/access/heap/heapam.c                   | 435 +++++++++------------
src/backend/access/heap/heapam_handler.c           | 324 +++++++++++++++
src/backend/access/heap/heapam_visibility.c        | 125 +++---
src/backend/access/heap/tuptoaster.c               |   2 +-
src/backend/access/table/tableam.c                 | 113 ++++++
src/backend/access/table/tableamapi.c              |  13 +
src/backend/commands/copy.c                        |   3 +-
src/backend/commands/trigger.c                     | 112 +++---
src/backend/executor/execIndexing.c                |   4 +-
src/backend/executor/execMain.c                    | 289 +-------------
src/backend/executor/execReplication.c             | 137 +++----
src/backend/executor/nodeLockRows.c                | 142 +++----
src/backend/executor/nodeModifyTable.c             | 435 ++++++++++++---------
src/backend/executor/nodeTidscan.c                 |   2 +-
src/include/access/heapam.h                        |  56 +--
src/include/access/tableam.h                       | 359 +++++++++++++++++
src/include/executor/executor.h                    |  12 +-
src/include/utils/snapshot.h                       |  13 -
.../isolation/expected/partition-key-update-1.out  |   4 +-
src/tools/pgindent/typedefs.list                   |   4 +-
21 files changed, 1527 insertions(+), 1063 deletions(-)


Re: pgsql: tableam: Add tuple_{insert, delete, update, lock} and use.

От
Alexander Korotkov
Дата:
Great work, thank you very much!

On Sun, Mar 24, 2019 at 6:02 AM Andres Freund <andres@anarazel.de> wrote:
> Previously both delete/update/lock call-sites and the EPQ mechanism had
> to have awareness of the specific tuple format to be able to fetch the
> latest version of a tuple. Obviously that needs to be abstracted
> away. To do so, move the logic that find the latest row version into
> the AM. lock_tuple has a new flag argument,
> TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
> version, rather than the current one.  It'd have been possible to do
> so via a separate callback as well, but finding the last version
> usually also necessitates locking the newest version, making it
> sensible to combine the two. This replaces the previous use of
> EvalPlanQualFetch().  Additionally HeapTupleUpdated, which previously
> signaled either a concurrent update or delete, is now split into two,
> to avoid callers needing AM specific knowledge to differentiate.

BTW, given your skepticism during PGCon 2018 discussions, I didn't
expect my lock tuple patch [1] to get in.  But if it finally got in,
I've expected to be listed as author...

1. https://www.postgresql.org/message-id/CAJrrPGc951F-R4Kfa4W47B5vHKeHsB2Y34zewp%3Db%2BAWSkF9RVA%40mail.gmail.com

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


Re: pgsql: tableam: Add tuple_{insert, delete, update, lock} and use.

От
Andres Freund
Дата:
Hi,

On March 24, 2019 11:17:15 AM PDT, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
>Great work, thank you very much!
>
>On Sun, Mar 24, 2019 at 6:02 AM Andres Freund <andres@anarazel.de>
>wrote:
>> Previously both delete/update/lock call-sites and the EPQ mechanism
>had
>> to have awareness of the specific tuple format to be able to fetch
>the
>> latest version of a tuple. Obviously that needs to be abstracted
>> away. To do so, move the logic that find the latest row version into
>> the AM. lock_tuple has a new flag argument,
>> TUPLE_LOCK_FLAG_FIND_LAST_VERSION, that forces it to lock the last
>> version, rather than the current one.  It'd have been possible to do
>> so via a separate callback as well, but finding the last version
>> usually also necessitates locking the newest version, making it
>> sensible to combine the two. This replaces the previous use of
>> EvalPlanQualFetch().  Additionally HeapTupleUpdated, which previously
>> signaled either a concurrent update or delete, is now split into two,
>> to avoid callers needing AM specific knowledge to differentiate.
>
>BTW, given your skepticism during PGCon 2018 discussions, I didn't
>expect my lock tuple patch [1] to get in.

Well, nobody came up with a better design, and after polishing it didn't look that bad...


>  But if it finally got in,
>I've expected to be listed as author...
>
>1.
>https://www.postgresql.org/message-id/CAJrrPGc951F-R4Kfa4W47B5vHKeHsB2Y34zewp%3Db%2BAWSkF9RVA%40mail.gmail.com

Sorry, I wasn't aware of the origin of they piece of work.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.