BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

Поиск
Список
Период
Сортировка
Искать

BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
PG Bug reporting form <noreply@postgresql.org>
Дата:
The following bug has been logged on the website:

Bug reference:      19760
Logged by:          Alperen Keles
Email address:      alpkeles99@gmail.com
PostgreSQL version: 18.6
Operating system:   macOS 26 arm64 (also seen on Ubuntu 24.04 x86_64)
Description:        

CLUSTER (and REPACK on master) silently drops a live row when the same
transaction inserted the row and then updated it inside a subtransaction
that was rolled back. No error is raised; the row is gone after COMMIT.

  CREATE TABLE t1(c0 int, c1 int);
  CREATE INDEX t1_i0 ON t1 (c1);
  BEGIN;
  INSERT INTO t1 VALUES (1, 1);
  SAVEPOINT s1; UPDATE t1 SET c0 = 2; ROLLBACK TO s1;
  CLUSTER t1 USING t1_i0;
  SELECT count(*) FROM t1;   -- 0, expected 1
  COMMIT;
  SELECT count(*) FROM t1;   -- 0, expected 1

Reproduced with default settings on 18.6 (REL_18_STABLE a85b5523e67),
14.24 and master 158b6d5109 (all --enable-cassert --enable-debug, macOS
arm64); also found on master on Ubuntu x86_64. Rechecked on 2026-10-15
(Ubuntu 24.04, x86_64, gcc 13.3, --enable-cassert --enable-debug, server
under ulimit -v 6 GB):
the same on 19rc1 (REL_19_STABLE 0fb192f1b02), master aa7b9cf132b and
REL_18_STABLE 4d62ef92fda. CLUSTER VERBOSE reports a
negative "dead row versions cannot be removed yet" count (-1 here).

Found with a query fuzzer.




Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Takeshi Ideriha <iderihatakeshi@gmail.com>
Дата:
Hi

Thank you for reporting this.

> 
> CLUSTER (and REPACK on master) silently drops a live row when the same
> transaction inserted the row and then updated it inside a subtransaction
> that was rolled back. No error is raised; the row is gone after COMMIT.
> 
>   CREATE TABLE t1(c0 int, c1 int);
>   CREATE INDEX t1_i0 ON t1 (c1);
>   BEGIN;
>   INSERT INTO t1 VALUES (1, 1);
>   SAVEPOINT s1; UPDATE t1 SET c0 = 2; ROLLBACK TO s1;
>   CLUSTER t1 USING t1_i0;
>   SELECT count(*) FROM t1;   -- 0, expected 1
>   COMMIT;
>   SELECT count(*) FROM t1;   -- 0, expected 1
> 
While I was reproducing the bug, I noticed that if we SELECT rows before
CLUSTER, the issue won't happen. This suggests that not setting hint bit
is very likely to be related to the issue.

----
BEGIN;
INSERT INTO t1 VALUES (1, 1);
SAVEPOINT s1;
UPDATE t1 SET c0 = 2;
ROLLBACK TO s1;

SELECT count(*) FROM t1; -- SELECT before CLUSTER

CLUSTER VERBOSE t1 USING t1_i0;
SELECT count(*) FROM t1;
---

-- RESULT --
 count
-------
     1
(1 row)

INFO:  repacking "public.t1" using index scan on "t1_i0"
INFO:  "public.t1": found 0 removable, 1 nonremovable row versions in 1
pages
DETAIL:  0 dead row versions cannot be removed yet.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
CLUSTER
 count
-------
     1
(1 row)

COMMIT
 count
-------
     1
(1 row)


> REL_18_STABLE 4d62ef92fda. CLUSTER VERBOSE reports a
> negative "dead row versions cannot be removed yet" count (-1 here).

The value is tups_recently_dead in heapam_relation_copy_for_cluster().
tups_recently_dead decreases only when rewrite_heap_dead_tuple()
returnes TRUE in rewriteheap.c. I guess there is something wrong with
visibility check invloving hint bits in rewriteheap.c. I'll look into
this further.

Regards,
Takeshi Ideriha


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Дата:
Hi,

On Fri, 16 Oct 2026 at 08:25, Takeshi Ideriha  wrote:
>
> Hi
>
> Thank you for reporting this.
>
> >
> > CLUSTER (and REPACK on master) silently drops a live row when the same
> > transaction inserted the row and then updated it inside a subtransaction
> > that was rolled back. No error is raised; the row is gone after COMMIT.
> >
> >   CREATE TABLE t1(c0 int, c1 int);
> >   CREATE INDEX t1_i0 ON t1 (c1);
> >   BEGIN;
> >   INSERT INTO t1 VALUES (1, 1);
> >   SAVEPOINT s1; UPDATE t1 SET c0 = 2; ROLLBACK TO s1;
> >   CLUSTER t1 USING t1_i0;
> >   SELECT count(*) FROM t1;   -- 0, expected 1
> >   COMMIT;
> >   SELECT count(*) FROM t1;   -- 0, expected 1
> >
> While I was reproducing the bug, I noticed that if we SELECT rows before
> CLUSTER, the issue won't happen. This suggests that not setting hint bit
> is very likely to be related to the issue.
>
> ----
> BEGIN;
> INSERT INTO t1 VALUES (1, 1);
> SAVEPOINT s1;
> UPDATE t1 SET c0 = 2;
> ROLLBACK TO s1;
>
> SELECT count(*) FROM t1; -- SELECT before CLUSTER
>
> CLUSTER VERBOSE t1 USING t1_i0;
> SELECT count(*) FROM t1;
> ---
>
> -- RESULT --
>  count
> -------
>      1
> (1 row)
>
> INFO:  repacking "public.t1" using index scan on "t1_i0"
> INFO:  "public.t1": found 0 removable, 1 nonremovable row versions in 1
> pages
> DETAIL:  0 dead row versions cannot be removed yet.
> CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
> CLUSTER
>  count
> -------
>      1
> (1 row)
>
> COMMIT
>  count
> -------
>      1
> (1 row)
>
>
> > REL_18_STABLE 4d62ef92fda. CLUSTER VERBOSE reports a
> > negative "dead row versions cannot be removed yet" count (-1 here).
>
> The value is tups_recently_dead in heapam_relation_copy_for_cluster().
> tups_recently_dead decreases only when rewrite_heap_dead_tuple()
> returnes TRUE in rewriteheap.c. I guess there is something wrong with
> visibility check invloving hint bits in rewriteheap.c. I'll look into
> this further.

I think you are right. It looks like the missing HEAP_XMAX_INVALID
hint in HeapTupleSatisfiesVacuumHorizon() lets the rewrite defer copying
the live original tuple and then discard it when it encounters the aborted
successor.

Regards,
Ayush


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
shihao zhong <zhong950419@gmail.com>
Дата:
Вложения:
Hi,

Here is a patch. rewrite_heap_dead_tuple() drops the held tuple when its
successor is dead, but that is only right if the successor's inserter
committed. If it aborted, the update never happened, so 0001 keeps the
tuple and adds its logical rewrite mapping.

Setting the hint in HeapTupleSatisfiesVacuum() is not enough. VACUUM
FULL on a catalog can see a transaction that is still open, and this
loses the pg_class row of a committed table:

  s1: BEGIN; CREATE TABLE t (a int);
  s1: SAVEPOINT sp; ALTER TABLE t ADD COLUMN b int; ROLLBACK TO sp;
  s2: VACUUM FULL pg_class;
  s1: COMMIT;

0002 is an isolation test for that case, it is optional. But I want it in tree to prevent
future regression in that area.

Thanks,
Shihao

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Takeshi Ideriha <iderihatakeshi@gmail.com>
Дата:
Вложения:
Hi Shihao, Ayush

On 2026/10/19 12:31, shihao zhong wrote:
> Hi,
> 
> Here is a patch. rewrite_heap_dead_tuple() drops the held tuple when its
> successor is dead, but that is only right if the successor's inserter
> committed. If it aborted, the update never happened, so 0001 keeps the
> tuple and adds its logical rewrite mapping.

> Setting the hint in HeapTupleSatisfiesVacuum() is not enough. VACUUM
> FULL on a catalog can see a transaction that is still open, and this
> loses the pg_class row of a committed table:
I was actually also writing a code to fix the issue and have just
noticed the patch was already submitted. Attached a patch for reference.
I haven't considered into logical_rewrite_heap_tuple(). I need to check
this... Thank you.

> 	{
>+		/*
>+		 * If the inserter of this tuple aborted, the update >never happened
>+		 * and the earlier tuple is not dead.  It will be >written at the end.
>+		 */
>+		if (!HeapTupleHeaderXminCommitted(old_tuple->t_data) &&
>+			!>TransactionIdDidCommit(HeapTupleHeaderGetXmin(old_tuple->t_data)))
>+			return false;

I am thinking like the below since HeapTupleSatisfiesVacuum set the xmin
invalid hint bit for the dead tuple. So I believe we can just rely on
HeapTupleHeaderXminInvalid hint bit.

+               if (HeapTupleHeaderXminInvalid(old_tuple->t_data))
+                       return false;

What do you think above?

>0002 is an isolation test for that case, it is optional. But I want it
>in tree to prevent
>future regression in that area.

+1. I also think this is optional but I believe the issue is easy to
oversight, so adding a new test would be helpful.

Regards,
Takeshi Ideriha

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
shihao zhong <zhong950419@gmail.com>
Дата:

Hi Takeshi,

> So I believe we can just rely on HeapTupleHeaderXminInvalid hint bit.

I tried it and it keeps the row in both cases. But it also needs the
logical_rewrite_heap_tuple() call in end_heap_rewrite(). Without it,
logical decoding after VACUUM FULL pg_class fails:

  ERROR:  could not map filenumber "base/16490/16491" to relation OID

Thanks,
Shihao

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Takeshi Ideriha <iderihatakeshi@gmail.com>
Дата:
Hi Shihao


>> So I believe we can just rely on HeapTupleHeaderXminInvalid hint bit.
> 
> I tried it and it keeps the row in both cases. But it also needs the
> logical_rewrite_heap_tuple() call in end_heap_rewrite(). Without it,
> logical decoding after VACUUM FULL pg_class fails:
> 
>   ERROR:  could not map filenumber "base/16490/16491" to relation OID
> 
Thank you for replying. Glad to know it worked for both two cases.

Regarding logical_rewrite_heap_tuple() part, you are completely right. I
tested by myself yesterday and hit the same error while decoding the
vacuum full scenario. Thank you for pointing out. I was not aware of DDL
case with logical replication.

Regards,
Takeshi Ideriha



Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Tue, Oct 20, 2026 at 11:32:44AM +0900, Takeshi Ideriha wrote:
> Regarding logical_rewrite_heap_tuple() part, you are completely right. I
> tested by myself yesterday and hit the same error while decoding the
> vacuum full scenario. Thank you for pointing out. I was not aware of DDL
> case with logical replication.

@@ -574,6 +576,14 @@ rewrite_heap_dead_tuple(RewriteState state, HeapTuple old_tuple)
 
 	if (unresolved != NULL)
 	{
+		/*
+		 * If the inserter of this tuple aborted, the update never happened
+		 * and the earlier tuple is not dead.  It will be written at the end.
+		 */
+		if (!HeapTupleHeaderXminCommitted(old_tuple->t_data) &&
+			!TransactionIdDidCommit(HeapTupleHeaderGetXmin(old_tuple->t_data)))
+			return false;

Having a recheck of the old tuple's state within
rewrite_heap_dead_tuple() feels very incorrect to me, based on the
bunch of comments at the top of rewriteheap.c.  It sounds to me that
rewrite_heap_dead_tuple() should be called when we know that the tuple
is dead, no?  Isn't heapam_relation_copy_for_cluster() the bit actually
incorrect here, marking the tuple with "isdead" to start with?  Your
solution of a recheck to make sure that an old tuple is actually dead
in a code path where an old tuple is assumed to be absolutely and
definitely dead looks like an API contract violation regarding
rewriteheap.c.
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Tue, Oct 20, 2026 at 04:41:49PM +0900, Michael Paquier wrote:
> Having a recheck of the old tuple's state within
> rewrite_heap_dead_tuple() feels very incorrect to me, based on the
> bunch of comments at the top of rewriteheap.c.  It sounds to me that
> rewrite_heap_dead_tuple() should be called when we know that the tuple
> is dead, no?  Isn't heapam_relation_copy_for_cluster() the bit actually
> incorrect here, marking the tuple with "isdead" to start with?  Your
> solution of a recheck to make sure that an old tuple is actually dead
> in a code path where an old tuple is assumed to be absolutely and
> definitely dead looks like an API contract violation regarding
> rewriteheap.c.

Looking a bit more at that before the end of the day, I think that the
suggestion of touching HeapTupleSatisfiesVacuumHorizon() and setting
some hint bits would be the correct thing to do.  One case that matters
is the "deleting subtransaction must have aborted" case.  I am pretty
sure that we should also worry about the case where we check for
TransactionIdIsInProgress(HeapTupleHeaderGetRawXmin(tuple)), which
looks reachable, and set an extra hint bit there..
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
shihao zhong <zhong950419@gmail.com>
Дата:
Вложения:
Hi Michael,

>> Isn't heapam_relation_copy_for_cluster() the bit actually
>> incorrect here, marking the tuple with "isdead" to start with?

Thanks for looking at this so closely. You are right that the recheck
broke the contract of rewriteheap.c.

> Looking a bit more at that before the end of the day, I think that
> the suggestion of touching HeapTupleSatisfiesVacuumHorizon() and
> setting some hint bits would be the correct thing to do.

Agreed, and thanks for pointing at both places. v2 does just that and
leaves rewriteheap.c alone.

With the hint set, the old version no longer waits in unresolved_tups,
so the logical mapping change from v1 is not needed anymore.

Thanks,
Shihao

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Tue, Oct 20, 2026 at 09:20:18PM -0400, shihao zhong wrote:
> Agreed, and thanks for pointing at both places. v2 does just that and
> leaves rewriteheap.c alone.
> 
> With the hint set, the old version no longer waits in unresolved_tups,
> so the logical mapping change from v1 is not needed anymore.

I can see that you are testing both code paths, at least.  The
coverage part looks good, but I'd suggest to make the test more
readable at least especially in the s1_alter and s1_plain that include
a bunch of queries.  There two are hard to parse.

+                if (!TransactionIdIsInProgress(xmax) &&
+                    !TransactionIdDidCommit(xmax))
+                    SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
+                                InvalidTransactionId)

It's also worth noting that these may make the code less efficient,
with more SLRU traffic.  What could be the class of worst case
possible?  A combination of an in-progress transaction with some
VACUUM activity while the in-progress transaction is still opened?
VACUUM would need to do more work with the patch in place, where I
guess it could be slower if we have a bunch of tuples that need to
have this hint bit set.

Or perhaps this is too narrow to worry about?  Anyway, we should be
super careful with such changes.  That can bite hard.

If others have any ideas and/or comments, please feel free..

Thanks,
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Zhijie Hou <houzhijie22@gmail.com>
Дата:
Вложения:
Hi,

On Wed, Oct 21, 2026 at 3:13 PM Michael Paquier  wrote:
>
> On Tue, Oct 20, 2026 at 09:20:18PM -0400, shihao zhong wrote:
> > Agreed, and thanks for pointing at both places. v2 does just that and
> > leaves rewriteheap.c alone.
> >
> > With the hint set, the old version no longer waits in unresolved_tups,
> > so the logical mapping change from v1 is not needed anymore.
>
> I can see that you are testing both code paths, at least.  The
> coverage part looks good, but I'd suggest to make the test more
> readable at least especially in the s1_alter and s1_plain that include
> a bunch of queries.  There two are hard to parse.
>
> +                if (!TransactionIdIsInProgress(xmax) &&
> +                    !TransactionIdDidCommit(xmax))
> +                    SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
> +                                InvalidTransactionId)
>
> It's also worth noting that these may make the code less efficient,
> with more SLRU traffic.  What could be the class of worst case
> possible?  A combination of an in-progress transaction with some
> VACUUM activity while the in-progress transaction is still opened?
> VACUUM would need to do more work with the patch in place, where I
> guess it could be slower if we have a bunch of tuples that need to
> have this hint bit set.
>
> Or perhaps this is too narrow to worry about?  Anyway, we should be
> super careful with such changes.  That can bite hard.
>
> If others have any ideas and/or comments, please feel free..

I haven't thought of a better idea yet, but I noticed a potential race condition
for MultiXacts in the current fix:

+ if (!(tuple->t_infomask & HEAP_XMAX_INVALID) &&
+ !HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask) &&
+ !HeapTupleHeaderIsOnlyLocked(tuple))
+ {
+ TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple);
+
+ if (!TransactionIdIsInProgress(xmax) &&
+ !TransactionIdDidCommit(xmax))
+ SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
+ InvalidTransactionId);

If a tuple's Xmax is a MultiXact and the updating subtransaction is still in
progress when HeapTupleHeaderIsOnlyLocked() is called, it returns false.
However, if that updating subtransaction aborts before the second
TransactionIdIsInProgress() check, the second check returns false and sets
HEAP_XMAX_INVALID on the tuple even though the MultiXact still has active
members, which I think violates the MultiXact hint-bit standard.

This can cause a problem if another transaction locked the tuple (e.g., FOR KEY
SHARE) beforehand. Once HEAP_XMAX_INVALID is set on the tuple, another backend
can lock the tuple FOR UPDATE even though the FOR KEY SHARE lock has not been
released.

I can reproduce this if a transaction updates a tuple, a second session locks
it FOR KEY SHARE, the first transaction updates it again in a subtransaction,
and a concurrent backend runs a SELECT that triggers on-access pruning
(heap_prune_satisfies_vacuum() -> HeapTupleSatisfiesVacuumHorizon()).

See the attached patch (which applies on top of the v2 patches and adds an
injection point) to reproduce this.

Best Regards,
Zhijie Hou

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Wed, Oct 21, 2026 at 06:01:28PM +0800, Zhijie Hou wrote:
> + if (!(tuple->t_infomask & HEAP_XMAX_INVALID) &&
> + !HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask) &&
> + !HeapTupleHeaderIsOnlyLocked(tuple))
> + {
> + TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple);
> +
> + if (!TransactionIdIsInProgress(xmax) &&
> + !TransactionIdDidCommit(xmax))
> + SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
> + InvalidTransactionId);
> 
> If a tuple's Xmax is a MultiXact and the updating subtransaction is still in
> progress when HeapTupleHeaderIsOnlyLocked() is called, it returns false.
> However, if that updating subtransaction aborts before the second
> TransactionIdIsInProgress() check, the second check returns false and sets
> HEAP_XMAX_INVALID on the tuple even though the MultiXact still has active
> members, which I think violates the MultiXact hint-bit standard.

Perhaps also check for HEAP_XMAX_IS_MULTI to skip the multixacts and
make sure that active members are not lost, then?  I am not completely
sure if this would be always right, though, these code paths are the
hard-level definition of tricky.

What I am really wondering is why do we actually need to fix *two*
problems at once?  The first SetHintBits() looks like a no-brainer to
me, and links to the second permutation in the test "s1_plain
s1_count" where we do a CLUSTER after an UPDATE rollbacked to a
previous savepoint.  I don't think that there is a need for an
isolation test on this one, either, wouldn't a plain SQL sequence be
sufficient?

That's not to mention that the first one can be hit with DDLs that do
some update on catalogs, as well.  Say, an ANALYZE with a CLUSTER
(this SQL sequence is a courtesy of Claude, I've just pointed to it
the possibility and the agent has confirmed the sequence):
BEGIN;
-- force the index-scan path
SET LOCAL random_page_cost = 0.0001;
SET LOCAL seq_page_cost = 1000;
CREATE TABLE s (a int);
INSERT INTO s VALUES (1);
ANALYZE s;
SAVEPOINT sp;
INSERT INTO s SELECT g FROM generate_series(2, 1000) g;
-- updates pg_statistic row
ANALYZE s;
ROLLBACK TO sp;
RELEASE sp;
CLUSTER VERBOSE pg_statistic USING pg_statistic_relid_att_inh_index;
COMMIT;
-- one row expected, none on HEAD.
SELECT count(*) FROM pg_statistic WHERE starelid = 's'::regclass;

The VACUUM FULL on pg_class is less exciting, points to the second
SetHintBits().  And as far as I can see, this one is actually the
source of a measurable regression.  To prove my point, see the SQL
scripts attached.  Use them like that:
1) Run setup.sql, for some relations.
2) Run holders.sql through a bunch of sessions (64 here, could be
more).  This is important to stress TransactionIdIsInProgress():
for i in $(seq 64); do
  psql -X -f holders.sql > /dev/null 2>&1 &
done
3) In a new session, run either subxact_updates.sql, delete.sql or
insert_only.sql.
3) Run vacuum.sql, measure its runtime.

Then with 500k rows, here are some VACUUM average runtimes in ms from
one of my laptops:
                HEAD  Shihao's v2
subxact_updates 1408         2123
delete           686         1369
insert_only      689          693

The subxact_updates is an edge case, fine, still it regresses badly.
The DELETE case is a much more common case.  It regresses even more
badly, and that does not look acceptable to me.  That should be
optimizable, of course, I don't really know yet how to do that.  I
know however that regressions like these ones are never good.

So the first step to me would be a "Divide and Conquer" strategy,
where we should look at each problem independently instead of trying
to do too much at once.
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Takeshi Ideriha <iderihatakeshi@gmail.com>
Дата:
Hi

> 
> What I am really wondering is why do we actually need to fix *two*
> problems at once?  The first SetHintBits() looks like a no-brainer to
> me, and links to the second permutation in the test "s1_plain
> s1_count" where we do a CLUSTER after an UPDATE rollbacked to a
> previous savepoint.  I don't think that there is a need for an
> isolation test on this one, either, wouldn't a plain SQL sequence be
> sufficient?
>

> So the first step to me would be a "Divide and Conquer" strategy,
> where we should look at each problem independently instead of trying
> to do too much at once.
Thank you for sorting the issue out and setting the direction. I was
thinking of leaving first SetHintBits() in
HeapTupleSatisfiesVacuumHorizon() and moving the second SetHintBits() to
heapam_relation_copy_for_cluster() to solve the performance issue.
However, it would still be another API violation, wouldn't it? We also
needs to figure out the multixact race issue. I'm trying to look into
multixact part as much as possible.

Regards,
Takeshi Ideriha


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Thu, Oct 22, 2026 at 11:53:29AM +0900, Takeshi Ideriha wrote:
> Thank you for sorting the issue out and setting the direction. I was
> thinking of leaving first SetHintBits() in
> HeapTupleSatisfiesVacuumHorizon() and moving the second SetHintBits() to
> heapam_relation_copy_for_cluster() to solve the performance issue.

Yeah, I don't think that we should begin using SetHintBits() outside
of heapam_visibility.c when it comes down to heap as a table AM.

> However, it would still be another API violation, wouldn't it? We also
> needs to figure out the multixact race issue. I'm trying to look into
> multixact part as much as possible.

What do you think about the first SetHintBits() added for the
subtransaction aborted case?  This one seems acceptable here, by
itself..
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Takeshi Ideriha <iderihatakeshi@gmail.com>
Дата:

On 2026/10/22 12:13, Michael Paquier wrote:
> Yeah, I don't think that we should begin using SetHintBits() outside> of
heapam_visibility.c when it comes down to heap as a table AM.

Sure, thank you for the reply.

>> However, it would still be another API violation, wouldn't it? We also
>> needs to figure out the multixact race issue. I'm trying to look into
>> multixact part as much as possible.
> 
> What do you think about the first SetHintBits() added for the
> subtransaction aborted case?  This one seems acceptable here, by
> itself..
+1 for this. Setting the hint bit for the aborted subtransaction inside
HeapTupleSatisfiesVacuumHorizon is safe and not harmful for the
performance.

Regards,
Takeshi Ideriha


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Takeshi Ideriha <iderihatakeshi@gmail.com>
Дата:
Hi

On 2026/10/22 11:22, Michael Paquier wrote:
> On Wed, Oct 21, 2026 at 06:01:28PM +0800, Zhijie Hou wrote:
>> + if (!(tuple->t_infomask & HEAP_XMAX_INVALID) &&
>> + !HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask) &&
>> + !HeapTupleHeaderIsOnlyLocked(tuple))
>> + {
>> + TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple);
>> +
>> + if (!TransactionIdIsInProgress(xmax) &&
>> + !TransactionIdDidCommit(xmax))
>> + SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
>> + InvalidTransactionId);
>>
>> If a tuple's Xmax is a MultiXact and the updating subtransaction is still in
>> progress when HeapTupleHeaderIsOnlyLocked() is called, it returns false.
>> However, if that updating subtransaction aborts before the second
>> TransactionIdIsInProgress() check, the second check returns false and sets
>> HEAP_XMAX_INVALID on the tuple even though the MultiXact still has active
>> members, which I think violates the MultiXact hint-bit standard.
> 
> Perhaps also check for HEAP_XMAX_IS_MULTI to skip the multixacts and
> make sure that active members are not lost, then?  I am not completely
> sure if this would be always right, though, these code paths are the
> hard-level definition of tricky.
> 
I'm +1 for fixing the subtransaction issue first but let me add a
comment regarding this multixact race issue.

It should be OK not to set the hint bit for a tuple whose xmax is
MultiXact containing a key locker and an aborted updating
subtransaction. This is because rewrite_heap_tuple filters out this
tuple by HeapTupleHeaderIsOnlyLocked and doesn't put it into unresolved
table. I'm not sure if there is another issue we need to consider in
addition to the performance issue.

Regards,
Takeshi Ideriha


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Zhijie Hou <houzhijie22@gmail.com>
Дата:
Hi,

On Thu, Oct 22, 2026 at 10:22 AM Michael Paquier  wrote:
>
> On Wed, Oct 21, 2026 at 06:01:28PM +0800, Zhijie Hou wrote:
> > + if (!(tuple->t_infomask & HEAP_XMAX_INVALID) &&
> > + !HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask) &&
> > + !HeapTupleHeaderIsOnlyLocked(tuple))
> > + {
> > + TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple);
> > +
> > + if (!TransactionIdIsInProgress(xmax) &&
> > + !TransactionIdDidCommit(xmax))
> > + SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
> > + InvalidTransactionId);
> >
> > If a tuple's Xmax is a MultiXact and the updating subtransaction is still in
> > progress when HeapTupleHeaderIsOnlyLocked() is called, it returns false.
> > However, if that updating subtransaction aborts before the second
> > TransactionIdIsInProgress() check, the second check returns false and sets
> > HEAP_XMAX_INVALID on the tuple even though the MultiXact still has active
> > members, which I think violates the MultiXact hint-bit standard.
>
> Perhaps also check for HEAP_XMAX_IS_MULTI to skip the multixacts and
> make sure that active members are not lost, then?  I am not completely
> sure if this would be always right, though, these code paths are the
> hard-level definition of tricky.

Right, I think we can skip setting flag for HEAP_XMAX_IS_MULTI tuples,
since they're
handled correctly later in rewrite_heap_tuple() by HeapTupleHeaderIsOnlyLocked()
and won't be added to unresolved_tups. (I see Takeshi San also mentioned that).

>
> What I am really wondering is why do we actually need to fix *two*
> problems at once?  The first SetHintBits() looks like a no-brainer to
> me, and links to the second permutation in the test "s1_plain
> s1_count" where we do a CLUSTER after an UPDATE rollbacked to a
> previous savepoint.

+1 to fix that part first. That part of the fix (patch) looks clean to me. It
also covers more common cases, which can happen for both user tables and
catalogs, while the other part (the complicated, performance-sensitive one) only
covers catalog cases IIUC, as user table DML holds locks until transaction end,
so concurrent CLUSTER can't touch it in that case.

> Then with 500k rows, here are some VACUUM average runtimes in ms from
> one of my laptops:
>                 HEAD  Shihao's v2
> subxact_updates 1408         2123
> delete           686         1369
> insert_only      689          693
>
> The subxact_updates is an edge case, fine, still it regresses badly.
> The DELETE case is a much more common case.  It regresses even more
> badly, and that does not look acceptable to me.  That should be
> optimizable, of course, I don't really know yet how to do that.  I
> know however that regressions like these ones are never good.

One idea to improve is to do these expensive checks only when xmin is not equal
to xmax. If xmin is still in progress, xmax cannot be aborted, so it's safe to
return early in that case. The DELETE test wouldn't regress after this change,
since xmin and xmax differ there. The subxact_update test would still regress,
but that's likely a rare case.

>
> So the first step to me would be a "Divide and Conquer" strategy,
> where we should look at each problem independently instead of trying
> to do too much at once.

Agreed.

Best Regards,
Zhijie Hou


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Manu <manuelreyesbravo@gmail.com>
Дата:
Вложения:
Hi Michael,

On Thu, Oct 22, 2026 at 11:22:08AM +0900, Michael Paquier wrote:
> The DELETE case is a much more common case.  It regresses even more
> badly, and that does not look acceptable to me.  That should be
> optimizable, of course, I don't really know yet how to do that.

I ran your scripts as attached upthread, on master 871dee0ccbe with
v2, 500k rows and 64 holders, 12 interleaved rounds per build.  The
ratios match yours: VACUUM takes 1.98x HEAD with delete.sql (95% CI
1.96..2.02) and 1.53x with subxact_updates.sql (1.51..1.55), and
insert_only.sql does not move.  Method and tables are in the attached
measurements.txt.

The two cases cost for different reasons.  In delete.sql xmax is the
inserter's own xid, so the new TransactionIdIsInProgress(xmax) is a
second proc array scan, and pg_stat_slru shows no extra access.  It
grows with the number of open transactions: 1.51x with no holder,
1.95x with 16, 2.02x with 128.  In subxact_updates.sql xmax is one of
500k subxids, the subxid cache has overflowed, and v2 doubles the
pg_subtrans and clog accesses of each VACUUM (999872 to 1999744 for
pg_subtrans, 499936 to 999872 for the clog).

Zhijie's idea, doing the check only when xmin != xmax, removes the
delete.sql regression: 1.00x (0.99..1.01).  In delete.sql xmax and
xmin are the same xid, so the check is skipped there.  subxact_updates
stays at 1.53x, as he expected.  The condition the other way around
loses the pg_class row again.  On its own the idea keeps the
MultiXact race of his repro; with the HEAP_XMAX_IS_MULTI skip you
suggested, the repro passes.  Both together are nocfbot-0001.

For subxact_updates, nocfbot-0002 on top of it.  In that branch xmin
is in progress, so xmax is the inserter's xid or one of its
subtransactions, and RecordTransactionAbort() writes a subtransaction
abort to the clog before the xids leave the proc array.  So
TransactionIdDidAbort(xmax) gives the same answer there as
!TransactionIdIsInProgress() && !TransactionIdDidCommit(), with one
clog lookup and no proc array or pg_subtrans access.  With both
diffs, subxact_updates is at 1.01x (1.00..1.03) with 64 holders and
1.00x with 128.  What 0002 costs is that clog lookup per tuple:
1.14x with no holder at all (50 ms to 57 ms).

With 0001 alone and with 0001 + 0002, these pass: the v2-0002 spec,
your pg_statistic sequence, VACUUM FULL pg_class with the CREATE
TABLE transaction open, logical decoding across it, Zhijie's
injection point repro, make check, isolation, and recovery 035.
Since 0002 changes when the hint is set, I added five cases where
the pg_class row must survive VACUUM FULL pg_class exactly once
(released, nested and prepared subtransactions).  A build that sets
the hint unconditionally passes the original SQL cases and fails
three of the new ones, with two pg_class rows for the table.

The diffs apply on top of v2-0001 and v2-0002.  Shihao, if they hold
up, feel free to fold them into a v3.

Regards,
Manu

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Thu, Oct 22, 2026 at 04:53:41PM +0800, Zhijie Hou wrote:
> +1 to fix that part first. That part of the fix (patch) looks clean to me. It
> also covers more common cases, which can happen for both user tables and
> catalogs, while the other part (the complicated, performance-sensitive one) only
> covers catalog cases IIUC, as user table DML holds locks until transaction end,
> so concurrent CLUSTER can't touch it in that case.

Please find attached a patch to do this part, for now.

> One idea to improve is to do these expensive checks only when xmin is not equal
> to xmax. If xmin is still in progress, xmax cannot be aborted, so it's safe to
> return early in that case. The DELETE test wouldn't regress after this change,
> since xmin and xmax differ there. The subxact_update test would still regress,
> but that's likely a rare case.

Yeah, the xmin <> xmax check would make sense here.  I'd like to
suspect that you should be able to make back for most of the
performance regression of v2 with tweaks like that.

How does the attached look to you?
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
shihao zhong <zhong950419@gmail.com>
Дата:
Вложения:
Hi,

Sorry for the slow reply. This turned out a bit more complex than I
expected, and I was busy with some personal stuff yesterday.

On Fri, Oct 23, 2026 at 08:42:14AM +0900, Michael Paquier wrote:
> How does the attached look to you?

Looks good to me. It fixes the report and your pg_statistic case,
and the test fails without the fix.

I think Manu's change for the second spot should go in too, then
both cases in this thread are fixed. I have an isolation test for
this case if that helps.

Thanks,
Shihao

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Дата:
Hi,

On Fri, 23 Oct 2026 at 05:12, Michael Paquier  wrote:
>
> On Thu, Oct 22, 2026 at 04:53:41PM +0800, Zhijie Hou wrote:
> > +1 to fix that part first. That part of the fix (patch) looks clean to me. It
> > also covers more common cases, which can happen for both user tables and
> > catalogs, while the other part (the complicated, performance-sensitive one) only
> > covers catalog cases IIUC, as user table DML holds locks until transaction end,
> > so concurrent CLUSTER can't touch it in that case.
>
> Please find attached a patch to do this part, for now.

v1 looks good to me.

Regards,
Ayush


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Zhijie Hou <houzhijie22@gmail.com>
Дата:
Hi,

On Fri, Oct 23, 2026 at 7:42 AM Michael Paquier  wrote:
>
> On Thu, Oct 22, 2026 at 04:53:41PM +0800, Zhijie Hou wrote:
> > +1 to fix that part first. That part of the fix (patch) looks clean to me. It
> > also covers more common cases, which can happen for both user tables and
> > catalogs, while the other part (the complicated, performance-sensitive one) only
> > covers catalog cases IIUC, as user table DML holds locks until transaction end,
> > so concurrent CLUSTER can't touch it in that case.
>
> Please find attached a patch to do this part, for now.
>
> > One idea to improve is to do these expensive checks only when xmin is not equal
> > to xmax. If xmin is still in progress, xmax cannot be aborted, so it's safe to
> > return early in that case. The DELETE test wouldn't regress after this change,
> > since xmin and xmax differ there. The subxact_update test would still regress,
> > but that's likely a rare case.
>
> Yeah, the xmin <> xmax check would make sense here.  I'd like to
> suspect that you should be able to make back for most of the
> performance regression of v2 with tweaks like that.
>
> How does the attached look to you?

Thanks for updating the patch, it looks good to me.

Best Regards,
Zhijie Hou


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Zhijie Hou <houzhijie22@gmail.com>
Дата:
Вложения:
Hi,

On Fri, Oct 23, 2026 at 4:44 AM Manu  wrote:
>
> On Thu, Oct 22, 2026 at 11:22:08AM +0900, Michael Paquier wrote:
> > The DELETE case is a much more common case.  It regresses even more
> > badly, and that does not look acceptable to me.  That should be
> > optimizable, of course, I don't really know yet how to do that.
>
>
> For subxact_updates, nocfbot-0002 on top of it.  In that branch xmin
> is in progress, so xmax is the inserter's xid or one of its
> subtransactions, and RecordTransactionAbort() writes a subtransaction
> abort to the clog before the xids leave the proc array.  So
> TransactionIdDidAbort(xmax) gives the same answer there as
> !TransactionIdIsInProgress() && !TransactionIdDidCommit(), with one
> clog lookup and no proc array or pg_subtrans access.

This doesn't look safe to me. Even if TransactionIdDidAbort() returns true, the
xmax may not have been removed from PGPROC yet. So this change would create the
risky case where the old tuple has been marked as XMAX_INVAL while the new
tuple's xmin is still considered in progress.

After some testing, I confirmed that this causes an issue with HOT updates:

When a tuple is HOT-updated in a subtransaction and the transaction then rolls
back, there is a window after the XIDs are marked aborted in CLOG but before
they are removed from ProcArray. If a concurrent ANALYZE scans the table during
that window, HeapTupleSatisfiesVacuumHorizon() marks the old tuple as
HEAP_XMAX_INVALID while still treating the new heap-only tuple as in progress.
If brin_summarize_range() then scans the page in that same window,
heap_get_root_tuples() does not follow the HOT chain from the old tuple because
HeapTupleHeaderIsHotUpdated() returns false once HEAP_XMAX_INVALID is set, while
the scan still tries to index the in-progress heap-only tuple and fails with:

ERROR: failed to find parent tuple for heap-only tuple at xx in table
"mxact_abort"

See the attached follow-up patch, which reproduces this. This may not be a
common case, but it looks risky to create such a race.

Best Regards,
Zhijie Hou

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Manu <manuelreyesbravo@gmail.com>
Дата:
Вложения:
Hi Zhijie,

On Sun, Oct 25, 2026 at 4:31 PM Zhijie Hou 
wrote:
> This doesn't look safe to me. Even if TransactionIdDidAbort() returns true, the
> xmax may not have been removed from PGPROC yet. So this change would create the
> risky case where the old tuple has been marked as XMAX_INVAL while the new
> tuple's xmin is still considered in progress.

You're right.  On master dfd7ede529f with v2 and nocfbot-0001, your
test fails in 3 of 3 runs once nocfbot-0002 is added ("failed to find
parent tuple for heap-only tuple at (0,3)"), and passes in 4 of 4
without it.

I took the order in RecordTransactionAbort() (clog first, proc array
later) as the reason the clog alone was enough, but that order is
what opens the window: until ProcArrayEndTransaction() runs, the new
version of the tuple, whose xmin is that xmax, is still in progress,
which is the race the note at the top of heapam_visibility.c is
about.

So the clog can only be a filter there.  The attached diff replaces
nocfbot-0002: it keeps TransactionIdDidAbort(xmax) first and then
confirms with !TransactionIdIsInProgress(xmax) before setting the
hint.  It passes your test in 4 of 4 runs, shihao's spec, cases 1-3,
5 and 7-11 from measurements.txt, and make check, the isolation suite,
injection_points and test_decoding.

It keeps what made nocfbot-0002 cheap in subxact_updates.sql.  There
xmax is a released subtransaction that is still running, so it is
not aborted in the clog and the proc array is never asked.  One
VACUUM makes the same SLRU accesses as with the old nocfbot-0002:
999937 clog and 999872 subtrans, against 999873 and 1999744 with
nocfbot-0001 alone.  When xmax did abort, it pays both lookups until
the hint is set.  I have not re-timed it.

nocfbot-0001 alone (your xmin != xmax idea plus the MULTI skip) is
unchanged and passes your test in 4 of 4 runs.

Thanks for the repro.

Regards,
Manu

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Sat, Oct 24, 2026 at 08:57:23PM +0800, Zhijie Hou wrote:
> Thanks for updating the patch, it looks good to me.

Thanks for the reviews.  Applied and backpatched this part down to
v14.
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Manu <manuelreyesbravo@gmail.com>
Дата:
Вложения:
Hi Michael,

On Mon, Oct 26, 2026 at 09:57:20AM +0900, Michael Paquier wrote:
> Thanks for the reviews.  Applied and backpatched this part down to
> v14.

Thanks for taking care of it.  The attached series adds the second
spot on top of e7f4463e869, as shihao asked upthread.

0001 sets HEAP_XMAX_INVALID in the branch of
HeapTupleSatisfiesVacuumHorizon() where the inserter is another
backend that is still running, once the updater has aborted.  It
skips xmax == xmin and MultiXacts, and it asks the clog first and
then the proc array, so it does not open the window Zhijie found in
my clog-only version.  It includes shihao's isolation spec, which
loses the pg_class row without the fix.  0002 is Zhijie's test for
that window.

VACUUM with your scripts, 64 holders, one run of 12 interleaved
rounds on 2026-10-25, ratios to HEAD:

  delete.sql: 1.00x
  subxact_updates.sql: 1.00x (with v2's check: 1.51x)
  aborted updates, first VACUUM: 1.07x, same as v2's check
  aborted updates, following VACUUMs: 1.00x

"Aborted updates" is a new script where each subtransaction rolls
back after its UPDATE.  It is the only case where 0001 asks the proc
array, once per tuple, in the VACUUM that sets the hint.  The
measured heapam_visibility.c matches 0001 except for two lines of a
test overlay.  Method and the script are in measurements.txt.

0001 and 0002 apply with git am on master and REL_19_STABLE.  On 14
to 18 the code applies as is, but the isolation_schedule line needs
another neighbor.  0002 only goes down to 17, where the injection
points it uses exist.  I ran make check and the isolation and
injection_points suites on master and 19, and make check and
isolation on 14.

The xmin != xmax check and the race are Zhijie's, the MultiXact skip
is yours, and the spec is shihao's.

Regards,
Manu

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Michael Paquier <michael@paquier.xyz>
Дата:
Вложения:
On Mon, Oct 26, 2026 at 12:41:19PM -0300, Manu wrote:
> Thanks for taking care of it.  The attached series adds the second
> spot on top of e7f4463e869, as shihao asked upthread.
>
> +            /*
> +             * If the updater of the tuple has already aborted (a rolled-back
> +             * subtransaction of the inserter), remember that, as callers like
> +             * rewriteheap.c rely on HEAP_XMAX_INVALID to tell a live tuple
> +             * from an updated one.
> +             */
> +            if (!(tuple->t_infomask & HEAP_XMAX_INVALID) &&
> +                !(tuple->t_infomask & HEAP_XMAX_IS_MULTI) &&
> +                !HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask) &&
> +                !HeapTupleHeaderIsOnlyLocked(tuple))
> +            {
> +                TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple);
> +
> +                /*
> +                 * xmax == xmin: xmax is in progress too, nothing to do.
> +                 * The clog is only a cheap filter here: a running xmax is not
> +                 * aborted there.  An abort reaches the clog before the xid
> +                 * leaves the proc array, and the new version of the tuple,
> +                 * whose xmin is this xmax, is in progress until then.  So the
> +                 * proc array must confirm it, as the note at the top of this
> +                 * file asks.
> +                 */
> +                if (!TransactionIdEquals(xmax, HeapTupleHeaderGetRawXmin(tuple)) &&
> +                    TransactionIdDidAbort(xmax) &&
> +                    !TransactionIdIsInProgress(xmax))
> +                    SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,
> +                                InvalidTransactionId);
> +            }

FWIW, this does not feel right.  Your patch is a gathering of
everybody's suggestions that have been fed to an LLM, without thinking
through the problem, resulting of a pile of changes which was itself a
pile of changes.  I'd suggest to take three deep breaths and not rely
on an LLM to try to think through all the potential holes that any of
the previous suggestions have.  Piling workarounds on top of
workarounds is no good for these code paths.  They just become more
complicated to think about long-term.
--
Michael

Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Manu <manuelreyesbravo@gmail.com>
Дата:
Hi Michael,

On Tue, Oct 27, 2026 at 08:34:41AM +0900, Michael Paquier wrote:
> Piling workarounds on top of workarounds is no good for these code
> paths.  They just become more complicated to think about long-term.

Fair enough. The patch grew by adding each condition as it came up in
the thread, not from a clear account of when that hint is safe to
set, and Zhijie's race shows how fragile that is. I'll withdraw it.
Your commit fixes the reported case. If I come back to the second
spot, it will start from that account, not from another variant.

Manu


Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction

От:
Manu <manuelreyesbravo@gmail.com>
Дата:
Вложения:
Hi Michael,

When I withdrew the patch, I said that if I came back to this, it
would start from an account of the problem and not from another
variant.  Here it is.

There is a case that no hint bit can prevent: the pg_class row of a
table committed long ago is lost when the subtransaction that
updated it aborts while VACUUM FULL pg_class is between the old and
the new version.  The attached script reproduces it with plain SQL,
using the progress view to time the ROLLBACK TO.  It loses the row
in every round on unmodified master, REL_18_STABLE and
REL_14_STABLE.

When the scan reads the old version, its updater is still running,
so there is nothing to set yet.  heapam_relation_copy_for_cluster()
is right to find the new version dead, as its inserter aborted.
What is wrong is what rewrite_heap_dead_tuple() concludes from that:
that the old version is dead too, which only holds if the update
committed.  pruneheap.c draws that conclusion only from a
RECENTLY_DEAD tuple.

So instead of one more condition in rewrite_heap_dead_tuple(), the
fix removes it: dead tuples are no longer passed to rewriteheap.c,
and a tuple left waiting for a version that was never written is
resolved in end_heap_rewrite(), once the outcome of its update is
final.  The code change removes more than it adds (71+, 77-); the
tests are a separate patch.  I can post both if you want to look.

Manu
FAQ