Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

Hi,

On 2021-05-21 11:01:03 -0400, Tom Lane wrote:
> It was a good thing I went through this code, too, because I noticed
> one serious bug (attcompression not checked in equalTupleDescs) and
> another thing that looks like a bug:

Grepping for attcompression while trying to understand the issue Tom
reported I found a substantial, but transient, memory leak:

During VACUUM FULL reform_and_rewrite_tuple() detoasts the old value if
it was compressed with a different method, while in
TopTransactionContext. There's nothing freeing that until
TopTransactionContext ends - obviously not great for a large relation
being VACUUM FULLed.

SET default_toast_compression = 'lz4';
DROP TABLE IF EXISTS wide CASCADE;
CREATE TABLE wide(data text not null);
INSERT INTO wide(data) SELECT random() || (SELECT string_agg(i::text, '') data FROM generate_series(1, 100000) g(i))
FROMgenerate_series(1, 1000);
 

\c

SET client_min_messages = 'log';
SET log_statement_stats = on;
VACUUM FULL wide;
...
DETAIL:  ! system usage stats:
!    0.836638 s user, 0.375344 s system, 1.268705 s elapsed
!    [2.502369 s user, 0.961681 s system total]
!    18052 kB max resident size
!    0/1789088 [0/3530048] filesystem blocks in/out
!    0/277 [0/205655] page faults/reclaims, 0 [0] swaps
!    0 [0] signals rcvd, 0/0 [0/0] messages rcvd/sent
!    22/1 [55/6] voluntary/involuntary context switches
LOCATION:  ShowUsage, postgres.c:4886
VACUUM
Time: 1269.029 ms (00:01.269)

\c
ALTER TABLE wide ALTER COLUMN data SET COMPRESSION pglz;
SET client_min_messages = 'log';
SET log_statement_stats = on;
VACUUM FULL wide;
...
DETAIL:  ! system usage stats:
!    19.816867 s user, 0.493233 s system, 20.320711 s elapsed
!    [19.835995 s user, 0.493233 s system total]
!    491588 kB max resident size
!    0/656032 [0/656048] filesystem blocks in/out
!    0/287363 [0/287953] page faults/reclaims, 0 [0] swaps
!    0 [0] signals rcvd, 0/0 [0/0] messages rcvd/sent
!    1/24 [13/26] voluntary/involuntary context switches

Note the drastically different "max resident size". This is with huge
pages (removing s_b from RSS), but it's visible even without.


Random fun note:
time for VACUUM FULL wide with recompression:
pglz->lz4: 3.2s
lz4->pglz: 20.3s

Greetings,

Andres Freund