> 17 июня 2021 г., в 06:19, Michael Paquier <michael@paquier.xyz> написал(а):
>
> On Wed, Jun 16, 2021 at 01:17:26PM +0500, Andrey Borodin wrote:
>> I agree that allowing just lz4 - is already a huge step ahead.
>
> Yeah, I am tempted to just add LZ4 as a first step as the patch
> footprint would be minimal, and we could come back to zstd once we
> have more feedback from the field, if that's necessary. As said
> upthread, we have more flexibility with WAL than for the relation
> data.
>
>> But I'd suggest supporting zstd as well. Currently we only compress
>> 8Kb chunks and zstd had no runaway to fully unwrap it's potential.
>> In WAL-G we observed ~3x improvement in network utilisation when
>> switched from lz4 to zstd in WAL archive compression.
>
> You mean full segments here, right? This has no need to be in core,
> except if we want to add more compression options to pg_receivewal and
> its friends? That would be a nice addition, saying that.
Konstantin, Daniil and Justin are working on compressing libpq [0]. That would make walsender compress WAL
automatically.
And we (at least I and Dan) are inclined to work on compressing on-disk WAL as soon as libpq compression will be
committed.
Zstd is much better at compressing long data sequences than lz4. I'm sure we will need such codec in core one day.
>> BTW we could get rid of whole hole-in-a-page thing if we would set
>> lz4 as default. This could simplify FPI code.
>
> Why would we do that? We still need to support pglz as fallback if a
> platform does not have LZ4, no?
Because compressing sequence of zeroes is cheap, even for pglz. But we still need to support 'no compression at all',
thismode benefits from hole-in-a-page a lot. Until we send and store WAL uncompressed, of cause.
Thanks!
Best regards, Andrey Borodin.
[0]https://www.postgresql.org/message-id/flat/aad16e41-b3f9-e89d-fa57-fb4c694bec25%40postgrespro.ru