Hi all,
Toast compression is supported for LZ4, and thanks to the refactoring
work done with compression methods assigned to an attribute, adding
support for more methods is straight-forward, as long as we don't
support more than 4 methods as the compression ID is stored within the
first 2 bits of the raw length.
Do we have an argument against supporting zstd for this stuff?
Zstandard compresses a bit more than LZ4 at the cost of some extra
CPU, outclassing easily pglz, but those facts are known, and zstd has
benefits over LZ4 when one is ready to pay more CPU for the extra
compression.
It took me a couple of hours to get that done. I have not added any
tests for pg_dump or cross-checks with the default compressions as
this is basically what compression.sql already does, so this patch
includes a minimum to look after the compression, decompression and
slice decompression. Another thing is that the errors generated by
SET default_toast_compression make the output generated
build-dependent, and that becomes annoying once there is more than one
compression option. The attached removes those cases for simplicity,
and perhaps we'd better remove from compression.sql all the LZ4-only
tests. ZSTD_decompress() does not allow the use of a destination
buffer lower than the full decompressed size, but similarly to base
backups, streams seem to handle the case of slices fine.
Thoughts?
--
Michael