Hi!
I’ve fixed an issue with compression level parsing in this PR https://github.com/postgrespro/libpq_compression/pull/4
Also, did a couple of pgbenchmarks to measure database resource usage with different compression levels.
Firstly, I measured the bidirectional compression scenario, i.e. database had to do both compression and decompression:
Database setup:
pgbench "host=xxx dbname=xxx port=5432 user=xxx” -i -s 500
Test run:
pgbench "host=xxx dbname=xxx port=5432 user=xxx compression=zstd:(1/3/5/7/9/11/13/15/17/19/20)" --builtin tpcb-like -t
50--jobs=64 --client=700
When using bidirectional compression, Postgres resource usage correlates with the selected compression level. For
example,here is the Postgresql application memory usage:
No compression - 1.2 GiB
ZSTD
zstd:1 - 1.4 GiB
zstd:7 - 4.0 GiB
zstd:13 - 17.7 GiB
zstd:19 - 56.3 GiB
zstd:20 - 109.8 GiB - did not succeed
zstd:21, zstd:22 > 140 GiB
Postgres process crashes (out of memory)
ZLIB
zlib:1 - 1.35 GiB
zlib:5 - 1.35 GiB
zlib:9 - 1.35 GiB
Full report with CPU/Memory/Network consumption graph is available here:
https://docs.google.com/document/d/1qakHcsabZhV70GfSEOjFxmlUDBe21p7DRoPrDPAjKNg
Then, I’ve disabled the compression for the backend and decompression for the frontend
and measured the resource usage for single-directional compression scenario (frontend compression only, backend
decompressiononly):
ZSTD
For all ZSTD compression levels, database host resource usage was roughly the same, except the Committed Memory
(Committed_AS):
no compression - 44.4 GiB
zstd:1 - 45.0 GiB
zstd:3 - 46.1 GiB
zstd:5 - 46.1 GiB
zstd:7 - 46.0 GiB
zstd:9 - 46.0 GiB
zstd:11 - 47.4 GiB
zstd:13 - 47.4 GiB
zstd:15 - 47.4 GiB
zstd:17 - 50.3 GiB
zstd:19 - 50.1 GiB
zstd:20 - 66.8 GiB
zstd:21 - 88.7 GiB
zstd:22 - 123.9 GiB
ZLIB
For all ZLIB compression level, database host resource usage was roughly the same.
Full report with CPU/Memory/Network consumption graph is available here:
https://docs.google.com/document/d/1gI7c3_YvcL5-PzeK65P0pIY-4BI9KBDwlfPpGhYxrNg
To sum up, there is actually almost no difference when decompressing the different compression levels, except the
Committed_ASsize.
—
Daniil Zakhlystov