Re: libpq compression

Поиск
Список
Период
Сортировка
От Andrey Borodin
Тема Re: libpq compression
Дата
Msg-id 799F4D34-7883-431C-9021-4B74120D6826@yandex-team.ru
обсуждение исходный текст
Ответ на Re: libpq compression  (Peter Eisentraut <peter.eisentraut@enterprisedb.com>)
Ответы Re: libpq compression
Список pgsql-hackers

> 6 нояб. 2020 г., в 00:22, Peter Eisentraut <peter.eisentraut@enterprisedb.com> написал(а):
>
> On 2020-11-02 20:50, Andres Freund wrote:
>> On 2020-10-31 22:25:36 +0500, Andrey Borodin wrote:
>>> But the price of compression is 1 cpu for 500MB/s (zstd). With a
>>> 20Gbps network adapters cost of recompressing all traffic is at most
>>> ~4 cores.
>> It's not quite that simple, because presumably each connection is going
>> to be handled by one core at a time in the pooler. So it's easy to slow
>> down peak throughput if you also have to deal with TLS etc.
>
> Also, current deployments of connection poolers use rather small machine sizes.  Telling users you need 4 more cores
perinstance now to decompress and recompress all the traffic doesn't seem very attractive. Also, it's not unheard of to
havemore than one layer of connection pooling.  With that, this whole design sounds a bit like a heat-generation
system.;-) 

User should ensure good bandwidth between pooler and DB. At least they must be within one availability zone. This makes
compressionbetween pooler and DB unnecessary. 
Cross-datacenter traffic is many times more expensive.

I agree that switching between compression levels (including turning it off) seems like nice feature. But
1. Scope of its usefulness is an order of magnitude smaller than compression of the whole connection.
2. Protocol for this feature is significantly more complicated.
3. Restarted compression is much less efficient and effective.

Can we design a protocol so that this feature may be implemented in future, currently focusing on getting things
compressed?Are there any drawbacks in this approach? 

Best regards, Andrey Borodin.


В списке pgsql-hackers по дате отправления:

Предыдущее
От: Konstantin Knizhnik
Дата:
Сообщение: Re: libpq compression
Следующее
От: Michael Paquier
Дата:
Сообщение: Refactor MD5 implementations and switch to EVP for OpenSSL