Re: libpq compression

Hi,

On 2019-02-08 23:38:12 +0100, Tomas Vondra wrote:
> On 2/8/19 11:10 PM, Konstantin Knizhnik wrote:
> > Does it mean that it is necessary to support multiple compression
> > algorithms and make it possible to perform switch between them at
> > runtime?
> 
> IMHO the negotiation should happen at connection time, i.e. the server
> should support connections compressed by different algorithms.

Exactly. And support client libraries with support for different
compression algorithms. We have large forward and backward compatibility
with libpq and even moreso with the wire protocol, we therefore
shouldn't require exactly matching client / server versions or
compilation options.


> Not sure if that's what you mean by runtime.

Same.


> AFAICS this is quite close to how negotiation of encryption algorithms
> works, in TLS and so on. Client specifies supported algorithms, server
> compares that to list of supported algorithms, deduces the encryption
> algorithm and notifies the client.
> 
> To allow fall-back to uncompressed connection, use "none" as algorithm.
> If there's no common algorithm, fail.

I'm somewhat inclined to think that not compressing is
preferrable. There's some reason to think that there's some
cryptographical issues around compression, therefore it could make a ton
of sense for specific servers to disable compression centrally.


> > Right now compression algorithm is linked statically.
> > Negotiation of compression type is currently performed but it only
> > checks that server and client are implementing the same algorithm and
> > disables compression if it is not true.

> I don't think we should automatically fall-back to disabled compression,
> when a client specifies compression algorithm.

Why?


> > If we are going to support multiple compression algorithms, do we need
> > dynamic loading of correspondent compression libraries or static linking
> > is ok? In case of dynamic linking we need to somehow specify information
> > about available compression algorithms.
> > Some special subdirectory for them so that I can traverse this directory
> > and try to load correspondent libraries?
> > 
> > Only I find it too complicated for the addressed problem?
> > 
> 
> I don't think we need dynamic algorithms v1, but IMHO it'd be pretty
> simple to do - just add a shared_preload_library which registers it in a
> list in memory.

I personally don't think that's something we need to support. There's a
lot of issues around naming registries that need to synchronized between
client/server.

Greetings,

Andres Freund