Re: libpq compression

On Mon, Jun 25, 2012 at 4:04 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Fri, Jun 22, 2012 at 12:38 PM, Euler Taveira <euler@timbira.com> wrote:
>> On 20-06-2012 17:40, Marko Kreen wrote:
>>> On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug <fgp@phlo.org> wrote:
>>>> I'm starting to think that relying on SSL/TLS for compression of
>>>> unencrypted connections might not be such a good idea after all. We'd
>>>> be using the protocol in a way it quite clearly never was intended to
>>>> be used...
>>>
>>> Maybe, but what is the argument that we should avoid
>>> on encryption+compression at the same time?
>>>
>>> AES is quite lightweight compared to compression, so should
>>> be no problem in situations where you care about compression.
>>>
>> If we could solve compression problem without AES that will turn things
>> easier. Compression-only via encryption is a weird manner to solve the problem
>> in the user's POV.
>>
>>> RSA is noticeable, but only for short connections.
>>> Thus easily solvable with connection pooling.
>>>
>> RSA overhead is not the main problem. SSL/TLS setup is.
>>
>>> And for really special compression needs you can always
>>> create a UDF that does custom compression for you.
>>>
>> You have to own the code to modify it; it is not always an option.
>>
>>> So what exactly is the situation we need to solve
>>> with postgres-specific protocol compression?
>>>
>> Compression only support. Why do I need to set up SSL/TLS just for compression?
>>
>> IMHO SSL/TLS use is no different from relying in another library to handle
>> compression for the protocol and more it is compression-specific. That way, we
>> could implement another algorithms in such library without needing to modify
>> libpq code. Using SSL/TLS you are bounded by what SSL/TLS software products
>> decide to use as compression algorithms. I'll be happy to maintain the code
>> iif it is postgres-specific or even as close as possible to core.
>
> I guess my feeling on this is that, so far as I can see, supporting
> compression via OpenSSL involves work and trade-offs, and supporting
> it without depending on OpenSSL also involves work, and trade-offs.

Nice summary :)

> So it's not real evident to me that we should prefer one to the other
> on general principle.  It seems to me that a lot might come down to
> performance.  If someone can demonstrate that using an external
> library involves gets significantly better compression, chews up
> significantly less CPU time, and/or is significantly less code than
> supporting this via OpenSSL, then maybe we ought to consider it.

I think we should, yes. But as you say, we need to know first. It's
also a question of if one of these compression schemes are trivial
enough that we could embed the code rather than rely on it externally
- I have no idea if that's even remotely possibe, but that would move
the goalposts a bit too.

> OpenSSL is kind of an ugly piece of code, and all things being equal
> depending on it more heavily would not be my first choice.

Indeed.

But we should really stop saying "rely on openssl" and start saying
"rely on the ssl library". There are other SSL libraries which are not
quite so ugly, which we should eventually support.

That said, it's *still* a bit ugly to rely on the SSL library for this, IMO.


> On the other hand, this does not seem to me to be a situation where we
> should accept a patch to use an external library just because someone
> takes the time to write one, because there is also a non-trivial cost
> for the entire project to depending on more things; or if the
> compression code gets put into the backend, then there's a cost to us
> to maintain that code inside our source base.  So I think we really
> need someone to try this both ways and compare.  Right now it seems
> like we're mostly speculating on how well either approach would work,
> which is not as good as having some experimental results.  If, for
> example, someone can demonstrate that an awesomebsdlz compresses 10x
> as fast as OpenSSL...  that'd be pretty compelling.

Or that it takes less code/generates cleaner code...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/