[BUGS] BUG #14686: OpenSSL 1.1.0+ breaks PostgreSQL's sslcompressionassumption, defaults to SSL_OP_NO_COMPRESSION
От | greenreaper@hotmail.com |
---|---|
Тема | [BUGS] BUG #14686: OpenSSL 1.1.0+ breaks PostgreSQL's sslcompressionassumption, defaults to SSL_OP_NO_COMPRESSION |
Дата | |
Msg-id | 20170603020554.1436.89380@wrigleys.postgresql.org обсуждение исходный текст |
Ответы |
Re: [BUGS] BUG #14686: OpenSSL 1.1.0+ breaks PostgreSQL's sslcompression assumption, defaults to SSL_OP_NO_COMPRESSION
|
Список | pgsql-bugs |
The following bug has been logged on the website: Bug reference: 14686 Logged by: Laurence Parry Email address: greenreaper@hotmail.com PostgreSQL version: 9.6.3 Operating system: Debian 9 'stretch', Linux 4.9.25, libc 2.24 Description: PostgreSQL is documented to compress data over SSL connections by default where possible: https://www.postgresql.org/docs/9.6/static/libpq-connect.html#LIBPQ-CONNECT-SSLCOMPRESSION PostgreSQL assumes that OpenSSL 0.9.8+ will compress with zlib if it can. However, OpenSSL 1.1.0+ - already in Manjaro, and the upcoming Debian 9 'Stretch', Fedora 26, and Tails 3.0 - sets SSL_OP_NO_COMPRESSION by default to discourage the CRIME attack: https://www.openssl.org/news/changelog.html#x7 *) CRIME protection: disable compression by default, even if OpenSSL is compiled with zlib enabled. Applications canstill enable compression by calling SSL_CTX_clear_options(ctx, SSL_OP_NO_COMPRESSION), or by using the SSL_CONFlibrary to configure compression. PostgreSQL does not anticipate this, and so fails to compress the connection even if zlib support is compiled into OpenSSL. This is not immediately obvious, unless you are watching networking graphs. Restoring compression in PostgreSQL requires patching out the new default in the depths of OpenSSL and recompiling (rather than just reconfiguring and recompiling). == Possible PostgreSQL code fix == If the connection string includes sslcompression=0, initialize_SSL() in pqlib disables SSL compression by setting SSL_OP_NO_COMPRESSION: https://doxygen.postgresql.org/fe-secure-openssl_8c_source.html#l01278 I believe there also needs to be an 'else' case to *clear* SSL_OP_NO_COMPRESSION with: SSL_clear_options(conn->ssl, SSL_OP_NO_COMPRESSION); in the case that sslcompression is unset or not '0', rather than rely on the default. PostgreSQL _could_ clear it if sslcompression is '1', rather than unset or not '0', but that would mean more people not using SSL compression. From 9.2 onwards, the documentation has 1 as the default; so many will not have set it, assuming that it will [continue to] work. Distributions will not use PostgreSQL 10 until it is released, while OpenSSL 1.1.0 is already in use with PostgreSQL 9.6 (and might be used with older versions): https://distrowatch.com/search.php?pkg=openssl&relation=similar&pkgver=1.1&distrorange=InAny#pkgsearch [In addition, PostgreSQL currently won't work as described if a distribution or end-user has made a similar change to the default in OpenSSL 1.0.x, although that seems less likely.] I therefore suggest leaving compression on by default, and back-patching a fix to 9.2 and above, where sslcompression was introduced, to clear this flag if it may have been set. The same code is in fe-secure.c rather than fe-secure-openssl.c in PostgreSQL 9.4 and below. This would be in line with the back-porting of other support for 1.1.0 in: https://www.postgresql.org/message-id/flat/20160627151604.GD1051%40msg.df7cb.de#20160627151604.GD1051@msg.df7cb.de According to a Python SSL comment: /* SSL_CTX_clear_options() and SSL_clear_options() were first added in* OpenSSL 0.9.8m but do not appear in some 0.9.9-devversions such the* 0.9.9 from "May 2008" that NetBSD 5.0 uses. */ Since SSL_OP_NO_COMPRESSION was added in 1.0.0, it should be fine to use SSL_clear_options() within the existing #ifdef. It may also help to mention in the documentation that this compression requires a version of openssl/libssl compiled with zlib[-dynamic], and potentially environment variables to enable it, such as OPENSSL_DEFAULT_ZLIB in Ubuntu 12.04 (but not 14.04+): https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/vivid/openssl/vivid/revision/95 This was suggested previously in: https://www.postgresql.org/message-id/flat/CAKwe89Cj7KQ3BZDoUXLF5KBZ8X6icKXHi2Y1mDzTut3PNrH2VA%40mail.gmail.com == Workaround: recompiling openssl == You can check whether openssl has been compiled with zlib support by checking for -DZLIB / -DZLIB_SHARED in: $ openssl version -f but either way, it will not be on by default with openssl 1.1.0+ For those wishing to fix compression on Debian stretch (or jessie, or Ubuntu 14.04+): # apt-get install build-essential # apt-get build-dep openssl From an unprivileged account: $ apt-get source openssl $ cd openssl-1.1.0f $ nano debian/rules [ replace 'no-zlib' with 'zlib' or 'zlib-dynamic' in CONFARGS ] For OpenSSL 1.1.0+ (stretch et. al), also apply the following patch to ssl/ssl_lib.c: ###START### --- ssl/ssl_lib.c.orig 2017-06-02 22:34:20.524405847 +0200 +++ ssl/ssl_lib.c 2017-06-02 22:34:36.764406166 +0200 @@ -2471,13 +2471,6 @@ * deployed might change this. */ ret->options |= SSL_OP_LEGACY_SERVER_CONNECT; - /* - * Disable compression by default to prevent CRIME. Applications can - * re-enable compression by configuring - * SSL_CTX_clear_options(ctx, SSL_OP_NO_COMPRESSION); - * or by using the SSL_CONF library. - */ - ret->options |= SSL_OP_NO_COMPRESSION; ret->tlsext_status_type = -1; ###END### $ nice dpkg-buildpackage -b -j4 -nc (note: build tests will fail if root) Then, from a privileged account [adjust for version/arch] # dpkg -I ../libssl1.1_1.1.0f-1_amd64.deb # dpkg -I ../openssl_1.1.0f-1_amd64.deb Check that ZLIB is no longer a disabled algorithm: # openssl list -disabled It should also be listed as -DZLIB and -DZLIB_SHARED in: # openssl version -f # service postgresql restart == Future thoughts == If compression is unavailable it may become a competitive issue (MySQL/MariaDB offer zlib/deflate-based compression toggled via slave_compressed_protocol), or a reason to avoid streaming physical replication - possibly logical as well, I don't have experience there. Uncompressed replication roughly triples the required bandwidth/transfer, which can translate into significant costs or performance issues. In my case, compression means I can remain within a 10TB/month budget on a server used for cascading replication. The feeling in 2012 was that transport compression isn't within PostgreSQL's wheelhouse (which is why it remains important to enable it where possible in OpenSSL): https://www.postgresql.org/message-id/flat/4FD9E70B.9040607%40timbira.com#4FD9E70B.9040607@timbira.com This is a fair argument; however, TLS 1.3 removes compression: https://tools.ietf.org/html/draft-ietf-tls-tls13-20 - "Other cryptographic improvements including the removal ofcompression and custom DHE groups..." ...and this will be included in OpensSSL 1.1.1+, likely this year: https://www.openssl.org/blog/blog/2017/05/04/tlsv1.3/ So it may become necessary to use *something* in PostgreSQL 11+, be it zlib, PGLZ ( http://paquier.xyz/postgresql-2/postgres-9-5-feature-highlight-pglz-compression-libpqcommon/ ), or another algorithm. Or specify TLS 1.2. But that would need further code changes, too - and you have to use an outdated security protocol for compression" does not sound compelling. Best regards, -- Laurence "GreenReaper" Parry - Inkbunny administrator greenreaper.co.uk - wikifur.com - flayrah.com - inkbunny.net "Eternity lies ahead of us, and behind. Have you drunk your fill?" -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
В списке pgsql-bugs по дате отправления:
Предыдущее
От: digoal@126.comДата:
Сообщение: [BUGS] BUG #14685: use ctid filter tuples will generate LOOP, very very slow