Обсуждение: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Heikki Linnakangas
Дата:
pgcrypto uses the old, deprecated, "low-level" functions for symmetric encryption, with algorithm-specific functions like AES_ecb_encrypt(), DES_ecb3_encrypt() and so forth. The recommended new API is the so-called EVP API, which has functions for initializing a "context" using a specific algorithm, and then that context is passed around to EVP_Encrypt*/Decrypt* functions. The EVP API has been around for ages, at least since OpenSSL 0.9.6. We should switch to the new API. Aside from being nicer, the low-level functions don't (necessarily) use hardware acceleration, while the EVP functions do. I could see a significant boost to pgcrypto AES encryption on my laptop, which has an Intel CPU that supports the special AES-NI instructions. That said, AES encryption is pretty fast anyway, so you need very large inputs to see any difference and it's actually pretty difficult to come up with a test case where the gains are not lost in the noise of e.g. toasting/detoasting the data. Nevertheless, it's a nice bonus. Test case is attached (aes-speedtest.sql). It runs in about 1.7s with the old API, and 1.3s with the new API. The real reason I started digging this, though, is that Pivotal was trying to use the FIPS-validated version of OpenSSL with PostgreSQL, and it turns out that the low-level APIs are disabled in "FIPS mode", and trip an assertion inside OpenSSL (that changed some time between 0.9.8 and 1.0.2, not sure when exactly). Switching to the EVP functions will avoid that problem. There is obviously a lot more you'd need to do before you could actually FIPS-certify PostgreSQL and pgcrypto, but this is one unnecessary hurdle. There was prior discussion on the EVP API in this old thread from 2007: http://www.postgresql.org/message-id/flat/46A5E284.7030402@sun.com#46A5E284.7030402@sun.com In short, pgcrypto actually used to use the EVP functions, but was changed to *not* use them, because in older versions of OpenSSL, some key lengths and/or padding options that pgcrypto supports were not supported by the EVP API. That was fixed in OpenSSL 0.9.7, however. The consensus in 2007 was that we could drop support for OpenSSL 0.9.6 and below, so that should definitely be OK by now, if we haven't already done that elsewhere in the code. Any objections to the attached two patches? - Heikki
Вложения
On 10/05/2015 06:02 AM, Heikki Linnakangas wrote: > There was prior discussion on the EVP API in this old thread from 2007: > http://www.postgresql.org/message-id/flat/46A5E284.7030402@sun.com#46A5E284.7030402@sun.com > > > In short, pgcrypto actually used to use the EVP functions, but was > changed to *not* use them, because in older versions of OpenSSL, some > key lengths and/or padding options that pgcrypto supports were not > supported by the EVP API. That was fixed in OpenSSL 0.9.7, however. The > consensus in 2007 was that we could drop support for OpenSSL 0.9.6 and > below, so that should definitely be OK by now, if we haven't already > done that elsewhere in the code. > > Any objections to the attached two patches? I haven't studied that patches themselves yet, but +1 for the concept. Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open Source Development
Re: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Alvaro Herrera
Дата:
Heikki Linnakangas wrote: > In short, pgcrypto actually used to use the EVP functions, but was changed > to *not* use them, because in older versions of OpenSSL, some key lengths > and/or padding options that pgcrypto supports were not supported by the EVP > API. That was fixed in OpenSSL 0.9.7, however. The consensus in 2007 was > that we could drop support for OpenSSL 0.9.6 and below, so that should > definitely be OK by now, if we haven't already done that elsewhere in the > code. I think we already effectively dropped support for < 0.9.7 with the renegotiation fixes; see https://www.postgresql.org/message-id/20130712203252.GH29206%40eldon.alvh.no-ip.org -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Andres Freund
Дата:
On 2015-10-05 12:16:05 -0300, Alvaro Herrera wrote: > Heikki Linnakangas wrote: > > > In short, pgcrypto actually used to use the EVP functions, but was changed > > to *not* use them, because in older versions of OpenSSL, some key lengths > > and/or padding options that pgcrypto supports were not supported by the EVP > > API. That was fixed in OpenSSL 0.9.7, however. The consensus in 2007 was > > that we could drop support for OpenSSL 0.9.6 and below, so that should > > definitely be OK by now, if we haven't already done that elsewhere in the > > code. > > I think we already effectively dropped support for < 0.9.7 with the > renegotiation fixes; see > https://www.postgresql.org/message-id/20130712203252.GH29206%40eldon.alvh.no-ip.org 9.5+ do again then :P But more seriously: Given the upstream support policies from https://www.openssl.org/policies/releasestrat.html : " Support for version 0.9.8 will cease on 2015-12-31. No further releases of 0.9.8 will be made after that date. Security fixesonly will be applied to 0.9.8 until then. Support for version 1.0.0 will cease on 2015-12-31. No further releases of 1.0.0 will be made after that date. Security fixesonly will be applied to 1.0.0 until then. We may designate a release as a Long Term Support (LTS) release. LTS releases will be supported for at least five years and we will specify one at least every four years. Non-LTS releases will be supported for at least two years. " and the amount of security fixes regularly required for openssl, I don't think we'd do anybody a favor by trying to continue supporting older versions for a long while. Note that openssl's security releases are denoted by a letter after the numeric version, not by the last digit. 0.9.7 was released 30 Dec 2002. Greetings, Andres Freund
Re: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Alvaro Herrera
Дата:
Andres Freund wrote: > But more seriously: Given the upstream support policies from > https://www.openssl.org/policies/releasestrat.html : > " > Support for version 0.9.8 will cease on 2015-12-31. No further releases of 0.9.8 will be made after that date. Securityfixes only will be applied to 0.9.8 until then. > Support for version 1.0.0 will cease on 2015-12-31. No further releases of 1.0.0 will be made after that date. Securityfixes only will be applied to 1.0.0 until then. > > We may designate a release as a Long Term Support (LTS) release. LTS > releases will be supported for at least five years and we will specify > one at least every four years. Non-LTS releases will be supported for at > least two years. > " > and the amount of security fixes regularly required for openssl, I don't > think we'd do anybody a favor by trying to continue supporting older > versions for a long while. > > Note that openssl's security releases are denoted by a letter after the > numeric version, not by the last digit. 0.9.7 was released 30 Dec 2002. Yeah. Last of the 0.9.7 line (0.9.7m) was in 2007: commit 10626fac1569ea37839c37b105681cd08dbe6658 Author: cvs2svn <cvs2svn> AuthorDate: Fri Feb 23 12:49:10 2007 +0000 CommitDate: Fri Feb 23 12:49:10 2007 +0000 This commit was manufactured by cvs2svn to create tag 'OpenSSL_0_9_7m'. Current 0.9.8 is 0.9.8zg, in June this year: commit 0823ddc56e9aaa1de6c4f57bb45457d5eeca404d Author: Matt Caswell <matt@openssl.org> AuthorDate: Thu Jun 11 15:20:22 2015 +0100 CommitDate: Thu Jun 11 15:20:22 2015 +0100 Prepare for 0.9.8zg release Reviewed-by: Stephen Henson <steve@openssl.org> -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Heikki Linnakangas
Дата:
Committed this patch now. - Heikki
Re: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Christoph Berg
Дата:
Re: Heikki Linnakangas 2016-10-17 <07ebd878-ff09-72d5-7df7-f7fde7b83824@iki.fi> > Committed this patch now. Hi, I've just taken up work again on PG 10 on Debian unstable. With openssl 1.1.0c-2, pgcrypto errors out with: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=.-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security-I/usr/include/mit-krb5 -fPIC -pie -fno-omit-frame-pointer -fpic -I. -I/<<PKGBUILDDIR>>/build/../contrib/pgcrypto-I../../src/include -I/<<PKGBUILDDIR>>/build/../src/include -Wdate-time -D_FORTIFY_SOURCE=2-D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/tcl8.6 -c -o openssl.o /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c:253:17: error: field 'evp_ctx' has incomplete type EVP_CIPHER_CTX evp_ctx; ^~~~~~~ /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c: In function 'bf_check_supported_key_len': /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c:373:17: error: storage size of 'evp_ctx' isn't known EVP_CIPHER_CTXevp_ctx; ^~~~~~~ /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c:373:17: warning: unused variable 'evp_ctx' [-Wunused-variable] <builtin>: recipe for target 'openssl.o' failed Reverting 5ff4a67f63fd6d3eb01ff9707d4674ed54a89f3b fixes compilation. (9.6 is fine.) Christoph
Re: Use EVP API pgcrypto encryption, dropping support for OpenSSL 0.9.6 and older
От
Heikki Linnakangas
Дата:
On 12/08/2016 05:51 PM, Christoph Berg wrote: > Re: Heikki Linnakangas 2016-10-17 <07ebd878-ff09-72d5-7df7-f7fde7b83824@iki.fi> >> Committed this patch now. > > Hi, > > I've just taken up work again on PG 10 on Debian unstable. > > With openssl 1.1.0c-2, pgcrypto errors out with: Yeah, sorry about that. It's already been discussed at https://www.postgresql.org/message-id/20161201014826.ic72tfkahmevpwz7%40alap3.anarazel.de. - Heikki
Re: [HACKERS] Use EVP API pgcrypto encryption, dropping support forOpenSSL 0.9.6 and older
От
Christoph Berg
Дата:
Re: Heikki Linnakangas 2016-10-17 <07ebd878-ff09-72d5-7df7-f7fde7b83824@iki.fi> > Committed this patch now. Hi, I've just taken up work again on PG 10 on Debian unstable. With openssl 1.1.0c-2, pgcrypto errors out with: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=.-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security-I/usr/include/mit-krb5 -fPIC -pie -fno-omit-frame-pointer -fpic -I. -I/<<PKGBUILDDIR>>/build/../contrib/pgcrypto-I../../src/include -I/<<PKGBUILDDIR>>/build/../src/include -Wdate-time -D_FORTIFY_SOURCE=2-D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/tcl8.6 -c -o openssl.o /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c:253:17: error: field 'evp_ctx' has incomplete type EVP_CIPHER_CTX evp_ctx; ^~~~~~~ /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c: In function 'bf_check_supported_key_len': /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c:373:17: error: storage size of 'evp_ctx' isn't known EVP_CIPHER_CTXevp_ctx; ^~~~~~~ /<<PKGBUILDDIR>>/build/../contrib/pgcrypto/openssl.c:373:17: warning: unused variable 'evp_ctx' [-Wunused-variable] <builtin>: recipe for target 'openssl.o' failed Reverting 5ff4a67f63fd6d3eb01ff9707d4674ed54a89f3b fixes compilation. (9.6 is fine.) Christoph