Обсуждение: Fwd: PostgreSQL: WolfSSL support

Hi,

Is anyone here interested in helping to evaluate an experimental patch
for wolfSSL support?

Attached please find a WIP patch for wolfSSL support in postgresql-12.
As a shortcut, you may find this merge request helpful:

    https://salsa.debian.org/postgresql/postgresql/-/merge_requests/4

I used Debian stable (buster) with backports enabled and preferred.

The wolfssl.patch in d/patches builds and completes all tests, as long
as libwolfssl-dev version 4.4.0+dfsg-2~bpo10+1 is installed and
patched with the included libwolfssl-dev-rename-types.patch.

You can do so as root with:

    cd /usr/include/wolfssl
    patch -p1 < libwolfssl-dev-rename-types.patch

Patching the library was easier than resolving type conflicts for
twenty-five files. An attempt was made but resulted in failing tests.

The offending types are called 'ValidateDate' and 'Hash'. They do not
seem to be part of the wolfSSL ABI.

The patch operates with the following caveats:

1. DH parameters are not currently loaded from a database-internal PEM
certificate. The function OBJ_find_sigid_algs is not available. The
security implications should be discussed with a cryptographer.

2. The contrib module pgcrypto was not compiled with OpenSSL support
and currently offers only native algorithms. wolfSSL's compatibility
support for OpenSSL's EVP interface is incomplete and offers only a
few algorithms. The module should work directly with wolfCrypt.

3. The error reporting in wolfSSL_set_fd seems to be different from
OpenSSL. I could not locate SSLerr and decided to return BAD_FUNC_ARG.
That is what the routine being mimicked does in wolfSSL. If you see an
SSL connection error, it may be wise to simply remove these two
statements in src/interfaces/libpq/fe-secure-openssl.c:

    ret = BAD_FUNC_ARG;

Unsupported functions or features can probably be replaced with
wolfSSL's or wolfCrypt's native interfaces. The company may be happy
to assist.

The patch includes modifications toward missing goals. Some parts
modify code, for example in util/pgpcrypto, that is not actually called.

Please note that the wolfSSL team prefers the styling of their brand
to be capitalized as recorded in this sentence. Thank you!

Kind regards
Felix Lechner

On 2020-06-27 00:33, Felix Lechner wrote:
> Is anyone here interested in helping to evaluate an experimental patch
> for wolfSSL support?

What would be the advantage of using wolfSSL over OpenSSL?

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Peter Eisentraut
> What would be the advantage of using wolfSSL over OpenSSL?

Avoiding the OpenSSL-vs-GPL linkage problem with readline.

Christoph

On Sat, Jun 27, 2020 at 02:50:27PM +0200, Christoph Berg wrote:
> Re: Peter Eisentraut
> > What would be the advantage of using wolfSSL over OpenSSL?
> 
> Avoiding the OpenSSL-vs-GPL linkage problem with readline.

Uh, wolfSSL is GPL2:

    https://www.wolfssl.com/license/

Not sure why we would want to lock Postgres into a GPL-style
requirement.  As I understand it, we don't normally ship readline or
openssl.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

Bruce Momjian <bruce@momjian.us> writes:
> On Sat, Jun 27, 2020 at 02:50:27PM +0200, Christoph Berg wrote:
>> Re: Peter Eisentraut
>>> What would be the advantage of using wolfSSL over OpenSSL?

>> Avoiding the OpenSSL-vs-GPL linkage problem with readline.

> Uh, wolfSSL is GPL2:
>     https://www.wolfssl.com/license/

Readline is GPLv3+ (according to Red Hat's labeling of that package
anyway, didn't check the source).  So they'd be compatible, while
openssl's license is nominally incompatible with GPL.  As I recall,
Debian jumps through some silly hoops to pretend that they're not
using openssl and readline at the same time with Postgres, so I
can definitely understand Christoph's interest in an alternative.

However, judging from the caveats mentioned in the initial message,
my inclination would be to wait awhile for wolfSSL to mature.

In any case, the patch as written seems to *remove* the option
to compile PG with OpenSSL.  The chance of it being accepted that
way is indistinguishable from zero.  We've made some efforts towards
separating out the openssl-specific bits, so the shape I'd expect
from a patch like this is to add some parallel wolfssl-specific bits.
There probably are more such bits to separate, but this isn't the
way to proceed.

            regards, tom lane

On Sat, Jun 27, 2020 at 10:56:46AM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Sat, Jun 27, 2020 at 02:50:27PM +0200, Christoph Berg wrote:
> >> Re: Peter Eisentraut
> >>> What would be the advantage of using wolfSSL over OpenSSL?
> 
> >> Avoiding the OpenSSL-vs-GPL linkage problem with readline.
> 
> > Uh, wolfSSL is GPL2:
> >     https://www.wolfssl.com/license/
> 
> Readline is GPLv3+ (according to Red Hat's labeling of that package
> anyway, didn't check the source).  So they'd be compatible, while
> openssl's license is nominally incompatible with GPL.  As I recall,
> Debian jumps through some silly hoops to pretend that they're not
> using openssl and readline at the same time with Postgres, so I
> can definitely understand Christoph's interest in an alternative.
> 
> However, judging from the caveats mentioned in the initial message,
> my inclination would be to wait awhile for wolfSSL to mature.

Also, wolfSSL is developed by a company and dual GPL/commerical
licenses, so it seems like a mismatch to me.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

Bruce Momjian <bruce@momjian.us> writes:
> Also, wolfSSL is developed by a company and dual GPL/commerical
> licenses, so it seems like a mismatch to me.

Yeah, that's definitely a factor behind my disinterest in
making wolfSSL be the only alternative.  However, as long as
it's available on GPL terms, I don't see a problem with it
being one alternative.

            regards, tom lane

On Sat, Jun 27, 2020 at 11:16:26AM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Also, wolfSSL is developed by a company and dual GPL/commerical
> > licenses, so it seems like a mismatch to me.
> 
> Yeah, that's definitely a factor behind my disinterest in
> making wolfSSL be the only alternative.  However, as long as
> it's available on GPL terms, I don't see a problem with it
> being one alternative.

Yeah, I guess it depends on how much Postgres code it takes to support
it.  Company-developed open source stuff usually goes into pay mode once
it gets popular, so I am not super-excited to be going in this
direction.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

Re: Tom Lane
> In any case, the patch as written seems to *remove* the option
> to compile PG with OpenSSL.

It's a WIP patch, meant to see if it works at all. Of course OpenSSL
would stay as the default option.

Christoph

Christoph Berg <myon@debian.org> writes:
> It's a WIP patch, meant to see if it works at all. Of course OpenSSL
> would stay as the default option.

Fair enough.  One thing that struck me as I looked at it was that
most of the #include hackery seemed unnecessary.  The configure
script could add -I/usr/include/wolfssl (or wherever those files
are) to CPPFLAGS instead of touching all those #includes.

            regards, tom lane

Re: Jonah H. Harris
> Somewhere, I recall seeing an open-source OpenSSL compatibility wrapper for
> WolfSSL. Assuming that still exists, this patch seems entirely unnecessary.

Unless you actually tried.

Christoph

Re: Ranier Vilela
> I'm curious, how do you intend to solve a linking problem with
> OpenSSL-vs-GPL-readline, with another GPL product?
> WolfSSL, will provide a commercial license for PostgreSQL?

It's replacing OpenSSL+GPL with GPL+GPL.

> Isn't LIbreSSL a better alternative?

I don't know.

Christoph

On Sat, Jun 27, 2020 at 04:22:51PM -0300, Ranier Vilela wrote:
> Em sáb., 27 de jun. de 2020 às 09:50, Christoph Berg <myon@debian.org>
> escreveu:
> 
>     Re: Peter Eisentraut
>     > What would be the advantage of using wolfSSL over OpenSSL?
> 
>     Avoiding the OpenSSL-vs-GPL linkage problem with readline.
> 
> I'm curious, how do you intend to solve a linking problem with
> OpenSSL-vs-GPL-readline, with another GPL product?

I assume you can use wolfSSL as long as the result is GPL, which is the
same requirement libreadline causes for Postgres, particularly if
Postgres is statically linked to libreadline.

> WolfSSL, will provide a commercial license for PostgreSQL?
> Isn't LIbreSSL a better alternative?

Seems it might be.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

On Sat, Jun 27, 2020 at 06:14:21PM -0300, Ranier Vilela wrote:
> Em sáb., 27 de jun. de 2020 às 16:40, Bruce Momjian <bruce@momjian.us>
> escreveu:
> 
>     On Sat, Jun 27, 2020 at 04:22:51PM -0300, Ranier Vilela wrote:
>     > Em sáb., 27 de jun. de 2020 às 09:50, Christoph Berg <myon@debian.org>
>     > escreveu:
>     >
>     >     Re: Peter Eisentraut
>     >     > What would be the advantage of using wolfSSL over OpenSSL?
>     >
>     >     Avoiding the OpenSSL-vs-GPL linkage problem with readline.
>     >
>     > I'm curious, how do you intend to solve a linking problem with
>     > OpenSSL-vs-GPL-readline, with another GPL product?
> 
>     I assume you can use wolfSSL as long as the result is GPL, which is the
>     same requirement libreadline causes for Postgres, particularly if
>     Postgres is statically linked to libreadline.
> 
> I don't want to divert the focus from the theread, but this subject has a
> controversial potential, in my opinion.
> I participated in a speech on another list, where I make contributions (IUP
> library: https://www.tecgraf.puc-rio.br/iup/).
> Where a user, upon discovering that two sub-libraries, were GPL licenses,
> caused an uproar, bringing the speech to Mr.Stallman himself.
> In short, the best thing for the project will be to remove the two GPL
> sub-libraries.

We aleady try to do that by trying to use BSD-licensed libedit if
installed:

    https://github.com/freebsd/freebsd/tree/master/lib/libedit
    https://certif.com/spec_print/readline.html

I would love to see libedit fully functional so we don't need to rely on
libreadline anymore, but I seem to remember there are a few libreadline
features that libedit doesn't implement, so we use libreadline if it is
already installed.  (I am still not clear if dynamic linking is a GPL
violation.)

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

On Sat, Jun 27, 2020 at 06:25:21PM -0300, Ranier Vilela wrote:
> Personally, the dynamic link does not hurt the GPL.
> But some people, do not think so, it was also unclear what Mr Stallman thinks
> of the subject (dynamic link).

I think Stallman says the courts have to decide, which kind of makes
sense.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

Christoph Berg <myon@debian.org> writes:
> Re: Ranier Vilela
>> Isn't LIbreSSL a better alternative?

> I don't know.

It should work all right --- it's the default ssl library on OpenBSD
and some other platforms, so we have some buildfarm coverage for it.
(AFAICT, none of the OpenBSD machines are running the ssl test, but
I tried that just now on OpenBSD 6.4 and it passed.)

However, I'm not exactly convinced that using LibreSSL gets you out
of the license compatibility bind.  LibreSSL is a fork of OpenSSL,
and IIUC a fairly hostile fork at that, so how did they get permission
to remove OpenSSL's problematic license clauses?  Did they remove them
at all?  A quick look at the header files on my OpenBSD installation
shows a whole lot of ancient copyright text.

            regards, tom lane

Bruce Momjian <bruce@momjian.us> writes:
> On Sat, Jun 27, 2020 at 06:25:21PM -0300, Ranier Vilela wrote:
>> Personally, the dynamic link does not hurt the GPL.
>> But some people, do not think so, it was also unclear what Mr Stallman thinks
>> of the subject (dynamic link).

> I think Stallman says the courts have to decide, which kind of makes
> sense.

This subject (openssl vs readline) has been discussed to death in the
past, with varying opinions --- for example, Red Hat's well-qualified
lawyers think building PG with openssl + readline poses no problem,
Debian's lawyers apparently think otherwise.  Please see the archives
before re-opening the topic.  And, if you're not a lawyer, it's quite
unlikely you'll bring any new insights.

            regards, tom lane

On Sat, Jun 27, 2020 at 05:46:17PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Sat, Jun 27, 2020 at 06:25:21PM -0300, Ranier Vilela wrote:
> >> Personally, the dynamic link does not hurt the GPL.
> >> But some people, do not think so, it was also unclear what Mr Stallman thinks
> >> of the subject (dynamic link).
> 
> > I think Stallman says the courts have to decide, which kind of makes
> > sense.
> 
> This subject (openssl vs readline) has been discussed to death in the
> past, with varying opinions --- for example, Red Hat's well-qualified
> lawyers think building PG with openssl + readline poses no problem,
> Debian's lawyers apparently think otherwise.  Please see the archives
> before re-opening the topic.  And, if you're not a lawyer, it's quite
> unlikely you'll bring any new insights.

I think the larger problem is that different jurisdictions, e.g., USA,
EU, could rule differently.  Also, the FSF is not the only organization
that can bring violation suits, e.g. Oracle with MySQL, so there
probably isn't one answer to this until it is thoroughly litigated.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

Christoph Berg <myon@debian.org> writes:
> Re: Ranier Vilela
>> Isn't LIbreSSL a better alternative?

> I don't know.

It should work all right --- it's the default ssl library on OpenBSD
and some other platforms, so we have some buildfarm coverage for it.
(AFAICT, none of the OpenBSD machines are running the ssl test, but
I tried that just now on OpenBSD 6.4 and it passed.)

However, I'm not exactly convinced that using LibreSSL gets you out
of the license compatibility bind.  LibreSSL is a fork of OpenSSL,
and IIUC a fairly hostile fork at that, so how did they get permission
to remove OpenSSL's problematic license clauses?  Did they remove them
at all?  A quick look at the header files on my OpenBSD installation
shows a whole lot of ancient copyright text.

                        regards, tom lane

On 2020-06-27 14:50, Christoph Berg wrote:
> Re: Peter Eisentraut
>> What would be the advantage of using wolfSSL over OpenSSL?
> 
> Avoiding the OpenSSL-vs-GPL linkage problem with readline.

We have added support for allegedly-OpenSSL compatible libraries such as 
LibreSSL before, so some tweaks for wolfSSL would seem acceptable. 
However, I doubt we are going to backpatch them, so unless you want to 
take responsibility for that as a packager, it's not really going to 
help anyone soon.  And OpenSSL 3.0.0 will have a new license, so for the 
next PostgreSQL release, this problem might be gone.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> On 27 Jun 2020, at 21:40, Bruce Momjian <bruce@momjian.us> wrote:
> On Sat, Jun 27, 2020 at 04:22:51PM -0300, Ranier Vilela wrote:

>> WolfSSL, will provide a commercial license for PostgreSQL?
>> Isn't LIbreSSL a better alternative?
> 
> Seems it might be.

That's not really an apples/apples comparison as the projects have vastly
different goals (wolfSSL offers FIPS certification for example).

cheers ./daniel

Personally I'm more interested in a library like Amazon's which is
trying to do less rather than more. I would rather a simpler, better
tested, easier to audit code-base than one with more features and more
complications.

https://github.com/awslabs/s2n

On Sun, Jun 28, 2020 at 10:18:12AM +0200, Peter Eisentraut wrote:
> On 2020-06-27 14:50, Christoph Berg wrote:
> > Re: Peter Eisentraut
> > > What would be the advantage of using wolfSSL over OpenSSL?
> > 
> > Avoiding the OpenSSL-vs-GPL linkage problem with readline.
> 
> We have added support for allegedly-OpenSSL compatible libraries such as
> LibreSSL before, so some tweaks for wolfSSL would seem acceptable. However,
> I doubt we are going to backpatch them, so unless you want to take
> responsibility for that as a packager, it's not really going to help anyone
> soon.  And OpenSSL 3.0.0 will have a new license, so for the next PostgreSQL
> release, this problem might be gone.

Oh, that is a long time coming --- it would be nice.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

Hi Greg,

On Sun, Jun 28, 2020 at 6:40 AM Greg Stark <stark@mit.edu> wrote:
>
> I'm more interested in a library like Amazon's

Does S2N support TLS 1.3?

    https://github.com/awslabs/s2n/issues/388

Kind regards
Felix Lechner

Hi Jonah,

On Sat, Jun 27, 2020 at 12:35 PM Jonah H. Harris <jonah.harris@gmail.com> wrote:
>
> Somewhere, I recall seeing an open-source OpenSSL compatibility wrapper for WolfSSL. Assuming that still exists, this
patchseems entirely unnecessary.
 

The patch uses the OpenSSL compatibility layer.

Kind regards
Felix Lechner

Hi Tom,

On Sat, Jun 27, 2020 at 11:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> The configure
> script could add -I/usr/include/wolfssl (or wherever those files
> are) to CPPFLAGS instead of touching all those #includes.

That does not work well when OpenSSL's development files are
installed. I did not think a segmentation fault was a good way to make
friends.

> However, as long as
> it's available on GPL terms, I don't see a problem with it
> [wolfSSL] being one alternative.

A minimal patch against -13 is on its way.

Kind regards
Felix Lechner

Hi Tom,

On Sat, Jun 27, 2020 at 7:56 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> However, judging from the caveats mentioned in the initial message,
> my inclination would be to wait awhile for wolfSSL to mature.

Please have a closer look. The library has been around since 2004 and
is popular in embedded systems. (It was previously known as cyaSSL.)
If you bought a car or an appliance in the past ten years, you may be
using it already.

wolfSSL's original claim to fame was that MySQL relied on it (I think
Oracle changed that). MariaDB still bundles an older, captive version.
The software is mature, and widely deployed.

Kind regards
Felix Lechner

Felix Lechner <felix.lechner@lease-up.com> writes:
> On Sat, Jun 27, 2020 at 7:56 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> However, judging from the caveats mentioned in the initial message,
>> my inclination would be to wait awhile for wolfSSL to mature.

> Please have a closer look. The library has been around since 2004 and
> is popular in embedded systems. (It was previously known as cyaSSL.)

I don't really care where else it's used.  If we have to hack the source
code before we can use it, it's not mature for our purposes.  Even when
(if) that requirement reduces to "you have to use the latest bleeding
edge release", it'll be problematic for people whose platforms supply
a less bleeding edge version.  I was signifying a desire to wait until
compatible versions are reasonably common in the wild before we spend
much time on this.

            regards, tom lane

On Sun, Jun 28, 2020 at 10:18:12AM +0200, Peter Eisentraut wrote:
> We have added support for allegedly-OpenSSL compatible libraries such as
> LibreSSL before, so some tweaks for wolfSSL would seem acceptable. However,
> I doubt we are going to backpatch them, so unless you want to take
> responsibility for that as a packager, it's not really going to help anyone
> soon.

That's a new feature to me.

> And OpenSSL 3.0.0 will have a new license, so for the next PostgreSQL
> release, this problem might be gone.

And there is this part too to consider, but I am no lawyer.

@@ -131,11 +131,11 @@ typedef union {
 #ifdef WOLFSSL_SHA3
      wc_Sha3 sha3;
 #endif
-} Hash;
+} WolfSSLHash;
[...]
 #endif
 #if !defined(XVALIDATE_DATE) && !defined(HAVE_VALIDATE_DATE)
     #define USE_WOLF_VALIDDATE
-    #define XVALIDATE_DATE(d, f, t) ValidateDate((d), (f), (t))
+    #define XVALIDATE_DATE(d, f, t) WolfSSLValidateDate((d), (f), (t))
 #endif
Looking at the patches, it seems to me that the part applying only to
WolfSSL should be done anyway, at least for the Hash part which is a
rather generic name, and that it may be better to do something as well
on the Postgres part for the same plan node to avoid conflicts, but
that's something old enough that it could vote (1054097).
ValidateTime() is present in the Postgres tree since f901bb5, but it
is always annoying to break stuff that could be used by external
plugins...

Regarding the Postgres part of the WIP, the hard part is that we need
more thinking about the refactoring bits, so as people compiling
Postgres can choose between OpenSSL or something else.  And as Tom
mentioned upthread there is no need for that:
-#include <openssl/x509.h>
-#include <openssl/x509v3.h>
-#include <openssl/asn1.h>
+#include <wolfssl/options.h>
+#include <wolfssl/openssl/x509.h>
+#include <wolfssl/openssl/x509v3.h>
+#include <wolfssl/openssl/asn1.h>

./configure should just append the correct path with -I.

-               my_bio_methods->bread = my_sock_read;
-               my_bio_methods->bwrite = my_sock_write;
+               my_bio_methods->readCb = my_sock_read;
+               my_bio_methods->writeCb = my_sock_write;
These parts could also be consolidated between OpenSSL and WolfSSL?

-       dh = PEM_read_DHparams(fp, NULL, NULL, NULL);
        FreeFile(fp);
+       return NULL;
This part is not acceptable as-is.  As a proof of concept, that's
fine of course.
--
Michael