Обсуждение: Will PostgreSQL 16 supports native transparent data encryption ?

On 8/21/23 09:02, Mostafa Fathy wrote:
> Hi there,
> 
> It is mentioned here 
> https://www.postgresql.org/about/press/faq/#:~:text=Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F
<https://www.postgresql.org/about/press/faq/#:~:text=Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F> that
nativetransparent data encryption is being worked on and it may be delivered with PostgreSQL 16.
 
> 
> Is PostgreSQL 16 beta version includes native transparent data 
> encryption or not ? because I checked the docs 
> https://www.postgresql.org/docs/16/index.html 
> <https://www.postgresql.org/docs/16/index.html>  and couldn't find 
> anything related to transparent data encryption.
> 
> If not supported yet in the beta version I would like to know if 
> PostgreSQL 16 final version will support native transparent data 
> encryption or not?

I can't find anything that says it is included and if it is not at this 
point(Beta) it will not be in the production release.

-- 
Adrian Klaver
adrian.klaver@aklaver.com

On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
> Hi there,
> 
> It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
> Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
> transparent data encryption is being worked on and it may be delivered with
> PostgreSQL 16.
> 
> Is PostgreSQL 16 beta version includes native transparent data encryption or
> not ? because I checked the docs https://www.postgresql.org/docs/16/index.html 
> and couldn't find anything related to transparent data encryption.
> 
> If not supported yet in the beta version I would like to know if PostgreSQL 16
> final version will support native transparent data encryption or not?

Not, PG 16 will not support it, and I am unclear if later major versions
will either.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

On 8/21/23 18:49, Bruce Momjian wrote:
> On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
>> Hi there,
>>
>> It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
>> Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
>> transparent data encryption is being worked on and it may be delivered with
>> PostgreSQL 16.
>>
>> Is PostgreSQL 16 beta version includes native transparent data encryption or
>> not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
>> and couldn't find anything related to transparent data encryption.
>>
>> If not supported yet in the beta version I would like to know if PostgreSQL 16
>> final version will support native transparent data encryption or not?
> Not, PG 16 will not support it, and I am unclear if later major versions
> will either.

That's disappointing, since TDE makes PCI audits that much simpler.

-- 
Born in Arizona, moved to Babylonia.

On 22 August 2023 06:52:10 CEST, Ron <ronljohnsonjr@gmail.com> wrote:
>On 8/21/23 18:49, Bruce Momjian wrote:
>> On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
>>> Hi there,
>>>
>>> It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
>>> Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
>>> transparent data encryption is being worked on and it may be delivered with
>>> PostgreSQL 16.
>>>
>>> Is PostgreSQL 16 beta version includes native transparent data encryption or
>>> not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
>>> and couldn't find anything related to transparent data encryption.
>>>
>>> If not supported yet in the beta version I would like to know if PostgreSQL 16
>>> final version will support native transparent data encryption or not?
>> Not, PG 16 will not support it, and I am unclear if later major versions
>> will either.
>
>That's disappointing, since TDE makes PCI audits that much simpler.
>

Sure. You can use EDB products (EPAS or Postgres Extended) with TDE.

Andreas

Greetings,

* Ron (ronljohnsonjr@gmail.com) wrote:
> On 8/21/23 18:49, Bruce Momjian wrote:
> > On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
> > > It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
> > > Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
> > > transparent data encryption is being worked on and it may be delivered with
> > > PostgreSQL 16.
> > >
> > > Is PostgreSQL 16 beta version includes native transparent data encryption or
> > > not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
> > > and couldn't find anything related to transparent data encryption.
> > >
> > > If not supported yet in the beta version I would like to know if PostgreSQL 16
> > > final version will support native transparent data encryption or not?
> > Not, PG 16 will not support it, and I am unclear if later major versions
> > will either.
>
> That's disappointing, since TDE makes PCI audits that much simpler.

There's ongoing work happening for TDE support and we'd love to hear
from folks who would like to see it included.  You can expect an updated
patch set for the September commitfest.  Getting more folks to test it
and use it and review it would certainly help move it forward.

Thanks,

Stephen

On 8/24/23 14:08, Stephen Frost wrote:
> Greetings,
>
> * Ron (ronljohnsonjr@gmail.com) wrote:
>> On 8/21/23 18:49, Bruce Momjian wrote:
>>> On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
>>>> It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
>>>> Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
>>>> transparent data encryption is being worked on and it may be delivered with
>>>> PostgreSQL 16.
>>>>
>>>> Is PostgreSQL 16 beta version includes native transparent data encryption or
>>>> not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
>>>> and couldn't find anything related to transparent data encryption.
>>>>
>>>> If not supported yet in the beta version I would like to know if PostgreSQL 16
>>>> final version will support native transparent data encryption or not?
>>> Not, PG 16 will not support it, and I am unclear if later major versions
>>> will either.
>> That's disappointing, since TDE makes PCI audits that much simpler.
> There's ongoing work happening for TDE support and we'd love to hear
> from folks who would like to see it included.

PgBackRest currently encrypts it's binary backups.

1. What kind of encryption would there be?  AES256 makes the auditors happy.
2. Would TDE-enabled pg_dump create encrypted dump files?
3. Would TDE obviate the need for PgBackRest's encryption?
4. How would encrypted "pg_dump --format=plain" work?  Or could it only work 
with the other formats (which is fine by me)?

>    You can expect an updated patch set for the September commitfest.

For that which will be Pg 17?

> Getting more folks to test it
> and use it and review it would certainly help move it forward.

By any chance, will binaries be created after the September commitfest?  
(Hoops must be jumped through to get development packages installed on the 
database servers I have access to, but I'd jump through them if needed.)

-- 
Born in Arizona, moved to Babylonia.

message from Stephen Frost <sfrost@snowman.net <mailto:sfrost@snowman.net>
> 
> 
> Greetings,
> 
> 
> * Ron (ronljohnsonjr@gmail.com <mailto:ronljohnsonjr@gmail.com>) wrote:
> > On 8/21/23 18:49, Bruce Momjian wrote:
> > > On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
> > > > It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
<https://www.postgresql.org/about/press/faq/#:~:text=>
> > > > Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
> > > > transparent data encryption is being worked on and it may be delivered with
> > > > PostgreSQL 16.
> > > > 
> > > > Is PostgreSQL 16 beta version includes native transparent data encryption or
> > > > not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
<https://www.postgresql.org/docs/16/index.html>
> > > > and couldn't find anything related to transparent data encryption.
> > > > 
> > > > If not supported yet in the beta version I would like to know if PostgreSQL 16
> > > > final version will support native transparent data encryption or not?
> > > Not, PG 16 will not support it, and I am unclear if later major versions
> > > will either.
> > 
> > That's disappointing, since TDE makes PCI audits that much simpler.
> 
> 
> There's ongoing work happening for TDE support and we'd love to hear
> from folks who would like to see it included. You can expect an updated
> patch set for the September commitfest. Getting more folks to test it
> and use it and review it would certainly help move it forward.
> 

Hello Stephen et all,

We have strong interest in TDE support and I would be happy to
test this with our Library Management System.

Thanks

    matthias

-- 
Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub

Greetings,

* Ron (ronljohnsonjr@gmail.com) wrote:
> On 8/24/23 14:08, Stephen Frost wrote:
> > * Ron (ronljohnsonjr@gmail.com) wrote:
> > > On 8/21/23 18:49, Bruce Momjian wrote:
> > > > On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
> > > > > It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
> > > > > Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
> > > > > transparent data encryption is being worked on and it may be delivered with
> > > > > PostgreSQL 16.
> > > > >
> > > > > Is PostgreSQL 16 beta version includes native transparent data encryption or
> > > > > not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
> > > > > and couldn't find anything related to transparent data encryption.
> > > > >
> > > > > If not supported yet in the beta version I would like to know if PostgreSQL 16
> > > > > final version will support native transparent data encryption or not?
> > > > Not, PG 16 will not support it, and I am unclear if later major versions
> > > > will either.
> > > That's disappointing, since TDE makes PCI audits that much simpler.
> > There's ongoing work happening for TDE support and we'd love to hear
> > from folks who would like to see it included.
>
> PgBackRest currently encrypts it's binary backups.

pgbackrest is optionally able to encrypt backups, sure, and that's
certainly a good thing, though having a way for the process performing
the backup to not be able to actually see the unencrypted data in the
first place eliminates that as an attack vector.

> 1. What kind of encryption would there be?  AES256 makes the auditors happy.

Supported options for AES would be 128, 192 and 256.

> 2. Would TDE-enabled pg_dump create encrypted dump files?

No, pg_dump is a client utility and hasn't got anything to do with TDE
really.

> 3. Would TDE obviate the need for PgBackRest's encryption?

The short answer to this is 'probably yes, when TDE is enabled on the
cluster'.  Clearly, pgbackrest would continue to support encryption and
there will be some things in PG that aren't encrypted that it might be
nice to have encrypted, depending on your particular security folks, but
you wouldn't need pgbackrest's encryption to ensure that the principle
user data is encrypted.

> 4. How would encrypted "pg_dump --format=plain" work?  Or could it only work
> with the other formats (which is fine by me)?

pg_dump isn't impacted by TDE.

> >    You can expect an updated patch set for the September commitfest.
>
> For that which will be Pg 17?

Probably still optimistic to be thinking about this for PG17, but
hopefully some of the preliminary work will be able to get into PG17
even if full TDE does not.

> > Getting more folks to test it
> > and use it and review it would certainly help move it forward.
>
> By any chance, will binaries be created after the September commitfest? 
> (Hoops must be jumped through to get development packages installed on the
> database servers I have access to, but I'd jump through them if needed.)

This would be something to ask the package maintainers... but I tend to
doubt they'd want the additional work; there's already lots and lots of
packages they're dealing with and to add on packages for every patch
that's posted to the PG mailing lists would be a huge task..

Thanks,

Stephen

Greetings,

* Matthias Apitz (guru@unixarea.de) wrote:
> message from Stephen Frost <sfrost@snowman.net <mailto:sfrost@snowman.net>
> > * Ron (ronljohnsonjr@gmail.com <mailto:ronljohnsonjr@gmail.com>) wrote:
> > > On 8/21/23 18:49, Bruce Momjian wrote:
> > > > On Mon, Aug 21, 2023 at 07:02:46PM +0300, Mostafa Fathy wrote:
> > > > > It is mentioned here https://www.postgresql.org/about/press/faq/#:~:text=
<https://www.postgresql.org/about/press/faq/#:~:text=>
> > > > > Q%3A%20What%20features%20will%20PostgreSQL%2016%20have%3F that native
> > > > > transparent data encryption is being worked on and it may be delivered with
> > > > > PostgreSQL 16.
> > > > >
> > > > > Is PostgreSQL 16 beta version includes native transparent data encryption or
> > > > > not ? because I checked the docs https://www.postgresql.org/docs/16/index.html
<https://www.postgresql.org/docs/16/index.html>
> > > > > and couldn't find anything related to transparent data encryption.
> > > > >
> > > > > If not supported yet in the beta version I would like to know if PostgreSQL 16
> > > > > final version will support native transparent data encryption or not?
> > > > Not, PG 16 will not support it, and I am unclear if later major versions
> > > > will either.
> > >
> > > That's disappointing, since TDE makes PCI audits that much simpler.
> >
> >
> > There's ongoing work happening for TDE support and we'd love to hear
> > from folks who would like to see it included. You can expect an updated
> > patch set for the September commitfest. Getting more folks to test it
> > and use it and review it would certainly help move it forward.
>
> We have strong interest in TDE support and I would be happy to
> test this with our Library Management System.

Great, glad to hear that.  Note that this is still very much a
development effort and so some familiarity with how to build PostgreSQL
from source, apply patches, and then run the resulting binaries is
necessary to test.  If you're still interested, there's patches that
have been posted, just let me know.

Thanks,

Stephen

El día jueves, septiembre 07, 2023 a las 12:33:06 -0400, Stephen Frost escribió:

> * Matthias Apitz (guru@unixarea.de) wrote:
> > > 
> > > There's ongoing work happening for TDE support and we'd love to hear
> > > from folks who would like to see it included. You can expect an updated
> > > patch set for the September commitfest. Getting more folks to test it
> > > and use it and review it would certainly help move it forward.
> > 
> > We have strong interest in TDE support and I would be happy to
> > test this with our Library Management System.
> 
> Great, glad to hear that.  Note that this is still very much a
> development effort and so some familiarity with how to build PostgreSQL
> from source, apply patches, and then run the resulting binaries is
> necessary to test.  If you're still interested, there's patches that
> have been posted, just let me know.

We ported our LMS from Sybase and Oracle to PostgreSQL some years ago
and compiled PostgreSQL on SuSE Linux always from source, IIRC starting
with version 11.0. We have even own modifications to improve the logging
of the ESQL/C layer of PostgreSQL , and for fixing a smaller bug.

I personally count with 30++ UNIX experience in FreeBSD, SVR4, Solaris
SPARC, AIX, HP-UX and Linux.

Said that, I don't see any problem applying the patches and compile the
binaries.

    matthias

-- 
Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub