Re: Proposed patch for key managment

Greetings Alastair,

* Alastair Turner (minion@decodable.me) wrote:
> On Sat, 19 Dec 2020 at 16:58, Bruce Momjian <bruce@momjian.us> wrote:
> > To enable the direct injection of keys into the server, we would need a
> > new command for this, since trying to make the passphrase command do
> > this will lead to unnecessary complexity.  The passphrase command should
> > do one thing, and you can't have it changing its behavior based on
> > parameters, since the parameters are for the script to process, not to
> > change behavior of the server.
> >
> > If we wanted to do this, we would need a new command, and one of the
> > parameters would be %k or something that would identify the key number
> > we want to retrieve from the KMS.  Stephen pointed out that we could
> > still validate that key;  the key would not be stored wrapped in the
> > file system, but we could store an HMAC in the file system, and use that
> > for validation.
>
> I think that this is where we have ended up talking at cross-purposes.
> My idea (after some refining based on Stephen's feedback) is that
> there should be only this new command (the cluster key command) and
> that the program for unwrapping stored keys should be called this way.
> As could a program to contact an HSM, etc. This moves the generating
> and wrapping functionality out of the core Postgres processes, making
> it far easier to add alternatives. I see this approach was discussed
> on the email thread you linked to, but I can't see where or how a
> conclusion was reached about it...

I do generally like the idea of having the single command able to be
used to either provide the KEK (where PG manages the keyring
internally), or called multiple times to retrieve the DEKs (in which
case PG wouldn't be managing the keyring).

However, after chatting with Bruce about it for a bit this weekend, I'm
not sure that we need to tackle the second case today.  I don't think
there's any doubt that there will be users who will want PG to manage
the keyring and to be able to work with just a passphrase, as Bruce has
worked to make happen here and which we have a patch for.  I'm hoping
Bruce will post a new patch soon based on the work that he and I
discussed today (mostly just clarifications around keys vs. passphrases
and having the PG side accept a key which the command that's run will
produce).  With that, I'm feeling pretty comfortable that we can move
forward and at least get this initial work committed.

> The pg_strong_random() function uses OpesnSSL's RAND_bytes() where
> available. With appropriate configuration of an OpenSSL engine
> supplying an alternative RAND, this could be handed off to a TRNG.

Sure.

> This is an example of the other option for providing flexibility to
> support specific key generation, wrapping, ... requirements - handing
> the operations off to a library like OpenSSL or nss tls which, in
> turn, can use pluggable providers for these functions. FWIW, the
> proprietary DBMSs use a pluggable engine approach to meet requirements
> in this category.

OpenSSL provides for this configuration and gives us a pluggable
architecture to make this happen, so I'm not sure what the concern here
is..?  I agree that eventually it'd be nice to allow the administrator
to, more directly, control the DEKs but we're still going to need to
have a solution for the user who wishes to just provide a passphrase or
a KEK and I don't see any particular reason to hold off on the
implementation of that.

> > My final point is that we can find ways to do what you are suggesting as
> > an addition to what we are adding now.  What we need is clear
> > justification of why these additional features are needed.  Requiring
> > the use of a true random number generator is a valid argument, but we
> > need to figure out, once this is implemented, who really wants that, and
> > how to implement it cleanly.
>
> Clean interfaces would be either "above" the database calling out to
> commands in user-land or "below" the database in the abstractions
> which OpenSSL, nss tls and friends provide. Since the conclusion seems
> already to have been reached that the keyring should be inside the
> core process and only the passphrase should pop out above, I'll review
> the patch in this vein and see if I can suggest any routes to carving
> out more manageable subsets of the patch.

There's no doubt that there needs to be a solution where the keyring is
managed by PG.  Perhaps there are options to allow an external keyring
as well, but we can add that in the future.

> "...once this is implemented..." changes become a lot more difficult.
> Particularly when the changes would affect what are regarded as the
> database's on-disk files. Which I suspect is a contributing factor to
> the level of engagement surrounding this feature.

Yes, it's true that after things are implemented it can be more
difficult to change them- but if you're concerned about the specific
on-disk format of the keyring then please help us understand what your
concern is there.  The concern you've raised so far is just that you'd
like an option to have an external keyring, and that's fine, but we are
also going to need to have an internal one and which comes first doesn't
seem particularly material to me.  I don't think having one or the other
in place first will really detract or make more difficult the adding of
the other later.

Thanks,

Stephen