Обсуждение: Have an encrypted pgpass file

Поиск
Список
Период
Сортировка

Have an encrypted pgpass file

От
Marco van Eck
Дата:
Hi,

Since .pgpass files contain plain-text passwords, I searched for an alternative.
In the attached patch I've added the possibility to run a command to produce the content of the pgpass file, in exactly the same format. In this way I could use gpg or any other command to decrypt a pgpass file. It will prefer the .pgpass file and will not call the command.

This would be my environment variable, to have no plain-text password:
    PGPASSCOMMAND="gpg -q -d pgpass.gpg"

Other usages of the variable:
    PGPASSCOMMAND="cat pgpass"
    PGPASSCOMMAND="curl http://passwords/really-unsecure-pgpass"
    PGPASSCOMMAND="my-own-secure-pgpass-script"

The submitted patch does it's job, though the command could throw errors. 

What do you think of this solution?


Best regards,
Marco van Eck

Вложения

Re: Have an encrypted pgpass file

От
Thomas Munro
Дата:
On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck <marco.vaneck@gmail.com> wrote:
> Since .pgpass files contain plain-text passwords, I searched for an
> alternative.
> In the attached patch I've added the possibility to run a command to produce
> the content of the pgpass file, in exactly the same format. In this way I
> could use gpg or any other command to decrypt a pgpass file. It will prefer
> the .pgpass file and will not call the command.
>
> This would be my environment variable, to have no plain-text password:
>     PGPASSCOMMAND="gpg -q -d pgpass.gpg"
>
> Other usages of the variable:
>     PGPASSCOMMAND="cat pgpass"
>     PGPASSCOMMAND="curl http://passwords/really-unsecure-pgpass"
>     PGPASSCOMMAND="my-own-secure-pgpass-script"

Hi Marco

I've heard requests for encrypted .pgpass files before, and I've
always been a bit confused about how an unattended system is supposed
to decrypt them.  If the key is in the configuration file or local
filesystem, it feels like you haven't really added much security over
a plaintext password, since an attacker who can steal the .pgpass file
can steal those things too.

There are other database systems out there where passwords are held in
an encrypted form but with only a small amount of digging on the
internet you can find out how to decrypt them.  Seems a bit bogus, at
first glance anyway.

Here you side step those questions completely and make that the end
user's problem.   I like it.

-- 
Thomas Munro
http://www.enterprisedb.com


Re: Have an encrypted pgpass file

От
Christophe Pettus
Дата:
> On Jul 18, 2018, at 14:33, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
> Here you side step those questions completely and make that the end
> user's problem.   I like it.

+1.  This is a clever solution, since any kind of key vault or other system could be dropped in there.

--
-- Christophe Pettus
   xof@thebuild.com



Re: Have an encrypted pgpass file

От
Tom Lane
Дата:
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck <marco.vaneck@gmail.com> wrote:
>> Since .pgpass files contain plain-text passwords, I searched for an
>> alternative.
>> In the attached patch I've added the possibility to run a command to produce
>> the content of the pgpass file, in exactly the same format.

> ... Here you side step those questions completely and make that the end
> user's problem.   I like it.

... but doesn't this just encourage people to build hacks that aren't
really any more secure than the unreadable-file approach?  In fact,
I'm afraid this would be an attractive nuisance, in that people would
build one-off hacks that get no security vetting and don't really work.

I'd like to see a concrete example of a use-case that really does add
security; preferably one short and useful enough to put into the docs
so that people might copy-and-paste it rather than rolling their own.
It seems possible that something of the sort could be built atop
ssh-agent or gpg-agent, for instance.

            regards, tom lane


Re: Have an encrypted pgpass file

От
Thomas Munro
Дата:
On Thu, Jul 19, 2018 at 9:52 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@enterprisedb.com> writes:
>> On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck <marco.vaneck@gmail.com> wrote:
>>> Since .pgpass files contain plain-text passwords, I searched for an
>>> alternative.
>>> In the attached patch I've added the possibility to run a command to produce
>>> the content of the pgpass file, in exactly the same format.
>
>> ... Here you side step those questions completely and make that the end
>> user's problem.   I like it.
>
> ... but doesn't this just encourage people to build hacks that aren't
> really any more secure than the unreadable-file approach?  In fact,
> I'm afraid this would be an attractive nuisance, in that people would
> build one-off hacks that get no security vetting and don't really work.
>
> I'd like to see a concrete example of a use-case that really does add
> security; preferably one short and useful enough to put into the docs
> so that people might copy-and-paste it rather than rolling their own.

+1

> It seems possible that something of the sort could be built atop
> ssh-agent or gpg-agent, for instance.

Another example would be the Apple keychain system.  I think the
command would be something like "/usr/bin/security
find-generic-password -a someaccount -s somekeychain -w", and you'd
have to have stored it with something like "/usr/bin/security
add-generic-password -a someaccount -s somekeychain -w".

-- 
Thomas Munro
http://www.enterprisedb.com


Re: Have an encrypted pgpass file

От
Alvaro Herrera
Дата:
On 2018-Jul-18, Marco van Eck wrote:

> Since .pgpass files contain plain-text passwords, I searched for an
> alternative.
> In the attached patch I've added the possibility to run a command to
> produce the content of the pgpass file, in exactly the same format. In this
> way I could use gpg or any other command to decrypt a pgpass file. It will
> prefer the .pgpass file and will not call the command.
> 
> This would be my environment variable, to have no plain-text password:
>     PGPASSCOMMAND="gpg -q -d pgpass.gpg"
> 
> Other usages of the variable:
>     PGPASSCOMMAND="cat pgpass"
>     PGPASSCOMMAND="curl http://passwords/really-unsecure-pgpass"
>     PGPASSCOMMAND="my-own-secure-pgpass-script"
> 
> The submitted patch does it's job, though the command could throw errors.
> 
> What do you think of this solution?

Seems to me that passing %-specifiers to the command would make it more
useful (%u for "user", "host" etc) -- your command could refuse to give
you a password for the superuser account for instance but grant one for
a read-only user.  Or grant a password for the (hypothetical) pg_backup
user to the account doing the backups, but not to anyone else.  Maybe if
the root/postgres user runs the program, all passwords are printed for
the instances in localhost/127.0.0.1.

That way, a client-side centralized security policy is just a SMOP.


Maybe there are reasons why this doesn't make sense and I'm not seeing
them -- if you do please point'em out.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Re: Have an encrypted pgpass file

От
Tom Lane
Дата:
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Seems to me that passing %-specifiers to the command would make it more
> useful (%u for "user", "host" etc) -- your command could refuse to give
> you a password for the superuser account for instance but grant one for
> a read-only user.

It would also provide a *very* fertile source of shell-script-injection
vulnerabilities.  (Whaddya mean, you tried to use a user name with a
quote mark in it?)

This is exactly the kind of area in which I'm concerned for the
possibility of sloppily-written scripts being a net negative for
security.

            regards, tom lane


Re: Have an encrypted pgpass file

От
"Joshua D. Drake"
Дата:
On 07/18/2018 04:25 PM, Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>> Seems to me that passing %-specifiers to the command would make it more
>> useful (%u for "user", "host" etc) -- your command could refuse to give
>> you a password for the superuser account for instance but grant one for
>> a read-only user.
> It would also provide a *very* fertile source of shell-script-injection
> vulnerabilities.  (Whaddya mean, you tried to use a user name with a
> quote mark in it?)
>
> This is exactly the kind of area in which I'm concerned for the
> possibility of sloppily-written scripts being a net negative for
> security.

Although I appreciate the concern, can we not worried about this? Your 
argument basically boils down to: Dumb will be Dumb. That will not 
change no matter what we do as is obvious by the number of people STILL 
using postgres as their connected web app user. The usability of this 
feature if fleshed out correctly is pretty large.

JD

>             regards, tom lane
>

-- 
Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc
***  A fault and talent of mine is to tell it exactly how it is.  ***
PostgreSQL centered full stack support, consulting and development.
Advocate: @amplifypostgres || Learn: https://postgresconf.org
*****     Unless otherwise stated, opinions are my own.   *****



Re: Have an encrypted pgpass file

От
Tom Lane
Дата:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> On 07/18/2018 04:25 PM, Tom Lane wrote:
>> This is exactly the kind of area in which I'm concerned for the
>> possibility of sloppily-written scripts being a net negative for
>> security.

> Although I appreciate the concern, can we not worried about this? Your 
> argument basically boils down to: Dumb will be Dumb. That will not 
> change no matter what we do as is obvious by the number of people STILL 
> using postgres as their connected web app user. The usability of this 
> feature if fleshed out correctly is pretty large.

Sorry, I don't buy that line of argument.  The *only* reason for this
feature to exist is if it allows ready creation of security solutions
that are actually more secure than a non-world-readable .pgpass file.
That's a much higher bar than many people realize to begin with ...
and if it comes along with huge risk of security foot-guns, I do not
think that it's going to be a net advance.

One reason I'd like to see a concrete use-case (or several concrete
use-cases) is that we might then find some design that's less prone
to such mistakes than "here, run this shell script" is going to be.
I'm vaguely imagining exec'ing a program directly without a layer
of shell quoting/evaluation in between; but not sure how far that
gets us.

Another question that ought to be asked somewhere along here is
"how well does this work on Windows?" ...

            regards, tom lane


Re: Have an encrypted pgpass file

От
Marco van Eck
Дата:
On Thu, Jul 19, 2018 at 5:19 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> On 07/18/2018 04:25 PM, Tom Lane wrote:
>> This is exactly the kind of area in which I'm concerned for the
>> possibility of sloppily-written scripts being a net negative for
>> security.

> Although I appreciate the concern, can we not worried about this? Your
> argument basically boils down to: Dumb will be Dumb. That will not
> change no matter what we do as is obvious by the number of people STILL
> using postgres as their connected web app user. The usability of this
> feature if fleshed out correctly is pretty large.

Sorry, I don't buy that line of argument.  The *only* reason for this
feature to exist is if it allows ready creation of security solutions
that are actually more secure than a non-world-readable .pgpass file.
That's a much higher bar than many people realize to begin with ...
and if it comes along with huge risk of security foot-guns, I do not
think that it's going to be a net advance.

One reason I'd like to see a concrete use-case (or several concrete
use-cases) is that we might then find some design that's less prone
to such mistakes than "here, run this shell script" is going to be.
I'm vaguely imagining exec'ing a program directly without a layer
of shell quoting/evaluation in between; but not sure how far that
gets us.

Another question that ought to be asked somewhere along here is
"how well does this work on Windows?" ...

                        regards, tom lane
 
The reason I wanted to have this feature since having an unencrypted .pgpass will give the administrator access to it's content. Nothing more nothing less. If the script can get the content the user can do it as well. 

In my case I use gpg to decrypt a pgpass-file, and use psql to connect to a bunch of remote postgresql databases. If the administrator trie the same she will be asked to present my yubikey. Since our security department forbids my to have unencrypted password on any filesystem, I have to type the password too many times.
My PGPASSCOMMAND is set to "gpg -q -d ~/etc/pgpass.gpg"

@Alvaro I also thought of adding  arguments (actually my first implementation) but it will make the usage more complex since you have to write a command to deliver the password. The content of the pgpass-file is already well defined, making it easy to use. 

I have no idea why it shouldn't work on Windows, it's just running a command or script, just like 'type .pgpass' or 'psql -At -F, -c "select 'localhost',5432,'db','db','db'"'

Regards, Marco van Eck

Re: Have an encrypted pgpass file

От
"Tels"
Дата:
Moin,

On Wed, July 18, 2018 7:25 pm, Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>> Seems to me that passing %-specifiers to the command would make it more
>> useful (%u for "user", "host" etc) -- your command could refuse to give
>> you a password for the superuser account for instance but grant one for
>> a read-only user.
>
> It would also provide a *very* fertile source of shell-script-injection
> vulnerabilities.  (Whaddya mean, you tried to use a user name with a
> quote mark in it?)

Little Bobby Tables, we call him. :)

I'm also concerned that that would let anybody who could alter the
environment then let arbitrary code be run as user postgres. Is this
something that poses a risk in addition to the current situation?

Best regards,

Tels


Re: Have an encrypted pgpass file

От
Isaac Morland
Дата:


On 20 July 2018 at 17:22, Tels <nospam-pg-abuse@bloodgate.com> wrote:
Moin,

> It would also provide a *very* fertile source of shell-script-injection
> vulnerabilities.  (Whaddya mean, you tried to use a user name with a
> quote mark in it?)

Little Bobby Tables, we call him. :)

I'm also concerned that that would let anybody who could alter the
environment then let arbitrary code be run as user postgres. Is this
something that poses a risk in addition to the current situation?

If I understand the proposal correctly, the pgpass program would run on the client, invoked by libpq when a password is needed for a connection. So the risk relates to strange things happening on the client when the client attempts to connect as a strangely-named user or to a strangely-named database or host, not to being able to break into the server.

Re: Have an encrypted pgpass file

От
Tom Lane
Дата:
Isaac Morland <isaac.morland@gmail.com> writes:
>>> It would also provide a *very* fertile source of shell-script-injection
>>> vulnerabilities.  (Whaddya mean, you tried to use a user name with a
>>> quote mark in it?)

> If I understand the proposal correctly, the pgpass program would run on the
> client, invoked by libpq when a password is needed for a connection. So the
> risk relates to strange things happening on the client when the client
> attempts to connect as a strangely-named user or to a strangely-named
> database or host, not to being able to break into the server.

Yeah.  The most obvious scenario for trouble is that somebody enters
a crafted user name on a website, and that results in bad things happening
on an application-server machine that tried to pass that user name to
a database server.  The DB server itself isn't compromised, but the app
server could be.

If we were putting this sort of feature into psql, it wouldn't be such
a risk, but if it's in libpq then I fear it is.  libpq underlies a lot
of client-side code.

            regards, tom lane


Re: Have an encrypted pgpass file

От
Marco van Eck
Дата:
Sorry Tom (and others), I didn't notice my change affected libpq. Though I would really like the possibility to have an encrypted way of presenting the pgpassfile. Would it be more secure if the script / command is passed as a compile option to libpg and the variable only contains the filename of the encrypted file or only the arguments to pass to the command and use the original pgpassfile to decrypt. As always the security of the library / command in the hands of the person who compiles it. 

So assume the library is compiled with PGPASSDECRYPTCOMMAND=/usr/bin/gpg

PGPASSFILE=pgpass.gpg
PGPASSARGUMENTS="-q -d" 

In the end libpq would call: '/usr/bin/gpg -q -d pgpass.gpg'

The only thing I'm wondering, is it flexible enough for use cases different than mine? Or should I make a static variable for it so the user of the libpq can define it if they want to use the feature, and if not defined ignore the feature? I can make a new patch, if this is the direction we want to go.


Best regards,
Marco van Eck




On Sat, Jul 21, 2018 at 7:29 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Isaac Morland <isaac.morland@gmail.com> writes:
>>> It would also provide a *very* fertile source of shell-script-injection
>>> vulnerabilities.  (Whaddya mean, you tried to use a user name with a
>>> quote mark in it?)

> If I understand the proposal correctly, the pgpass program would run on the
> client, invoked by libpq when a password is needed for a connection. So the
> risk relates to strange things happening on the client when the client
> attempts to connect as a strangely-named user or to a strangely-named
> database or host, not to being able to break into the server.

Yeah.  The most obvious scenario for trouble is that somebody enters
a crafted user name on a website, and that results in bad things happening
on an application-server machine that tried to pass that user name to
a database server.  The DB server itself isn't compromised, but the app
server could be.

If we were putting this sort of feature into psql, it wouldn't be such
a risk, but if it's in libpq then I fear it is.  libpq underlies a lot
of client-side code.

                        regards, tom lane

Re: Have an encrypted pgpass file

От
Robert Haas
Дата:
On Wed, Jul 18, 2018 at 11:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Sorry, I don't buy that line of argument.  The *only* reason for this
> feature to exist is if it allows ready creation of security solutions
> that are actually more secure than a non-world-readable .pgpass file.
> That's a much higher bar than many people realize to begin with ...
> and if it comes along with huge risk of security foot-guns, I do not
> think that it's going to be a net advance.

I don't think I agree with this objection.  First, not doing anything
won't be a net advance, either.  Second, your objection seems akin to
saying "we're not going to let you drive because you might crash the
car".  There are *some* people who should not be allowed to get behind
the wheel, but your proposal seems analogous to banning *everyone*
from driving on the theory that car crashes are bad.  I think that's
an overreaction.  I agree that there's probably a risk, but why can't
we just document best practices?  Really, I'm not sure that it's right
to suppose that you're calling a shell script specifically.  If it's a
Perl, Python, Ruby, etc. script the risk is probably much less --
you're going to take $ARGV[1] or the equivalent and shove it in a
string variable, and after that it's not really any more or less risky
than any other string variable you've got.  You could of course
perform an ill-considered interpolation into a shell command, but
that's true of any string that originates from a user in any
situation, and if you're a somewhat-knowledgeable programmer you
probably won't.  Generally you have to do *extra* work to make things
safe in the shell, whereas in a scripting language you just have to
not screw up.  foo($thingy) is safe in Perl; foo $thingy is unsafe in
the shell.  Of course mistakes are possible and we can avoid all the
mistakes by not providing the feature, but to me, that doesn't seem
like the way to go.

> One reason I'd like to see a concrete use-case (or several concrete
> use-cases) is that we might then find some design that's less prone
> to such mistakes than "here, run this shell script" is going to be.

I think that the most common use case is likely to be to get the data
from a local or remote keyserver.  For example, when I'm logged in, my
keychain is available to provide passwords; when I log out, those
passwords aren't accessible any more.  Or, when the server is in the
datacenter where it's supposed to be located, it can pull the data
from some other machine in that data center whose job it is provide
said data; when the server is physically stolen from the datacenter
and taken to some other location, the other machine isn't there and
necessary credentials are no longer available (or even if the other
machine *is* there, it probably requires a manually-entered password
to start the key service, which the thief may not have).

> I'm vaguely imagining exec'ing a program directly without a layer
> of shell quoting/evaluation in between; but not sure how far that
> gets us.

It's not a bad thought, although it might not help that much if it
causes somebody who would have written PGPASSCOMMAND="this that" to
instead set PGPASSCOMMAND="thisthat.sh" where that file contains

#!/bin/bash
this that

...which seems like a likely outcome.

> Another question that ought to be asked somewhere along here is
> "how well does this work on Windows?" ...

True.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Have an encrypted pgpass file

От
Jeff Janes
Дата:
On Wed, Jul 18, 2018 at 5:52 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck <marco.vaneck@gmail.com> wrote:
>> Since .pgpass files contain plain-text passwords, I searched for an
>> alternative.
>> In the attached patch I've added the possibility to run a command to produce
>> the content of the pgpass file, in exactly the same format.

> ... Here you side step those questions completely and make that the end
> user's problem.   I like it.

... but doesn't this just encourage people to build hacks that aren't
really any more secure than the unreadable-file approach?  In fact,
I'm afraid this would be an attractive nuisance, in that people would
build one-off hacks that get no security vetting and don't really work.

I'd like to see a concrete example of a use-case that really does add
security; preferably one short and useful enough to put into the docs
so that people might copy-and-paste it rather than rolling their own.
It seems possible that something of the sort could be built atop
ssh-agent or gpg-agent, for instance.

If the goal is not unattended operation but just unannoying operation, I think the first example he provided is already that use-case.  If you already have gpg configured to use gpg-agent, then it just works.  You get encryption-at-rest, and you don't have to type in your password repeatedly in the same continuous shell session.

Cheers,

Jeff

Re: Have an encrypted pgpass file

От
Craig Ringer
Дата:
On 24 July 2018 at 05:53, Jeff Janes <jeff.janes@gmail.com> wrote:
On Wed, Jul 18, 2018 at 5:52 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck <marco.vaneck@gmail.com> wrote:
>> Since .pgpass files contain plain-text passwords, I searched for an
>> alternative.
>> In the attached patch I've added the possibility to run a command to produce
>> the content of the pgpass file, in exactly the same format.

> ... Here you side step those questions completely and make that the end
> user's problem.   I like it.

... but doesn't this just encourage people to build hacks that aren't
really any more secure than the unreadable-file approach?  In fact,
I'm afraid this would be an attractive nuisance, in that people would
build one-off hacks that get no security vetting and don't really work.

I'd like to see a concrete example of a use-case that really does add
security; preferably one short and useful enough to put into the docs
so that people might copy-and-paste it rather than rolling their own.
It seems possible that something of the sort could be built atop
ssh-agent or gpg-agent, for instance.

If the goal is not unattended operation but just unannoying operation, I think the first example he provided is already that use-case.  If you already have gpg configured to use gpg-agent, then it just works.  You get encryption-at-rest, and you don't have to type in your password repeatedly in the same continuous shell session.

... and the attacker steals the key from gpg-agent.

Grabbing it from a process's memory is a bit harder than grabbing contents of a file, but not much harder. If the agent is remote then that's harder, but you can just ask the script to decrypt the pgpass for you, so again, not much of a win.

Even with a hardware crypto offload device the advantage here seems to be mainly limited to making it harder to capture data from backups or file-lifting attacks. Anything that can execute code or commands on the host can still get the credentials.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: Have an encrypted pgpass file

От
Craig Ringer
Дата:
On 24 July 2018 at 10:08, Craig Ringer <craig@2ndquadrant.com> wrote:
On 24 July 2018 at 05:53, Jeff Janes <jeff.janes@gmail.com> wrote:
On Wed, Jul 18, 2018 at 5:52 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck <marco.vaneck@gmail.com> wrote:
>> Since .pgpass files contain plain-text passwords, I searched for an
>> alternative.
>> In the attached patch I've added the possibility to run a command to produce
>> the content of the pgpass file, in exactly the same format.

> ... Here you side step those questions completely and make that the end
> user's problem.   I like it.

... but doesn't this just encourage people to build hacks that aren't
really any more secure than the unreadable-file approach?  In fact,
I'm afraid this would be an attractive nuisance, in that people would
build one-off hacks that get no security vetting and don't really work.

I'd like to see a concrete example of a use-case that really does add
security; preferably one short and useful enough to put into the docs
so that people might copy-and-paste it rather than rolling their own.
It seems possible that something of the sort could be built atop
ssh-agent or gpg-agent, for instance.

If the goal is not unattended operation but just unannoying operation, I think the first example he provided is already that use-case.  If you already have gpg configured to use gpg-agent, then it just works.  You get encryption-at-rest, and you don't have to type in your password repeatedly in the same continuous shell session.

... and the attacker steals the key from gpg-agent.

Grabbing it from a process's memory is a bit harder than grabbing contents of a file, but not much harder. If the agent is remote then that's harder, but you can just ask the script to decrypt the pgpass for you, so again, not much of a win.

Even with a hardware crypto offload device the advantage here seems to be mainly limited to making it harder to capture data from backups or file-lifting attacks. Anything that can execute code or commands on the host can still get the credentials.


To be clear I'm not saying not to do this. I think it'd make more sense to do it via an agent and socket, where libpq learns to ask the agent for credentials (and certs!). That way we could finally support libnss, etc. It's not perfect, and it doesn't magically make unattended storage secure, but it sure helps slow down file-based password theft.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: Have an encrypted pgpass file

От
Thomas Munro
Дата:
On Tue, Jul 24, 2018 at 2:10 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
>> Grabbing it from a process's memory is a bit harder than grabbing contents
>> of a file, but not much harder. If the agent is remote then that's harder,
>> but you can just ask the script to decrypt the pgpass for you, so again, not
>> much of a win.
>>
>> Even with a hardware crypto offload device the advantage here seems to be
>> mainly limited to making it harder to capture data from backups or
>> file-lifting attacks. Anything that can execute code or commands on the host
>> can still get the credentials.
>
> To be clear I'm not saying not to do this. I think it'd make more sense to
> do it via an agent and socket, where libpq learns to ask the agent for
> credentials (and certs!). That way we could finally support libnss, etc.
> It's not perfect, and it doesn't magically make unattended storage secure,
> but it sure helps slow down file-based password theft.

Yeah, the idea that you can defend yourself against root is obviously
not a good one.  But the Apple keychain example (see the commands I
showed earlier) does seem to protect you against some threat
scenarios: if your computer is stolen, that password can't be
extracted from the disk.  You'd need to unlock the keychain first by
logging in.  It's not 'unattended': you have to be in attendance to
unlock the keychain.

I'm deeply suspicious of unattended use cases for encrypted passwords
though.  I have heard requests for this.  Encrypting it with a key
that is present in the configuration file, derived from well known
things, or stored in a nearby file doesn't change that.  The encrypted
secret + easily available key is still a secret you must protect
exactly as hard.  I suspect it is a misapplication of the advice that
you should never store (other people's) passwords in your database --
instead you should store something that allows you to check if *they*
have the password, without storing the password itself.  But that
doesn't mean that you should throw away your own passwords: instead
you should recognise that they are secrets, and treat them as such.

An idea I wondered about: if the goal is to allow psql (not libpq) to
use a secret from <insert your favourite password keeper technology>,
then perhaps psql could have a machine-friendly
--read-password-from-stdin feature for use by wrapper scripts (it
seems to be hard to send passwords to -W, though maybe I'm just doing
something stupid).  Or if you could wrap it in a script that provides
one end of a named pipe as PGPASSFILE (is that 'on disk'?).  Or uses a
temporary file in tmpfs with swap not configured (is that 'on disk'
yet?  Yeah, I bet that's against the rules...)  Of course you could
write a wrapper script that sets PGPASSWORD, but some people don't
like putting secrets in environment variables.  Supposedly there are
operating systems where anyone can see your environment (which?), but
on the systems I know only root can.  Then you run into the thorny
question of why you don't trust root not to peer at your
/proc/{$PID}/environ (or local equivalent), but do trust them not to
core dump your whole process and kernel.  (It is interesting that
Linux pam_exec.so chooses to pass the username via env var PAM_USER
but send the password via a pipe connected to stdin, considering all
the ways root could intercept that.)

-- 
Thomas Munro
http://www.enterprisedb.com


Re: Have an encrypted pgpass file

От
Marco van Eck
Дата:
Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W) around is making my auditors unhappy, and forcing me to enter the password over and over again. With a simple test it seems the password entered by the user also stays in memory, since it is able to reset a broken connection. Finding the password in memory is not trivial, but prevention is always preferred. 

It might be an idea to wipe the password after the login, and decrypt/read it again if it needs to reconnect. Would this make the solution more secure? I had a quick look at the code and the patch would stay compact. Please let me know of doing this would make sense.


Regards, Marco



On Tue, Jul 24, 2018 at 4:56 AM Thomas Munro <thomas.munro@enterprisedb.com> wrote:
On Tue, Jul 24, 2018 at 2:10 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
>> Grabbing it from a process's memory is a bit harder than grabbing contents
>> of a file, but not much harder. If the agent is remote then that's harder,
>> but you can just ask the script to decrypt the pgpass for you, so again, not
>> much of a win.
>>
>> Even with a hardware crypto offload device the advantage here seems to be
>> mainly limited to making it harder to capture data from backups or
>> file-lifting attacks. Anything that can execute code or commands on the host
>> can still get the credentials.
>
> To be clear I'm not saying not to do this. I think it'd make more sense to
> do it via an agent and socket, where libpq learns to ask the agent for
> credentials (and certs!). That way we could finally support libnss, etc.
> It's not perfect, and it doesn't magically make unattended storage secure,
> but it sure helps slow down file-based password theft.

Yeah, the idea that you can defend yourself against root is obviously
not a good one.  But the Apple keychain example (see the commands I
showed earlier) does seem to protect you against some threat
scenarios: if your computer is stolen, that password can't be
extracted from the disk.  You'd need to unlock the keychain first by
logging in.  It's not 'unattended': you have to be in attendance to
unlock the keychain.

I'm deeply suspicious of unattended use cases for encrypted passwords
though.  I have heard requests for this.  Encrypting it with a key
that is present in the configuration file, derived from well known
things, or stored in a nearby file doesn't change that.  The encrypted
secret + easily available key is still a secret you must protect
exactly as hard.  I suspect it is a misapplication of the advice that
you should never store (other people's) passwords in your database --
instead you should store something that allows you to check if *they*
have the password, without storing the password itself.  But that
doesn't mean that you should throw away your own passwords: instead
you should recognise that they are secrets, and treat them as such.

An idea I wondered about: if the goal is to allow psql (not libpq) to
use a secret from <insert your favourite password keeper technology>,
then perhaps psql could have a machine-friendly
--read-password-from-stdin feature for use by wrapper scripts (it
seems to be hard to send passwords to -W, though maybe I'm just doing
something stupid).  Or if you could wrap it in a script that provides
one end of a named pipe as PGPASSFILE (is that 'on disk'?).  Or uses a
temporary file in tmpfs with swap not configured (is that 'on disk'
yet?  Yeah, I bet that's against the rules...)  Of course you could
write a wrapper script that sets PGPASSWORD, but some people don't
like putting secrets in environment variables.  Supposedly there are
operating systems where anyone can see your environment (which?), but
on the systems I know only root can.  Then you run into the thorny
question of why you don't trust root not to peer at your
/proc/{$PID}/environ (or local equivalent), but do trust them not to
core dump your whole process and kernel.  (It is interesting that
Linux pam_exec.so chooses to pass the username via env var PAM_USER
but send the password via a pipe connected to stdin, considering all
the ways root could intercept that.)

--
Thomas Munro
http://www.enterprisedb.com

Re: Have an encrypted pgpass file

От
Tom Lane
Дата:
Marco van Eck <marco.vaneck@gmail.com> writes:
> Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W)
> around is making my auditors unhappy, and forcing me to enter the password
> over and over again. With a simple test it seems the password entered by
> the user also stays in memory, since it is able to reset a broken
> connection. Finding the password in memory is not trivial, but prevention
> is always preferred.

> It might be an idea to wipe the password after the login, and decrypt/read
> it again if it needs to reconnect. Would this make the solution more
> secure? I had a quick look at the code and the patch would stay compact.
> Please let me know of doing this would make sense.

We're basically not going to accept any added complication that's designed
to prevent memory-inspection attacks, because in general that's a waste
of effort.  All you're doing is (slightly) reducing the attack window.

            regards, tom lane


Re: Have an encrypted pgpass file

От
Marco van Eck
Дата:
After explaining the patch to a college we identified potentially execution of another user when it is defined in as a command parameter. To protect agains it I've removed the possibility to pass the 'passcommand'. With the result libpq only allows the PGPASSCOMMAND environment variable, which can only be defined by the executing user, and will be executed by the same user. It only reduces the need of unencrypted password's in a file. 

I think this solution is secure enough, shall we solve this feature-request?


Regards, Marco

On Tue, Jul 24, 2018 at 4:00 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Marco van Eck <marco.vaneck@gmail.com> writes:
> Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W)
> around is making my auditors unhappy, and forcing me to enter the password
> over and over again. With a simple test it seems the password entered by
> the user also stays in memory, since it is able to reset a broken
> connection. Finding the password in memory is not trivial, but prevention
> is always preferred.

> It might be an idea to wipe the password after the login, and decrypt/read
> it again if it needs to reconnect. Would this make the solution more
> secure? I had a quick look at the code and the patch would stay compact.
> Please let me know of doing this would make sense.

We're basically not going to accept any added complication that's designed
to prevent memory-inspection attacks, because in general that's a waste
of effort.  All you're doing is (slightly) reducing the attack window.

                        regards, tom lane
Вложения

Re: Have an encrypted pgpass file

От
Jeremy Schneider
Дата:
On 7/24/18 03:25, Marco van Eck wrote:
> Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W)
> around is making my auditors unhappy, and forcing me to enter the
> password over and over again. 

I'm late to the party here, but I just noticed this thread and I'm
excited about it. Security is not about perfection; it's about managing
risk, about layers of mitigations, about iterative improvements, about
the difficulty of attack against the value of assets protected. That
plain text pgpass file does irk auditors, and it's always driven me
crazy too.

On 8/1/18 08:33, Marco van Eck wrote:
> With the result libpq only allows the PGPASSCOMMAND
> environment variable, which can only be defined by the executing user,
> and will be executed by the same user. It only reduces the need of
> unencrypted password's in a file.
>
> I think this solution is secure enough, shall we solve this
> feature-request?

I'm happy with this. I think another useful question is what parameters
are needed for someone to write a callout-program that itself integrates
with something like HashiCorp Vault or the KMS solutions from all of the
major cloud providers or various token-based authentication protocols.
But we can always make the case later for adding some particular parameters.

Sockets sound nice (Craig), as does deeper database integration (so for
example postgres_fdw or pg10 native replication getting credentials
based on connection parameters, object owners, etc). But this idea
already helps with things like making plain-text password files less
common in system backups.

I'm hopeful that this moves forward.  :)

-- 
Jeremy Schneider
Database Engineer
Amazon Web Services


Re: Have an encrypted pgpass file

От
Kyotaro HORIGUCHI
Дата:
Hello.

I have had complaints several times on lack of this kind of
feature.

At Wed, 1 Aug 2018 17:33:39 +0200, Marco van Eck <marco.vaneck@gmail.com> wrote in
<CAE35ztPKvPE4xA7+jCGY+O5kL_9FtZ-owPDrUpXMvpU168pGgA@mail.gmail.com>
> After explaining the patch to a college we identified potentially execution
> of another user when it is defined in as a command parameter. To protect
> agains it I've removed the possibility to pass the 'passcommand'. With the
> result libpq only allows the PGPASSCOMMAND environment variable, which can
> only be defined by the executing user, and will be executed by the same
> user. It only reduces the need of unencrypted password's in a file.

Myabe we don't need the new environment variable by just allowing
.pgpass be a script. If we put faith in the security of .pgpass,
this won't be a problem, putting aside what the script actually
does.

> I think this solution is secure enough, shall we solve this feature-request?
> 
> 
> Regards, Marco
> 
> On Tue, Jul 24, 2018 at 4:00 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> > Marco van Eck <marco.vaneck@gmail.com> writes:
> > > Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W)
> > > around is making my auditors unhappy, and forcing me to enter the
> > password
> > > over and over again. With a simple test it seems the password entered by
> > > the user also stays in memory, since it is able to reset a broken
> > > connection. Finding the password in memory is not trivial, but prevention
> > > is always preferred.
> >
> > > It might be an idea to wipe the password after the login, and
> > decrypt/read
> > > it again if it needs to reconnect. Would this make the solution more
> > > secure? I had a quick look at the code and the patch would stay compact.
> > > Please let me know of doing this would make sense.
> >
> > We're basically not going to accept any added complication that's designed
> > to prevent memory-inspection attacks, because in general that's a waste
> > of effort.  All you're doing is (slightly) reducing the attack window.
> >
> >                         regards, tom lane

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



Re: Have an encrypted pgpass file

От
Geoff Winkless
Дата:
On Tue, 24 Jul 2018 at 11:25, Marco van Eck <marco.vaneck@gmail.com> wrote:
Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W) around is making my auditors unhappy,

With the greatest of respect, perhaps you need to get auditors who understand crypto better.​

​Having a user that has the minimal permissions ​to perform the required tasks with a stored password that only the automation user can read is perfectly valid. Encrypting it with a key that must (perforce) be accessible using the same permissions that the user would need in order to to read the unencrypted password file is no more valid (look up "security through obscurity").

Perhaps you could make your auditors happier by restricting that user's permissions to only run a defined function, and make that function do the work that the automation script wants? So even if the attacker can access the password he will still only be able to run that function? (You could even add DOS protection into the function to ensure it's only run so often, if you were worried about that.)

Geoff

Re: Have an encrypted pgpass file

От
Geoff Winkless
Дата:
On Thu, 2 Aug 2018 at 10:41, I wrote:
Perhaps you could make your auditors happier by restricting that user's permissions to only run a defined function, and make that function do the work that the automation script wants? So even if the attacker can access the password he will still only be able to run that function? (You could even add DOS protection into the function to ensure it's only run so often, if you were worried about that.)


​I realise (of course, after I sent this) that I ​misunderstood the thrust of your requirement, and that you want the ability to log in your own user without entering your own password. Apologies. Ignore me.

Geoff

Re: Have an encrypted pgpass file

От
Bruce Momjian
Дата:
On Wed, Aug  1, 2018 at 05:33:39PM +0200, Marco van Eck wrote:
> After explaining the patch to a college we identified potentially execution of
> another user when it is defined in as a command parameter. To protect agains it
> I've removed the possibility to pass the 'passcommand'. With the result libpq
> only allows the PGPASSCOMMAND environment variable, which can only be defined
> by the executing user, and will be executed by the same user. It only reduces
> the need of unencrypted password's in a file. 
> 
> I think this solution is secure enough, shall we solve this feature-request?

[Toshi and Nico added as CC's]

I think we need to step back and understand where we are going with our
various encryption options.  We have this feature under consideration,
and we have a proposal for data-at-rest encryption, which encrypts
writes to the file system:

    https://www.postgresql.org/message-id/CA%2BCSw_tb3bk5i7if6inZFc3yyf%2B9HEVNTy51QFBoeUk7UE_V%3Dw%40mail.gmail.com

So, until now, we have only supported encryption at the "optimal" level,
e.g. encrypted file system which encrypts the storage.  We have relied
on file permissions to protect access to the mounted file system, and
have a similar approach to securing .pgpass.  So, the question is
whether we continue to offer only encryption at the "optimal" level, or
whether we allow encryption at other levels.

There are two reasons to support non-optimal encryption levels.  First,
there is the issue that different threat models require different
encryption levels for protection.  For example, someone might have the
ability to _read_ a directory, but not the ability to modify it and
therefore attack it by grabbing passwords as they are typed.  An
encrypted .pgpass would be secure from that attack, but a .pgpass stored
on an encrypted file system would not.

Second, there should always be some kind of unlocking action, either
with a password, a hardware encryption device, or connection to a key
server.  Sometimes this unlocking action only makes sense at a certain
level, e.g. a personal Yubikey works for .pgpass but might be odd for
file system encryption.

I sympathize with concerns that the encryption key might be stored
unencrypted, or stored with the key that decrypts the encryption key. 
However, I don't think we can assume that there are no valid uses for
encryption at non-optimal levels, and that they would only be used in
insecure ways.

As a practical example, this presentation:

    http://momjian.us/main/writings/crypto_hw_use.pdf

shows how you might use a Yubikey to encrypt .pgpass.  The private key
can't be copied out of the Yubikey, but the private key wouldn't be used
to encrypt .pgpass --- instead a key would be encrypted using the
Yubikey's private key.  An attacker wouldn't need to read the Yubikey
private key but read the .pgpass password once it is decrypted.  That
could be done by modifying the gpg binary, the psql binary, or the
gpg-agent script, but those all require modification of something, with
proper permissions and possible detection.  I don't know how valuable
that would be to the average user, but I am sure it is useful for some
users.

I think with proper documentation and warnings, we could make these
non-optimal encryption levels useful for the users who have threat
models where they are useful, and hopefully minimize their misuse.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +


Re: Have an encrypted pgpass file

От
Nico Williams
Дата:
On Tue, Jul 24, 2018 at 12:25:31PM +0200, Marco van Eck wrote:
> Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W)
> around is making my auditors unhappy, and forcing me to enter the password
> over and over again. With a simple test it seems the password entered by
> the user also stays in memory, since it is able to reset a broken
> connection. Finding the password in memory is not trivial, but prevention
> is always preferred.

Sometimes the auditors are just wrong.  Say you're using Kerberos, so
you put service keys in... "keytab files" -- that's just like putting a
password in a file because they are equivalent.  Or server certificates
and their private keys... -- where are you going to put them if not in
some file?

Sure, you could put keys in a TPM (except they are orders of magnitude
too slow) or some other hardware token that doesn't suck (except those
are expensive).  But now you still need unattended access to the token,
and who cares what the keys _are_ when you can just _use_ them to
escalate privilege anyways??

Forcing attended operation of otherwise automatic systems is not a good
idea, and it is very expensive too.

A quick search turns up tools for finding cryptographic keys in memory.
Passwords can't be much harder.

> It might be an idea to wipe the password after the login, and decrypt/read
> it again if it needs to reconnect. Would this make the solution more
> secure? I had a quick look at the code and the patch would stay compact.
> Please let me know of doing this would make sense.

But you still wanted to automate things, no?

You can't protect from local hosts compromises.  You have to protect the
host.  Please, no security theater.  Do you think the day after a
massive data breach at your company you can tell the press "gee, I dunno
how they got the password, it was only loaded in memory!" and save it
from liability?  (If there's no liability either way, that's a different
problem.)

Nico
-- 


Re: Have an encrypted pgpass file

От
Jeremy Schneider
Дата:
On 7/23/18 08:07, Robert Haas wrote:
> This objection seems akin to
> saying "we're not going to let you drive because you might crash the
> car".  There are *some* people who should not be allowed to get behind
> the wheel, but this proposal seems analogous to banning *everyone*
> from driving on the theory that car crashes are bad.  I think that's
> an overreaction.

I would second this. There will always be lots of ways people can shoot
themselves in the foot. Our goal should be helping packagers make sure
the out-of-box setup is secure, and providing an extensible and flexible
product which can be customized to meet both mainstream and eclectic use
cases.

On 7/23/18 08:07, Robert Haas wrote:
> I think that the most common use case is likely to be to get the data
> from a local or remote keyserver.

This was also my thought. In fact, in the case of token-based
authentication schemes, today you'd have to have a cron job get a new
token every N minutes and rewrite the pgpass file. This patch enables
users to build far more elegant solutions under those schemes.

I gave the patch a spin on a linux box, and it works as expected. If we
can address the windows bit, then I'd support the idea of adding this
capability to libpq.

-Jeremy

-- 
Jeremy Schneider
Database Engineer
Amazon Web Services