Обсуждение: Amazon RDS auth tokens in .pgpass

Поиск
Список
Период
Сортировка

Amazon RDS auth tokens in .pgpass

От
Nicholas Chammas
Дата:
I suspect there are some restrictions on the kind of data you can put in .pgpass that are not documented sufficiently here: https://www.postgresql.org/docs/12/libpq-pgpass.html

I am trying to connect to a Postgres database on Amazon RDS using IAM authentication. This works by having IAM generate an authentication token that you use as the password for the database user you're connecting as. You can read more about this here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.AWSCLI.PostgreSQL.html

Here's an example of what one of these auth tokens looks like (with some of the sections redacted/altered from a real token):

```
```

That whole thing is the password for `someuser`.

Now, I know that .pgpass works for me because I can use it to connect to another user that has a regular-looking password like `some-password-123`. And I know that using these auth tokens with `someuser` also works because psql successfully connects to that user if I provide the auth token via the `PGPASSWORD` environment variable.

It's only the combination of a) .pgpass and b) RDS auth token that doesn't work. This leads me to believe something is going wrong on a code path specific to .pgpass, or that there are some undocumented restrictions on what can go in .pgpass.

The docs do instruct:

> If an entry needs to contain : or \, escape this character with \.

So I tried escaping the one `:` character in the auth token with a backslash, but that didn't help. Thinking that perhaps there are other characters that might need escaping, I also tried escaping every character in the auth token -- i.e. `\s\o\m\e\-\h\o\s\t\.\u\s...` -- but that didn't work either. psql fails to connect with:

```
psql: error: could not connect to server: FATAL:  PAM authentication failed for user "someuser"
```

Are the docs on .pgpass missing something? Is there perhaps a bug in how .pgpass is being parsed?

Nick

Re: Amazon RDS auth tokens in .pgpass

От
Tom Lane
Дата:
Nicholas Chammas <nicholas.chammas@gmail.com> writes:
> I am trying to connect to a Postgres database on Amazon RDS using IAM
> authentication. This works by having IAM generate an authentication token
> that you use as the password for the database user you're connecting as.
> You can read more about this here:
> https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.AWSCLI.PostgreSQL.html
> Here's an example of what one of these auth tokens looks like (with some of
> the sections redacted/altered from a real token):

> ```
>
some-host.us-east-1.rds.amazonaws.com:5432/?DBUser=someuser&Action=connect&X-Amz-SignedHeaders=host&X-Amz-Security-Token=FwoGZXIvYXdzEHcaDD1hC2q3GGNNaftOvCLRAetaDArXOt6kpr1Ac83hzwtPxEojvZzARJN%2Ftys%2BkLnxsP6FmHmIMmMERWeGBiJmNcUyXWYY%2BSU9oduSSeAv%2BCpYy028Cep%2Bpyl1Km3B5axPAA2q0L4NWa41LQOayWF8F7%2FlB540%2B0aSkZ%2BIucM%2BLZXTcQl3Q0nJiIgu65lhuME4q3Mvst1ZEaZWfUegCWGaX0npajiKbNU2Ut3FH%2F6046RxlNwO4jg4vteTcPk%2BfgMjuAkf5gFR9EYMTziRx1dJGJn8VGFLTugGVITQI%2FEC1iuqKNb%2Fs%2FoFMisfzgF1nd7kxcZYYAmQtugBnDuJuIboYwYAXI2qn7HDbhgQ5v%2FwPWfZieWO&X-Amz-Credential=ABCDEFGHIJKLMNOPQRST%2F20200831%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=900&X-Amz-Date=20200831T163708Z&X-Amz-Signature=fb41c59dcaf4e013eda6885a4d456549366d2612ecf06307d2443b6a44461ab7
> ```

Ugh :-(

> It's only the combination of a) .pgpass and b) RDS auth token that doesn't
> work. This leads me to believe something is going wrong on a code path
> specific to .pgpass, or that there are some undocumented restrictions on
> what can go in .pgpass.

Looking at the source code, there's a hard-wired restriction that lines of
.pgpass can't be more than 320 characters long (well, NAMEDATALEN*5, but
very few builds don't have NAMEDATALEN=64).  I see that somebody very
recently added code to make libpq print a warning for overlength lines,
but I wonder why they didn't just, um, remove the restriction.  We had
not previously heard of a use-case for passwords with hundreds of
characters in them, but I guess we need to cope.

If you're in a position to rebuild libpq, could you check that changing
LINELEN in fe-connect.c to something large enough (like 1K) fixes your
problem?  While that's clearly one issue, it'd be good to verify that
there's not another one lurking behind it.

            regards, tom lane



Re: Amazon RDS auth tokens in .pgpass

От
Tom Lane
Дата:
Stephen Frost <sfrost@snowman.net> writes:
> My memory might be faulty, but I've got some specific recollection of
> people complaining about this before and the community response being
> "who would ever need such very long things?!"

Yeah, so I guessed wrong ;-).  Never too late to absorb new facts though.

That thread does point out that passwordFromFile() is far from the
only place that assumes passwords aren't going to be longer than
what would be sane to enter manually.  I wonder whether we need to
worry about the other bottlenecks.

            regards, tom lane



Re: Amazon RDS auth tokens in .pgpass

От
Ron
Дата:
On 8/31/20 1:21 PM, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
>> My memory might be faulty, but I've got some specific recollection of
>> people complaining about this before and the community response being
>> "who would ever need such very long things?!"
> Yeah, so I guessed wrong ;-).  Never too late to absorb new facts though.
>
> That thread does point out that passwordFromFile() is far from the
> only place that assumes passwords aren't going to be longer than
> what would be sane to enter manually.  I wonder whether we need to
> worry about the other bottlenecks.

Would it be better to add a .pgtoken file which gets examined (and bypasses 
those other length restrictions) if the user isn't in .pgpass?  Or is that a 
feature which would have to wait for v14?

-- 
Angular momentum makes the world go 'round.



Re: Amazon RDS auth tokens in .pgpass

От
Tom Lane
Дата:
Stephen Frost <sfrost@snowman.net> writes:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> That thread does point out that passwordFromFile() is far from the
>> only place that assumes passwords aren't going to be longer than
>> what would be sane to enter manually.  I wonder whether we need to
>> worry about the other bottlenecks.

> Well, as I said in that thread two years ago, seems like we should make
> it work everywhere and be consistent between frontend and backend
> regarding what's supported.  Perhaps even clearly document what the
> limit is too...

In the case of passwordFromFile(), the line doesn't only contain a
password.  There's also a hostname that has no a-priori upper length,
and some other fields too; not to mention that if we have a convention
for comments then it's unfriendly to have an a-priori upper length for
comment lines.  So I'm thinking that the correct thing to do in
passwordFromFile() is use an expansible buffer and fail only on OOM.
There remains, though, the question of whether any of the other limits
are problematic.

            regards, tom lane



Re: Amazon RDS auth tokens in .pgpass

От
Nicholas Chammas
Дата:
On Mon, Aug 31, 2020 at 2:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Looking at the source code, there's a hard-wired restriction that lines of
.pgpass can't be more than 320 characters long (well, NAMEDATALEN*5, but
very few builds don't have NAMEDATALEN=64).  I see that somebody very
recently added code to make libpq print a warning for overlength lines,
but I wonder why they didn't just, um, remove the restriction.  We had
not previously heard of a use-case for passwords with hundreds of
characters in them, but I guess we need to cope.

Just FYI, the auth tokens generated by Amazon RDS appear to be 796 bytes long.

```
$ aws rds generate-db-auth-token --hostname "some-host.us-east-1.rds.amazonaws.com" --port 5432 --region us-east-1 --username someuser | wc -c
     796
```

If you're in a position to rebuild libpq, could you check that changing
LINELEN in fe-connect.c to something large enough (like 1K) fixes your
problem?  While that's clearly one issue, it'd be good to verify that
there's not another one lurking behind it.

I'm not in an easy position to do that (having not contributed code to the project before), but I'd be happy to provide more information about my use case or about how IAM-based authentication works as it relates to Postgres.

Re: Amazon RDS auth tokens in .pgpass

От
Tom Lane
Дата:
Nicholas Chammas <nicholas.chammas@gmail.com> writes:
> On Mon, Aug 31, 2020 at 2:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Looking at the source code, there's a hard-wired restriction that lines of
>> .pgpass can't be more than 320 characters long (well, NAMEDATALEN*5, but
>> very few builds don't have NAMEDATALEN=64).  I see that somebody very
>> recently added code to make libpq print a warning for overlength lines,
>> but I wonder why they didn't just, um, remove the restriction.  We had
>> not previously heard of a use-case for passwords with hundreds of
>> characters in them, but I guess we need to cope.

> Just FYI, the auth tokens generated by Amazon RDS appear to be 796 bytes
> long.

Thanks.  I've pushed a fix to remove libpq's undocumented restriction
on the length of a .pgpass line.  It will be in November's releases.

            regards, tom lane



Re: Amazon RDS auth tokens in .pgpass

От
Nicholas Chammas
Дата:
Thanks Tom. It looks like this is the commit corresponding to your fix: https://github.com/postgres/postgres/commit/b55b4dad99e99d5306744a4e8ef8021fa3a922e4

On Tue, Sep 1, 2020 at 2:25 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Nicholas Chammas <nicholas.chammas@gmail.com> writes:
> On Mon, Aug 31, 2020 at 2:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Looking at the source code, there's a hard-wired restriction that lines of
>> .pgpass can't be more than 320 characters long (well, NAMEDATALEN*5, but
>> very few builds don't have NAMEDATALEN=64).  I see that somebody very
>> recently added code to make libpq print a warning for overlength lines,
>> but I wonder why they didn't just, um, remove the restriction.  We had
>> not previously heard of a use-case for passwords with hundreds of
>> characters in them, but I guess we need to cope.

> Just FYI, the auth tokens generated by Amazon RDS appear to be 796 bytes
> long.

Thanks.  I've pushed a fix to remove libpq's undocumented restriction
on the length of a .pgpass line.  It will be in November's releases.

                        regards, tom lane