Re: RM1849: Auto-generating security keys

On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:

On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

Hi Fahar,

Please log the case on redmine.

https://redmine.postgresql.org/issues/1871

Please find the attached patch, please apply it locally, and test it.

And, please update the case, and this mail chain accordingly.

This is resolved now and no error message displayed when we apply the patch that is already shared.

Sure Will test the patch and update the status accordingly.

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi

On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:

Here is the output of if we copy config_local.py and execute python setup.py
pgAdmin 4 - Application Initialisation
======================================


The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering initial setup mode...
NOTE: Configuring authentication for SERVER mode.


    Enter the email address and password to use for the initial pgAdmin user     account:

Email address: fahar.abbas@enterprisedb.com
Password:
Retype password:
Traceback (most recent call last):
  File "setup.py", line 449, in <module>
    do_setup(app)
  File "setup.py", line 96, in do_setup
    password = encrypt_password(p1)
  File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password
    signed = get_hmac(password).decode('ascii')
  File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac
    'set to "%s"' % _security.password_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512"
python setup.py
pgAdmin 4 - Application Initialisation
======================================

User can not do any setup for web based now.

The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering initial setup mode...
NOTE: Configuring authentication for SERVER mode.


    Enter the email address and password to use for the initial pgAdmin user     account:

Email address: fahar.abbas@enterprisedb.com
Password:
Retype password:
Traceback (most recent call last):
  File "setup.py", line 449, in <module>
    do_setup(app)
  File "setup.py", line 96, in do_setup
    password = encrypt_password(p1)
  File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password
    signed = get_hmac(password).decode('ascii')
  File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac
    'set to "%s"' % _security.password_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512"

On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:

Dave,

Testing Environment

 

Ubuntu 16.04 Linux 64:
--------------------------------

pg-AdminIV Development Environment Setup for Ubuntu  :

1) Install GIT

sudo apt-get install git

2) Install pip3

sudo apt-get install python3-pip

3) Install virtualenv

sudo pip3 install virtualenv

4) install below dependency as it is required for psycopg2 & pycrypto module

sudo apt-get install libpq-dev

sudo apt-get install python3-dev

5) Create virtual environment

virtualenv -p python3 venv

6) Create mkdir Projects

7) Clone git repo in Projects

git clone http://git.postgresql.org/git/pgadmin4.git

8) activate virtual environment

source venv/bin/activate

9) Install modules

pip3 install -r requirements_py3.txt

10) Edit the config.py file to config_local.py  resides in Projects\pgAdmin4\web  

11)Now run setup.py file  (\Projects\pgAdmin4\web)

    python setup.py

If user does not create config_local.py and do Python setup.py for new Development then SECURITY_PASSWORD_SALT message is also displayed:

Here is the output:
-------------------------

python setup.py
pgAdmin 4 - Application Initialisation
======================================


The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering initial setup mode...
NOTE: Configuring authentication for SERVER mode.


    Enter the email address and password to use for the initial pgAdmin user     account:

Email address: fahar.abbas@enterprisedb.com
Password:
Retype password:
Traceback (most recent call last):
  File "setup.py", line 449, in <module>
    do_setup(app)
  File "setup.py", line 96, in do_setup
    password = encrypt_password(p1)
  File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password
    signed = get_hmac(password).decode('ascii')
  File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac
    'set to "%s"' % _security.password_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512"
(venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$

Is this expected?

On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:

Sure,

Will test this thoroughly after complete investigation.

Kind Regards,

On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <dpage@pgadmin.org> wrote:

Patch applied.

Fahar, can you please test this thoroughly in desktop and server modes, with both fresh and upgraded installations?

https://redmine.postgresql.org/issues/1849

Packagers: This change means that packages are no longer forced to create a config_local.py file, and there is no longer any need to explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config (in fact, they should be removed for new installations, if you have included them in 1.0)

Thanks.

On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

Hi Dave,

On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Friday, October 14, 2016, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Thursday, October 13, 2016, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

Hi Dave,

On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi Ashesh,

Can you please review the attached patch, and apply if you're happy with it?

Overall the patch looked good to me.

But - I encounter an issue in 'web' mode, which wont happen with 'runtime'.

Steps for reproduction on existing pgAdmin 4 environment with 'web' mode.

- Apply the patch

- Start the pgAdmin4 application (stand alone application).

- Open pgAdmin home page.

- Log out (if already login).

And, you will see an exception.

I have figure out the issue with the patch.

We were setting the SECURITY_PASSWORD_SALT, after initializing the Security object.

Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT properly.

Hmm.

 

I had moved the Security object initialization after fetching these configurations from the database.

I have attached a addon patch for the same.

OK, thanks.

 

Now - I run into another issue.

Because - the existing password was hashed using the old SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4.

I think - we need to think about different strategy for upgrading the configuration file in the 'web' mode.

I was thinking - we can store the existing security configurations in the database during upgrade process in 'web' mode.

My concern with that is that we'll likely be storing the default config values in many cases, thus for those users, perpetuating the problem.

I guess what we need to do is re-encrypt the password during the upgrade - however, that makes me think; we then have both the key and the encrypted passwords in the same database which is clearly not a good idea. Sigh... Needs more thought. 

OK, so I've been thinking about this and experimenting for a couple of hours, as well as annoying the crap out of Magnus by thinking out loud in his general direction, and it looks like this isn't a major problem as from what I can see,  SECURITY_PASSWORD_SALT is (aside from really being a key not a salt) not the only salting that's done. 

It looks like it's used system-wide as the key to generate an HMAC of the users password, which is then passed to passlib which salts and hashes it. I did some testing, and found that two users with the same password end up with different hashes in the database, so clearly there is also per-user salting happening. I also created two users, then dropped the database and created the same user accounts with the same passwords again, and found that the resulting hashes were different in both databases - thus there is something else ensuring the hashes are unique across different installations/databases.

So, I believe we can do as you suggest and migrate existing values for SECURITY_PASSWORD_SALT, given that there's clearly some other per user and per installation/database salting going on anyway. New installations can have the random value for SECURITY_PASSWORD_SALT.

We do not need to generate the random SECURITY_PASSWORD_SALT during upgrade mode, which was wrong added in my addon patch.

Please find the updated patch.

Otherwise - looks good to me.

Please commit the new patch (if you're ok with the change).

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi  

I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as they're used for purposes that are essentially ephemeral, and thus can be changed during an upgrade.

Adding Magnus as I'd appreciate any thoughts he may have.

Patch attached - please review (Ashesh, but others too would be appreciated)!

Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Syed Fahar Abbas

Quality Management Group

EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com

--

Syed Fahar Abbas

Quality Management Group

EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com

--

Syed Fahar Abbas

Quality Management Group

EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com

--

Syed Fahar Abbas

Quality Management Group

EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com

--

Syed Fahar Abbas

Quality Management Group

EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com