Re: Feature tests on M1 Mac fail

Hi,

Could you please provide a pip freeze output for python 3.10, 3.11?

Somehow I feel this is some package related issue only. I did not get an error on fresh 3.11 venv though.

Thanks,

Yogesh Mahajan

EnterpriseDB

On Wed, May 10, 2023 at 6:38 PM Dave Page <dpage@pgadmin.org> wrote:

Hi

On Wed, 10 May 2023 at 13:24, Yogesh Mahajan <yogesh.mahajan@enterprisedb.com> wrote:

Hi,

Errors occurring for python 3.8, 3.10, 3.11 are due the latest Werkzeug version 2.3.4/2.3.3.

Could you please check Werkzeug version getting downloaded in venv? It should be Werkzeug 2.2.3

They are all completely fresh venvs, so:

for x in 38 39 310 311; do

workon pgadmin4-py$x

pip freeze | grep Werkzeug    

for> done

Werkzeug==2.2.3

Werkzeug==2.2.3

Werkzeug==2.2.3

Werkzeug==2.2.3

 

I have seen error occurring with python 3.9, but could not reproduce locally. My guess is, it is again due to some dependent package version upgrade.

Didn't Pravesh report the same thing? In any case, as the venv is built from requirements.txt specifically for this test, I'd expect it to be correct. This is what the venv looks like:

% workon pgadmin4-py39

% python3 --version

Python 3.9.13

% pip freeze

alembic==1.10.4

async-generator==1.10

attrs==23.1.0

Authlib==1.2.0

azure-common==1.1.28

azure-core==1.26.4

azure-identity==1.12.0

azure-mgmt-core==1.4.0

azure-mgmt-rdbms==10.1.0

azure-mgmt-resource==23.0.0

azure-mgmt-subscription==3.1.1

Babel==2.12.1

bcrypt==4.0.1

bidict==0.22.1

blinker==1.6.2

boto3==1.26.131

botocore==1.29.131

Brotli==1.0.9

cachetools==5.3.0

certifi==2023.5.7

cffi==1.15.1

charset-normalizer==3.1.0

click==8.1.3

coverage==7.2.3

cryptography==40.0.2

decorator==5.1.1

dnspython==2.3.0

email-validator==2.0.0.post2

eventlet==0.33.3

exceptiongroup==1.1.1

extras==1.0.0

fixtures==4.0.1

Flask==2.2.5

flask-babel==3.1.0

Flask-Compress==1.13

Flask-Gravatar==0.5.0

Flask-Login==0.6.2

Flask-Mail==0.9.1

Flask-Migrate==4.0.4

Flask-Paranoid==0.3.0

Flask-Principal==0.4.0

Flask-Security-Too==5.1.2

Flask-SocketIO==5.3.4

Flask-SQLAlchemy==3.0.3

Flask-WTF==1.1.1

google-api-core==2.11.0

google-api-python-client==2.86.0

google-auth==2.17.3

google-auth-httplib2==0.1.0

google-auth-oauthlib==1.0.0

googleapis-common-protos==1.59.0

greenlet==1.1.2

gssapi==1.8.2

h11==0.14.0

httpagentparser==1.9.5

httplib2==0.22.0

idna==3.4

importlib-metadata==6.6.0

isodate==0.6.1

itsdangerous==2.1.2

Jinja2==3.1.2

jmespath==1.0.1

ldap3==2.9.1

linecache2==1.0.0

Mako==1.2.4

MarkupSafe==2.1.2

msal==1.22.0

msal-extensions==1.0.0

msrest==0.7.1

oauthlib==3.2.2

outcome==1.2.0

paramiko==3.1.0

passlib==1.7.4

pbr==5.11.1

Pillow==9.5.0

portalocker==2.7.0

protobuf==4.23.0

psutil==5.9.5

psycopg==3.1.8

psycopg-c==3.1.8

pyasn1==0.5.0

pyasn1-modules==0.3.0

pycodestyle==2.10.0

pycparser==2.21

PyJWT==2.7.0

PyNaCl==1.5.0

pyotp==2.8.0

pyparsing==3.0.9

pypng==0.20220715.0

PySocks==1.7.1

python-dateutil==2.8.2

python-engineio==4.4.1

python-mimeparse==1.6.0

python-socketio==5.8.0

pytz==2023.3

qrcode==7.4.2

requests==2.28.2

requests-oauthlib==1.3.1

rsa==4.9

s3transfer==0.6.1

selenium==4.8.3

six==1.16.0

sniffio==1.3.0

sortedcontainers==2.4.0

speaklater3==1.4

SQLAlchemy==2.0.12

sqlparse==0.4.4

sshtunnel==0.4.0

testscenarios==0.5.0

testtools==2.6.0

traceback2==1.4.0

trio==0.22.0

trio-websocket==0.10.2

typing_extensions==4.5.0

ua-parser==0.16.1

uritemplate==4.1.1

urllib3==1.26.15

user-agents==2.2.0

Werkzeug==2.2.3

wsproto==1.2.0

WTForms==3.0.1

zipp==3.15.0

 

And both the errors should not be specific to feature tests. It should be for API/ReSQL tests too.

Yes, that does seem to be the case. I'm just working on feature tests so that's where I happened to see the problem.

Setting up the venv as I have done is exactly how it's done for all the Python tests we run on GitHub, on Windows, Mac, and Linux, which work just fine. It makes me wonder if there's something architecture specific going on here. It's quite odd.

 

Thanks,

Yogesh Mahajan

EnterpriseDB

On Wed, May 10, 2023 at 5:38 PM Dave Page <dpage@pgadmin.org> wrote:

I get the following errors when trying to run the feature tests on an M1 Mac, with Python 3.8 through 3.11. The Python 3.9 failure seems to be a known issue with eventlet (though, our tests should still be able to run), and whilst the others can be fixed with the strategic addition of a .decode('UTF-8'), it does beg the question "how do these tests pass on our buildfarm".

Yogesh, can you take a look please?

############

~/git/pgadmin4/web  % workon pgadmin4-py38

~/git/pgadmin4/web  % python regression/runtests.py --pkg feature_tests

2023-05-10 12:49:05,666: INFO   pgadmin:        ########################################################

2023-05-10 12:49:05,666: INFO   pgadmin:        Starting pgAdmin 4 v7.1...

2023-05-10 12:49:05,666: INFO   pgadmin:        ########################################################

2023-05-10 12:49:05,666: DEBUG  pgadmin:        Python syspath: ['/Users/dpage/git/pgadmin4/web/regression', '/Users/dpage/git/pgadmin4/web', '/Users/dpage/git/pgadmin4/web/regression', '/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip', '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8', '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload', '/Users/dpage/.virtualenvs/pgadmin4-py38/lib/python3.8/site-packages']

NOTE: Configuring authentication for SERVER mode.

2023-05-10 12:49:06,738: INFO   pgadmin:        Registering blueprint module: <AboutModule 'about'>

2023-05-10 12:49:06,739: INFO   pgadmin:        Registering blueprint module: <AuthenticateModule 'authenticate'>

2023-05-10 12:49:06,740: INFO   pgadmin:        Registering blueprint module: <BrowserModule 'browser'>

2023-05-10 12:49:07,662: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:49:07,668: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:49:07,668: INFO   pgadmin:        Registering blueprint module: <HelpModule 'help'>

2023-05-10 12:49:07,668: INFO   pgadmin:        Registering blueprint module: <MiscModule 'misc'>

2023-05-10 12:49:08,769: INFO   pgadmin:        Registering blueprint module: <PreferencesModule 'preferences'>

2023-05-10 12:49:08,771: INFO   pgadmin:        Registering blueprint module: <PgAdminModule 'redirects'>

2023-05-10 12:49:08,771: INFO   pgadmin:        Registering blueprint module: <SettingsModule 'settings'>

2023-05-10 12:49:08,774: INFO   pgadmin:        Registering blueprint module: <ToolsModule 'tools'>

pgAdmin 4 - Application Initialisation

======================================

Starting pgAdmin 4. Please navigate to http://127.0.0.1:34575 in your browser.

 * Serving Flask app 'pgadmin'

 * Debug mode: off

=============Running the test cases for 'PostgreSQL 15' =============

Traceback (most recent call last):

  File "regression/runtests.py", line 486, in execute_test

    test_utils.create_parent_server_node(server_passed)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 691, in create_parent_server_node

    srv_id = create_server(server_info)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 634, in create_server

    server['type'] = get_server_type(server)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 1189, in get_server_type

    if "EnterpriseDB" in version_string:

TypeError: a bytes-like object is required, not 'str'

a bytes-like object is required, not 'str'

Exception in 6208811008 Thread-3

=============Completed the test cases for 'PostgreSQL 15'=============

======================================================================

Test Result Summary

======================================================================

======================================================================

Please check output in file: /Users/dpage/git/pgadmin4/web/regression/regression.log

~/git/pgadmin4/web  % workon pgadmin4-py39

~/git/pgadmin4/web  % python regression/runtests.py --pkg feature_tests

2023-05-10 12:49:30,701: INFO   pgadmin:        ########################################################

2023-05-10 12:49:30,701: INFO   pgadmin:        Starting pgAdmin 4 v7.1...

2023-05-10 12:49:30,701: INFO   pgadmin:        ########################################################

2023-05-10 12:49:30,701: DEBUG  pgadmin:        Python syspath: ['/Users/dpage/git/pgadmin4/web/regression', '/Users/dpage/git/pgadmin4/web', '/Users/dpage/git/pgadmin4/web/regression', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/Users/dpage/.virtualenvs/pgadmin4-py39/lib/python3.9/site-packages']

NOTE: Configuring authentication for SERVER mode.

2023-05-10 12:49:31,591: INFO   pgadmin:        Registering blueprint module: <AboutModule 'about'>

2023-05-10 12:49:31,592: INFO   pgadmin:        Registering blueprint module: <AuthenticateModule 'authenticate'>

2023-05-10 12:49:31,593: INFO   pgadmin:        Registering blueprint module: <BrowserModule 'browser'>

2023-05-10 12:49:32,297: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:49:32,303: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:49:32,303: INFO   pgadmin:        Registering blueprint module: <HelpModule 'help'>

2023-05-10 12:49:32,304: INFO   pgadmin:        Registering blueprint module: <MiscModule 'misc'>

2023-05-10 12:49:33,355: INFO   pgadmin:        Registering blueprint module: <PreferencesModule 'preferences'>

2023-05-10 12:49:33,357: INFO   pgadmin:        Registering blueprint module: <PgAdminModule 'redirects'>

2023-05-10 12:49:33,358: INFO   pgadmin:        Registering blueprint module: <SettingsModule 'settings'>

2023-05-10 12:49:33,361: INFO   pgadmin:        Registering blueprint module: <ToolsModule 'tools'>

pgAdmin 4 - Application Initialisation

======================================

Starting pgAdmin 4. Please navigate to http://127.0.0.1:49511 in your browser.

 * Serving Flask app 'pgadmin'

 * Debug mode: off

=============Running the test cases for 'PostgreSQL 15' =============

Traceback (most recent call last):

  File "/Users/dpage/git/pgadmin4/web/regression/runtests.py", line 486, in execute_test

    test_utils.create_parent_server_node(server_passed)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 691, in create_parent_server_node

    srv_id = create_server(server_info)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 634, in create_server

    server['type'] = get_server_type(server)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 1172, in get_server_type

    connection = get_db_connection(

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 55, in get_db_connection

    connection = psycopg.connect(

  File "/Users/dpage/.virtualenvs/pgadmin4-py39/lib/python3.9/site-packages/psycopg/connection.py", line 723, in connect

    rv = cls._wait_conn(

  File "/Users/dpage/.virtualenvs/pgadmin4-py39/lib/python3.9/site-packages/psycopg/connection.py", line 972, in _wait_conn

    return waiting.wait_conn(gen, timeout=timeout)

  File "/Users/dpage/.virtualenvs/pgadmin4-py39/lib/python3.9/site-packages/psycopg/waiting.py", line 86, in wait_conn

    sel.register(fileno, s)

  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/selectors.py", line 527, in register

    self._selector.control([kev], 0, 0)

TypeError: changelist must be an iterable of select.kevent objects

changelist must be an iterable of select.kevent objects

Exception in 6213283840 Thread-3

=============Completed the test cases for 'PostgreSQL 15'=============

======================================================================

Test Result Summary

======================================================================

======================================================================

Please check output in file: /Users/dpage/git/pgadmin4/web/regression/regression.log

~/git/pgadmin4/web  % workon pgadmin4-py310

~/git/pgadmin4/web  % python regression/runtests.py --pkg feature_tests

2023-05-10 12:49:51,253: INFO   pgadmin:        ########################################################

2023-05-10 12:49:51,254: INFO   pgadmin:        Starting pgAdmin 4 v7.1...

2023-05-10 12:49:51,254: INFO   pgadmin:        ########################################################

2023-05-10 12:49:51,254: DEBUG  pgadmin:        Python syspath: ['/Users/dpage/git/pgadmin4/web/regression', '/Users/dpage/git/pgadmin4/web', '/Users/dpage/git/pgadmin4/web/regression', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python310.zip', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10', '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload', '/Users/dpage/.virtualenvs/pgadmin4-py310/lib/python3.10/site-packages']

NOTE: Configuring authentication for SERVER mode.

2023-05-10 12:49:52,218: INFO   pgadmin:        Registering blueprint module: <AboutModule 'about'>

2023-05-10 12:49:52,219: INFO   pgadmin:        Registering blueprint module: <AuthenticateModule 'authenticate'>

2023-05-10 12:49:52,219: INFO   pgadmin:        Registering blueprint module: <BrowserModule 'browser'>

2023-05-10 12:49:52,912: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:49:52,916: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:49:52,917: INFO   pgadmin:        Registering blueprint module: <HelpModule 'help'>

2023-05-10 12:49:52,917: INFO   pgadmin:        Registering blueprint module: <MiscModule 'misc'>

2023-05-10 12:49:54,010: INFO   pgadmin:        Registering blueprint module: <PreferencesModule 'preferences'>

2023-05-10 12:49:54,012: INFO   pgadmin:        Registering blueprint module: <PgAdminModule 'redirects'>

2023-05-10 12:49:54,012: INFO   pgadmin:        Registering blueprint module: <SettingsModule 'settings'>

2023-05-10 12:49:54,015: INFO   pgadmin:        Registering blueprint module: <ToolsModule 'tools'>

pgAdmin 4 - Application Initialisation

======================================

Starting pgAdmin 4. Please navigate to http://127.0.0.1:53161 in your browser.

 * Serving Flask app 'pgadmin'

 * Debug mode: off

=============Running the test cases for 'PostgreSQL 15' =============

Traceback (most recent call last):

  File "/Users/dpage/git/pgadmin4/web/regression/runtests.py", line 486, in execute_test

    test_utils.create_parent_server_node(server_passed)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 691, in create_parent_server_node

    srv_id = create_server(server_info)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 634, in create_server

    server['type'] = get_server_type(server)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 1189, in get_server_type

    if "EnterpriseDB" in version_string:

TypeError: a bytes-like object is required, not 'str'

a bytes-like object is required, not 'str'

Exception in 6139408384 Thread-3 (execute_test)

=============Completed the test cases for 'PostgreSQL 15'=============

======================================================================

Test Result Summary

======================================================================

======================================================================

Please check output in file: /Users/dpage/git/pgadmin4/web/regression/regression.log

~/git/pgadmin4/web  % workon pgadmin4-py311

~/git/pgadmin4/web  % python regression/runtests.py --pkg feature_tests

2023-05-10 12:50:21,962: INFO   pgadmin:        ########################################################

2023-05-10 12:50:21,963: INFO   pgadmin:        Starting pgAdmin 4 v7.1...

2023-05-10 12:50:21,963: INFO   pgadmin:        ########################################################

2023-05-10 12:50:21,963: DEBUG  pgadmin:        Python syspath: ['/Users/dpage/git/pgadmin4/web/regression', '/Users/dpage/git/pgadmin4/web', '/Users/dpage/git/pgadmin4/web/regression', '/Library/Frameworks/Python.framework/Versions/3.11/lib/python311.zip', '/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11', '/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload', '/Users/dpage/.virtualenvs/pgadmin4-py311/lib/python3.11/site-packages']

NOTE: Configuring authentication for SERVER mode.

2023-05-10 12:50:22,483: INFO   pgadmin:        Registering blueprint module: <AboutModule 'about'>

2023-05-10 12:50:22,484: INFO   pgadmin:        Registering blueprint module: <AuthenticateModule 'authenticate'>

2023-05-10 12:50:22,484: INFO   pgadmin:        Registering blueprint module: <BrowserModule 'browser'>

2023-05-10 12:50:22,947: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:50:22,951: INFO   pgadmin:        Registering blueprint module: <DashboardModule 'dashboard'>

2023-05-10 12:50:22,951: INFO   pgadmin:        Registering blueprint module: <HelpModule 'help'>

2023-05-10 12:50:22,951: INFO   pgadmin:        Registering blueprint module: <MiscModule 'misc'>

2023-05-10 12:50:23,568: INFO   pgadmin:        Registering blueprint module: <PreferencesModule 'preferences'>

2023-05-10 12:50:23,569: INFO   pgadmin:        Registering blueprint module: <PgAdminModule 'redirects'>

2023-05-10 12:50:23,570: INFO   pgadmin:        Registering blueprint module: <SettingsModule 'settings'>

2023-05-10 12:50:23,571: INFO   pgadmin:        Registering blueprint module: <ToolsModule 'tools'>

pgAdmin 4 - Application Initialisation

======================================

Starting pgAdmin 4. Please navigate to http://127.0.0.1:30355 in your browser.

 * Serving Flask app 'pgadmin'

 * Debug mode: off

=============Running the test cases for 'PostgreSQL 15' =============

Traceback (most recent call last):

  File "/Users/dpage/git/pgadmin4/web/regression/runtests.py", line 486, in execute_test

    test_utils.create_parent_server_node(server_passed)

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 691, in create_parent_server_node

    srv_id = create_server(server_info)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 634, in create_server

    server['type'] = get_server_type(server)

                     ^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/dpage/git/pgadmin4/web/regression/python_test_utils/test_utils.py", line 1189, in get_server_type

    if "EnterpriseDB" in version_string:

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: a bytes-like object is required, not 'str'

a bytes-like object is required, not 'str'

Exception in 6212005888 Thread-3 (execute_test)

=============Completed the test cases for 'PostgreSQL 15'=============

======================================================================

Test Result Summary

======================================================================

======================================================================

Please check output in file: /Users/dpage/git/pgadmin4/web/regression/regression.log

############

--

Dave Page
Blog: https://pgsnake.blogspot.com
Twitter: @pgsnake

EDB: https://www.enterprisedb.com

--

Dave Page
Blog: https://pgsnake.blogspot.com
Twitter: @pgsnake

EDB: https://www.enterprisedb.com