Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4]

Поиск

Список

Период

Сортировка

От	Murtuza Zabuawala
Тема	Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4]
Дата	17 февраля 2017 г. 16:52:51
Msg-id	CAKKotZQfkjwy873qxScyy3gBCQDsBYcYU7mNmNA2_vyXkGpeWA@mail.gmail.com обсуждение исходный текст
Ответ на	Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4] (Dave Page <dpage@pgadmin.org>)
Ответы	Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4]
Список	pgadmin-hackers

Дерево обсуждения

Hi Harshal,

Python3.5

1) I installed PG_9.4 with locale: Arabic, Saudi Arabia,

- When I try to create new database with name "السلام" -> with 'template 0' as template , it fails to open, see screenshot

- Backup/Restore fails, refer screenshot

I tested INSET/UPDATE/DELETE and other common operations and seems to be working.

Attaching screenshots.

2) I installed PG_9.5 with locale: Chinese, China

- I am not able to create new DB with Chinese name same as above mentioned

Same as (1), most parts are working with patch

Attaching screenshots.

3) I installed PG_9.5 with locale: Japanese, Japan

- I am not able to create new DB with Japanese name same as above mentioned

Same as (1), most parts are working with patch

Attaching screenshots.

Python2.7 with 3 server mentioned above

Works fine most the places but also fails at some places with Unicode errors, As I observed it fails when we receive the data from DB server (by running properties.sql) and we try to render the data in template as we received from DB server.

Screenshots attached.

Common Exceptions:

{{conn|qtIdent(c.name)}} {{ GET_TYPE.CREATE_TYPE_SQL(conn, c.cltype, c.attlen, c.attprecision, c.hasSqrBracket) }}{% if c.collspcname %} COLLATE {{c.collspcname}}{% endif %}{% if c.attnotnull %} NOT NULL{% endif %}{% if c.defval %} DEFAULT {{c.defval}}{% endif %}

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 10: ordinal not in range(128)

Traceback (most recent call last):

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 2000, in __call__

return self.wsgi_app(environ, start_response)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1991, in wsgi_app

response = self.make_response(self.handle_exception(e))

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1567, in handle_exception

reraise(exc_type, exc_value, tb)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app

response = self.full_dispatch_request()

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request

rv = self.dispatch_request()

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request

return self.view_functions[rule.endpoint](**req.view_args)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/views.py", line 84, in view

return self.dispatch_request(*args, **kwargs)

File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/utils.py", line 235, in dispatch_request

return method(*args, **kwargs)

File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/__init__.py", line 341, in wrap

return f(*args, **kwargs)

File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/__init__.py", line 1021, in sql

query_for="sql_panel")

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template

context, ctx.app)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/flask/templating.py", line 116, in _render

rv = template.render(context)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render

return self.environment.handle_exception(exc_info, True)

File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception

reraise(exc_type, exc_value, tb)

File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/templates/function/pg/sql/9.5_plus/create.sql", line 7, in top-level template code

CREATE{% if query_type is defined %}{{' OR REPLACE'}}{% endif %} FUNCTION {{func_def}}

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 0: ordinal not in range(128)

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Feb 9, 2017 at 4:34 PM, Dave Page <dpage@pgadmin.org> wrote:

Harshal,

I had a brief discussion with Ashesh about this. He's going to setup a
call next week for us to discuss a plan of action for eliminating any
remaining encoding issues including this one. I'm putting this on hold
until then.

Thanks.

On Mon, Feb 6, 2017 at 2:54 PM, Harshal Dhumal
<harshal.dhumal@enterprisedb.com> wrote:
> Hi Dave,
>
> --
> Harshal Dhumal
> Software Engineer
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Mon, Feb 6, 2017 at 6:48 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Hi
>>
>> On Mon, Feb 6, 2017 at 12:57 PM, Harshal Dhumal
>> <harshal.dhumal@enterprisedb.com> wrote:
>> > Hi,
>> >
>> > Please find attached patch for RM 1983.
>> >
>> > This issue only occurs when database encoding is other than utf-8
>> >
>> > Also other issue was when we use connection of database with encoding
>> > other
>> > than utf-8 to retrieve data from cluster table/s which has encoding
>> > utf-8
>> > (e.g. pg_database) then data was not decoded properly.
>>
>> The code makes an assumption that pg_database is always utf-8 encoded.
>> I don't believe that is correct - I believe it's the encoding used in
>> the database from which the new database was created. The general
>> advice is that users should avoid using non-7bit ASCII characters in
>> shared catalogs, e.g. databases and comments etc.
>
>
> Ok.
>
> Let me split this into two issues:
> i) RM1983 for which I have attached updated patch. (basically I removed
> changes related to decode data retried from pg_database when connection
> encoding is other than utf-8)
> ii) Support to allow user to use non-&bit ASCII characters in shared
> catalogs with the help of pgAdmin4.
>
> Regarding your statement about pg_database "I believe it's the encoding used
> in the database from which the new database was created.". I found it
> little-bit confusing for me (correct me if i'm wrong); As mentioned here
> there is only one copy of pg_database per cluster. So I assume pg_database
> is created when we initialize database cluster and not when we create new
> database.
>
>
>>
>>
>> See
>> https://www.postgresql.org/message-id/flat/20160216163833.GF31273%40momjian.us#20160216163833.GF31273@momjian.us
>> for more info for example.
>>
>> Did pgAdmin 3 just assume it was UTF-8? I suspect it did - and that
>> just happened to work in most cases.
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>
>

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers