Re: [pgAdmin4][Patch]: Fixes #1940 - Handle non-ascii characters in file/folder names

Hi

On Thu, Nov 24, 2016 at 10:44 AM, Surinder Kumar
<surinder.kumar@enterprisedb.com> wrote:
> Hi
>
> This patch handles encode/decode issues for files names when following
> operations are performed such as "create/rename/delete/download/upload" or
> browsing a file etc.
>
> Changes in code:
> 1) Removed cleanstring() js function which converts ascii characters into
> its alphabetical characters.
> We support ascii characters for file/folder name.
> 2) Read/Write file in bytes mode(rb/wb) as file need not be always text
> file.
> 3) The title of query tab is currently shown encoded, it should be decoded
> while displaying in tab title.
>
> Please find attached patch and review.

On macOS, with the Apple supplied Python 2.7.10.

- If I try to create a new folder with non-ASCII chars it the name, I get:

2016-11-25 10:42:55,074: INFO werkzeug: 127.0.0.1 - - [25/Nov/2016
10:42:55] "GET /file_manager/filemanager/7000827/?mode=addfolder&path=/&name=u%CC%88brig_wei%C3%9F
HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 2000, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1991, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1567, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1641, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1544, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1639, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1625, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
line 792, in decorated_view
    return func(*args, **kwargs)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 842, in file_manager
    return getattr(myFilemanager, mode)(**kwargs)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 756, in addfolder
    newPath = dir + '/' + path + newName + '/'
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position
1: ordinal not in range(128)

- If I try to open a folder (by double clicking it) with non-ASCII
chars in the name, the location bar is updated but the file/folder
list is not redrawn. I get the following exception:

2016-11-25 10:46:15,651: INFO werkzeug: 127.0.0.1 - - [25/Nov/2016
10:46:15] "POST /file_manager/filemanager/7000827/ HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 2000, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1991, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1567, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1641, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1544, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1639, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1625, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
line 792, in decorated_view
    return func(*args, **kwargs)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 842, in file_manager
    return getattr(myFilemanager, mode)(**kwargs)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 561, in getfolder
    filelist = self.list_filesystem(dir, path, trans_data, file_type)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 433, in list_filesystem
    orig_path = "{0}{1}".format(dir, path)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
1-6: ordinal not in range(128)

As I cannot even open a directory, there's not much more testing I can do here.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company