Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

On 06/07/12 10:14, Jan Urbański wrote:
> On 06/07/12 10:05, Heikki Linnakangas wrote:
>> In the backend elog routines, there is a global variable
>> 'recursion_depth', which is incremented when an error-handling routine
>> is entered, and decremented afterwards. Can we use a similar mechinism
>> in PLy_elog() to detect and stop recursion?
>
> I guess we can, I'll try to do some tests in order to see if there's an
> easy user-triggereable way of causing PLy_elog to recurse and if not
> then a guard like this should be enough as a safety measure against as
> yet unknown conditions (as opposed to something we expect to happen
> regularly).

Attached is a patch that stores the recursion level of PLy_traceback and 
prevents it from running if it's too deep (PLy_traceback is the one 
doing heavy lifting, that's why I chose to put the logic to skip running 
there).

I tried a few things and was not able to easily invoke the infinite 
recursion condition, but I did notice that there are two more encodings 
that have different names in Postgres and in Python (KOI8-R and KOI8-U) 
and added them to the switch.

There's still trouble with EUC_TW and MULE_INTERNAL which don't have 
Python equivalents. EUC-TW has been discussed in 
http://bugs.python.org/issue2066 and rejected (see 
http://bugs.python.org/issue2066#msg113731).

If you use any of these encodings, you *will* get into the recursion 
trouble described eariler, just as before the path you'd get into it 
with CP1252 as your encoding.

What shall we do about those? Ignore them? Document that if you're sing 
one of these encodings then PL/Python with Python 2 will be crippled and 
with Python 3 just won't work?

Cheers,
Jan

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

On 13/07/12 13:38, Jan Urbański wrote:
> On 12/07/12 11:08, Heikki Linnakangas wrote:
>> On 07.07.2012 00:12, Jan Urbański wrote:
>>> So you're in favour of doing unicode -> bytes by encoding with UTF-8 and
>>> then using the server's encoding functions?
>>
>> Sounds reasonable to me. The extra conversion between UTF-8 and UCS-2
>> should be quite fast, and it would be good to be consistent in the way
>> we do conversions in both directions.
>>
>
> I'll implement that than (sorry for not following up on that eariler).

Here's a patch that always encodes Python unicode objects using UTF-8 
and then uses Postgres's internal functions to produce bytes in the 
server encoding.

Cheers,
Jan

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

On 18/07/12 17:17, Heikki Linnakangas wrote:
> On 14.07.2012 17:50, Jan Urbański wrote:
>
> If pg_do_encoding_conversion() throws an error, you don't get a chance
> to call Py_DECREF() to release the string. Is that a problem?
>
> If an error occurs in PLy_traceback(), after incrementing
> recursion_depth, you don't get a chance to decrement it again. I'm not
> sure if the Py* function calls can fail, but at least seemingly trivial
> things like initStringInfo() can throw an out-of-memory error.

Of course you're right (on both accounts).

Here's a version with a bunch of PG_TRies thrown in.

Cheers,
Jan

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

On 20/07/12 08:59, Jan Urbański wrote:
> On 18/07/12 17:17, Heikki Linnakangas wrote:
>> On 14.07.2012 17:50, Jan Urbański wrote:
>>
>> If pg_do_encoding_conversion() throws an error, you don't get a chance
>> to call Py_DECREF() to release the string. Is that a problem?
>>
>> If an error occurs in PLy_traceback(), after incrementing
>> recursion_depth, you don't get a chance to decrement it again. I'm not
>> sure if the Py* function calls can fail, but at least seemingly trivial
>> things like initStringInfo() can throw an out-of-memory error.
>
> Of course you're right (on both accounts).
>
> Here's a version with a bunch of PG_TRies thrown in.

Silly me, playing tricks with postincrements before fully waking up.

Here's v3, with a correct inequality test for exceeding the traceback 
recursion test.

J

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.