Обсуждение: improve type conversion of SPI_processed in Python

Here is a patch to improves how PL/Python deals with very large values
of SPI_processed.  The previous code converts anything that does not fit
into a C long into a Python float.  But Python long has unlimited
precision, so we should be using that instead.  And in Python 3, int and
long as the same, so there is no need to deal with any variations anymore.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> Here is a patch to improves how PL/Python deals with very large values
> of SPI_processed.  The previous code converts anything that does not fit
> into a C long into a Python float.  But Python long has unlimited
> precision, so we should be using that instead.  And in Python 3, int and
> long as the same, so there is no need to deal with any variations anymore.

I took a quick look at this.  +1 for returning Python long all the time,
but I wonder why the Python version dependency.  Our existing function
PLyLong_FromInt64() believes that PyLong_FromLongLong is unconditionally
available.  I'd be inclined to code PLyObject_FromUint64() as an exact
analog of PLyLong_FromInt64(), ie

    /* on 32 bit platforms "unsigned long" may be too small */
    if (sizeof(uint64) > sizeof(unsigned long))
        return PyLong_FromUnsignedLongLong(DatumGetUInt64(d));
    else
        return PyLong_FromUnsignedLong(DatumGetUInt64(d));

and let Python worry about how to optimize the conversion.

So far as I can tell from
https://docs.python.org/2/c-api/long.html
these functions are available as far back as we could need.
If the buildfarm tells us otherwise, we could deal with it then.

            regards, tom lane

On 1/9/18 15:54, Tom Lane wrote:
> Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
>> Here is a patch to improves how PL/Python deals with very large values
>> of SPI_processed.  The previous code converts anything that does not fit
>> into a C long into a Python float.  But Python long has unlimited
>> precision, so we should be using that instead.  And in Python 3, int and
>> long as the same, so there is no need to deal with any variations anymore.
> 
> I took a quick look at this.  +1 for returning Python long all the time,
> but I wonder why the Python version dependency.

To keep returning an int in Python 2 in the cases it fits.  Maybe that's
overkill.

> Our existing function
> PLyLong_FromInt64() believes that PyLong_FromLongLong is unconditionally
> available.

Interesting.  I had coded this to account for the possibility that long
long does not exist on a 64-bit platform, but that situation probably
died with the first Alpha or something.

> I'd be inclined to code PLyObject_FromUint64() as an exact
> analog of PLyLong_FromInt64(), ie
> 
>     /* on 32 bit platforms "unsigned long" may be too small */
>     if (sizeof(uint64) > sizeof(unsigned long))
>         return PyLong_FromUnsignedLongLong(DatumGetUInt64(d));
>     else
>         return PyLong_FromUnsignedLong(DatumGetUInt64(d));
> 
> and let Python worry about how to optimize the conversion.

Would that even be necessary?  Why not use the LongLong variant all the
time then?

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 1/9/18 15:54, Tom Lane wrote:
>> I'd be inclined to code PLyObject_FromUint64() as an exact
>> analog of PLyLong_FromInt64(), ie
>> 
>> /* on 32 bit platforms "unsigned long" may be too small */
>> if (sizeof(uint64) > sizeof(unsigned long))
>>     return PyLong_FromUnsignedLongLong(DatumGetUInt64(d));
>> else
>>     return PyLong_FromUnsignedLong(DatumGetUInt64(d));
>> 
>> and let Python worry about how to optimize the conversion.

> Would that even be necessary?  Why not use the LongLong variant all the
> time then?

I've not looked at the Python internals to see if one is noticeably faster
than the other, but if it isn't, yeah we could simplify that.  In any case
my main point is let's keep these two functions using similar approaches.

            regards, tom lane

On 1/12/18 11:06, Tom Lane wrote:
>> Would that even be necessary?  Why not use the LongLong variant all the
>> time then?
> 
> I've not looked at the Python internals to see if one is noticeably faster
> than the other, but if it isn't, yeah we could simplify that.  In any case
> my main point is let's keep these two functions using similar approaches.

I ran extensive tests on this and went ahead with the simplified versions.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services