Обсуждение: adding import in pl/python function

Поиск
Список
Период
Сортировка

adding import in pl/python function

От
Szymon Guz
Дата:
Hi,
I'm wondering if it would be OK to change the procedure code before execution. I'm thinking about adding magically an import at the beginning of a function.

Currently numeric arguments passed to the procedure are converted into floats. This is not good, as it causes loss of information.

The proposed solution in code comment is "maybe use a string?".

I'm thinking about something else. We could convert it into Decimal (http://docs.python.org/2/library/decimal.html) class in Python. Unfortunately this class requires import like `from decimal import Decimal` from a standard Python library.

I'm wondering if it would be a good idea to do it like this. It shouldn't fail even with the trusted version of pl/python, as I'd rather see the trusted version to allow importing packages from standard library.

regards,
Szymon

Re: adding import in pl/python function

От
Claudio Freire
Дата:
On Fri, May 24, 2013 at 4:10 PM, Szymon Guz <mabewlun@gmail.com> wrote:
>
> I'm thinking about something else. We could convert it into Decimal
> (http://docs.python.org/2/library/decimal.html) class in Python.
> Unfortunately this class requires import like `from decimal import Decimal`
> from a standard Python library.
>
> I'm wondering if it would be a good idea to do it like this. It shouldn't
> fail even with the trusted version of pl/python, as I'd rather see the
> trusted version to allow importing packages from standard library.


Why would passing a decimal require an import?

The extension itself needs a reference to Decimal, to build them, but
the procedure's context doesn't need to have it.



Re: adding import in pl/python function

От
Szymon Guz
Дата:

On 24 May 2013 21:14, Claudio Freire <klaussfreire@gmail.com> wrote:
On Fri, May 24, 2013 at 4:10 PM, Szymon Guz <mabewlun@gmail.com> wrote:
>
> I'm thinking about something else. We could convert it into Decimal
> (http://docs.python.org/2/library/decimal.html) class in Python.
> Unfortunately this class requires import like `from decimal import Decimal`
> from a standard Python library.
>
> I'm wondering if it would be a good idea to do it like this. It shouldn't
> fail even with the trusted version of pl/python, as I'd rather see the
> trusted version to allow importing packages from standard library.


Why would passing a decimal require an import?

The extension itself needs a reference to Decimal, to build them, but
the procedure's context doesn't need to have it.

Hm... maybe you're right. I think I don't understand fully how the procedures are executed, and I need to read more to get it.


thanks,
Szymon

Re: adding import in pl/python function

От
Claudio Freire
Дата:
On Fri, May 24, 2013 at 4:22 PM, Szymon Guz <mabewlun@gmail.com> wrote:
> Hm... maybe you're right. I think I don't understand fully how the
> procedures are executed, and I need to read more to get it.


Well, it's easy.

Instead of PLyFloat_FromNumeric[0], you can make a
PLyDecimal_FromNumeric. There, you'd do with the Python/C[1]:

PyObject *decimal = PyImport_Import("decimal");
PyObject *decimal_dict = PyModule_GetDict(decimal);
PyObject *decimal_ctor = PyDict_GetItemString(decimal_dict, "Decimal");

And invoke it with a string rep of your Numeric:

PyObject *value = PyObject_CallFunction(decimal_ctor, "S", string_value);

Add of course all kinds of error checking and reference count boiler
plate, and you'd have a very dumb version of it.

To make it more "pro", you'd want to do all that stuff to get
decimal_ctor only at initialization time. Especially since you don't
want to fumble with the import lock right there in _FromNumeric.

And to make it totally "pro", you can even freeze Decimal (using
pyfreeze) if you'd like. I would only do this in contexts where you
don't have a stdlib of course. Not sure whether windows falls into
that category. Linux doesn't.


[0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518
[1] http://docs.python.org/2/c-api/import.html



Re: adding import in pl/python function

От
Szymon Guz
Дата:

On 24 May 2013 21:46, Claudio Freire <klaussfreire@gmail.com> wrote:
On Fri, May 24, 2013 at 4:22 PM, Szymon Guz <mabewlun@gmail.com> wrote:
> Hm... maybe you're right. I think I don't understand fully how the
> procedures are executed, and I need to read more to get it.


Well, it's easy.

Instead of PLyFloat_FromNumeric[0], you can make a
PLyDecimal_FromNumeric. There, you'd do with the Python/C[1]:

PyObject *decimal = PyImport_Import("decimal");
PyObject *decimal_dict = PyModule_GetDict(decimal);
PyObject *decimal_ctor = PyDict_GetItemString(decimal_dict, "Decimal");

And invoke it with a string rep of your Numeric:

PyObject *value = PyObject_CallFunction(decimal_ctor, "S", string_value);

Add of course all kinds of error checking and reference count boiler
plate, and you'd have a very dumb version of it.

To make it more "pro", you'd want to do all that stuff to get
decimal_ctor only at initialization time. Especially since you don't
want to fumble with the import lock right there in _FromNumeric.

And to make it totally "pro", you can even freeze Decimal (using
pyfreeze) if you'd like. I would only do this in contexts where you
don't have a stdlib of course. Not sure whether windows falls into
that category. Linux doesn't.


[0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518
[1] http://docs.python.org/2/c-api/import.html


Thanks, I will take a look at this, looks pretty easy. However testing on Windows will be pretty funny :)

thanks,
Szymon

Re: adding import in pl/python function

От
Peter Eisentraut
Дата:
On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote:
> Well, it's easy.
> 
> Instead of PLyFloat_FromNumeric[0], you can make a
> PLyDecimal_FromNumeric. 

Please send a patch.  This would be a welcome addition.





Re: adding import in pl/python function

От
Claudio Freire
Дата:
On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote:
>> Well, it's easy.
>>
>> Instead of PLyFloat_FromNumeric[0], you can make a
>> PLyDecimal_FromNumeric.
>
> Please send a patch.  This would be a welcome addition.


I can write it blind as I have more than enough experience with
CPython, but I don't use PLPython so I can't perform extensive
testing.
If someone's willing to do the testing, by all means.



Re: adding import in pl/python function

От
Peter Eisentraut
Дата:
On Mon, 2013-05-27 at 20:43 -0300, Claudio Freire wrote:
> On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote:
> >> Well, it's easy.
> >>
> >> Instead of PLyFloat_FromNumeric[0], you can make a
> >> PLyDecimal_FromNumeric.
> >
> > Please send a patch.  This would be a welcome addition.
> 
> 
> I can write it blind as I have more than enough experience with
> CPython, but I don't use PLPython so I can't perform extensive
> testing.
> If someone's willing to do the testing, by all means.

Yes please.




Re: adding import in pl/python function

От
Szymon Guz
Дата:
On 28 May 2013 01:55, Peter Eisentraut <peter_e@gmx.net> wrote:
On Mon, 2013-05-27 at 20:43 -0300, Claudio Freire wrote:
> On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote:
> >> Well, it's easy.
> >>
> >> Instead of PLyFloat_FromNumeric[0], you can make a
> >> PLyDecimal_FromNumeric.
> >
> > Please send a patch.  This would be a welcome addition.
>
>
> I can write it blind as I have more than enough experience with
> CPython, but I don't use PLPython so I can't perform extensive
> testing.
> If someone's willing to do the testing, by all means.

Yes please.


I'm working on that.

- Szymon