Обсуждение: adding import in pl/python function
Hi,
I'm wondering if it would be OK to change the procedure code before execution. I'm thinking about adding magically an import at the beginning of a function.
Currently numeric arguments passed to the procedure are converted into floats. This is not good, as it causes loss of information.
The proposed solution in code comment is "maybe use a string?".
I'm thinking about something else. We could convert it into Decimal (http://docs.python.org/2/library/decimal.html) class in Python. Unfortunately this class requires import like `from decimal import Decimal` from a standard Python library.
I'm wondering if it would be a good idea to do it like this. It shouldn't fail even with the trusted version of pl/python, as I'd rather see the trusted version to allow importing packages from standard library.
regards,
Szymon
On Fri, May 24, 2013 at 4:10 PM, Szymon Guz <mabewlun@gmail.com> wrote: > > I'm thinking about something else. We could convert it into Decimal > (http://docs.python.org/2/library/decimal.html) class in Python. > Unfortunately this class requires import like `from decimal import Decimal` > from a standard Python library. > > I'm wondering if it would be a good idea to do it like this. It shouldn't > fail even with the trusted version of pl/python, as I'd rather see the > trusted version to allow importing packages from standard library. Why would passing a decimal require an import? The extension itself needs a reference to Decimal, to build them, but the procedure's context doesn't need to have it.
On 24 May 2013 21:14, Claudio Freire <klaussfreire@gmail.com> wrote:
On Fri, May 24, 2013 at 4:10 PM, Szymon Guz <mabewlun@gmail.com> wrote:Why would passing a decimal require an import?
>
> I'm thinking about something else. We could convert it into Decimal
> (http://docs.python.org/2/library/decimal.html) class in Python.
> Unfortunately this class requires import like `from decimal import Decimal`
> from a standard Python library.
>
> I'm wondering if it would be a good idea to do it like this. It shouldn't
> fail even with the trusted version of pl/python, as I'd rather see the
> trusted version to allow importing packages from standard library.
The extension itself needs a reference to Decimal, to build them, but
the procedure's context doesn't need to have it.
Hm... maybe you're right. I think I don't understand fully how the procedures are executed, and I need to read more to get it.
thanks,
Szymon
On Fri, May 24, 2013 at 4:22 PM, Szymon Guz <mabewlun@gmail.com> wrote:
> Hm... maybe you're right. I think I don't understand fully how the
> procedures are executed, and I need to read more to get it.
Well, it's easy.
Instead of PLyFloat_FromNumeric[0], you can make a
PLyDecimal_FromNumeric. There, you'd do with the Python/C[1]:
PyObject *decimal = PyImport_Import("decimal");
PyObject *decimal_dict = PyModule_GetDict(decimal);
PyObject *decimal_ctor = PyDict_GetItemString(decimal_dict, "Decimal");
And invoke it with a string rep of your Numeric:
PyObject *value = PyObject_CallFunction(decimal_ctor, "S", string_value);
Add of course all kinds of error checking and reference count boiler
plate, and you'd have a very dumb version of it.
To make it more "pro", you'd want to do all that stuff to get
decimal_ctor only at initialization time. Especially since you don't
want to fumble with the import lock right there in _FromNumeric.
And to make it totally "pro", you can even freeze Decimal (using
pyfreeze) if you'd like. I would only do this in contexts where you
don't have a stdlib of course. Not sure whether windows falls into
that category. Linux doesn't.
[0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518
[1] http://docs.python.org/2/c-api/import.html
On 24 May 2013 21:46, Claudio Freire <klaussfreire@gmail.com> wrote:
On Fri, May 24, 2013 at 4:22 PM, Szymon Guz <mabewlun@gmail.com> wrote:Well, it's easy.
> Hm... maybe you're right. I think I don't understand fully how the
> procedures are executed, and I need to read more to get it.
Instead of PLyFloat_FromNumeric[0], you can make a
PLyDecimal_FromNumeric. There, you'd do with the Python/C[1]:
PyObject *decimal = PyImport_Import("decimal");
PyObject *decimal_dict = PyModule_GetDict(decimal);
PyObject *decimal_ctor = PyDict_GetItemString(decimal_dict, "Decimal");
And invoke it with a string rep of your Numeric:
PyObject *value = PyObject_CallFunction(decimal_ctor, "S", string_value);
Add of course all kinds of error checking and reference count boiler
plate, and you'd have a very dumb version of it.
To make it more "pro", you'd want to do all that stuff to get
decimal_ctor only at initialization time. Especially since you don't
want to fumble with the import lock right there in _FromNumeric.
And to make it totally "pro", you can even freeze Decimal (using
pyfreeze) if you'd like. I would only do this in contexts where you
don't have a stdlib of course. Not sure whether windows falls into
that category. Linux doesn't.
[0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518
[1] http://docs.python.org/2/c-api/import.html
Thanks, I will take a look at this, looks pretty easy. However testing on Windows will be pretty funny :)
thanks,
Szymon
On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote: > Well, it's easy. > > Instead of PLyFloat_FromNumeric[0], you can make a > PLyDecimal_FromNumeric. Please send a patch. This would be a welcome addition.
On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote: >> Well, it's easy. >> >> Instead of PLyFloat_FromNumeric[0], you can make a >> PLyDecimal_FromNumeric. > > Please send a patch. This would be a welcome addition. I can write it blind as I have more than enough experience with CPython, but I don't use PLPython so I can't perform extensive testing. If someone's willing to do the testing, by all means.
On Mon, 2013-05-27 at 20:43 -0300, Claudio Freire wrote: > On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote: > >> Well, it's easy. > >> > >> Instead of PLyFloat_FromNumeric[0], you can make a > >> PLyDecimal_FromNumeric. > > > > Please send a patch. This would be a welcome addition. > > > I can write it blind as I have more than enough experience with > CPython, but I don't use PLPython so I can't perform extensive > testing. > If someone's willing to do the testing, by all means. Yes please.
On 28 May 2013 01:55, Peter Eisentraut <peter_e@gmx.net> wrote:
Yes please.On Mon, 2013-05-27 at 20:43 -0300, Claudio Freire wrote:
> On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote:
> >> Well, it's easy.
> >>
> >> Instead of PLyFloat_FromNumeric[0], you can make a
> >> PLyDecimal_FromNumeric.
> >
> > Please send a patch. This would be a welcome addition.
>
>
> I can write it blind as I have more than enough experience with
> CPython, but I don't use PLPython so I can't perform extensive
> testing.
> If someone's willing to do the testing, by all means.
I'm working on that.
- Szymon