Re: Possible major bug in PlPython (plus some other ideas)

* Kevin Jacobs (jacobs@penguin.theopalgroup.com) [011109 10:53]:
> 
> On Fri, 9 Nov 2001, Bradley McLean wrote:
> 
> Actually, I'm mostly unaware of how createlang works, but I assumed that
> plpython was installed as a trusted backend.  Can anyone clarify _exactly_
> what is meant by a 'trusted'?

IANA Expert here.  createlang (shell script, just read) defines plpython
as "TRUSTED", but you can install it either way with CREATE LANGUAGE.

Looking at the pl/tcl and pl/perl implementations, plpython.c needs to
look at the Form_pg_language->lanpltrusted element to get the flag.

Then, plprocedure_compile needs to be modified to set up either exec or
r_exec as appropriate.  Also looks like PLy_init and it's sub procedures
need some rework as well.

Note that I'm just another newcomer who has submitted one more patch
than you, but has a keen interest in a strong python PL.

> To fix this issue, is it better to create a C-API method override to RExec
> or should we implement a real plpy.py module that does this (and some of the
> rest of my suggestions).  I'd prefer the module, since it is _much_ easier
> to do, though it does incur some extra overhead.

I think we need to continue to work in the C-API, and not at the Python
level.  It's not *that* much harder, and I don't see a continuing need
to work at the python level after this situation is fixed.

> Virtually all the other basic types will usually do the right thing.
> We could add support for array conversions:
> 
>    Python [1,2,3] => Postgres INTEGER[] {1,2,3}
> 
> Can you think of any others?

No, and I couldn't before; just asking the design question.

> > Three weeks ago, I really wanted a feature like this, but I've slowly
> > been convincing myself that it's a bad idea.  Consider the effects of
> > a race condition in user generated threaded python code that results
> > in a deadlock within the backend.
> 
> I'm not completely sold on this either.  I'm going to let this idea
> percolate for a while and wait to see if I can find a need for it.

Sounds like a plan.

> Other than not having the same modules available, what is annoying you most?

CREATE FUNCTION one() returns int4 AS '
result = plpy.execute("SELECT 1 as one")
return result[0]["one"]
' language 'plpython';

vs.

from pg import DB
def one():   db = DB(dbname="foo")   result = db.query("SELECT 1 as one")   return result.getresult()[0][0]

----- 

query vs execute, getresult() in one case and not the other,
results by column name vs column index.

Again, I'm not sure I have a strong proposal to fix it, since
the two APIs run in very different environments, and one is
influenced by the python database api, the other by the postgres
SPI api.  I'm just whining because when they're up in two
emacs windows, I'm forever putting the wrong one in the wrong
place.

> > 7)  Would it be completely impossible to make an extra-global
> > dictionary that was shared between multiple back-ends (place
> > in shared memory)?  Anything I'm likely to place in GD, I'm likely
> > to initialize once and want in all backends.
> 
> It will be very tricky to do in full generality.  Anything that is inserted
> into the "shared global" dictionary will have to be copied, re-allocated on
> the shared heap, and hidden from the garbage collector.  This is fairly easy
> to do for a simple types {int,float,string}; much harder for more complex
> types.

Agreed it would be tricky.  But is it useful?  Would write-once (constant)
semantics help?  

-Brad