plpython

Greetings,
I've recently been spending some quality time with the plpython module, and I think I'm well on the road to an improved
versionof it(although, nothing about a trusted variant).  By improved, I mostly mean cleaned up, and reorganized..
 

Here are some of the changes that I have made in my own version:
Compilation and execution have been greatly simplified and should be faster(at least execution should be).Caching of
compiledcode no longer references a Python dictionary(PLyProcedureCache). The handler keeps its own vector of procedure
structs(shouldbe faster, and is trivial).Removal of plpython generated dictionaries SD and GD. They don't seem be very
useful,as they are forgotten when the postmaster exits and not remembered when a new one starts. SD is questionable,
does/didanyone find SD very useful? GD seems almost pointless as the global keyword should be sufficient. Although, I
dothink there was a mention of GD being "safe globals", but I don't know why it would be safer than "global
var".Removalof the built-in "plpy" python module that plpython creates. This is done because it provides interfaces to
pgsqlfunctions that I feel should be located elsewhere; elsewhere being another python module. I've already generated a
preliminaryinterface to elog and SPI_* with SWIG that at first glance seems quite functional(it links, and is at least
ableto properly call elog, I haven't really tested SPI).Improvement to tracebacks, as it now NOTICE's the python
tracebacks(Thereis already an ERROR, so I don't think WARNING is necessary). PLy_traceback, originally, seemed to
ignorethe tb of the PyErr_Fetch.Removal of plpython type conversion routines and data structures. This was done because
Ifelt that there was a better way to do it. Not sure what yet, as it is one of my questions to the list, but it will
probablyend up being a similar implementation.I also plan to make some changes to trigger handling, but I haven't done
anythingworth mentioning yet..
 


Type conversion
plpython's current type conversion implementation appears to be dependent on strings as the common format. This is
fine,but not very extensible as is, unless you don't mind explicitly parsing strings inside each function that takes an
unsupporteddata type.I was thinking that a better solution would be creating a python object type inside the database.
Thusallowing users to write casts to and from non-standard or unimplemented data types with little difficulty(well,
maybesome :). This would allow conversion in an extensible way, which doesn't require modification to plpython. Storage
couldbe easily achieved by pickling the object.Another thought would be to just pass valid PyObject pointers in and out
ofconversion procedures, effectively disallowing storage(outside the process in which the object was created in),
unlessit is possible to have a persistent storage mechanism that makes it possible to go through pickle?.?..(yeah, I'm
newto pgsql dev).
 


Python PostgreSQL Interface
plpython, currently, implements its own built-in module to interface with a few pgsql routines, and it works, but I
feelit should be located elsewhere, as I said before.For the most part, I can only see most people using elog, and SPI
withinplpy, but perhaps that is too narrow of a view. Perhaps it would be useful to many to have access to some backend
routinesthrough plpy, but I'm not sure and that is why I'm asking the list.How far should such an PostgreSQL interface
modulego?What should its name be if full/semi-full interface is created? I was thinking simply py-pgsql as the package
name,and the module name, of course, would be pgsql.What should the name be if it was only elog and SPI? py-pgspi?I'm
leaningtowards py-pgsql, a partial interface consisting of elog and SPI and perhaps a few other useful routines. But
havethe module as a package as to allow easy extensions to the package as subpackages..From this interface, a DB-API
2.0compatible SPI interface will come as well.
 

My version has a short ways to go before it is ready for usage, but if you want to see what I've done, just drop me an
e-mail.


Comments? Criticisms? Feature suggestions?
Anyone else doing significant work on plpython?


-James