Обсуждение: PQntuples for older version (2.2.1) of psycopg2 with ctypes
I am using cursor.copy_expert and I would like to be able to use cursor.rowcount but most of the systems I am targeting have older psycopg2 versions on them.
According to the following URL rowcount was updated for use with copy_expert in 2.5.3...
https://psycopg.lighthouseapp.com/projects/62710/tickets/180-return-number-of-rows-for-copy-operationsInspired by:
I wonder if it is possible to use ctypes to do this... Something like this maybe:
import ctypes
libpq = ctypes.CDLL('libpq.so.5')
conn = psycopg2.connect(foo)
cursor = conn.cursor()
cursor.copy_expert(bar)
rowcount = libpq.PQntuples(???)
I can "read" the psycopg2 source but I am not fluent. I am having trouble working out what 'pgres' is. Is it the fileno() the backend_process_id() etc...
So far all I have manged to do is segfault my python processes. Woot!
Is 'pgres' exposed by the Cursor or Connection classes somewhere?
Regards,
-Alan
On Fri, Feb 19, 2016 at 11:34 PM, Alan Evans <alanwevans@gmail.com> wrote: > I am using cursor.copy_expert and I would like to be able to use > cursor.rowcount but most of the systems I am targeting have older psycopg2 > versions on them. > > According to the following URL rowcount was updated for use with copy_expert > in 2.5.3... > https://psycopg.lighthouseapp.com/projects/62710/tickets/180-return-number-of-rows-for-copy-operations > > Inspired by: > https://github.com/sunlightlabs/psycopg2-ctypes/blob/master/psycopg2ct/_impl/cursor.py > > I wonder if it is possible to use ctypes to do this... Something like this > maybe: > > import psycopg2 > import ctypes > libpq = ctypes.CDLL('libpq.so.5') > > conn = psycopg2.connect(foo) > cursor = conn.cursor() > cursor.copy_expert(bar) > > rowcount = libpq.PQntuples(???) > > I can "read" the psycopg2 source but I am not fluent. I am having trouble > working out what 'pgres' is. Is it the fileno() the backend_process_id() > etc... > > So far all I have manged to do is segfault my python processes. Woot! > > Is 'pgres' exposed by the Cursor or Connection classes somewhere? Funny hack :) The pgres is on the cursor: <https://github.com/psycopg/psycopg2/blob/2_2_1/psycopg/cursor.h#L60>. Thankfully (at least in 2.2.1 but I guess it never changed) it is located right after the cur.description pointer, so we can use it to find the offset pgres offset in the structure: import psycopg2 cnn = psycopg2.connect('') cur = cnn.cursor() cur.execute("select 1") import ctypes, struct, sys curdump = ctypes.string_at(id(cur), sys.getsizeof(cur)) descptr = struct.pack('P', id(cur.description)) resoff = curdump.find(descptr) + len(descptr) The offset is fixed for a given build so it won't change at runtime. You can inspect the cursor after the COPY operation to find the result structure and pass it to the libpq: libpq = ctypes.cdll.LoadLibrary("libpq.so") cur.execute("select generate_series(1,42)") # cur.copy_expert here curdump = ctypes.string_at(id(cur), sys.getsizeof(cur)) resptr = struct.unpack('P', curdump[resoff:resoff+len(descptr)])[0] print libpq.PQntuples(resptr) # 42 Hope this helps :) -- Daniele