Re: [psycopg] speed concerns with executemany()

On 01/09/2017 11:45 AM, Jim Nasby wrote:
> On 1/9/17 10:04 AM, mike bayer wrote:
>> SQLAlchemy can definitely ignore the aggregated rowcount as most DBAPIs
>> don't support it anyway, so we can flip the flag off if we know exactly
>> what psycopg version breaks it.
>
> ISTM it'd be better to add an attribute to indicate whether rowcount was
> available or not, especially since it would be available with a page
> size of 1?


so it's a dialect-level flag which on a SQLAlchemy engine you'd see as
engine.dialect.supports_sane_multi_rowcount.  When psycopg2 first adds
the page size API, we can begin supporting it by detecting the
availability of the flag via psycopg2 version info, as well as the
default for this value - if psycopg2 is setting it to a number greater
than 1 by default, we'd want to keep that default.   If these two things
are true then the supports_sane_multi_rowcount flag would be turned off
for the dialect that imports this version of psycopg2.    SQLAlchemy at
the very least needs to respond if/when psycopg2 changes this default
upstream since if we don't, it will break our own system and tests.

The next addition to SQLAlchemy would be explicit API that allows this
value to be configured both at dialect startup time as well as a
per-statement execution option.   supports_sane_multi_rowcount would be
flagged based on how this is set up.   There may need to be changes to
ensure supports_sane_multi_rowcount is consulted local to a statement
execution context, if we allow on-the-fly modifications to it.