Обсуждение: [HACKERS] Why does plpython delay composite type resolution?
Why do functions that accept composite types delay type resolution until execution? I have a naive patch that speeds up plpy.execute() by 8% by caching interred python strings for the dictionary key names (which are repeated over and over). The next step is to just pre-allocate those strings as appropriate for the calling context, but it's not clear how to handle that for input arguments. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532)
On 12/21/2016 04:14 AM, Jim Nasby wrote: > Why do functions that accept composite types delay type resolution until > execution? I have a naive patch that speeds up plpy.execute() by 8% by > caching interred python strings for the dictionary key names (which are > repeated over and over). The next step is to just pre-allocate those > strings as appropriate for the calling context, but it's not clear how > to handle that for input arguments. Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate guess would be that it could be a cache invalidation thing. Andreas
On 12/21/16 1:55 AM, Andreas Karlsson wrote: > On 12/21/2016 04:14 AM, Jim Nasby wrote: >> Why do functions that accept composite types delay type resolution until >> execution? I have a naive patch that speeds up plpy.execute() by 8% by >> caching interred python strings for the dictionary key names (which are >> repeated over and over). The next step is to just pre-allocate those >> strings as appropriate for the calling context, but it's not clear how >> to handle that for input arguments. > > Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate > guess would be that it could be a cache invalidation thing. Won't that only happen at end of transaction? After reading the tuple queue code I'm wondering if part of the issue is anonymous records, though that doesn't make much sense since plpython doesn't support those. Given the lackluster support for arrays and composites in plpython, I suspect this is just a wart that hasn't been removed yet... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532)
Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> On 12/21/16 1:55 AM, Andreas Karlsson wrote:
>> Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate
>> guess would be that it could be a cache invalidation thing.
> Won't that only happen at end of transaction?
No.
BEGIN;SELECT plpython_function();ALTER TYPE ...;SELECT plpython_function();COMMIT;
For that matter, the plpython function could execute the ALTER itself
through SPI, or call another function that does so.
(I'm not claiming that the existing code, either in plpython or other
PLs, necessarily handles such all scenarios nicely. But we shouldn't
make it worse.)
regards, tom lane
On 12/21/16 8:39 AM, Tom Lane wrote: > Jim Nasby <Jim.Nasby@BlueTreble.com> writes: >> On 12/21/16 1:55 AM, Andreas Karlsson wrote: >>> Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate >>> guess would be that it could be a cache invalidation thing. > >> Won't that only happen at end of transaction? > > No. > > BEGIN; > SELECT plpython_function(); > ALTER TYPE ...; > SELECT plpython_function(); > COMMIT; > > For that matter, the plpython function could execute the ALTER itself > through SPI, or call another function that does so. > > (I'm not claiming that the existing code, either in plpython or other > PLs, necessarily handles such all scenarios nicely. But we shouldn't > make it worse.) Hmm... so I guess the only way we could safely handle this is if any caching of type info happened via fcinfo->flinfo->fn_extra? Would it also work if we verified pg_type.(tid,xmin) hadn't changed? (That's what plpython currently does to verify a row in pg_procedure hasn't changed.) -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532)