Re: [HACKERS] Faster methods for getting SPI results
От
Jon Nelson
Тема
Re: [HACKERS] Faster methods for getting SPI results
Дата
Msg-id
CAKuK5J2JXi5tGe-XB6gt1PT=hwatWGPOhV2=-XP7sxRUL67gAw@mail.gmail.com
Ответ на
Re: [HACKERS] Faster methods for getting SPI results (Peter Eisentraut)
Список
Дерево обсуждения
[HACKERS] Faster methods for getting SPI results Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results Jim Nasby <jim.nasby@openscg.com>
Re: [HACKERS] Faster methods for getting SPI results Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results Jon Nelson <jnelson+pgsql@jamponi.net>
Re: [HACKERS] Faster methods for getting SPI results Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results Chapman Flack <chap@anastigmatix.net>
Re: [HACKERS] Faster methods for getting SPI results Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Faster methods for getting SPI results Chapman Flack <chap@anastigmatix.net>
Re: [HACKERS] Faster methods for getting SPI results Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Faster methods for getting SPI results Daniel Gustafsson <daniel@yesql.se>
Re: [HACKERS] Faster methods for getting SPI results Chapman Flack <chap@anastigmatix.net>
Re: Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results (460% improvement) Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Faster methods for getting SPI results (460% improvement) Craig Ringer <craig@2ndquadrant.com>
Re: [HACKERS] Faster methods for getting SPI results (460% improvement) Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: [HACKERS] Faster methods for getting SPI results (460% improvement) Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Andres Freund <andres@anarazel.de>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <jim@nasby.net>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Andres Freund <andres@anarazel.de>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Jim Nasby <jim.nasby@openscg.com>
Re: [HACKERS] Faster methods for getting SPI results (460%improvement) Andres Freund <andres@anarazel.de>
Re: [HACKERS] Faster methods for getting SPI results (460% improvement) Jim Nasby <jim.nasby@openscg.com>
Re: [HACKERS] Faster methods for getting SPI results Jim Nasby <Jim.Nasby@BlueTreble.com>
On Thu, Mar 2, 2017 at 10:03 AM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
-- On 12/20/16 23:14, Jim Nasby wrote:
> I've been looking at the performance of SPI calls within plpython.
> There's a roughly 1.5x difference from equivalent python code just in
> pulling data out of the SPI tuplestore. Some of that is due to an
> inefficiency in how plpython is creating result dictionaries, but fixing
> that is ultimately a dead-end: if you're dealing with a lot of results
> in python, you want a tuple of arrays, not an array of tuples.
There is nothing that requires us to materialize the results into an
actual list of actual rows. We could wrap the SPI_tuptable into a
Python object and implement __getitem__ or __iter__ to emulate sequence
or mapping access.
Python objects have a small (but non-zero) overhead in terms of both memory and speed. A built-in dictionary is probably one of the least-expensive (memory/cpu) choices, although how the dictionary is constructed also impacts performance. Another choice is a tuple.
Avoiding Py_BuildValue(...) in exchange for a bit more complexity (via PyTuple_New(..) and PyTuple_SetItem(...)) is also a nice performance win in my experience.
Jon
В списке pgsql-hackers по дате отправления
От: Peter Eisentraut
Дата: