Обсуждение: Re: VOPS-2.0
On Wed, Nov 28, 2018 at 01:01:03PM +0300, Konstantin Knizhnik wrote: > Hi, > > I want to introduce new version of VOPS extension for Postgres (vectorized > operations) providing new, more convenient way of usage: > auto-substitution of projections. [Announce post moved to hackers.] I remember the good performance numbers from this feature. How does this interact with the JIT executor feature, which is also designed to speed up the executor? Is it something that can be combined with JIT? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 2018-Nov-28, Bruce Momjian wrote: > On Wed, Nov 28, 2018 at 01:01:03PM +0300, Konstantin Knizhnik wrote: > > Hi, > > > > I want to introduce new version of VOPS extension for Postgres (vectorized > > operations) providing new, more convenient way of usage: > > auto-substitution of projections. > > [Announce post moved to hackers.] > > I remember the good performance numbers from this feature. How does > this interact with the JIT executor feature, which is also designed to > speed up the executor? Is it something that can be combined with JIT? ISTM that VOPS is a temporary hack that we'll need to include in some form eventually, but that basing it on top of pluggable storage would be a better way forward than the current proposal. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 28.11.2018 16:18, Bruce Momjian wrote:
> On Wed, Nov 28, 2018 at 01:01:03PM +0300, Konstantin Knizhnik wrote:
>> Hi,
>>
>> I want to introduce new version of VOPS extension for Postgres (vectorized
>> operations) providing new, more convenient way of usage:
>> auto-substitution of projections.
> [Announce post moved to hackers.]
>
> I remember the good performance numbers from this feature. How does
> this interact with the JIT executor feature, which is also designed to
> speed up the executor? Is it something that can be combined with JIT?
>
JIT and vector execution are more or less alternative solutions of the
same problem: eliminate interpretation overhead.
In JIT it was done be replacing interpretation with native code execution.
In VOPS - by processing more elements by each operation.
This is the implementation of binary operations in VOPS:
PG_FUNCTION_INFO_V1(vops_##TYPE##_##OP); \
Datum vops_##TYPE##_##OP(PG_FUNCTION_ARGS) \
{ \
vops_##TYPE* left = (vops_##TYPE*)PG_GETARG_POINTER(0); \
vops_##TYPE* right = (vops_##TYPE*)PG_GETARG_POINTER(1); \
vops_##TYPE* result = (vops_##TYPE*)palloc(sizeof(vops_##TYPE));\
int i; \
for (i = 0; i < TILE_SIZE; i++) result->payload[i] =
left->payload[i] COP right->payload[i]; \
So it is just loop through TILE_SIZE elements (128 by default) for which
compiler generates optimal code.
Interpretation overhead is not eliminate but is divided by 128.
In principle this two approaches can be combined (as it was done for
example in HyPer).
But practically, at least for expressions evaluation is has not so much
sense.
Because code generated by compiler for vector function mentioned above
at better (or at least not worser) than one generated by LLVM.
And 100 times reduced interpretation overhead is not noticeable.
Concerning results: now Postgres with JIT is executing Q1 about 30%
faster than without JIT.
VOPS is executing Q1 about 10 times faster.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
On 28.11.2018 16:45, Alvaro Herrera wrote: > On 2018-Nov-28, Bruce Momjian wrote: > >> On Wed, Nov 28, 2018 at 01:01:03PM +0300, Konstantin Knizhnik wrote: >>> Hi, >>> >>> I want to introduce new version of VOPS extension for Postgres (vectorized >>> operations) providing new, more convenient way of usage: >>> auto-substitution of projections. >> [Announce post moved to hackers.] >> >> I remember the good performance numbers from this feature. How does >> this interact with the JIT executor feature, which is also designed to >> speed up the executor? Is it something that can be combined with JIT? > ISTM that VOPS is a temporary hack that we'll need to include in some > form eventually, but that basing it on top of pluggable storage would be > a better way forward than the current proposal. > I am not sure that pluggable storage API will solve all problems. Right now there are the following challenges with VOPS: 1. Provide convenient way of defining projections for user: not doable now at extension level. 2. Vector executor: in principle it is possible to reimplement all (or most) nodes to use vector operations and somehow make optimizer to generate them. But it requires huge amount of work. And I am not sure that all can be done et extension level just by defining new custom nodes. 3. Update projections. It can be efficiently done only using bulk inserts, but it means that projections are not up-to-date with original table. Hiding internals inside pluggable storage doesn't automatically solve this problem. Actually VOPS approach has least of all problems with storage layer, because VOPS projections are stored as normal Postgres tables. The main challenge is incorporating vector operations in executor. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company