Expression based aggregate transition / combine function invocation

Поиск
Список
Период
Сортировка
От Andres Freund
Тема Expression based aggregate transition / combine function invocation
Дата
Msg-id 20171128003121.nmxbm2ounxzb6n2t@alap3.anarazel.de
обсуждение исходный текст
Ответы Re: Expression based aggregate transition / combine functioninvocation
Список pgsql-hackers
Hi,

this is part of my work to make expression evaluation JITable. In a lot
of analytics queries the major bottleneck is transition function
invocation (makes sense, hardly anyone wants to see billions of
rows). Therefore for JITing to be really valuable transition function
stuff needs to be JITable.

Excerpt from the preliminary commit message:

  Previously aggregate transition and combination functions were invoked
  by special case code in nodeAgg.c, evaluting input and filters
  separately using the expression evaluation machinery. That turns out
  to not be great for performance for several reasons:
  - repeated expression evaluations have some cost
  - the transition functions invocations are poorly predicted
  - filter and input computation had to be done separately
  - the special case code made it hard to implement JITing of the whole
    transition function invocation

  Address this by building one large expression that computes input,
  evaluates filters, and invokes transition functions.

  This leads to moderate speedups in queries bottlenecked by aggregate
  computations, and enables large speedups for similar cases once JITing
  is done.


While this gets rid of a substantial amount of duplication between the
infrastructure for transition and combine functions, it still increases
codesize a bit.

Todo / open Questions:
- Location of transition function building functions. Currently they're
  in execExpr.c. That allows not to expose a bunch of functions local to
  it, but requires exposing some aggregate structs to the world. We
  could go the other way round as well.

- Right now we waste a bunch of time by having to access transition
  states indexed by both grouping set number and the transition state
  offset therein. It'd be nicer if we could cheaply reduce the number of
  indirections, but I can't quite see how without adding additional
  complications.


Here's some example tpch Q01 timings:
master: 11628 ms (best of three)
patches: 10330 ms (best of three)

other tpch queries are similar, aggregate improvement is a factor of
x 1.04.

Greetings,

Andres Freund

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Masahiko Sawada
Дата:
Сообщение: Re: [HACKERS] Challenges preventing us moving to 64 bit transactionid (XID)?
Следующее
От: David Steele
Дата:
Сообщение: Re: [HACKERS] Timeline ID in backup_label file