Re: parallel joins, and better parallel explain

Поиск
Список
Период
Сортировка
От Simon Riggs
Тема Re: parallel joins, and better parallel explain
Дата
Msg-id CANP8+jJfV8HP7ZNvfbKNd5tDd5fRLw_BwnLV-4EuDmk+VAn9Dw@mail.gmail.com
обсуждение исходный текст
Ответ на Re: parallel joins, and better parallel explain  (Robert Haas <robertmhaas@gmail.com>)
Список pgsql-hackers
On 30 November 2015 at 17:52, Robert Haas <robertmhaas@gmail.com> wrote:
 
My idea is that you'd end up with a plan like this:

Gather
-> Hash Join
  -> Parallel Seq Scan
  -> Parallel Hash
    -> Parallel Seq Scan

Not only does this build only one copy of the hash table instead of N
copies, but we can parallelize the hash table construction itself by
having all workers insert in parallel, which is pretty cool. 

Hmm. If the hash table is small it should be OK to keep it locally. If its larger, we need the shared copy. Seems like we can do the math on when to use each kind of hash table.... build it in shmem and then copy locally if needed. 

Another way might to force the hash table into N batches, then give each scan the task of handling one batch. That would allow a much larger hash table to still be kept locally, moving the problem towards routing the data.

I'm not immediately convinced by the coolness of loading the hash table in parallel. A whole class of bugs could be avoided if we choose not to, plus the hash table is frequently so small a part of the HJ that its not going to gain us very much. 

The other way to look at what you've said here is that you don't seem to have a way of building the hash table in only one process, which concerns me.

What I can confirm at this point is that
I've thought about the problem you're asking about here, and that
EnterpriseDB intends to contribute code to address it.

Good 

--
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Alvaro Herrera
Дата:
Сообщение: Re: [DOCS] max_worker_processes on the standby
Следующее
От: Alvaro Herrera
Дата:
Сообщение: Re: Re: In-core regression tests for replication, cascading, archiving, PITR, etc.