Deadlock between concurrent index builds on different tables

Поиск
Список
Период
Сортировка
От Jeremy Finzel
Тема Deadlock between concurrent index builds on different tables
Дата
Msg-id CAMa1XUhHjCv8Qkx0WOr1Mpm_R4qxN26EibwCrj0Oor2YBUFUTg@mail.gmail.com
обсуждение исходный текст
Ответы Re: Deadlock between concurrent index builds on different tables  (Alban Hertroys <haramrae@gmail.com>)
Re: Deadlock between concurrent index builds on different tables  (Martin Marques <martin.marques@2ndquadrant.com>)
Re: Deadlock between concurrent index builds on different tables  (Jeff Janes <jeff.janes@gmail.com>)
Список pgsql-general
Дерево обсуждения
I am attempting to build several indexes in parallel, guaranteeing that I never build one on the same table twice.  I understand I can't build two on the same table at once or I will get a deadlock.  However, I am also getting a deadlock when doing several in parallel on different tables.

Here is an example of the error I am getting:

2017-12-22 15:48:07.669 CST,"CREATE INDEX",2017-12-22 15:48:02 CST,8/32,0,ERROR,40P01,"deadlock detected","Process 4470 waits for ShareLock on virtual transaction 4/262; blocked by process 4466.
Process 4466 waits for ShareLock on virtual transaction 8/32; blocked by process 4470.
Process 4470: CREATE INDEX CONCURRENTLY index_foo_on_created_at ON foo USING btree (created_at);
Process 4466: CREATE INDEX CONCURRENTLY index_bar_on_id ON bar USING btree (id);","See server log for query details.",,,,"CREATE INDEX CONCURRENTLY index_foo_on_created_at ON foo USING btree (created_at);",,,""

Here is my process:
  • Kick off one index build and background 1 second apart, using a queue table to determine what indexes to build and what is finished
  • When I determine if a index can be built, I first check if there is another already in build on target table before starting another
  • After the index is built in each workflow, I mark it as built in a queue table
I assume that one of the indexes (the earlier transaction) is building and taking so long that several finished indexes end up waiting, and there is perhaps a limit to that?

Any insight appreciated.

Thanks,
Jeremy

В списке pgsql-general по дате отправления:

Предыдущее
От: "Joshua D. Drake"
Дата:
Сообщение: Re: Does postgresq database supports reading data from the same tablefrom many observable simultanousely?
Следующее
От: Alban Hertroys
Дата:
Сообщение: Re: Deadlock between concurrent index builds on different tables