Re: tables with 300+ partitions

Поиск
Список
Период
Сортировка
От Heikki Linnakangas
Тема Re: tables with 300+ partitions
Дата
Msg-id 47277E72.6010101@enterprisedb.com
обсуждение исходный текст
Ответ на tables with 300+ partitions  (Pablo Alcaraz <pabloa@laotraesquina.com.ar>)
Список pgsql-performance
Pablo Alcaraz wrote:
> I had a big big big table. I tried to divide it in 300 partitions with
> 30M rows each one. The problem was when I used the table to insert
> information: the perfomance was LOW.

That's very vague. What exactly did you do? Just inserted a few rows, or
perhaps a large bulk load of millions of rows? What was the bottleneck,
disk I/O or CPU usage? How long did the operation take, and how long did
you expect it to take?

> I did some testing. I created a 300 partitioned empty table. Then, I
> inserted some rows on it and the perfomance was SLOW too.
>
> SLOW = 1% perfomance compared with a non partitioned table. That is too
> much.
>
> Then, I did a 10 partitioned table version with 30M rows each one and I
> inserted rows there. The performance was the same that the no
> partitioned table version.

That suggests that the CPU time is spent in planning the query, possibly
in constraint exclusion. But that's a very different scenario from
having millions of rows in each partition.


> I suspect there is a lock problem there. I think every SQL command do a
> lock to ALL the partitions so the perfomance with concurrent inserts and
> updates are far worst than the no partitioned version.

Every query takes an AccessShareLock on each partition, but that doesn't
prevent concurrent inserts or updates, and acquiring the locks isn't
very expensive. In other words: no, that's not it.

--
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

В списке pgsql-performance по дате отправления:

Предыдущее
От: Pablo Alcaraz
Дата:
Сообщение: tables with 300+ partitions
Следующее
От: Guillaume Lelarge
Дата:
Сообщение: Re: Optimizing PostgreSQL for Windows